Project Profile

This project addresses our nation's urgent need for an agile and diverse science and technology workforce by innovating cost-effective and scalable hands-on chemistry and biochemistry laboratory training using virtual reality (VR) and mixed reality (MR) educational technologies. Learning is optimized when theory and practice are seamlessly integrated and when multiple senses (sight, sound, and touch) reinforce each other to promote long-term knowledge retention. Nowhere is this more evident than in traditional hands-on laboratory classes where students apply theoretical concepts, carry out experiments and procedures, and analyze data in an active, guided, and often open-ended manner using their eyes, ears, and hands. Unfortunately, such hands-on learning experiences are highly resource intensive and thus are severely limited in a number of ways. Dangerous, long, complicated, or expensive experiments are often difficult to offer. Students are often not free to make mistakes, redo experiments, iteratively refine hypotheses or master techniques. Lack of scheduling flexibility in traditional labs can reduce participation by low-income and underrepresented students who often work while in school. Lab instructors, industry trainers, and research mentors have limited time with students at the bench preventing them from being able to correct the most nuanced types of mistakes that students make when learning new techniques. Virtual labs, in contrast, can be designed to be less resource-intensive and allow the immediate illustration of the molecular basis for observations made at the virtual bench. Unfortunately, virtual labs lack the authentic tactile feel of doing real science experiments with real laboratory instruments and tools. In this project, researchers will use emerging high-resolution active and passive optical motion-tracking, 3D printing, and mixed reality (MR) technologies to "virtualize" actual physical lab tools that can be held and manipulated by a user in a completely naturalistic manner while interacting with the virtual world with real-time feedback and just-in-time dynamic visualizations. This project will examine whether the additional level of tactile feedback offered by such authentic hands-on MR labs together with dynamic molecular visualizations can improve student intrinsic motivation, engagement, self-efficacy, knowledge acquisition, and long-term retention in undergraduate chemistry and biochemistry courses. The project will reach out to local secondary school science teachers and their students, offer professional development, and test the scalable deployment of hands-on MR lab hardware and software to underserved universities and secondary schools, addressing the outstanding problem of disproportionate science, technology, engineering and mathematics attrition amongst these groups of learners.

This project will address the following learning research questions: 1) To what extent does authentic tactile sensory feedback of MR chemistry and biochemistry lab activities impact student learning, intrinsic motivation, and self-efficacy, and are the effects significantly different for students with different incoming levels of practical or theoretical experience with course content? 2) Do conceptually illustrative dynamic molecular visualizations presented to students immediately as they are performing "hands-on" MR experiments afford any advantages for student learning within the context of chemistry and biochemistry, and are such effects significantly different for students with different incoming levels of practical or theoretical experience with course content? Researchers will produce twelve user-selectable hands-on MR lab sub-modules distributed across four subject areas: introductory, general, organic, and biological chemistry. The content and assessments in each submodule will be user-selectable at either an introductory, intermediate, or advanced level, making it possible to examine the impact of each sub-module within multiple courses. Impact on student learning outcomes in ten different courses, most of which have existing traditional, take-home, or virtual lab activities while others are lecture-only courses, will be assessed. Formative assessments will include: standardized and validated items embedded within class quizzes and tests; internally controlled pre/mid/post/real-time assessments administered before, during, and after completion of the MR activities; online student surveys; and, for traditional lab courses, the time-to-completion or data quality achieved for real lab activities when the hands-on MR labs are used as a pre-lab activity. Results from the assessments will inform the contexts and student populations for which hands-on MR labs and just in-time dynamic molecular visualizations should be targeted and further developed. This project will enable internet-scale hands-on MR investigations across disciplines. The deliverables of the project will also include the hardware, software, workflows, and professional development tools required to enable the distributed and scalable creation of analogous "hands-on" MR labs in other disciplines and educational settings. The outcomes of this project will contribute to STEM workforce development among college and high school students at scale.

Lessons Learned & Insights Gained

Our publications detailing our findings are listed below:

1. “Framework for scalable content development in hands-on virtual and mixed reality science labs”. Kambiz M. Hamadani, Yuanyuan Jiang, Ali Ahmadinia, Ahmad Hadaegh, Juan Moraleja-Garcia, Alan Mendez, Arshia Shaikh, Andres Lozano, Jane Huang, Ariel Aquino, Ryan Palacio, and Maxwell Sheperd. 2022 8th International Conference of the Immersive Learning Research Network (iLRN), 2022. Vienna, Austria. https://ieeexplore.ieee.org/document/9815945. This paper won the best short paper award at the iLRN conference in 2022

2. “Tracking Objects using QR codes and Deep Learning”. Ali Ahmadinia, Atika Singh, Kambiz Hamadani, Yuanyuan Jiang. Proc. SPIE 12701, Fifteenth International Conference on Machine Vision (ICMV 2022), 1270106 (7 June 2023); https://doi.org/10.1117/12.2679910

3. “Miniaturization and geometric optimization of SteamVR active optical trackers”. Sean Gow, Jared Macshane, Ali Ahmadinia, Kambiz Hamadani, Yuanyuan Jiang. SPIE Volume 12449, Optical Architectures for Displays and Sensing in Augmented, Virtual, and Mixed Reality (AR, VR, MR) IV; 1244921 (2023) https://doi.org/10.1117/12.2652267

4. “Developing scalable hands-on virtual and mixed-reality science labs”. Yuanyuan Jiang, Kambiz M. Hamadani, Ali Ahmadinia, Karno Ng, Ahmad Hadaegh, et al. Virtual Reality (accepted). 

Equity

To make our approach and mixed-reality science lab technologies in general more equitable and affordable, hardware costs must be brought down. We have worked successfully over the past 12 month to further this goal.

New Challenges & Next Steps

Our assessment strategy and agile development model has needed constant iterative evolution to ensure success of our grant objectives and goals. We are now scaling up development and testing of the chemistry and biochemistry modules we’ve developed to address the learning science objectives of our project.

Our efforts are also translating into intellectual property and commercial products that we hope will make our technology and approach available more broadly to educators, industry partners, and the general public.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.