2019 NSF ITEST Summit Online Program

Main Image: 

2019 Summit Program Cover

NSF ITEST PI & Evaluator Summit 2019

Living, Learning, and Working in the Digital Age

Thursday, June 13 - Friday, June 14, 2019
Westin Alexandria, 400 Courthouse Square, Alexandria, VA


Twitter Event Hashtags: #ITEST2019  #STELAR2019



2019 PI & Evaluator Summit Information





Sarita Pillai, Education Development Center
Brianna Roche, Education Development Center




Melody Hachey, Technology Associate, mhachey@edc.org, 617-618-2801
Sarah MacGillivray, Project Associate, smacgillivray@edc.org, 617-618-2931
Joyce Malyn-Smith, Senior Staff, jmsmith@edc.org, 617-618-2386
Caroline Parker, STELAR Co-PI, cparker@edc.org, 617-618-2740
Sarita Pillai, STELAR PI, spillai@edc.org, 617-618-2164
Brianna Roche, Research Associate, broche@edc.org, 617-618-2760
Becca Schillaci, Research Associate, bschillaci@edc.org, 617-618-2772


The meeting chairs and STELAR staff thank the program committee for the innovative ideas and thoughtful input that helped shape the agenda for this meeting.

Winnie Black, Central Susquehanna Intermediate Unit, Center for Schools and Communities
Melissa K. Demetrikopoulos, Institute for Biomedical Philosophy
Stacey Forsyth, University of Colorado Boulder
Dana Franz, Mississippi State University
Janelle M. Johnson, Metropolitan State University of Denver
Lisa Kaczmarczyk, Lisa Kaczmarczyk PhD Consulting, LLC
Keliann LaConte, National Center for Interactive Learning, Space Science Institute
C. James Liu, New York Hall of Science
Kavita Mittapalli, MN Associates, Inc.
Sharon Nelson-Barber, WestEd
Julie Poncelet, Julie Poncelet Consulting, LLC
Helen Zhang, Boston College



The meeting chairs, STELAR staff and Program Committee would especially like to thank Amy Wilson-Lopez, Alejandra Sorto, Brenda Carpenter, and the entire NSF ITEST team of Program Officers and staff for their support and guidance that made this meeting possible.




National Science Foundation Featured Speakers, ITEST Program Officers, and Key Personnel




Sylvia James, Deputy Assistant Director, Directorate for Education and Human Resources
Karen Marrongelle, Assistant Director, Directorate for Education and Human Resources
Elizabeth VanderPutten, Acting Deputy Director of the Division of Research on Learning




Amy Baylor, abaylor@nsf.gov
Julia Clark, jclark@nsf.gov
Michael Ford, miford@nsf.gov
David Haury, dhaury@nsf.gov
Celestine Pea, cpea@nsf.gov
Joseph Reed, jreed@nsf.gov
Ann Rivet, arivet@nsf.gov
Monya Ruffin-Nash, mruffin@nsf.gov
Robert Russell, rlrussel@nsf.gov
Chia Shen, cshen@nsf.gov
M. Alejandra Sorto, msorto@nsf.gov
Michael Steele, msteele@nsf.gov
Amy Wilson-Lopez, awilsonl@nsf.gov

For general inquiries about ITEST, please write to DRLITEST@nsf.gov



Brenda Carpenter Einstein Fellow, brcarpen@nsf.gov
Daniel McEnrue, Grants Management Specialist, Division of Grants and Agreements
Sheryl Miller Program Specialist, smiller@nsf.gov




Thursday, June 13
Start Description
7:30 Registration and Breakfast
8:30 Summit Welcome*
9:00 Keynote*
9:45 Tribute to Julio E. López-Ferrao
10:00 Break
10:30 ITEST and Beyond
11:30 Flash Talks*
12:30 Lunch
1:00 Panel*
2:15 Breakout Sessions Part 1
3:15 Break
3:45 Breakout Sessions Part 2
4:45 Break & Expo Set Up
5:30 Project Expo
7:00 End of Day 1
Friday, June 13
Start Description
7:30 Networking & Breakfast
8:30 Day 2 Orientation*
8:45 NSF Remarks*
9:00 Keynote*
10:15 Break
10:30 Roundtables
12:00 Lunch & Program Officer Meetings
1:45 Breakout Session Part 3
3:00 Closing Reflections & NSF Town Hall*
4:00 Working Groups & Program Officer Meetings
5:00 End of Day 2

*Plenary sessions will be live-streamed and recorded.

(Events and sessions subject to change, start times are firm.)



Keynote Speaker Bios

(Listed in order of appearance)



Joshua Brown is the Founder, President, and CEO of Icarus Aerospace Inc. The focus of Joshua and his company has been to develop and integrate future concepts to foster a next generation workforce. He is a Doctoral student in Global Security at the American Military University, holds a Master’s degree in Intelligence Studies, and a Bachelor’s degree in Emergency & Disaster Management. He is a Service-Disabled Veteran who trained and supervised Naval Special Warfare/Special Operations candidates & Aviation Water Survival for pilots and aircrews. Joshua has extensive experience in Military, Law Enforcement, Fire, and EMS including supervisory/instructor roles as well as operational roles. He holds almost three dozen certifications in various disciplines along with copyrights and other pending intellectual property which include being the architect of the UAS for Public Safety IADLEST Certification Program.


Day 1 Lunch Panel: Preparing All Learners for the Future of Work


Ritu Raman is an engineer, writer, and educator with a passion for introducing bio-hybrid materials into the toolbox of every inventor. She grew up in India, Kenya, and the United States where she learned to appreciate and thrive in diverse and dynamic environments. Her life experiences have forged the belief that technical innovation can drive positive social change, and this inspires her work to democratize and diversify STEM education around the world. Ritu is currently a postdoctoral fellow in the Langer Lab at MIT, funded by a Ford Foundation Fellowship by The National Academies of Sciences, Engineering, and Medicine. She holds many awards and honors, including being named a Forbes 30 Under 30 awardee for Science, an Innovation and Technology Delegate to the International Achievement Summit, and a L'Oreal USA For Women in Science Fellow.

Emily Reid is a computer scientist, educator, and entrepreneur. Emily is the VP of the Open Learning Program at AI4ALL and the owner of E. E. Reid Consulting, LLC, where she advises and creates curriculum for schools and businesses to help them build inclusive computing education programs. Previously, Emily was the founding Director of Education at Girls Who Code, where she was responsible for ensuring that GWC programs were unrivaled in delivering quality computer science education. Before Girls Who Code, Emily was a Senior Cyber Security Engineer at the MITRE Corporation and has published research in computational linguistics and cybersecurity. She speaks and writes on diversity, education, and technology. Emily holds an M.S. in Computer Science from Columbia University and a B.S. in Mathematics from Tufts University.

Sheryl Sorby is a Professor of Engineering Education at the University of Cincinnati and was a Fulbright Scholar at Dublin Institute of Technology. She is a former Program Director within the Division of Undergraduate Education at the NSF. She has been an investigator on more than $13M in external funding and is the author of numerous publications and textbooks. She was the recipient of the Betty Vetter research award through WEPAN for her work in improving spatial skills and ultimately the success of women engineering students. Dr. Sorby was recently elected President-Elect of the American Society of Engineering Education and will begin her term as President in June 2020.


Day 2 Keynote Panel: The Future of Work


Journel Joseph is the author of Data & Analytics 4.0:The Future of Work, Privacy and Trust in the Age of Artificial Intelligence. As a passionate, multi-disciplinary, multilingual, hands-on data & analytics professional, he brings over a decade of experience to the industry and consulting arena. His interest in leveraging the advances in technology in order to harness the power of data began as an Intern at the former ITEST Learning Resource Center (LRC), currently known as STELAR. Being well-versed in both the public and private sectors, Journel advises clients on the most efficient use of technology in addition to data and analytics as it pertains to their business models. He is the founder of Analytics for Lunch: a masterclass series on the business of data & analytics, and the co-founder of Insert Analytics™. Journel Joseph is a graduate of Tufts University, The Fletcher School of Law and Diplomacy (MA, International Economics and Trade), the Department of Urban and Environmental Policy & Planning (Master of Public Policy), the Graduate School of Art and Science (Program Evaluation), and the University of Massachusetts (BA, Technology and Community Media).

Joylin Kirk is the Director of Public Sector Partnerships at Burning Glass Technologies. At Burning Glass, she connects community and social services agencies with leading edge labor market data and analytics solutions that improve organizational and systems-level performance, promote economic vitality of businesses and communities, and help individuals take greater control over their own career success. Prior to joining Burning Glass, Joylin was the senior director of mission services for Goodwill Industries International. At Goodwill, she led the career navigation, financial capacity building portfolios with focuses on veterans and military families, and youth services. Joylin has more than 20 years of experience leading programs for behavioral health and nonprofit workforce development organizations. She holds a B.A. in Psychology and Human Resource Management from King’s College and an M.S. in Experimental Psychology from State University of New York: College at Cortland.

James Tracy is a nationally renowned educator and speaker on the implications of technology inflections for education, workforce development, spirituality, and social policy. With Greg Toppo, he has just completed a book on these issues, under contract with MIT Press. Jim received a Ph.D. in American History from Stanford University as well as an M.B.A. (with a concentration in nonprofit management) and an Ed.M. (in Higher Education Administration) from Boston University. He has been the head of several Independent schools and, most recently, President of the Woodrow Wilson Graduate School of Teaching and Learning, an innovative teacher training college founded in collaboration with MIT. Jim has been a Board Member of the Bostonian Society, an education advisor to Singularity University, a Visiting Fellow at Yale University, a Klingenstein Fellow at Columbia University, an Affiliate of the Berkman Center for Internet and Society at Harvard Law School, and a Board member of Boston University’s Pardee Center for the Study of the Longer-Range Future. He is currently Co-Chair of the MassRobotics Work of the Future Committee, a Member of the MassRobotics Advisory Board, a Board member of the Boston edtech convener and accelerator LearnLaunch, a member of the Advisory Board for Boston University’s Digital Education Initiative, a member of the MIT Subgroup on Skills and Training (within the MIT Future of Work Taskforce), a Board member of the Massachusetts Association of Nonprofit Schools and Colleges, and a Trustee of the Boston Athenaeum, among other endeavors.



Breakout Sessions

1st Breakout Session: Thursday 6/13 @ 2:15 – 3:15pm



Technical Assistance Session: Logic Models
Although logic models are not required components of an ITEST submission, they are nonetheless helpful for many reasons. They help the PI team articulate and implement the educational and research components of their project in relation to anticipated outcomes; they clearly communicate a coherent rationale behind different project elements to reviewers; and they help the evaluators write useful formative and summative evaluations. In this interactive technical assistance session, which will be facilitated by a senior NSF advisor with expertise in evaluation, PIs and evaluators will explore the elements of a compelling and useful logic model, and they will discuss how logic models can be used to empower projects from their conceptualization to their dissemination. Session participants are encouraged to bring a logic model for a forthcoming project, and they will have opportunities to receive and provide feedback on this model and others based on the principles learned in this session. 

Presenter: Sarah-Kay McDonald, National Science Foundation

What Does it Mean to be a Scientist? How Culture, Language, and Gender Affect Equity in STEM
How do we navigate learner-centric and culturally responsive strategies to improve equity and access in our projects? Listen to 5-minute presentation each from Keisha Varma, Jaymee Nanasi Davis, and Allison Master who are working with a diverse group of learners that include high school girls, Native Hawaiian and immigrant students and their families in HI, MN, and WA. Their presentations will be followed by short group discussions that will help explore what STEM means to the learners; what skills and knowledge learners bring that already align with STEM; and how to give agency and value to what learners already know. Furthermore, what challenges learners face will also be addressed.

Facilitators: Keliann LaConte, National Center for Interactive Learning C. James Liu, New York Hall of Science Presenters Keisha Varma, University of Minnesota Jaymee Nanasi Davis, University of Hawaii - Maui College Allison Master, University of Washington

How Technology is Used to Support Relationships with Partners
Technology is a key part of all ITEST projects, the presentations in this session will focus on the innovative ways that technology can be used to cultivate, support, and sustain relationships with project partners. Attendees will learn about the innovative use of technologies in presenting projects, have the opportunity to discuss the role of technology in their work, and think about how they can leverage technology to solidify partnerships in future work.

Facilitator: Helen Zhang, Boston College
Presenters: Jennifer Jocz, Education Development Center, David Reider, Education Design, INC., Lori Rubino-Hare, Northern Arizona University

Multiple Pathways to New Collar Jobs
What will the careers of the future look like, and how do we best prepare students for these “New Collar Jobs?” This session focuses on the growing sector of technology-based careers often attained through non-traditional educational pathways. Participants will hear brief presentations from projects highlighting workforce development initiatives, then will break into small groups to discuss innovations and challenges in meeting the needs of tomorrow’s workforce, including frameworks and practices that support increased engagement with underrepresented groups.

Facilitators: Stacey Forsyth, CU Science Discovery and Janelle Johnson, Metropolitan State University of Denver
Presenters: Berri Jacque, Tufts University Medical School, Connie Flanagan, University of Wisconsin Madison, Kristin Searle, Utah State University Elizabeth Radday, EdAdvance

Q&A with NSF Program Officers about 2019 Solicitation
This session provides attendees the opportunity to meet with NSF Program Officers to ask questions about the upcoming 2019 ITEST Solicitation. The session will be set up in an informal town-hall format. Note: this session is being offered twice, during Breakout Sessions 1 and 3 in order to allow attendees multiple opportunities to attend.

Facilitator: Becca Schillaci, STELAR
Presenters: Michael Ford and Ann Rivet, National Science Foundation


2nd Breakout Session: Thursday 6/13 @ 3:45 – 4:45pm


Technical Assistance Session: Research and Evaluation
In this interactive technical assistance session, program officers will explain distinctions between evaluation and research in light of the new ITEST solicitation. The new solicitation encourages PIs to upload copies of evaluation reports as part of annual and final reports. This session will review the components of a quality evaluation plan and specify what might be included in evaluation reports. Discussion will also describe elements of a quality research plan, which is at the heart of all successful ITEST proposals. To prepare for this session, participants are encouraged to bring a 1-page outline of a research plan, describing the research question, data sources, and proposed analyses; as well as a 1-page outline of an accompanying evaluation plan, describing the types of questions the evaluators will ask, the evidence that will be gathered in relation to those questions, and how this evidence will provide formative feedback to the project efforts.

Presenters: Karen King and Michael Ford, National Science Foundation

Biohybrid Design: A New Era in Bioethics
The emerging field of biohybrid design, or building machines with partbiological and part-synthetic components, is interesting because it harnesses the dynamic adaptive behavior of biological materials to produce machines capable of complex functional behaviors such as self-assembly, selfmaintenance, and self-healing. This concurrently raises questions around the bioethics of building such machines, and the regulatory guidelines that must accompany their design, manufacture, and implementation. This workshop will introduce biohybrid design, present a few vignettes asking ethical questions related to this field, and drive an active and engaged discussion on how an interdisciplinary groups of scientists, educators, and policy makers can guide the safe, effective, and ethical use of this technology in the future.

Presenter: Ritu Raman, Massachusetts Institute of Technology

Removing barriers to STEM Success: Enhancing 3-D Spatial Skills
This workshop will begin with a brief presentation regarding the need for and impact of spatial skills training for success in STEM. The remainder of the session will include hands-on practice with a module from a curriculum designed to help students improve their spatial skills. The session will conclude with a brief discussion among participants regarding models for curricular implementation.

Presenter: Sheryl Sorby, University of Cincinnati 

The Next Generation: Student Voices in STEM
In this student-led session, youth from the Chief Science Officers (CSO) project will present the changing dynamics of learning from a student perspective. CSOs will share their experiences in the program, including working with industry partners, STEM professionals and opportunities that encouraged them to visualize themselves as the future. They will share strategies on how PIs can include students in the planning process and curriculum development, and how PIs can create learning environments that place students in control of their efforts. They will also share how they are scaling up the program, and how projects can benefit from including students as partners for the impact to be more meaningful.

Facilitators: Jeremy Babendure, Institute for Learning Innovation Research Fellow and SciTech Institute Executive Director Kelly Greene, COO SciTech Institute and International Director of Student Success
Presenting Youth from Chief Science Officers: Caillou, Marissa, Moira, and Shalae 

New Proposal Pitches: ITEST Shark Tank
Members of the ITEST community will give 5-minute “Shark Tank” style pitches of their ITEST project ideas to get valuable feedback from the ITEST community. Presenters will make their pitches in front of a panel of judges and an audience of their peers. Following each pitch, presenters will receive feedback from the judges. The session will conclude with informal Q&A from the audience.

Facilitator: Brianna Roche, STELAR
Judges: Monya Ruffin-Nash, Robert Russell, Chia Shen, and Michael Steele, National Science Foundation
Presenters: Joselina Cheng, University of Central Oklahoma, Alec Bodzin, Lehigh University, Eugene Cordero, San Jose State University, Mary Dussault, and Erika Wright, Smithsonian Astrophysical Observatory 


3rd Breakout Session: Friday 6/14 @ 1:45 – 2:45pm 


Technical Assistance Session: Meeting the Solicitation-Specific Criteria
This technical assistance session addresses ITEST’s solicitation-specific review criteria, which review panels must consider in addition to the NSF-wide merit review criteria. These criteria represent a significant change from the previous solicitation in the sense that they emphasize the importance of strengths-based approaches to innovative technologies, instructional strategies, and research designs in projects whose goals are to broaden participation. In this session, led by two ITEST program officers, participants will read brief mock examples of educational activities and research plans that might be found in proposals, and they will evaluate and discuss these examples in relation to the solicitationspecific review criteria. This session is designed to increase PIs’ capacity to engage diverse populations in educational and research activities in rigorous and responsive ways. Given that ITEST is a program explicitly designated for broadening participation, this session will provide practical insights on communicating plans for working with groups who are underserved and underrepresented in STEM fields.

Presenters: M. Alejandra Sorto and Amy Wilson-Lopez, National Science Foundation

Building Equity in STEM by Cultivating a Lifetime of Learning Through Lasting Partnerships
Diverse community-based, government, and industry partnerships are essential for sustaining the impacts of STEM learning. Presenters Josie Cheng, Winnie Black, Martin Reardon and Anne Haugh will showcase innovative ways humans and technology converge, interact, and engage with young STEM learners in OK, NC and PA. Their 5-minute project overviews will provide examples of how learners are developing transferable technological skills to prepare them for an evolving workforce. Their presentations will be followed by group discussions that will help explore the challenges in bringing technologies to various audiences and generate strategies for engaging partners to achieve equity goals.

Facilitator(s): Kavita Mittapalli, MN Associates, Inc.
Presenters: Joselina Cheng, University of Central Oklahoma, Winnie Black, Central Susquehanna Intermediate Unit - Center for Schools and Communities, Martin Reardon, East Carolina University, Anne Haugh, Duplin County Schools

How Technology is Used to Promote Student Engagement
Technology is a key part of all ITEST projects, the presentations in this session will focus on the innovative ways that technology can be used to attract, sustain, and deepen student engagement. Attendees will learn about the innovative use of technologies in presenting projects, have the opportunity to discuss the role of technology in their work, and think about how they can leverage technology to engage and retain students in future work. Facilitator Dana Franz, Mississippi State University Presenters Megan Littrell, CIRES Education and Outreach Stacey Forsyth, University of Colorado-Boulder Keliann LaConte, National Center for Interactive Learning, Space Science Institute Edison E Ethical and Social Considerations: What would, could, should you have done and why? This interactive session will engage participants of all experience levels in exploring how they can proactively incorporate ethical and social considerations into their ITEST projects. This session will go beyond content and mechanics and include the perspective of process.

Facilitators: Melissa Demetrikopoulos, Institute for Biomedical Philosophy, Lisa Kaczmarczyk, Lisa Kaczmarczyk PhD Consulting, LLC, Sharon Nelson-Barber, WestEd

Building Capacity to Support & Scale-up ITEST Projects
This session will explore current and alternative approaches to building capacity in ITEST projects, including ways to scale-up. Following three 5-minute ignitestyle presentations, participants will engage in an interactive discussion (called “World Cafe” activity) in order to share and harvest insights organized around three guiding questions: what are current approaches to capacity building? What alternative approaches should be considered? How to scale-up and sustain ITEST projects to support more equitable, localized ownership, and foster deeper partnerships to amplify outcomes?

Facilitators: Julie Poncelet, Center for Social Innovation Becca Schillaci, STELAR
Presenters: Janelle Johnson, Metropolitan State University of Denver James Van Haneghan, STEMWorksLLC Moses Ochanji, California State University San Marcos Jennifer Yu, SRI International

Q&A with NSF Program Officers about 2019 Solicitation
This session provides attendees the opportunity to meet with NSF Program Officers to ask questions about the upcoming 2019 ITEST Solicitation. The session will be set up in an informal town-hall format. Note: this session is being offered twice, during Breakout Sessions 1 and 3 in order to allow attendees multiple opportunities to attend.
Facilitator: Sarita Pillai, STELAR
Presenters:  Michael Ford, National Science Foundation Ann Rivet, National Science Foundation




Presentation Slides and Video Links

Thursday, June 13
Description Materials
Summit Welcome & Karen Marrongelle, NSF

Presentation Slides

Keynote: Joshua Brown, Icarus Aerospace

Presentation Slides

ITEST 2019 and Beyond: Solicitation Updates

Presentation Slides

In Memory of Julio López-Ferrao

Memorial Slides

Plenary Flash Talks

Presentation Slides

Lunch Panel: Preparing All Learners for the Future of Work

Presentation Slides

Friday, June 13
Description Materials
Day 2 Orientation & Silvia James, NSF

Presentation Slides

Keynote Panel: The Future of Work

Presentation Slides

NSF Town Hall: Elizabeth VanderPutten, NSF

Presentation Slides