Blog

The Management Information System (MIS) survey is distributed to ITEST projects annually every fall. This blog post was written about the 2015-2016 MIS, our eighth running of the MIS. In fall of 2019 we will collect information from projects on their 2018-2019 activities - our 11th year of ITEST project data collected!

What is the MIS, and why does STELAR ask projects to complete it? 

In the early years of the ITEST program, NSF would request periodic “highlights” or examples of project work from the ITEST Learning Resource Center (2003-2013). This was initially carried out by issuing a series of questions to Principal Investigators (PIs) on the topic of interest. As the ITEST program grew, it became clear that we needed a more comprehensive information system to gather more accurate data describing ITEST projects. The MIS was thus created in 2009 and is revised and administered to all active ITEST projects annually in order to inform NSF, the ITEST projects, the field, and other stakeholders as to the state of the ITEST program: who participates, in what kinds of activities, with what objectives and targeted outcomes.

What is the MIS data used for? 

The MIS information provided by projects shapes much of the work of the STELAR team for the following year. As soon as the MIS survey is closed, the team begins synthesizing the data to produce products that provide an overview of the ITEST Program like the Snapshot, the 2015 Projects Overview, and data briefs that describe want ITEST projects do, like this year’s Research Methods in ITEST and ITEST in Action. These publications are used by a myriad of audiences for a variety of reasons. For example, the 2016 ITEST in Action data brief highlights that only 14% of projects focus specifically on students with disabilities. This insight could be used to focus recruitment efforts on a current project, or help inform a new proposal. 

The MIS also allows STELAR to help disseminate projects’ incredible work by updating the STELAR Resources section with projects’ publications, curricular materials, and news, and sharing it in our newsletters and on our social media networks (Twitter/Facebook). That information is also tied to each Project Profile, keeping it current and accessible to those seeking to learn about the ITEST projects’ important work. In addition, the valuable Instrument Database is also updated by learning about how ITEST projects measure their program outcomes. 

The MIS data also helps to determine what technical assistance (TA) activities would be the most beneficial to the community. For example, by examining project’s dissemination strategies, we are able to review what projects have used in the past year, and see whether it aligns with NSF’s preferences. If projects aren’t submitting to very many peer-reviewed journals, we can host a webinar providing TA on how to get published. 

Last, but certainly not least, the stories provided by projects in their open-ended responses are used by STELAR to highlight the ITEST program. These stories make the program more visible, and help attract attention to the work of the entire community. 

How is the MIS different from NSF annual reports? 

The annual report is an avenue in which projects provide NSF with a detailed description of their activities and accomplishments. However, the format does not allow effective synthesis across the many separate projects engaged in uniquely innovative activities. The purpose of the MIS is for STELAR to gather similar information across dissimilar projects in order to tell the story of all the work carried out in the ITEST Program in a way that is meaningful to a larger audience.  

What do I, Becca Schillaci, enjoy about the MIS?

I really enjoy learning about the projects and getting to know them from their responses. I also love the challenge of organizing all of the data. Collecting data from ITEST projects, making the questions meaningful for our reporting but also understandable to those answering, requires quite a bit of creativity. This year, 96 projects received the MIS, and as we know all ITEST projects are structured very differently. I like the challenge of figuring out how to get the most accurate data from everyone! 

What changes about the MIS each year? 

First and foremost, the information STELAR receives changes! This is because every annual MIS asks projects to reflect upon their previous year's activities. We also revise the MIS from year to year to remove outdated questions and include new questions that capture the interest of NSF and the ITEST community.  With this fall’s distribution of the MIS we will have collected 8 years of data from ITEST projects. We ask many of the same questions of projects each year, building a longitudinal file that we can use to describe how the program has changed over time. This information is incredibly valuable to NSF, as well as to the larger STEM education community.

Becca Schillaci is a research associate with Education Development Center, and a team member of the NSF-funded STEM Learning and Research (STELAR) Center, the resource center for the Innovative Technology Experiences for Students and Teachers (ITEST) program. With expertise in research design, quantitative and qualitative methodologies, survey design, statistics, and data analysis, Becca is committed to conducting high-quality research of educational programs. Through her work on webinars and as a project specialist, she provides technical assistance to ITEST projects and helps inform NSF and other stakeholders of ITEST’s impact by collecting and synthesizing data from those projects.