Evaluation in ITEST

Evaluation in the Context of ITEST Projects

A key component of the technical support offered by STELAR (and by the LRC from 2003-2012) has been in helping projects to develop rigorous and relevant project evaluations that can provide formative and summative feedback to project leaders to improve their project and determine impact.

The resources collected here include a guide on evaluation, targeted to project directors, that helps build common understanding between project implementers and evaluators about the value and importance of evaluation. Also included are two articles providing specific examples of evaluation results, articles from the STELAR archives about ITEST evaluation resources, and articles looking at evaluation more generally. Finally, we have included an example of one of the instruments available on the STELAR website, the STEM Semantic Survey. The survey was developed by three ITEST PIs as part of an ITEST project, and has demonstrated validity and reliability as a measure of perception of STEM disciplines. For access to other validated instruments used by ITEST projects, go to the STELAR resources page and click on 'Instruments.'

Partner NSF Resource Centers have also compiled resources for evaluation, including:

The Center for the Advancement of Informal Science Education (CAISE) has produced an Evaluation Guide for PIs and has compiled additional resources.

The Center for Advancing Research & Communication (ARC) has compiled evaluation resources.  

Resources

Ten Years of Youth Programs at The American Museum of Natural History: An Independent Perspective and Lessons Learned

(Publication)

In this report, Inverness Research summarizes their findings based on evaluating the American Museum of Natural History (AMNH)'s high school program over 10 years. The report includes:

  • A discussion of the theory of action and the rationale that underlies the AMNH youth programs
  • A description of three of the programs that have been offered and the evolution in program design that has taken place over the years
  • A description of the core common features of the programs

 

Read More

A Guide to Evaluation Primers

(Publication)

Produced by the Association for the Study and Development of Community (ASDC) for the Robert Wood Johnson Foundation, this document offers an orientation to handbooks and basic primers on evaluation. These resources are designed to meet the needs of the non-expert, explaining some of the central issues in evaluation and why they are important. The Guide itself helps organizations to assess which evaluation primer might be most useful for their needs and what to expect, in terms of information, from each.

Read More

How Can Multi-Site Evaluations Be Participatory?

(Publication)

Multi-site evaluations are becoming increasingly common in federal funding portfolios. Although much thought has been given to multi-site evaluation, there has been little emphasis on how it might interact with participatory evaluation. Therefore, this paper reviews several National Science Foundation educational, multi-site evaluations for the purpose of examining the extent to which these evaluations are participatory. Based on this examination, the paper proposes a model for implementing multi-site, participatory evaluation.

Read More

The ITEST Learning Resource Center's Online Evaluation Database: Examples from the Collection

(Publication)

The ITEST Learning Resource Center at EDC developed an online database of instruments developed by ITEST project evaluators and researchers from 2003 to 2007. This 2007 article details the purpose and development of that database and highlights three instruments from it that represent the kind of evaluation tools archived there. While the database is no longer available, the three instruments described in the article provide useful examples of project-developed evaluation tools.

Read More

STEM Semantics Survey

(Instruments)

The STEM Semantics Survey measures student interest in science, technology, engineering and mathematics as well as interest in STEM careers. The survey is comprised of five scales each with five items measured on a 7-point scale. The STEM Semantic Survey is appropriate for administration to middle school students and teachers.

The link below provides access to the instrument as well as information about the instrument.

Authors provide instrument validity and/or reliability information.

Read More

Recent Highlights