UTOP for Mathematics and Science Training Guide

Training Guide

UTOP for Mathematics and Science

The UTeach Observation Protocol (UTOP) for Mathematics and Science is an observational instrument that can be used to assess the overall quality of classroom instruction in math and science from kindergarten to the undergraduate level. The UTOP was designed to allow individuals to evaluate teaching effectiveness while valuing different modes of instruction. We have specifically considered instruction in a spectrum from inquiry-based instruction to direct instruction.

The UTOP was created and piloted by faculty, master teachers, and research assistants in the UTeach College of Natural Sciences program at the University of Texas Austin.[1]

The development of UTOP was informed by the following resources:

  • National Council for the Teaching of Mathematics: Principles and Standards for School Mathematics
  • National Academy of Science: National Science Education Standards
  • American Association for the Advancement of Science: Project 2061, Benchmarks for Scientific Literacy
  • National Research Council: How People Learn: Brain, Mind, Experience, and School and Knowing What Students Know
  • Arizona Collaborative for Excellence in the Preparation of Teachers: Reformed Teaching Observation Protocol
  • Horizon Research’s Inside the Classroom Observation Protocol and Study

Further revisions of the UTOP have been informed by the following:

  • Common Core State Standards for Mathematics (2012)
  • Framework for K–12 Science Education (2011)
  • Next Generation Science Standards Framework (2012)
  • Feedback from UTOP workshop training participants (July 2012–May 2013)

The UTOP is a criterion-referenced instrument, requiring training and recalibration to maintain the integrity and fair use of the instrument. This training guide is intended to provide resources to instruct, develop, and sustain inter-rater reliability. Interactive discussion of terminology used to define and describe each indicator is essential for groups of observers in order to ensure consensus and consistency in the rating process. As much as is humanly possible, raters’ personal opinions and/or directive judgments and suggestions for improvements should not play a role in their choice of ratings; evidence for ratings must be based on what was observed or discussed in the classroom. The UTOP can be used to evaluate quality of instruction and provide meaningful feedback for improvement at any level of education, in a variety of settings. It is composed of 26 items rated on a 5-point Likert scale (1 to 5), with an NA (Not Applicable) rating option for a few items where sufficient information may not be accessible during the observation session.

As stated, the UTOP is intended for use by raters trained in its application. This training guide provides future raters with specific information relating to the various elements of the UTOP. Particularly, this guide aims to clarify the rating standards for each indicator in order to improve the accuracy and reliability of raters’ ratings.

In addition, the UTOP is an instrument that is designed for use over multiple observations of a teacher’s practices over the course of time—several weeks, at minimum—thereby reducing the impact of a low score on any other single indicator.

[1] Those involved in creating and piloting the UTOP for Mathematics and Science include Mary Walker, Gail Dickinson, Mark Daniels, Denise Ekberg, Kelli Allen, Larry Abraham, Michael Marder, Candace Walkington, Prerna Arora, Jessica Gordon, and Shasta Ihorn. Audrey De Zeeuw and Paige Bauerkemper have assisted in further revisions.