What is practical measurement?
Practical measurement is the deliberate and routine gathering, analysis, and interpretation of information with the distinct purpose of enhancing the learning of system actors as they test changes and improve processes that are at the heart of their work. Measures are “practical” in that they can be collected, analyzed, and used within the daily work lives of practitioners. They are also “practical” in that they reflect practice – in that they act as sensing mechanisms at the level at which work is carried out.
What contextual factors support the use of the measures in the repository?
In order for measures in the repository to be used optimally, users:
- Are members of a professional learning community
- Have a clear vision of instruction & an explicit focus on equity
- Are engaged in student centered mathematics instruction
- Will utilize data as part of a continuous improvement effort
- Will NOT use the data to evaluate educators
What criteria did we use to select measures for this repository?
To qualify as a practical measure, measures needed to minimally satisfy three over-arching criteria of being useful, easy, and consequential. Practical measures in this repository are useful in a continuous improvement effort; they are relatively easy for practitioners to enact and use in their work; and they capture something that matters for the improvement of math teaching and learning. Each of these three criteria should be viewed as continua; different measures will fall at different points along each of the three continua. Users will need to weigh and (in the spirit of continuous improvement!) “test out” how well each measure meets their specific needs. With these criteria in mind, we sought out measures that:
- Are clearly connected to math learning or best practices recognized by the field of math educators
- Have demonstrated valid evidence of support for instructional improvement (either at a single site or in a more comprehensive study)
- Yield actionable data
- Can be used across multiple contexts and settings
- Are easy to administer and produce data that is easy to analyze/ interpret
- Can fit into educators’ regular routines
- Require minimal training
- Not overly time-consuming
- Are free or low-cost
What is student centered mathematics instruction?
Students learn best when they are actively engaged in a full range of mathematical activities—exploring, noticing, questioning, analyzing, struggling, problem-solving, justifying, reasoning, communicating, making connections, and explaining. Powerful mathematics classrooms thrive when students feel a sense of agency and understand that the intellectual authority in mathematics rests in mathematical reasoning itself; it is the mathematics and whether or not it makes sense that tells us if a solution or idea is correct, not a teacher or textbook. These factors support students as they develop their own identities as powerful mathematics learners and users.
Mathematics teaching that supports access and equity also requires being responsive to students’ backgrounds, experiences, cultural perspectives, traditions, and knowledge when designing and implementing a mathematics program and assessing its effectiveness. It includes holding high expectations for all students, ensuring access to high-quality mathematics curriculum and instruction, allowing adequate time for students to learn, placing appropriate emphasis on differentiated processes that broaden students’ productive engagement with mathematics, and making strategic use of human and material resources. As educators increase access for all students to engage with and thrive in mathematics and value the different ways questions and problems can be approached and learned, many more students view themselves as belonging to the mathematics community.
The term student-centered mathematics instruction does not refer to a single instructional method and instead includes an array of complementary approaches to teaching and learning that draws from multiple theories, disciplines, and trends in the field of education. As the term implies, these approaches place all students at the center of the learning process, preparing them to be successful both inside and outside of the classroom.
What is continuous improvement?
Continuous improvement efforts generally involve some variation of the following iterative cycle of activities:
- Facilitating investigative processes (e.g., empathy interviews, root-cause exercises) to understand a problem and the system that produces it
- Focusing learning efforts to develop shared aims for improvement
- Generating or gathering ideas for change
- Iteratively testing changes at a small scale before eventually bringing them to full and reliable implementation
There are many definitions for continuous improvement across the literature, but continuous improvement is often used as an umbrella term to capture a range of disciplined system-improvement methodologies, including improvement science, networked improvement communities, Lean, Six Sigma, and Design-Based Implementation Research. These methodologies share a common approach of empirically testing changes to a system by focusing on improving the interactions of people, processes, materials, and norms toward a common goal. This approach calls for centrally including the voices of those at the front lines of the system (e.g., teachers) and of those who use the system (e.g., students, families) in order to guide the work. With this focus on including the voices of key stakeholders, and in some other aspects, continuous improvement draws heavily on design-thinking approaches to innovation. Design-thinking approaches have at their core a search for new ideas and prototypes through understanding users’ experiences and through developing empathy. Organizations that adopt continuous improvement as their approach to learning and improvement are often characterized by the following:
- Shared, evidence-based processes and practices
- Shared responsibilities, organizational goals, and priorities
- A shared improvement methodology
- A data infrastructure that provides feedback tied to organizational outcomes
- A culture and discipline of learning from failures and near-failures
- Leadership practices that build and sustain a continuous-improvement culture
References:
Hough, H., Willis, J., Grunow, A., Krausen, K., Kwon, S., Mulfnger, L., & Park, S. (2017). Continuous improvement in practice. Policy Analysis for California Education. https://www.edpolicyinca.org/
LeMahieu, P. G., Bryk, A. S., Grunow, A., & Gomez, L. M. (2017). Working to improve: Seven approaches to improvement science in education. Quality Assurance in Education, 25(1), p. 2-4.
Valdez, A., Takahashi, S., Krausen, K., Bowman, A., & Gurrola, E. (2020). Getting Better at Getting More Equitable: Opportunities and Barriers for Using Continuous Improvement to Advance Educational Equity. WestEd. (Note: the above definition of continuous improvement has been adapted from this report)
What is the role of practical measurement in a continuous improvement effort?
Practical measurement aims to be useful, easy, and consequential in the context of a continuous improvement effort, and any given practical measure works in concert with other measures that comprise a system of measurement. For those applying continuous improvement methods to their work, the measures in this repository are likely to function as measures at the level of “drivers” (e.g. in a driver diagram) or “processes.” They attend to aspects of teaching and learning of 6-9th grade math. In some cases, the measures may be useful at the level of small-scale Plan-Do-Study-Act cycles, although we would generally recommend that the frequency of use be considered in light of any burden of measurement. Any decision about what measures to use should be based on an explicit connection to the theory of improvement of the particular improvement effort in its specific context.
How can I use these practical measures to understand and advance equity in math classrooms?
The math practical measures repository includes some measurement instruments explicitly designed to capture information about equity concerns in middle grades math classrooms. The field of math education research has identified that less favorable experiences and outcomes can be particularly acute for students whose identities are Black, Latino/a, Indigenous, female, emergent multilingual, or living in poverty, among other identities and intersections of these identities. EQUIP and Equitable Classrooms, for example, are both tools to gather information about which students are participating and how, and whether this participation varies by key identity characteristics. Many of the student-facing socioemotional learning surveys can shed light on students’ experiences and perspectives that may be connected to key identity characteristics. The PERTS Copilot Student Survey, for example, includes items addressing dimensions of socioemotional learning, such as affirming cultural identity, classroom belonging, and teacher caring, that can be concerns about the classroom context for historically underserved student groups. Other measures of core aspects of math classrooms can also illuminate equity concerns. PMRR’s whole class discussion and small group work surveys, for example, are used to understand how students perceive and experience classroom discourse. The facets of discourse that are addressed in this tool – such as students’ perceptions that their ideas and voices matter and are heard in the classroom – are also ways in which equity is enacted (or not) in specific contexts. Importantly, critically assessing classroom inequities and taking steps to advance equity requires much more than the right measurement instruments – they also require a larger support structure and system that enables critical reflection and professional learning, where educators are supported and have opportunities to learn from expert others.