Student engagement in the math processes of making connections, creating justifications, and problem-solving.
Use this measure:
Access a copy of the BMTN survey
Measure of...
Measurement instrument overview
This is a student survey consisting of 14 questions asking students about the frequency with which they enact behaviors associated with deep engagement in mathematics. The survey takes approximately five minutes for students to complete. For each question, students are asked how often they participate in various activities.
Connection to student learning
The BMTN Student Survey was created to gather data on the frequency with which students enact behaviors associated with deep engagement in mathematics.
Items in the survey are aligned with behaviors associated with deep engagement, and they fall into three categories:
- Connect: Making connections among mathematical algorithms, concepts, and application to real-world contexts (for example, “How often do you make connections to math concepts you used previously in this class?”)
- Justify: Communicating and justifying mathematical thinking as well as critiquing the reasoning of others (for example, “How often do you explain your answers to others in the class?”)
- Solve: Making sense of and solving challenging math problems that extend beyond rote application of algorithms (for example, “How often do you keep trying different ways to solve math problems even when they are hard?”)
What we know about how well this measure works for its intended use
The BMTN team drew these survey items from existing surveys that had been validated in their own contexts, including the University of Chicago’s 5Essentials Survey and survey items from aprior project that focused on student-centered math teaching and was led by BMTN’s hub leaders.
Analyses were conducted with BMTN data to assess the reliability of survey items across the survey and per individual construct. Reliability statistics (Cronbach’s Alpha) for the survey: .91 (All items), .80 (Connect items), .82 (Justify items), and .78 (Solve items).
Frequency
BMTN administered surveys each semester to analyze semester-to-semester and year-to-year growth. The survey could be administered at greater frequency (e.g. twice a semester) to better understand student growth in math engagement.
Measurement routine details
BMTN teachers used Google Forms to administer the survey to their classes online once per semester. Survey responses fed into a central Excel database.
With the support of a data analyst, network leadership analyzed aggregated survey data to understand the progress of the network as a whole and to understand strengths and weaknesses in learning environments across the network.
Additionally, BMTN provided individual teachers with a classroom-level report to better understand student engagement in their individual classrooms. These reports provided teachers with before and after data to better understand the efficacy of the change ideas teachers tested in their classrooms between survey periods.
Data analysis details
For each question, students report whether they never (1), rarely (2), sometimes (3), often (4) or almost every class (5) experience each of the conditions described in the survey items. Overall, and by construct, average scores of 3 or higher were considered to provide moderate evidence of deep engagement, and average scores of 4 or higher were considered to provide strong evidence.
Below is an example of how BMTN created visuals of the survey results. They used simple bar graphs to demonstrate the depth of engagement (i.e., moderate or strong) overall and across each of the three constructs.
Individual classroom-level reports for each teacher also compared the teacher’s classroom-level scores to that of the network as a whole and showed changes in average responses from one survey administration to the next.
Conditions that support use
BMTN was led by math education researchers (and former math practitioners) and leveraged this expertise to support teachers to use the student survey. For example, the network did the following:
- Built a strong sense of community and buy-in
- Compensated teachers for their involvement in the network
- Organized regular, facilitated small-group meetings for teachers to identify and discuss change ideas
- Developed common network rubrics designed to assess student engagement across a continuum for the network’s three focal dimensions (connect, justify, and solve)
- Owned survey analysis, reporting, and visualization
BMTN began as a small planning grant before being launched in full, allowing network leadership ample time to establish this system of support.
Challenges
Some teachers reported that students had trouble understanding one of the items under “Solve,” which focused on solving extended math problems. The network decided to drop this item from the survey given this feedback.
Other tools and resources to support use
In addition to the student survey, BMTN created a teacher-facing version of the survey with the same items but which asked about the frequency with which teachers perceive that they enact these learning opportunities or conditions. Teachers utilized findings from the teacher survey to identify areas of focus in their teaching related to student engagement and to determine if there were discrepancies in their perceptions of learning conditions, compared to their students. The network also used the aggregated teacher survey results to compare networkwide learning opportunities to what students reported experiencing.
Find out more about BMTN’s use of the student and teacher survey by viewing its full report. Learn more about BMTN by visiting its website.
The BMTN is a Networked Improvement Community (NIC) of teachers and researchers who have the shared goal of deepening and expanding student-centered instruction in high school mathematics. The collaborative approach of the network includes specifying a common problem related to student-centered math teaching; testing out potential ways to address the problem; sharing how it went with others in the NIC; and continuing with additional cycles of testing, sharing, and refinement.
The math education and improvement science experts forming the BMTN “hub” designed the BMTN student survey to understand how the network as a whole was progressing toward its aim of increasing the number of New England students who “connect, justify, and solve with depth.” The surveys offered the network a high-level, student-centered perspective on how students were experiencing teachers’ shifts toward more student-centered instruction. Surveys were administered to students twice during the year. The network hub analyzed the survey results and provided each network teacher with an individual report displaying both their classroom-level data and data from across the network as a whole.
While valuable, the surveys were just one component of the network’s larger efforts to support teachers in improving student-centered instruction and deepening student engagement. Teachers applied network-designed rubrics to student work to capture ongoing data on the impact of their instructional changes on student mathematical thinking. Teachers met regularly in small, facilitated affinity groups to discuss whether to adopt, adapt, or abandon the particular instructional routine of focus. A complementary teacher survey allowed hub leadership to see if teacher self-perceptions of the opportunities they were providing for students aligned with what students experienced, as reflected in the student surveys.
Interviewee: Kirk Walters, BMTN Principal Investigator and Director of Mathematics at WestEd
Did your use and/or analysis include student students with IEPs, particularly with an LD classification?
Cheryl,
Thanks for your question, and apologies for the slow response! Kirk Walters, Principal Investigator for the BMTN, says “We did not have data on students with IEPs, so sadly no.”.
Best,
Andrew