In the summer of 2020, in the midst of a global pandemic, ROCKiT led 42 MMSD staff through a virtual and abbreviated five-day design sprint professional development opportunity, from which staff would leave with meaningful learnings about and outcomes from Liberatory and human-centered design and its key components, and a viable prototype for innovation to build out and test at their schools. A team of four sixth grade teachers from Toki Middle School joined this ROCKiT Space Camp specifically wanting to explore redesigning their daily Crew (or homeroom) time and structure at Toki Middle School.
The Design Challenge
We will provide student choice and voice by redesigning the Crew experience for Toki Middle School 6th graders on the Foxes team. In a world where students are often left out of the conversation and do not have control over their learning experience, we aim to provide opportunities for students to take ownership over their learning and Crew activities. We need to test the assumption that students don’t want to or don’t have the skills to advocate in their school experience.
Empathy Inspiration to Insight Generation
The Toki Crew design team began by digging into the needs, wants, desires and greatest pain points within the Crew experience of Black and ELL middle school youth. The team interviewed a handful of 7th and 8th grade students at Toki and asked the students to describe a positive experience they had in school last year, a time when they felt their voice was heard, a challenging day at school in the past year, and to tell them a story about a time they had choice in their learning.
The design team synthesized all the data from the interviews in order to generate insights. Students shared that they wanted more teacher support, help with time management and organizing assignments, strong relationships with friends, peers and teachers, a safe space to fail, more choice in their learning, and time to talk about things they are passionate about. Based on the key insights and themes from empathy work, the team created a persona canvas and defined their client, Abi.
Point-of-View Statement: Abi, a black and/or bilingual 6th grade student in an English Language Learner (ELL) cluster, needs a way to get teacher support and build authentic relationships with peers and staff due to limited access to support at home and feeling disconnected in the virtual space.
After engaging in empathy and defining the problem, the team moved to the ideation phase of the innovation process. Using the Point of View statements and How Might We statements to guide their brainstorming, the team came up with a variety of solutions from changing the Crew schedule, to providing student-choice Crew activities, to creating small cooperative learning groups. Ultimately the team chose to build out a new Crew schedule wherein each day incorporated the different needs, wants and desires of the students.
Build, Measure, Learn (Prototyping) Phase 1
In order to gain valuable feedback on their design, the team first set out to test the critical assumptions and desirability of their solution. Since schools were on summer break, the team reached out to former students to get their feedback on the new Crew design. Some of the critical learnings the team gained form these sessions were:
Kids are excited to see their friends, play games, have choices in the Crew activities and build relationships with their peers and teachers.
Students are worried about the time that Crew is held, their ability to attend, the expectations if they miss Crew.
Students were also worried about who is in attendance and not having any of their friends online.
Students needed to know more about the activities they will be engaging in each day.
“What if none of my friends show up?”
- 7th grade student, Toki Middle School
From these focus group conversations, the team went back to their design and made some changes. They built out clear expectations for when a student missed a Crew session and tried to create smaller learning pods to support the student desire to be online with their friends and the need for smaller learning environments. They also attempted to build out different Crew times throughout the day to be responsive to student scheduling needs but quickly learned that the Toki school day structure made this design change unfeasible.
With early evidence showing that students desired the new Crew schedule, the team knew they needed to more fully test their prototype to learn more about the feasibility and viability of their idea, specifically focusing on longer-term desirability and whether this new crew schedule would increase student engagement, sense of belonging and academic success.
At their first Innovation Council check-in, made up of Toki staff, MMSD Central Office staff, and two community partners (American Family Institute for Corporate and Social Impact and Madison Public Libraries), the design team presented what they had accomplished over the summer, important learnings and pivots they made based on learnings and their recommendation to Council leadership on how to proceed. Based on the early indicators of success the Innovation Council gave the team the go-ahead to persevere and move to Phase 2 of testing.
Prototype Testing-Phase 2
During the fall of 2020 the team ran and tested out their new Crew schedule with the goal to more rigorously test the hypothesis that a more intentional Crew space that was designed with students needs, wants and desires would have a positive impact on students’ sense of belonging, engagement in virtual learning and academic achievement and success.
Over nine weeks the team gathered evidence and data around the desirability, feasibility and viability of their design. To better understand student desirability, the team observed engagement, defined as camera on, uses the chat for content related purposes, unmutes and talks and completes work; sense of belonging; attendance; and grades. At the end of the semester, engagement data indicated that 28% of their target students always engaged and 38% sometimes engaged in Crew and their first core class of the day (Math, Science/Social Studies and English Language Arts). When they asked students about their engagement with the Crew experience, they learned of structural barriers, such as internet issues (access, bandwidth, etc.) that prevented full engagement. From student feedback the team also learned that students want more time to connect with each other, so they started a daily virtual cafeteria to complement the morning Crew redesign.
At the end of the first semester, after 18 weeks of testing the Crew structure prototype, team data indicated that student engagement over Quarter 2 improved or stayed the same for 87% of their target students in comparison to Quarter 1. They also found that 60% of the target students' grades improved or stayed the same over Quarter 2. The sixth grade team also gained important insights from additional empathy interviews that students want to continue to make connections with their peers and teachers and enjoy working in small groups. For students who struggled to engage, they found the barriers to their success to include home distractions, technology, and disruptive sleep patterns.
Based on promising evidence, the Toki Innovation Council decided to proceed forward and move this program from prototype to a pilot phase during the Spring of 2021. During the pilot phase the team, supported by MMSD’s Research and Program Evaluation Office (RPEO) will prove their concept by gathering additional data and the design’s impact on Black students' sense of belonging, engagement and academic success. The desired outcomes for the pilot program are that data will further validate the hypothesis that this new Crew structure will increase target students sense of belonging, sense of ownership over their learning and Increase their academic engagement.
What It Took:
From the Design Team:
4 participants - 4 6th grade teachers from Toki Middle School
22 days over 8 months for a total of 44 hours per team member
1 Innovation Strategist to coach, plan, facilitate, and guide the team
From the Design Council:
7 participants - 3 Toki Middle School Staff, 2 MMSD Central Office leaders, 1 American Family Institute Partner, 1 Madison Public Library community partner
3 hours - 3 meetings with design team
1 Innovation Strategists to plan facilitate, and guide the team through the process
70 hours - over 8 months