As some high schools across the country try new ways to engage and educate students, they often turn to innovative approaches that are still being evaluated to gauge their effectiveness.
These include “personalized learning,” creating small schools, and focusing entire campuses on STEM (science, technology, engineering and math). Two researchers who have studied these concepts in U.S. high schools recently shared what they’ve learned – as well as the challenges they faced in trying to measure success – during an Education Writers Association seminar at High Tech High in San Diego.
“There’s not one defining characteristic or model of an innovative school,” said Barbara Means, the executive director of learning sciences research for Digital Promise, a nonprofit organization authorized by Congress to stimulate innovation in education.
Curriculum, Assessments, School Culture
Means points to five major components of schooling where innovative practices can be applied. These include the curriculum, instructional assessments and practices, norms and practice around the social culture in schools, the staffing model, and the use of time and space.
“I think you have to be different on at least four of those [areas],” she said, “because they’re all interrelated.”
The researchers said two types of studies are especially helpful in gauging efforts to innovate in high school.
“Implementation studies,” which look at how new ideas are carried out, are interesting to those who may be thinking of adopting similar ideas or practices in their schools, said Elizabeth Steiner, of the RAND Corporation. Steiner has researched personalized learning in 40 schools throughout the country and “Opportunity by Design” high schools funded by the Carnegie Corporation of New York. (Carnegie provided major funding for this EWA seminar. EWA retains sole editorial control.)
Sometimes, the “vision may be different from the reality,” Steiner said.
The second study type is summative, which evaluates the academic effect of specific innovative strategies, Means said. She is researching the “impacts of inclusive STEM high schools on student outcomes” in Texas, North Carolina and Ohio for SRI International.
Conventional high schools, Means said, typically require prescribed “seat time,” change class periods every 50 minutes, and assign teachers to one specific subject area in which they work with about 150 students per year, preparing them to pass state tests or Advanced Placement exams.
Steiner, who studied schools that received Next Generation Learning Challenge (NGLC) grants, said it is sometimes difficult to find comparison groups when studying schools.
“A lot of practices considered innovative are being done in lots of different schools,” she said. “So, we can’t say we’re comparing a bucket of innovative schools to a bucket of non-innovative schools.”
In RAND’s studies, she said, “We saw some differences, but not as many as we might have expected. That was a little bit of a surprise.”
Putting Personalized Learning Under the Microscope
The personalized learning study found that students in the NGLC schools reported higher levels of personalization than their national counterparts in non-NGLC schools. RAND highlighted practices such as one-on-one tailored support, use of up-to-date data on student progress to group students and personalize instruction, students keeping track of their own progress, “competency-based practices” that allow students to move through coursework based on mastery, and flexible use of staff, space and time.
But Steiner and her team found that some “more-difficult-to-implement aspects” of personalized learning were the same nationally as in the NGLC schools, including student discussions with teachers on goals and progress; the collection of data related to students’ strengths, weaknesses and goals; and students’ ability to choose topics and materials.
Overall, RAND found NGLC students scored about 3 percentage points higher in math compared to a group of similar students at other schools. And although researchers found a similar trend in reading, they did not consider those gains to be statistically significant.
When the EWA panel’s moderator, journalist Beth Hawkins of The 74, asked about the paucity of evidence in support of personalized learning and other innovations, Steiner said she was not aware of many rigorous “whole school” studies. While the NGLC study tried to look at entire schools, Steiner said “it could be difficult to say which piece worked” and which didn’t, since every school was doing something different.
So far, she said, the evidence base for integrating social and emotional learning with academics “is pretty strong” and evidence for “mastery-based” student progress is “more robust” at the secondary level. Evidence supporting some other elements of personalized learning – such as developing student choice and helping to engage students in their learning – is “very mixed,” she said.
Similarly, Means said it’s very difficult to “disentangle the things creating different outcomes.” Further complicating her research is the time her team must wait to find out whether STEM-focused schools are really preparing students for a STEM college major, she said.
“Many of the schools start with 9th grade, then add 10th grade, and so on,” she said. “So, it takes a lot of patience and time. I think that’s why there’s not a lot of places to look for research on innovative schools.”
Challenges to innovation, the researchers said, include the need for teachers to develop new, personalized curriculum, especially in math. Accountability is another thorny issue, since schools are expected to administer state tests and meet state four-year graduation requirements, even though they’re striving to allow students to work at their own pace and move on only after they’ve mastered the content.
“You might say, ‘Students should have as much time as they need,’” Steiner said. “If you do that, you might imagine high school could take more than four years for some kids. Is the world ready for that – especially in states where the four-year graduation rate is an accountability metric? Some principals and teachers said that was something they were struggling with.”
Means said math was the “biggest challenge” in non-selective STEM schools because some students came into 9th grade working at a 7th grade level or below, even though they needed to be ready to take calculus in their first year of college to pursue a STEM major. This means they need to get through a good pre-calculus course before graduating from high school, she said.
“They’re continuing to work on it,” Means said.
Vision and Reality
Journalists interested in writing about personalized learning and its impact in a given school should try to understand the students’ and teachers’ perspectives, the researchers said. In addition, it’s important to find out how many students and teachers – and which students and teachers – are involved in innovative programs.
“Ask about the pervasiveness of innovation within the school,” Means said. “Schools like High Tech High are very clear about having a philosophy and set of norms that are very pervasive.”
Steiner warned that “there’s not really widespread agreement in the field” about what terms such as “mastery” and “personalized learning” mean. Figuring out the difference between the school’s vision and reality, she said, is “the crux of education research.”
More information about the RAND study can be found in an EdSource post based on a follow-up interview with Steiner focused on NGLC schools, including 12 in California.
