A plunge in the number of students participating in standardized tests during COVID-19 is complicating the state’s ability to gauge how well its schools are doing on student achievement.
A long-term review of student test scores in Colorado shows a small but steady rise over the last six years until the pandemic hit, when scores of standardized tests took a nosedive as participation in the tests — which is optional for students — took a beating.
In which direction are your school district's test scores going?
Assessment scores from the 2020-2021 academic year show student performance fell, more significantly in math than literacy. Just 27.4% of students who tested that year met or exceeded expectations in math, when 32.7% met that benchmark the year before the pandemic hit. Performance in literacy also declined, with 43% of students meeting or exceeding expectations in the 2020-2021 academic year, down from 44.5% the year before.
However, education officials who spoke to The Denver Gazette said this data set might not be telling the complete story as participation rates dropped from over 90% to as low as 57.9% across the state, depending on the school district.
New test scores will be released this month, and there are mixed opinions on what those results will show.
“For English Language Arts, I think pre-pandemic we were seeing stair-step improvement,” Joyce Zurkowski, the chief assessment officer at CDE said. “What we saw (with math) was not as clearly stair-step, but there was improvement from 2015 to 2019.
“Different parts of the state were impacted differently (by the pandemic), but it’s fair to say education systems were impacted significantly in 2020 and 2021.”
The highest-achieving schools in the state, such as Cheyenne Mountain District 12 in Colorado Springs and Steamboat Springs RE-2, rode the pandemic out without huge drops in achievement, while scores in some of the lowest-achieving districts such as and Springfield RE-4 and Adams County 14 plunged dramatically during the pandemic.
The 2021 assessment showed Colorado students on average throughout the state regressed in gains that had been made during the first four years the state used the annual Colorado Measures of Academic Success assessment.
Eighth graders who met or exceeded the Colorado Department of Education’s expectations on its English, language arts (ELA) assessment rose nearly 10 percentage points between the 2014-2015 and 2018-2019 academic school years. At the same time, the number of eighth graders who met those same expectations on the state’s math assessment rose by 18 percentage points, according to CDE data.
This positive trend wasn’t limited to only higher-grade levels as Colorado students statewide between third and eighth grade improved by a minimum of 3 percentage points over the four-year period on the ELA assessment. Similarly, all but one grade level had higher achievement on the math assessment, according to the data.
Although the data shows improvement by Colorado students up until the pandemic struck, Tyler Sandberg, the co-founder and consultant at Ready Colorado, said there have always been problems that the pandemic only exacerbated.
“The pandemic really just took some serious gulps of achievement gaps and widened them to become Grand Canyon size,” Sandberg said. “What the pandemic and school closures effectively did was widen achievement gaps and gaps of opportunity, and those who were poorly served by our school system to begin with were served the worst during COVID, so it really created a system of have and have nots in terms of access to high quality education.”
CMAS tests are aligned to the state’s academic standards and designed to gauge how well Colorado students are mastering the standards and how prepared they are for the next grade level. Results provide a snapshot of schools, districts, teachers, parents and the community to show how a specific student, grade level, or district is progressing academically.
Colorado Assessment History
The state education department has administered three different end-of-the-year assessments to Colorado students since the turn of the century with the Colorado Student Assessment Program, Transitional Colorado Assessment Program and now CMAS.
CSAP was administered between 1997 and 2011 and was replaced by a temporary assessment, the Transitional Colorado Assessment Program through the 2013-2014 school year, said Jeremy Meyer, a spokesman for CDE.
The change in assessment began after the passage of Senate Bill 2008-212, which ordered CDE to update its assessment standards. At the time, the Colorado Student Assessment Program did not have grade specific standards and instead was “grade-span based,” Zurkowski said.
Each test varies in a multitude of ways, and multiple officials said it was not possible to compare the current CMAS data to previous assessments. Amie Baca-Oehlert, president of the Colorado Education Association said such a comparison “would be dangerous.”
“If you try to compare those scores, you’ll see this huge drop between 2014 and 2015 and it’s not that suddenly our kids shifted dramatically in terms of their ability and achievement, but what we were asking students to be able to do shifted in that year,” Zurkowski said. “The Colorado model content standards were not preparing kids adequately for college and career.”
After the inaugural wave of CMAS exams had been turned in, the state began analyzing how Colorado students fared on a new assessment that asked for much more than in years past. CDE officials reported a significant drop in performance.
“When we look at the last 22 years there’s been one change of significance and that was 2015,” Zurkowski said.
That year, students were asked to showcase their knowledge on several new skills and abilities that had not been mandated before the 2014-2015 year. At the same time, teachers were still attempting to get a grasp on the new requirements, officials said.
“Math was always a sticky (subject),” said Rep. Barbara McLachlan, who is the chair of Colorado House of Representative’s Education Committee and a former teacher of two decades.
McLachlan, a Democrat from Durango, said teacher shortages and changing student populations contribute to the difficulty in teaching math and may explain some of the decline in test scores.
“Year one you have to complete certain tasks, and they build on those in years two, three and so on,” she said.
Although sixth grade declined over the four-year period starting in 2014, Zurkowski said there was somewhat of a caveat as seventh and eighth graders had the option to take the high school assessment instead of their grade-level test. This option began to be phased out during the 2017-2018, which resulted in higher achievement for that specific year.
This increase in math scores was recorded in all seven districts The Denver Gazette analyzed, ranging from an increase of .3 percentage points in Denver Public Schools to 25.7 percentage points in Jefferson County Public Schools, according to the data.
Even though overall achievement improved across the state prior to the COVID-19 pandemic, some districts saw more growth than others. Each of the seven districts analyzed by The Denver Gazette reported lower scores on at least one assessment — sixth-grade math —on the 2018-2019 CMAS assessment when compared to 2014-2015.
Adams County School District 14 regressed 4.6 percentage points on the sixth-grade math assessment. However, the district also reported a 7.8 percentage point increase on the ELA assessment.
The Douglas County School District increased achievement the most when compared to the other six districts. While the district reported a minuscule .5 percentage point drop in sixth grade math, it also reported a 31.4 percentage point increase in eighth grade math and saw improvement on each ELA assessment, according to the data.
To the north, Weld County RE-8 struggled the most, as all, but two classes regressed in math achievement scores and overall scores declined by as much as 6.7 percentage points. However, their reading scores ranged between a decline of 4.2 percentage points and an increase of 15.3 percentage points, depending on the specific grade level, according to the data.
To the south, Colorado Springs 11 saw substantial increases. Like most districts, there was regression in two assessments with those being in sixth grade math and third grade reading dipping 2.1 and .5 percentage points, respectively.
Meanwhile, Denver Public Schools increased their achievement in all but one assessment — seventh grade ELA. The grade dropped 3.3 percentage points, but the district also saw progress as high as 11.8 percentage points for fourth grade literacy.
Additionally, Denver schools and Douglas County schools decreased the gap in achievement between students who receive free and reduced lunch and those who do not.
Students enrolled in Denver schools who receive free or reduced lunch reported higher scores following the four-year period on every ELA assessment. Students who did not receive free or reduced lunch only made gains on the fourth, seventh and eighth grade math assessments during that period, according to the data.
Dr. Tony Smith, a deputy superintendent at DPS, said district officials “doubled-down” on a myriad of programs, specifically innovation and project-based learning in schools. He compared these innovative schools to the Nebraska Cornhuskers triple-option offense run during the 1990s, when the opposing team knew, Nebraska was going to run the ball but still couldn’t stop an innovative game plan.
“Individual schools have really strong plans to grow the teachers to lead and for leaders to lead in a way that overcame and provided grade level access to our kids,” Smith said. “We understand that grade level access and rigorous course material is an accelerator of progress and achievement, and so, while some people may be against (innovation), what we know is true is if you have a system and you follow it to fidelity, success usually follows.”
Innovation schools are slightly different than ordinary schools as they have flexibility in their curriculum that aims to increase achievement levels in non-traditional ways, according to DPS. They also allow teachers and administrators to use resources in a way that directly impacts families, the district stressed.
Ready Colorado’s co-founder Sandberg said these innovation platforms should be expanded and used more frequently across the state for achievement improvement.
“I’ve never heard (from a parent with more than one kid) that their children all learn identically,” Sandberg said. “Kids learn in different ways even within the same family … and we need to have districts that provide as many opportunities as possible as district schools or innovation schools. Choice is a really great solution to a very diverse set of student needs and challenges.”
Virtual learning lags
Douglas County schools also closed the gap, as students on free and reduced lunch improved more than their fellow students on half of the assessments.
While students made progress on the CMAS assessment between the 2014-2015 and 2018-2019 academic years, the current state of learning is unknown. In March 2020, students in Colorado and across the country went from sitting at a desk surrounded by their classmates to sitting in front of a computer for eight hours of virtual learning.
The CMAS assessment was canceled because of the pandemic during the 2019-2020 year, however, it was administered in the spring of 2021.
From afar, the results looked bleak as reading scores for third, fifth and seventh graders fell between 1.2 and 3.9 percentage points, while more than 70% of students in fourth, sixth and seventh grades who took the test failed to meet or exceed expectations on the math assessment, according to the state.
But officials said that these numbers don’t tell the entire story as the number of students who took each assessment changed as certain grade levels took either the ELA or math test, and not both like in years past, and the overall participation rates plummeted to all-time lows, Zurkowski said earlier this year.
“It’s so hard to tell what is actually happening with CMAS, because you can look at just the scores and draw one conclusion,” said Baca-Oehlert, CEA president. “I would urge caution to jumping to conclusions just using the CMAS scores alone, because if you think about it, it’s very hard to draw comparisons right now.”
The biggest impact on these scores, and the main reason officials say the numbers may not be telling the whole story, is the decline in participation. Families have had the option to opt their student out of the CMAS test since the mid-2010s. Despite this, participation rates remained at or around 90%, until the pandemic hit.
Zurkowski said participation dropped between 20% and 30% in 2021, which has made it difficult for officials to compare them to previous assessments. And just how there was variance in the progress made in the CMAS between 2015 and 2019, there was a variance in participation rates in 2021.
For example, Adams County 14 saw a 56% decline in the number of seventh graders who took the ELA assessment compared to 2019. Denver Public Schools saw participation rates drop from an average of 97.5% in 2018-2019 to under 50% the following year.
Zurkowski said students of color, specifically Black and Hispanic students, opted out more than other students, which goes against the historical trend.
“I think in part it’s a (reflection) of how our communities were impacted (by the pandemic,) and it was disproportional,” she previously said. “Our students who were Hispanic and Black communities were impacted more by COVID than our students who were not in those communities. And I’m not saying that students who were in the suburbs weren’t impacted, but we know our Hispanic and Black communities were impacted more.”
The change in participation prompted Zurkowski and her team to create two data sets comparing past results to the 2021 scores and a “matched” data set that showed what 2021 would have looked like based on historical trends.
The matched data showed that if there had been greater participation the scores would have marginally decreased between .6 and 1.7%, according to a presentation given to the State Board of Education last August.
Education officials now are focused on next month when the 2021-2022 CMAS assessment results will be released. Zurkowski said despite this past school year still being interrupted by the COVID-19 pandemic, professionals still expect they’ll get a better idea on the true impacts of the pandemic this month.
But she and others said they do not expect the results to return to pre-pandemic levels.
“We don’t have expectations for what the data will look like, but we hope we don’t see a continued lowering of achievement,” Zurkowski said. “We wanted to stop that lowering of achievement and at least see an increase and I think it’s fair to say that we were not expecting to return to 2019 achievement levels between 2021 and 2022. The hope is that within the next couple of years we’ll be able to return to that achievement level, but recovery is going to take time.”
Paul Teske, the dean of the school of public affairs at the University of Colorado Denver, said he believes like on past assessments there will be a lot of variation, though he’s not sure to what extent.
“It’ll be interesting to see the Colorado specific data, but there’s some hope of a possible rebound,” Teske said. “We’re going to learn whether students are starting to catch back up to where students were a few years ago or if there was a big dip. If it’s the latter we’re going to have to figure out what that means for people graduating and going to college.”
Sen. Tammy Story, D-Conifer, who’s the vice chair of the Senate Education Committee, said she wouldn’t be surprised to see lower achievement scores than in 2020-2021 due to the number of interruptions, such as remote learning or students causing disruptions in classrooms, occurring because of the pandemic.
Story said these disruptions have disturbed student learning, and she believes CMAS should not be given out until school environments are back to “normal.”
“I don’t see these (tests) as a good baseline to compare to when there is so much disruption,” Story said. “I think we should be waiting to do testing until we’ve gone through an entire school year, and it is determined we’ve concluded a normal year, even if it doesn’t look like a 2018-2019 year, but there’s no more quarantines, less mental health issues and attendance is more stable.”
Story said she couldn’t say when these conditions might return but said teachers should be focusing on instructional time versus administering a test.
Although no one is certain of what the data will show come August, Smith said he and the rest of the DPS staff would be proud if the scores improved. But that’s not their only goal as they’re aiming to improve scores to all-time highs.
“The pandemic offered us opportunities and we want to see our students surpass what we did in the past, and I think the greatest fault that we can have, is think of the pre-pandemic (scores) as the benchmark,” Smith said. “We still have a long way to go, and yes, we need to get back to grade level, but we also have to exceed what we did in 2018-2019.”