I try and keep a level head when discussing social problems, but the hardships that young adults and children faced during the pandemic really riles me up. Here in the Twin Cities, the negative consequences of shutting down public schools became pretty obvious pretty quickly. Why did I bold public? Because as far as I can tell, private schools opened back up much more quickly. Here’s the best data I’ve seen at the national level, collected by the NCES:
This aligns with what I saw on the ground here in the Twin Cities. And qualitatively, in person schooling went just fine, while online schooling, especially for young people, was a disaster. Hey, does anyone want to guess whether these kinds of trends might translate into things like, I don’t know, massive inequality?
To very little surprise, yes, it turns out that the last two years have been an inequality production machine unlike anything we’ve really seen in recent generations. For example, see this New York Times article on learning loss.
The three big points, in my opinion.
National test results released on Thursday showed in stark terms the pandemic’s devastating effects on American schoolchildren, with the performance of 9-year-olds in math and reading dropping to the levels from two decades ago.
and
The declines spanned almost all races and income levels and were markedly worse for the lowest-performing students. While top performers in the 90th percentile showed a modest drop — three points in math — students in the bottom 10th percentile dropped by 12 points in math, four times the impact.
and
In math, Black students lost 13 points, compared with five points among white students, widening the gap between the two groups. Research has documented the profound effect school closures had on low-income students and on Black and Hispanic students, in part because their schools were more likely to continue remote learning for longer periods of time.
Why might this matter?
“Student test scores, even starting in first, second and third grade, are really quite predictive of their success later in school, and their educational trajectories overall,” said Susanna Loeb, the director of the Annenberg Institute at Brown University, which focuses on education inequality.
“The biggest reason to be concerned is the lower achievement of the lower-achieving kids,” she added. Being so far behind, she said, could lead to disengagement in school, making it less likely that they graduate from high school or attend college.
But this is probably just some silly MTurk study or low quality survey, right?
The National Assessment of Educational Progress is considered a gold standard in testing. Unlike state tests, it is standardized across the country, has remained consistent over time and makes no attempt to hold individual schools accountable for results, which experts believe makes it more reliable.
I was surprised that the New York Times didn’t have a spiffy visualization of the trends, given how many of their other articles include such things. In fact, the NYT didn’t even link to the report! Luckily, the Washington Post did, and Alec MacGillis selected some nice summary measures of the decline.
Here is one:
Massive declines in reading and math scores. Decades of real progress wiped away in a few short years.
Unsurprisingly, these reversals did not occur uniformly across the population. You can see that groups differed in the decline.
You can see striking racial gaps, particularly among math scores. Black students saw a 13 point decline, Hispanic 9, and White and Asian around 6.
The pandemic didn’t produce learning loss equally. We can look at the top and bottom percentiles of test scores to see more spread of educational outcomes. Below’s a visualization of the spread growing, from a nice explainer thread:
Loss at the bottom, stability at the top. More education inequality after the pandemic than before. But this doesn’t really do justice to the profound shift of educational inequality that has occurred. Below is a table of the gap between high and low test scores from the NAEP.
Here we see 9 year old math test scores at the 90th (high scores) and 10th (low scores) percentiles. The 90ths percentile was not hit particularly hard, staying at around the 280s. But the low scores dropped to 1986 levels, producing a spread that' hasn’t seen seen in the past half century. And by the way, the last had century has had itself a fair amount of educational inequality.
The production of more inequality along established tracks was also found by Raj Chetty in his Economic Tracker. Below is the relative change in math scores among high, middle, and low income schools.
These are relative changes to the baseline. And the baseline had, unsurprisingly, significant gaps in performance across high-, middle-, and low-income schools. So The already existing SES gaps were exacerbated in the pandemic.
On the ground reports seem to show that students are really struggling. A quote from a recent New York Times article about New York City students:
Data on how New York City students are faring academically has been scarce. The state has not yet released the last school year’s test results, and the city has not made public data on how students performed on tests it administered during the school year.
But a survey of more than 100 New York City teachers found that the vast majority believe students are behind academically compared with how they fared before the pandemic. And national test results released Sept. 1 found that 9-year-olds fell far behind students who took the test in years past.
“What I’ve seen is astonishing,” said Aaron Worley, a social worker at P.S. 243 and P.S. 262 in Bedford-Stuyvesant, Brooklyn. “Kids in fifth grade that are struggling with their reading, their writing, their sentence comprehension — it’s alarming.”
The schools chancellor, David C. Banks, joined the mayor on Thursday at P.S. 161, the site of one of the city’s new dyslexia programs. He described the issue as a central piece of a larger challenge: “the fundamental way in which we teach our kids how to read.”
A quote from a story from the Los Angeles Times on LA students:
A Unified test scores released Friday showed the harsh reality of the pandemic’s effects on learning across all grade levels, with about 72% of students not meeting state standards in math and about 58% not meeting standards in English, deep setbacks for a majority of Los Angeles schoolchildren who were already far behind.
The scores show that about five years of gradual academic progress in the nation’s second-largest school district have been reversed, L.A. schools Supt. Alberto Carvalho said Friday.
“The pandemic deeply impacted the performance of our students,” said Carvalho, who spoke at a news conference at Aragon Avenue Elementary School in Cypress Park. “Particularly kids who were at risk, in a fragile condition, prior to the pandemic, as we expected, were the ones who have lost the most ground.”
and
The types of students who fared the worst “are not small subgroups in California, particularly in L.A.,” said UCLA education professor Tyrone Howard. “This has consequences for us as a state, as a city [and] I think it poses significant challenges about who we are and who we want to be if we’re not intentional about who is being left behind.”
I’m not sure if this equals the tragedy of the pandemic, but it sure seems close. A stunning disaster that fell on the most marginalized and least powerful members of our society: poor and minority children. In my humble opinion, anybody who professes to care about equality, justice, or fairness should be feeling their blood boiling right about now.
The above is the outcome of closing schools, closing public schools more than private ones, and keeping them closed for a very long time. I haven’t linked to any study for this claim because, well, it’s so obvious that I don’t think we need a study on it. But school closure and all the school disruption that has occurred in the last two years is such the obvious reason why this happened. Really, any parent or young person could probably let you know that online school was, well, not exactly the optimal route, particularly for students from more difficult social origins.
Prominent Academics React Surprisingly
One might think: “prominent sociologists on twitter are big into education, race, and disparities. I bet this is a big deal to them!” That’s what I thought. But the reactions to these learning loss findings have been pretty surprising. Below are examples from very prominent academics who focus on education, race, and inequality:
Here’s one example:
Here’s another:
Here’s another:
Responses have been much hand-wavy-er than I would have anticipated. The responses above have very 80’s guy execu-speak vibes that I don’t think rise to the occasion that we’re in.
In the interest of being constructive, I’d like to call on academics to take the issue of learning loss a tad more seriously. I’d ask folks to think of this as a generational disaster in terms of its production of educational inequality. I think there are a few things people should keep in mind if they’ve been reacting in the kinds of ways shown above.
It might be shortsighted and overly postmodern to question these findings.
The learning loss findings seem to be rooted in pretty mainstream and straightforward metrics of education. I’d argue that denying learning loss may be a pretty postmodern stance that can be easily mobilized to logically argue for the dismantling of public education and the Department of Education. If this stuff isn’t real and measurable, why throw money at such a non-thing as learning?
If you think test scores are too rigid and the squishy side of education is more important, I’d argue the squishier part of education was worse during the pandemic (anyone else sitting in online kindergarten would surely agree, and feel free to email me if you want some concrete examples).
There’s no reason to try and provide cover for the United States’ decisions.
That there might be some massive unanticipated outcome of a radical, experimental, and ad hoc policy choice seems…pretty obvious. There’s little need to try and deny it. One could argue that the entire discipline of history is the study of a cascade of unintended consequences.
Perhaps you’d argue that no other outcome was possible. I’d ask you to think long and hard about the implications of such an argument.
Learning loss looks to be very, very consequential for justice and equality.
I’ll admit that I haven’t read his book closely, but wouldn’t school shutdowns be defined as racist in a Kendi-style framework? The race and class gaps are just so massive. I’d argue that it does little good to try and downplay a massive production of inequality.
Perhaps you might argue that the costs were worth it. But I’d ask that you think very seriously about this argument.
The production of generational inequality is not ok, even in hard times.
Public education expansion occurred during the following events: the Great Depression, World War 2, the Vietnam War, Civil Rights, 9/11, the opioid epidemic. Which is to say: history is littered with hardship, either at the macro or local levels. We won’t reach history’s end, so educational attainment will arguably always occur in difficult times. I’d argue that hardship is not a reasonable explanation or excuse for producing a generational-level of education inequality. Or…if it is, then we necessarily must shut off educational equality as a goal or possibility, since there’s no reason to think we’ll be able to successfully avoid hardship for a sufficiently long enough time to close educational gaps.
There were other options playing out at the time.
European countries provided many different sets of decisions, including opening schools much earlier and restricting masks to older children and adults. One of the people I mention wrote a prominent article in the New York Times titled something like, “European countries have the welfare state. America has women.” So academics definitely have the wherewithal to compare policy decisions across social contexts. We might not like what we did, or we might think that it was worth it, but…this was at least partially the outcome of a decision made from a broader suite of choices.
Denying or downplaying learning loss undermines your legitimacy.
I’m a little worried that academics are operating like PR officers providing cover for a company’s massive oil spill. The learning loss experienced during the pandemic is very obvious in the data. It’s obvious in it importance. I suspect that folks are equivocating because school closure was primarily energized by the left side of the political left. I realize this makes the outcomes of school closure embarrassing, but denial, or omission, or equivocation of the issue simply calls into question one’s broader legitimacy and trustworthiness.
We probably won’t reverse these inequalities.
I’d recommend settling into the mess we’re in. Or at least, I’d strongly consider that these gaps and depths of learning loss will not be easily, or broadly, made up. In September 2020, Alec MacGillis wrote a great ProPublica / New Yorker article. I recommend reading it. He cites research showing that kids have real difficulty making up learning loss following things like natural disasters.
There will be new kinds of inequalities that exist between generations.
Kids going through school right now will be disadvantaged compared to those who came before and those being born right now. I have no idea what this means for social policy, employment opportunities, the distribution of social welfare, or issues of mobility and status attainment. But I suspect it’s going to be weird. It seems reasonable to anticipate a whole bunch of odd social problems and inequality coming down the pipe.
One might suggest that social policies should begin to be designed to assist today’s children.
There are going to be very serious consequences carried forward by millions of children.
Sometimes I’ve seen people say, “It’s ok because everyone lost a year.” Or whatever. But that’s not the case. When we look at the data, we see that Black, Hispanic, Native American, and poor children fared the worst. Remember the public/private split that began this post. School closures and the resulting learning loss could be argued to be the most consequential intergenerational inequality change of our lifetimes. It could be argued as the most consequential amplifier of racial inequality of our lifetime (or at least, it could rival mass incarceration).
I suspect that the consequences of producing race and class inequalities on steroids will be very predictable. I can think of few reasons to deny, justify, or hand wave these issues.
I don’t think we can just shut up or ad hoc our way out of this. There are snowball risks.
I’d be shocked if consequences didn’t continue to snowball. For example: public school enrollment has been seriously hit. Here’s an example from Minnesota:
May not look huge, but I’m seeing similar 3-7% declines in enrollment in reports from different cities and suburbs. Enrollment declines of these kinds of magnitudes have massive implications for public school funding. For example, an article from the New York Times a few months ago:
ORANGE COUNTY, Calif. — In New York City, the nation’s largest school district has lost some 50,000 students over the past two years. In Michigan, enrollment remains more than 50,000 below prepandemic levels from big cities to the rural Upper Peninsula.
In the suburbs of Orange County, Calif., where families have moved for generations to be part of the public school system, enrollment slid for the second consecutive year; statewide, more than a quarter-million public school students have dropped from California’s rolls since 2019.
And since school funding is tied to enrollment, cities that have lost many students — including Denver, Albuquerque and Oakland — are now considering combining classrooms, laying off teachers or shutting down entire schools.
All together, America’s public schools have lost at least 1.2 million students since 2020, according to a recently published national survey. State enrollment figures show no sign of a rebound to the previous national levels any time soon.
So…going forward we’ll have kids with more needs and we’ll have fewer resources to handle them. We’ll see lower funding for public schools with more resourced kids in private, charter, and homeschool systems. That might well be a new force of more and more serious inequality. That doesn’t seem like something that can be easily ignored or discounted.
So maybe we could just, like, recognize this to be a big deal?
Maybe I’m misreading the scene. But I’d like to use this space as a call to maybe pay a bit of attention to the learning loss that was created by school closure. I’d like to gently push back against academics who are downplaying these findings. I realize it’s embarrassing when one’s “side” creates a disaster. But the stakes are too high to ignore the issue. I’m increasingly convinced that we’re facing a once-in-a-century social problem that holds the potential to snowball into something truly awful. We really can’t afford to downplay it.