Learning Outcome Losses Were Greater for Students Who Were Fully Virtual during the Pandemic

During the height of the COVID-19 pandemic, schools and students across the country faced drastic changes to how classes and instruction were administered. To mitigate the spread of COVID-19, school districts used different schooling modes, from fully virtual learning to full-time in-person classes, including “hybrid” schooling that involved a mix of the two.

The range of schooling modes students experienced may have contributed to different learning outcomes. For students of color and students from families with low incomes who are more likely to live in disinvested neighborhoods with lower-quality housing and schools, having to conduct school remotely may have had disproportionate negative effects. This study examines the relationship between differences in schooling modes across school districts and students’ test scores.

Analyzing data across 11 states during the 2020–21 school year, the authors used the COVID-19 School Data Hub to document each school district’s schooling mode and calculate the share of the school year conducted under the categories of either “in-person,” “virtual,” or “hybrid.” They then combined data on district-level schooling modes with state assessment data for math and English language arts (ELA) standardized exams for grades 3 through 8. To conduct their analysis, the authors looked at changes in the passing rates of standardized exams from 2016–19 compared with 2021. Their model controlled for local contexts, such as county-level unemployment, school district demographics, and enrollment and test participation rates. They also examined how the observed changes in passing rates varied across states, schooling mode, and student demographics.

Key Findings

• Overall, the average pass rate on state standardized assessments declined by 12.8 percentage points in math and 6.8 percentage points in ELA between 2019 and 2021.
• Districts with higher baseline pass rates, fewer Black and Latine students, and smaller populations of students who are English-language learners or eligible for free and reduced priced lunch were more likely to have access to in-person learning.
• Districts that offered full in-person learning during the 2020–21 school year saw smaller declines in passing rate by about 13 percentage points in math and 8 percentage points in ELA compared with districts with full virtual learning.
• Districts that offered a hybrid learning model reduced loss in passing rates by 7 percentage points in math and between 5 and 6 percentage points in ELA compared with a fully virtual approach.
• On average, the benefits of in-person learning were larger for grades K–3 than for higher grade levels.
• While districts with more Black students were less likely to have access to in-person schooling, the benefits of in-person learning were found to be more significant for districts with a higher percentage of Black students.

Policy Implications

• Policymakers can target additional funding to districts and students, such as those with higher shares of Black or Latine students, that had less in-person learning during the 2020–21 school year to address the larger declines in learning in those areas.
• The varying effects of schooling mode on learning outcomes should be considered a critical factor in school policy responses to pandemics or other large-scale schooling changes.