Advertisement

Estimating COVID-19 Vaccination and Booster Effectiveness Using Electronic Health Records From an Academic Medical Center in Michigan

  • Emily K. Roberts
    Affiliations
    Department of Biostatistics, School of Public Health, University of Michigan, Ann Arbor, Michigan

    Center for Precision Health Data Science, School of Public Health, University of Michigan, Ann Arbor, Michigan
    Search for articles by this author
  • Tian Gu
    Affiliations
    Department of Biostatistics, Harvard T.H. Chan School of Public Health, Boston, Massachusetts
    Search for articles by this author
  • Abram L. Wagner
    Affiliations
    Department of Epidemiology, School of Public Health, University of Michigan, Ann Arbor, Michigan
    Search for articles by this author
  • Bhramar Mukherjee
    Correspondence
    Address correspondence to: Bhramar Mukherjee, PhD, Department of Biostatistics, School of Public Health, University of Michigan, 1415 Washington Heights, Ann Arbor MI 48109.
    Affiliations
    Department of Biostatistics, School of Public Health, University of Michigan, Ann Arbor, Michigan

    Center for Precision Health Data Science, School of Public Health, University of Michigan, Ann Arbor, Michigan

    Department of Epidemiology, School of Public Health, University of Michigan, Ann Arbor, Michigan

    Michigan Institute for Data Science, University of Michigan, Ann Arbor, Michigan
    Search for articles by this author
  • Lars G. Fritsche
    Correspondence
    Address correspondence to: Lars G. Fritsche, PhD, Department of Biostatistics, School of Publich Health, University of Michigan, 1415 Washington Heights, Ann Arbor MI 48109.
    Affiliations
    Department of Biostatistics, School of Public Health, University of Michigan, Ann Arbor, Michigan

    Center for Precision Health Data Science, School of Public Health, University of Michigan, Ann Arbor, Michigan
    Search for articles by this author
Open AccessPublished:July 26, 2022DOI:https://doi.org/10.1016/j.focus.2022.100015

      HIGHLIGHTS

      • Studying vaccine effectiveness is crucial for informing public health interventions.
      • COVID-19 vaccines lowered infection rates by >60% in 2021.
      • COVID-19 vaccines lowered severe outcomes rates by >74% in 2021.
      • Vaccines’ protection against severe outcomes was persistent throughout 2021.
      • Boosters remarkably improved protection against both severe outcomes and infections.

      Introduction

      Observational studies of COVID-19 vaccines’ effectiveness can provide crucial information regarding the strength and durability of protection against SARS-CoV-2 infection and whether the protective response varies across different patient subpopulations and in the context of different SARS-CoV-2 variants.

      Methods

      We used a test-negative study design to assess vaccine effectiveness against SARS-CoV-2 infection and severe COVID-19 resulting in hospitalization, intensive care unit admission, or death using electronic health records data of 170,741 adults who had been tested for COVID-19 at the University of Michigan Medical Center between January 1 and December 31, 2021. We estimated vaccine effectiveness by comparing the odds of vaccination between cases and controls during each 2021 calendar quarter and stratified all outcomes by vaccine type, patient demographic and clinical characteristics, and booster status.

      Results

      Unvaccinated individuals had more than double the rate of infections (12.1% vs 4.7%) and >3 times the rate of severe COVID-19 outcomes (1.4% vs 0.4%) than vaccinated individuals. COVID-19 vaccines were 62.1% (95% CI=60.3, 63.8) effective against a new infection, with protection waning in the last 2 quarters of 2021. The vaccine effectiveness against severe disease overall was 73.7% (95% CI=69.6, 77.3) and remained high throughout 2021. Data from the last quarter of 2021 indicated that adding a booster dose augmented effectiveness against infection up to 87.3% (95% CI=85.0, 89.2) and against severe outcomes up to 94.0% (95% CI=89.5, 96.6). Pfizer-BioNTech and Moderna vaccines showed comparable performance when controlling for vaccination timing. Vaccine effectiveness was greater in more socioeconomically affluent areas and among healthcare workers; otherwise, we did not detect any significant modification of vaccine effectiveness by covariates, including gender, race, and SES.

      Conclusions

      COVID-19 vaccines were highly protective against infection and severe COVID-19 resulting in hospitalization, intensive care unit admission, or death. Administration of a booster dose significantly increased vaccine effectiveness against both outcomes. Ongoing surveillance is required to assess the durability of these findings.

      Keywords

      Abbreviations:

      VE (vaccine effectiveness), MM (Michigan Medicine), RT-PCR (reverse transcription-polymerase chain reaction), SES (socioeconomic status), NDI (neighborhood disadvantage index), HCW (healthcare worker)

      INTRODUCTION

      A total of 3 coronavirus disease 2019 (COVID-19) vaccines were developed, assessed for efficacy against symptomatic COVID-19 disease in placebo-controlled trials, and approved under emergency use authorization in the U.S. by February 2021: mRNA-1273 (Moderna), BNT162b2 (Pfizer-BioNTech), and Ad26.COV2.S (Johnson & Johnson-Janssen).
      • Baden LR
      • El Sahly HM
      • Essink B
      • et al.
      Efficacy and safety of the mRNA-1273 SARS-CoV-2 vaccine.
      • Polack FP
      • Thomas SJ
      • Kitchin N
      • et al.
      Safety and efficacy of the BNT162b2 mRNA Covid-19 vaccine.
      • Sadoff J
      • Gray G
      • Vandebosch A
      • et al.
      Safety and efficacy of single-dose Ad26.COV2.S vaccine against COVID-19.
      As of June 16, 2022, a total of 592 million doses had been administered, and 222 million people had been fully vaccinated in the primary series, meaning that they received at least 1 dose of Janssen or 2 doses of Pfizer-BioNTech or Moderna vaccine in the U.S.

      COVID-19 vaccinations in the United States. Centers for Disease Control and Prevention. https://covid.cdc.gov/covid-data-tracker/#vaccinations_vacc-people-onedose-pop-12yr. Updated July 29, 2022. Accessed November 18, 2021.

      Administration of booster doses has been shown to provide more protection than the primary series,
      • Bar-On YM
      • Goldberg Y
      • Mandel M
      • et al.
      Protection of BNT162b2 vaccine booster against COVID-19 in Israel.
      ,
      • Arbel R
      • Hammerman A
      • Sergienko R
      • et al.
      BNT162b2 vaccine booster and mortality due to Covid-19.
      but only 47.2% in the U.S. had received a booster as of June 16, 2022.

      Our world in data. Coronavirus (COVID-19) vaccinations; 2022 2022/02/10; https://ourworldindata.org/covid-vaccinations.

      Observational studies of vaccine effectiveness (VE) can assess real-world effectiveness, estimate duration of protection, identify protection against new variants that arise, and provide guidance on booster requirements.
      • Dagan N
      • Barda N
      • Kepten E
      • et al.
      BNT162b2 mRNA Covid-19 vaccine in a nationwide mass vaccination setting.
      • Thompson MG
      • Stenehjem E
      • Grannis S
      • et al.
      Effectiveness of COVID-19 vaccines in ambulatory and inpatient care settings.
      • Bruxvoort KJ
      • Sy LS
      • Qian L
      • et al.
      Effectiveness of mRNA-1273 against delta, mu, and other emerging variants of SARS-CoV-2: test negative case-control study.
      • Tenforde MW
      • Self WH
      • Naioti EA
      • et al.
      Sustained effectiveness of Pfizer-BioNTech and Moderna vaccines against COVID-19 associated hospitalizations among adults - United States, March–July 2021.
      • Bajema KL
      • Dahl RM
      • Prill MM
      • et al.
      Effectiveness of COVID-19 mRNA vaccines against COVID-19-associated hospitalization - five Veterans Affairs Medical Centers, United States, February 1–August 6, 2021.
      • Rosenberg ES
      • Holtgrave DR
      • Dorabawila V
      • et al.
      New COVID-19 cases and hospitalizations among adults, by vaccination status - New York, May 3–July 25, 2021.
      • Lin DY
      • Gu Y
      • Wheeler B
      • et al.
      Effectiveness of COVID-19 vaccinesover a 9-month period in North Carolina.
      • Andrews N
      • Tessier E
      • Stowe J
      • et al.
      Duration of protection against mildand severe disease by Covid-19 vaccines.

      Results of COVID-19 vaccine effectiveness studies: an ongoing systematic review. International Vaccine Access Center, Johns Hopkins Bloomberg School of Public Health, World Health Organization.https://view-hub.org/sites/default/files/2022-03/COVID19%20Vaccine%20Effectiveness%20Transmission%20Studies%20-%20Summary%20Tables_20220310.pdf. Updated March 10, 2022. Accessed March 19, 2022.

      Observational studies conducted early in 2021 through COVID-19‒Associated Hospitalization Surveillance Network, a surveillance network in 13 states, including Michigan, found that 2 doses of Pfizer and Moderna provided 96% effectiveness in protecting against hospitalization,
      • Moline HL
      • Whitaker M
      • Deng L
      • et al.
      Effectiveness of COVID-19 vaccines in preventing hospitalization among adults aged ≥65 Years – COVID-NET, 13 States, February-April 2021.
      and partial vaccination was 64% effective against hospitalization.
      • Tenforde MW
      • Olson SM
      • Self WH
      • et al.
      Effectiveness of Pfizer-BioNTech and moderna vaccines against COVID-19 among hospitalized adults aged ≥65 Years – United States, January-March 2021.
      The Food and Drug Administration approved vaccination boosters,

      Do I qualify for a COVID-19 vaccine booster and which one? United States Food and Drug Administration.https://www.fda.gov/emergency-preparedness-and-response/coronavirus-disease-2019-covid-19/do-i-qualify-covid-19-vaccine-booster-and-which-one. Updated May 18, 2022. Accessed August 10, 2022.

      but the protection that the booster doses provide over time and among different vaccines remains unclear. It is critical to assess whether VE and durability of protection vary by patient demographics, comorbidities, history of previous severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection, SARS-CoV-2 exposure risk, and other variables that may impact vaccine performance.
      • Gu T
      • Mack JA
      • Salvatore M
      • et al.
      Characteristics associated with racial/ethnic disparities in COVID-19 outcomes in an academic health care system.
      Retrospective analyses using large health systems’ electronic health record (EHR), and other similar databases, provide rich data for this type of research.
      We used EHR data from the University of Michigan Health System (i.e., Michigan Medicine [MM]), a large, nationally ranked healthcare center in Michigan, to conduct a time-stratified retrospective cohort study to evaluate VE for the 3 major COVID-19 vaccines that were available in the U.S. Michigan is a state with both large urban population centers and rural communities and with extensive socioeconomic disparities across counties―particularly between Detroit, the most populous Black-majority city in the U.S., and its suburbs. In 32 (39%) of 83 counties in Michigan, vaccination primary series completion rates were below 50%.

      State of Michigan. COVID-19 vaccine dashboard. 2022. https://www.michigan.gov/coronavirus/resources/covid-19-vaccine/covid-19-dashboard. Accessed February 10, 2022.

      ,

      QuickFacts Michigan. United States Census Bureau. https://www.census.gov/quickfacts/MI. Updated July 1, 2021. Accessed January 3, 2022.

      The primary study goals were to determine the following:
      • 1.
        VE against infection and severe disease across 2021;
      • 2.
        VE stratified by the 2 most common vaccines―Pfizer-BioNTech and Moderna―and by sociodemographic and clinical characteristics that are associated with COVID-19 outcomes; and
      • 3.
        VE of booster doses on the basis of the data from the last quarter of 2021.

      METHODS

      Study Sample

      In a test-negative design cohort, eligible individuals included adults (aged ≥18 years) who received primary or other health care at MM and had a reverse transcription-polymerase chain reaction (RT-PCR) test for SARS-CoV-2 infection performed or recorded at MM between January 1 and December 31, 2021. All individuals of this cohort were MM patients; however, some were also healthcare staff.

      Measures

      RT-PCR Test Data

      The RT-PCR test data were collected as part of patient screening on hospital check-in or before hospital admission and healthcare staff screening, regardless of symptom status. Only RT-PCR test results were included in the data set (details are in Appendix Text 1, available online). Antigen test results were excluded because of the large range of test performance characteristics among test types, which limited comparison with RT-PCR tests. This cohort includes individuals who tested positive (i.e., at least 1 RT-PCR‒positive test) and individuals who tested negative (i.e., only negative tests). The test results were considered per quarter of the year 2021, that is, if an individual was tested multiple times within a quarter, we used the first positive test date as the index date for individuals who tested positive at least once in that quarter, whereas we used the last negative test as the index date if an individual never tested positive in that quarter. Because some individuals were tested across multiple quarters of 2021, we carried forward only the latest quarter in which an individual was tested when we report on the overall cohort to ensure maximum follow-up time in the overall analysis.

      Vaccination Status

      We obtained records of vaccinations for individuals who received a vaccination at MM or who had a recorded vaccination record in the Michigan Care Improvement Registry (MCIR). Michigan's immunization providers are encouraged to report the administration of COVID-19 vaccines to MCIR within 72 hours, and records can be sent from a hospital EHR to MCIR through HL7 messages.

      How to be a COVID-19 immunizing provider. Michigan Care Improvement Registry.https://mcir.org/2021/02/16/how-to-be-a-covid-19-immunizing-provider/. Updated February 16, 2021. Accessed May 9, 2022.

      Among all individuals, 78,002 had at the date of the corresponding test no documented vaccination and thus were considered unvaccinated. A total of 92,485 individuals had at least 1 documented dose of a COVID-19 vaccine. According to the Centers for Disease Control and Prevention (CDC)’s vaccination guideline,

      Interim clinical considerations for use of COVID-19 vaccines currently approved or authorized in the United States. Centers for Disease Control and Prevention.https://www.cdc.gov/vaccines/covid-19/clinical-considerations/interim-considerations-us.html. Updated April 21, 2022. Accessed April 29, 2022.

      we categorized 74,060 individuals as fully vaccinated in the primary series, meaning documentation of 2 doses of Moderna or Pfizer-BioNTech vaccine or a single dose of Janssen vaccine at least 21 days before the corresponding test date.
      • Baden LR
      • El Sahly HM
      • Essink B
      • et al.
      Efficacy and safety of the mRNA-1273 SARS-CoV-2 vaccine.
      • Polack FP
      • Thomas SJ
      • Kitchin N
      • et al.
      Safety and efficacy of the BNT162b2 mRNA Covid-19 vaccine.
      • Sadoff J
      • Gray G
      • Vandebosch A
      • et al.
      Safety and efficacy of single-dose Ad26.COV2.S vaccine against COVID-19.
      The remaining 18,425 vaccinated individuals were excluded because they were partially vaccinated (i.e., had not completed the vaccination primary series [n = 18,321] or received a mixed sequence of vaccines outside of CDC guidelines

      Interim clinical considerations for use of COVID-19 vaccines currently approved or authorized in the United States. Centers for Disease Control and Prevention.https://www.cdc.gov/vaccines/covid-19/clinical-considerations/interim-considerations-us.html. Updated April 21, 2022. Accessed April 29, 2022.

      [n = 104]) or received unspecified or other vaccine brands (i.e., Astra Zeneca, Sinopharm, Sinovac, or Novavac) (n = 254) (Appendix Table 1 and Appendix Figure 1B, available online). Individuals were classified as being boosted if they received any additional vaccination at least 21 days after a completed vaccination series. We conducted the booster-related VE analysis only in Quarter (Q) 4 of 2021, covering the period before the CDC guideline for boosters was announced.

      Interim clinical considerations for use of COVID-19 vaccines currently approved or authorized in the United States. Centers for Disease Control and Prevention.https://www.cdc.gov/vaccines/covid-19/clinical-considerations/interim-considerations-us.html. Updated April 21, 2022. Accessed April 29, 2022.

      ,

      COVID-19 vaccines for moderately or severely immunocompromised people. Who is moderately or severely immunocompromised? Centers for Disease Control and Prevention.https://www.cdc.gov/coronavirus/2019-ncov/vaccines/recommendations/immuno.html. Updated February 17, 2022. Accessed March 19, 2022.

      COVID-19 Outcomes

      n this study, we focused on 2 COVID-19‒associated outcomes: (1) SARS-CoV-2 infection and (2) severe disease or death. SARS-CoV-2 infection was defined as any documented positive result on an RT-PCR test (symptomatic or asymptomatic, nonsevere or severe disease), whereas severe disease or death was defined as either hospitalization or intensive care unit (ICU) admission between −7 and +30 days relative to the date of a documented positive SARS-CoV-2 RT-PCR test result or death between −7 and +60 days relative to the date of a positive RT-PCR test (Appendix Figure 1A, available online). Data on hospitalizations, ICU admissions, and deaths were obtained from MM's EHR databases as well as from the Michigan Death Registry.

      Demographics, Socioeconomic Status, and Other Covariates

      We have access to all patients’ structured deidentified EHR data that had been added by an MM healthcare provider. We performed additional analyses to examine whether VE varied by patient demographics (i.e., age, sex, race/ethnicity), weight status (i.e., BMI), clinical characteristics (e.g., comorbidities, history of SARS-CoV-2 infection, or history of cancer), occupational exposure (i.e., documented healthcare worker [HCW] or not), and SES (i.e., neighborhood disadvantage index [NDI]

      Clarke P, Melendez R. National neighborhood data archive (NaNDA): neighborhood socioeconomic and demographic characteristics by tract, United States, 2000–2010. Ann Arbor, MI: Inter-university Consortium for Political and Social Research. https://www.openicpsr.org/openicpsr/project/111107/. Published August 28, 2019. Accessed March 19, 2022.

      ,

      Melendez R, Clarke P, Khan A, Gomez-Lopez I, Li M, Chenoweth M. National Neighborhood Data Archive (NaNDA): Socioeconomic Status and Demographic Characteristics of ZIP Code Tabulation Areas, United States, 2008–2017. Ann Arbor, MI: Inter-university Consortium for Political and Social Research.https://www.openicpsr.org/openicpsr/project/119451/view. Published December 14, 2020. Accessed March 19, 2022.

      and population density). Age, gender, and race/ethnicity were directly extracted from the EHR, whereas history of SARS-CoV-2 infection, history of cancer, immunosuppressed status,

      COVID-19 vaccines for moderately or severely immunocompromised people. Who is moderately or severely immunocompromised? Centers for Disease Control and Prevention.https://www.cdc.gov/coronavirus/2019-ncov/vaccines/recommendations/immuno.html. Updated February 17, 2022. Accessed March 19, 2022.

      ,
      • Wu P
      • Gifford A
      • Meng X
      • et al.
      Mapping ICD-10 and ICD-10-CM codes to phecodes: workflow development and initial evaluation.
      BMI, the Elixhauser comorbidity score,
      • Gasparini A.
      comorbidity: an R package for computing comorbidity scores.
      ,
      • Elixhauser A
      • Steiner C
      • Harris DR
      • Coffey RM.
      Comorbidity measures for use with administrative data.
      HCW status, NDI, and population density were derived on available variables. Definitions for derived variables are available in Appendix Text 2 (available online). We assumed that covariates included in our adjusted analyses were missing completely at random and performed complete case analyses for each adjustment. The sample sizes of the complete case analyses for various sets of covariates are listed in Appendix Table 4 (available online).
      Ethical review and approval were waived for this study because of its qualification for a federal exemption as secondary research for which consent is not required. Determination for exemption was made by the University of Michigan Medical School IRB.

      Statistical Analysis

      We fit Firth's bias‒corrected logistic regression for each COVID-19‒related outcome YCOVID, considering several sets of covariates:
      logitP(YCOVID=1|I(Vaccine),Covariates)=β0+βVacI(Vaccine)+βCovCovariates,


      where logit(x)=log(x1x), I(Vaccine) is the indicator of vaccination status, either without or with a booster, that is,
      I(Vaccine)={01unvaccinatedvaccinatedfor the models comparing fully vaccinated individuals, and I(Vaccine)={01unvaccinatedvaccinated and received booster


      for the models comparing boosted individuals, and covariates denotes the vector of covariates adjusted. A total of 5 nested sets of covariates were evaluated, denoted as Adjustments 1–5 (defined in Appendix Table 4, available online). We reported the final model adjusted for age, gender, race/ethnicity, BMI, Elixhauser score, NDI, population density (persons per square mile), past COVID-19 infection, history of cancer, and HCW status (Adjustment 5).
      We estimated the VE using the following formula
      • Venmani A.
      Comparison of regression models on estimation of vaccine efficacy in anti-leprosy vaccination trial-a large prospective vaccination trial.
      :
      VE^=1incidencerateratio^=P^VaccinatedP^Unvaccinated,


      where incidencerateratio, the ratio of incidence between the vaccinated group and the unvaccinated group, can be estimated using the Michigan 12-month COVID-19 incidence rate and the estimated OR from the logistic regression mentioned earlier through the following formula
      • Zhang J
      • Yu KF.
      What's the relative risk? A method of correcting the odds ratio in cohort studies of common outcomes.
      :
      Incidencerateratio=OR(1IR)+(IR*OR).


      We estimated an incidence rate of 0.088 on the basis of the cumulative case count through December 29, 2021 (1,710,325 cumulative cases; JHU COVID-19 Dashboard

      United States Michigan: overview. Johns Hopkins University of Medicine. https://coronavirus.jhu.edu/region/us/michigan. Updated December 20, 2021. Accessed January 3, 2022.

      and the resident population of Michigan: census 2020: 10,077,331).

      QuickFacts Michigan. United States Census Bureau. https://www.census.gov/quickfacts/MI. Updated July 1, 2021. Accessed January 3, 2022.

      We conducted a quarter-stratified analysis to examine the time-varying changes in VE over 2021: Q1, January 1–March 31; Q2, April 1–June 30; Q3, July 1–September 30; and Q4, October 1–December 31. For the estimation of the overall VE in 2021, we used the latest quarter in which a patient was tested (see above).
      Finally, we estimated VE separately for individuals who received Pfizer-BioNTech only or Moderna only and provided estimates for the overall cohort as well as by quarter.
      To assess the potentially different VE across different strata of risk factors (denoted as X) associated with COVID-19 outcomes, we further conducted interaction analysis by vaccine status using the following model:
      logitP(YCOVID=1|I(vaccine),X,covariates)=β0+βVacI(vaccine)+βXX+βIntX×I(vaccine)+βCovcovariates


      All analyses were performed in R statistical software, Version 4.1.2.
      R Core Team
      R: a language and environment for statistical computing.
      For each model, we reported VE along with the corresponding 95% Wald CIs and p-values. For the interaction analysis, we obtained the vaccination OR for each stratum of X on the basis of βVac+βInt and reported the strata-specific subgroup VE along with the p-values, Pint., by testing the difference of subgroup effects through the null hypothesis H0:βint.=0. We performed a multiple testing correction on the basis of a Bonferroni-corrected threshold of p<0.00294 (0.05/17) to maintain the overall significance level at 0.05 across the total of 17 performed interaction tests.

      RESULTS

      Participant Characteristics

      In our cohort of 170,487 adults who were tested for SARS-CoV-2 infection between January 1 and December 31, 2021, 74,060 (43.4%) were fully vaccinated, and 78,002 (45.8%) were unvaccinated. In addition, 7,187 (9.7%) individuals from the vaccinated group received at least 1 additional booster vaccine dose after completing the primary series. In the overall cohort, the average time between being fully vaccinated and a recorded test was 136 days. More than 66.5% of individuals were tested within 3 months of being fully vaccinated.
      The characteristics of individuals by vaccination status are summarized in Table 1. Vaccinated individuals were slightly older (37.6% of the vaccinated group were aged ≥65 years compared with only 20% of the unvaccinated), were more likely to be established MM primary care patients (i.e., received a primary care visit at MM within the last 24 months; 45% vs 31.9%), and generally resided in less socioeconomically disadvantaged areas.
      Table 1Characteristics of the Michigan Medicine Cohort of 170,487 Individuals Who Were Tested for COVID-19 Between January and December 2021
      VariablesVaccination status
      Unvaccinated or unknownFully vaccinated
      All
      Includes individuals who received booster.
      ≥1 booster
      n78,00274,0607,187
      Patient demographics
      Age, median (IQR)46 (31.8)58.2 (32.8)64.5 (26)
      Age, years, n (%)
       18–49.943,430 (55.7)28,130 (38)1,981 (27.6)
       50–64.918,986 (24.3)18,094 (24.4)1,670 (23.2)
       ≥6515,586 (20)27,836 (37.6)3,536 (49.2)
      Male gender, n (%)33,023 (42.3)30,924 (41.8)3,122 (43.4)
      Race ethnicity, n (%)
       Non-Hispanic White55,111 (70.7)56,883 (76.8)5,830 (81.1)
       Non-Hispanic Black9,187 (11.8)5,287 (7.1)349 (4.9)
       Other or unknown13,704 (17.6)11,890 (16.1)1,008 (14)
      Primary care at MM, n (%)24,847 (31.9)33,192 (44.8)3,697 (51.4)
      Clinical variables
      BMI category, n (%)
       <18.520,007 (29.7)20,063 (29.7)2044 (30)
       18.5–24.91,497 (2.2)1,136 (1.7)94 (1.4)
       25–29.920,027 (29.8)21,348 (31.7)2,210 (32.4)
       ≥3025,777 (38.3)24,894 (36.9)2,470 (36.2)
       Elixhauser score AHRQ, mean (SD)
      Lower score indicates fewer/milder comorbidities.
      3.6 (10.7)5.9 (12.5)8.9 (14.2)
      Elixhauser score AHRQ category, n (%)
      Lower score indicates fewer/milder comorbidities.
       <019,887 (30)18,374 (27.4)1,543 (22.8)
       018,177 (27.4)15,036 (22.4)1,135 (16.8)
       1–47,324 (11.1)7,036 (10.5)706 (10.5)
       ≥520,837 (31.5)26,624 (39.7)3,370 (49.9)
      Past COVID-19 infection, n (%)1,728 (2.2)1,401 (1.9)79 (1.1)
      Healthcare worker, n (%)571 (0.7)2,786 (3.8)465 (6.5)
      Past immunosuppression, n (%)5,217 (6.7)7,249 (9.8)1,267 (17.6)
      Past cancer diagnosis, n (%)12,326 (15.8)18,552 (25)2,584 (36)
      Socioeconomic variables
      NDI, n (%)
      Quartile #1 indicates the least socioeconomically deprived.
       Quartile 122,022 (34.9)27,826 (45.4)3,116 (50.9)
       Quartile 214,860 (23.6)14,545 (23.7)1,359 (22.2)
       Quartile 313,808 (21.9)11,920 (19.4)1,128 (18.4)
       Quartile 412,390 (19.6)7,038 (11.5)515 (8.4)
      Persons per square mile, n (%)
      Quartile #1 indicates the lowest population density. AHRQ, Agency for Healthcare Research and Quality; MM, Michigan Medicine; NDI, neighborhood disadvantage index.
       Quartile 118,615 (29.5)15,058 (24.6)1,340 (21.9)
       Quartile 218,826 (29.8)20,076 (32.7)2139 (35)
       Quartile 319,742 (31.3)20,778 (33.9)2,107 (34.4)
       Quartile 45,897 (9.3)5,417 (8.8)532 (8.7)
      Note: For individuals who were tested across multiple quarters, only the vaccination status at their last observed quarter is shown.
      a Includes individuals who received booster.
      b Lower score indicates fewer/milder comorbidities.
      c Quartile #1 indicates the least socioeconomically deprived.
      d Quartile #1 indicates the lowest population density.AHRQ, Agency for Healthcare Research and Quality; MM, Michigan Medicine; NDI, neighborhood disadvantage index.
      The vaccinated subgroup had a slightly higher proportion of individuals with multiple comorbidities (Elixhauser Index ≥5). Furthermore, nearly twice as many vaccinated participants had a recent history of immunosuppression and cancer diagnosis as unvaccinated participants (9.8% vs 6.7% and 25% vs 15.8%, respectively) (Table 1 and Appendix Table 1, available online). We report the variables’ missingness by vaccination status in Appendix Table 2 (available online).
      Over 62% of the study population lived in the 4 Michigan counties surrounding MM (Washtenaw, Wayne, Livingston, and Oakland), whereas 5.7% lived outside of Michigan. Although Detroit resides in Wayne County, the population of Washtenaw County, where MM is located and most individuals lived, had over 72.6% vaccinated (Figure 1 and Appendix Table 3, available online).
      Figure 1
      Figure 1Geographic representation of the origin of individuals (left) and the rate of the fully vaccinated population (right; aged ≥12 years) across all MI counties at the end of 2021. The location of Michigan Medicine in Washtenaw County is indicated by a white dot.
      MI, Michigan.

      Rates of Infection and Severe COVID-19 Outcomes Among Unvaccinated and Vaccinated Individuals

      Unvaccinated individuals were significantly more likely to test positive for SARS-CoV-2 infection (12.1%) than vaccinated individuals (4.7%). The positivity rate among vaccinated individuals reached a valley with low community transmission in Q2 (Q1: 1.3% and Q2: 1.1%) and surged to 3.2% in Q3 and to 6.7% in Q4 (Table 2). In contrast, the positivity rate among unvaccinated individuals was lowest in Q1 (7.6%), increased to 11.8% in Q2, and decreased to 7.9% in Q3, before reaching a peak of 19.3% in Q4.
      Table 2COVID-19 Outcome Summary of Included Individuals Who Were Tested for COVID-19 Between January and December 2021
      VariablesOverallQ1/2021Q2/2021Q3/2021Q4/2021
      Unvaccinated or unknown
      N78,00256,95323,64013,50711,101
      Outcome, n (%)
       Positive9,464 (12.1)4,342 (7.6)2,781 (11.8)1,073 (7.9)2,143 (19.3)
       Nonsevere
      Includes asymptomatic and symptomatic patients who were not hospitalized −7 to +30 days relative to their test date. ICU, intensive care unit; Q, quarter.
      8,359 (10.7)4,016 (7.1)2,476 (10.5)932 (6.9)1,802 (16.2)
       Severe1,105 (1.4)326 (0.6)305 (1.3)141 (1)341 (3.1)
        Hospitalized798 (1)228 (0.4)222 (0.9)97 (0.7)255 (2.3)
        ICU admission166 (0.2)50 (0.1)36 (0.2)31 (0.2)50 (0.5)
        Death141 (0.2)48 (0.1)47 (0.2)13 (0.1)36 (0.3)
      Fully vaccinated
      n74,0604,07525,66130,23730,424
      Outcome, n (%)
       Positive3,495 (4.7)51 (1.3)278 (1.1)1,056 (3.5)2,204 (7.2)
        Nonsevere
      Includes asymptomatic and symptomatic patients who were not hospitalized −7 to +30 days relative to their test date. ICU, intensive care unit; Q, quarter.
      3,207 (4.3)49 (1.2)247 (1)979 (3.2)2,026 (6.7)
        Severe288 (0.4)2 (<0.1)31 (0.1)77 (0.3)178 (0.6)
         Hospitalized212 (0.3)2 (<0.01)23 (0.1)50 (0.2)137 (0.5)
         ICU admission36 (<0.1)0 (0)5 (<0.01)14 (<0.1)17 (0.1)
         Death40 (0.1)0 (0)3 (<0.01)13 (<0.1)24 (0.1)
      Fully vaccinated + booster
      N7,187151115116,755
      Outcome n (%)
       Positive245 (3.4)1 (6.7)1 (0.9)11 (2.2)232 (3.4)
        Nonsevere
      Includes asymptomatic and symptomatic patients who were not hospitalized −7 to +30 days relative to their test date. ICU, intensive care unit; Q, quarter.
      227 (3.2)1 (6.7)1 (0.9)10 (2)215 (3.2)
        Severe18 (0.3)0 (0)0 (0)1 (0.2)17 (0.3)
         Hospitalized15 (0.2)0 (0)0 (0)1 (0.2)14 (0.2)
         ICU admission1 (<0.1)0 (0)0 (0)0 (0)1 (<0.1)
         Death2 (<0.1)0 (0)0 (0)0 (0)2 (<0.1)
      Note: Overall numbers are based on the latest quarter in which a patient was tested. For individuals who were tested across multiple quarters, the vaccination status at their last observed quarter is summarized in the overall cohort.
      a Includes asymptomatic and symptomatic patients who were not hospitalized −7 to +30 days relative to their test date.ICU, intensive care unit; Q, quarter.
      The overall rate of the severe disease, a composite of COVID-19‒related hospitalization, ICU admission, or death, was higher in unvaccinated (1.4%) than in vaccinated individuals (0.4%). Across the 4 quarters of 2021, the rate of severe disease among vaccinated individuals remained relatively low (Q1: <0.01%, Q2: 0.1%, Q3: 0.3%, and Q4: 0.6%) compared with that among the unvaccinated group (Q1: 0.6%, Q2: 1.3%, Q3: 1.0%, Q4: 3.1%). Only 18 severe cases (0.3%) were observed among the 7,187 vaccinated individuals who received a booster vaccination (Table 2).

      Vaccine Effectiveness

      In Figure 2, we present VE estimates for the overall cohort (using data only from the latest calendar quarter in which each participant was tested) and time-stratified cohort (Q1, Q2, Q3, and Q4). Because the VE estimates were reasonably consistent across adjustments, for example, in the overall cohort, VE against SARS-CoV-2 infection (VE against infection) ranged between 59.4 and 62.1% (Appendix Table 6, available online), we only discuss the estimates from the most conservative setting (Adjustment 5, which included all the covariates mentioned earlier) in this study. Results from all models are listed in the Appendix Material (Appendix Tables 6 and 7, available online). VE against infection in Q1 of 2021 was 82.0% (76.0%, 86.5%), increased in Q2 to 90.9% (89.5%, 92.1%), and was substantially reduced in Q3 (60.1% [55.9%, 64.0%]) and Q4 (68.8% [66.3%, 71.1%]) (Figures 2A and 3
      • Hadfield J
      • Megill C
      • Bell SM
      • et al.
      Nextstrain: real-time tracking of pathogen evolution.
      ). This pattern was also observed for VE against the composite outcome of severe COVID-19 (VE against severe outcome) estimates, although they were generally higher and did not decrease as dramatically (Q1: 87.4% [58.1%, 96.3%]; Q2: 92.2% [88.3%, 94.8%]; Q3: 74.4% [64.8%, 81.5%], Q4: 83.0% [78.8%, 86.4%]) (Figures 2B and 3
      • Hadfield J
      • Megill C
      • Bell SM
      • et al.
      Nextstrain: real-time tracking of pathogen evolution.
      ,

      Genomic epidemiology of SARS-CoV-2 with global subsampling focus globally over the past 6 months. Nextstrain. https://nextstrain.org/ncov/open/global. Updated April 29, 2022. Accessed April 29, 2022.

      and Appendix Table 6, available online).
      Figure 2
      Figure 2VE and the rates of (A) RT-PCR‒confirmed SARS-CoV-2 Infection and (B) the composite severe COVID-19 outcome (hospitalization + ICU admission + death) among vaccinated and unvaccinated as well as (C) a comparison of these rates between vaccinated without or with booster in the last quarter of 2021.
      Note: Overall vaccine effectiveness is based on the latest quarter in which a patient was tested (Q1: n=39,782; Q2: n=36,058, Q3: n=34,697, Q4: n=41,525; shown in the Methods section). + denotes logistic regression. Adjustment 1: age, gender, race/ethnicity. Adjustment 5: Adjustment 1 + Elixhauser score AHRQ + Persons per square mile + neighborhood disadvantage index + past COVID-19 infection + healthcare worker status.
      AHRQ, Agency for Healthcare Research and Quality; ICU, intensive care unit; Q, quarter; RT-PCR, reverse transcription-polymerase chain reaction; VE, vaccine effectiveness.
      Figure 3
      Figure 3VEs in 2021 against infection and severe outcomes and SARS-CoV-2 variant frequencies in the U.S. across the 4 quarters of 2021. The variant frequencies are based on 420 SARS-CoV-2 genomes sampled in the U.S. in 2021. VE estimates and their 95% CIs are shown (VE against SARS-CoV-2 infection: solid line, purple; VE against the composite outcome of severe COVID-19 disease: dashed line, pink). The variants are colored by their WHO variant names, whereas the Nextstrain clade names are shown in the figure.
      • Hadfield J
      • Megill C
      • Bell SM
      • et al.
      Nextstrain: real-time tracking of pathogen evolution.
      ,

      Genomic epidemiology of SARS-CoV-2 with global subsampling focus globally over the past 6 months. Nextstrain. https://nextstrain.org/ncov/open/global. Updated April 29, 2022. Accessed April 29, 2022.

      VE, vaccine effectiveness.
      A stratified analysis by booster status in Q4 showed additional protection of a booster against infection (VE against infection: no booster: 64.0% [61.1%, 66.7%] vs booster: 87.3% [85.0%, 89.2%]) and severe outcomes (VE against severe outcome: no booster: 78.8% [73.5%, 83.0%] vs booster: 94.0% [89.5%, 96.6%]) (Figures 2C and Appendix Figure 2 and Appendix Table 7, available online).
      We also performed a sensitivity analysis using only data on individuals who received primary care at MM and thus were more likely to have documented vaccination data. In this analysis, estimates were substantially higher for VE against infection (Q1: 88.7% [81.5%, 93.1%]; Q2: 93.8% [92.3%, 95.0%]; Q3: 68.7% [63.7%, 73.0%]; Q4: 72.3 [68.9%, 75.4%]) (Appendix Figure 2A, available online) and VE against severe outcome (Q1: not available or no severe case among vaccinated; Q2: 95.2% [91.1%, 97.4%]; Q3: 79.4% [68.0%, 86.7%]; Q4: 80.8% [73.8%, 85.9%]) (Appendix Figure 2B, available online).

      Comparing Vaccine Effectiveness Between Pfizer-BioNTech and Moderna

      Because most of the 74,060 fully vaccinated individuals completed a vaccine series with either Pfizer-BioNTech (n=45,168; 61.0%) or Moderna (n=25,267; 34.1%), we compared the effectiveness of the 2 vaccines. To address that the Pfizer-BioNTech vaccine was available before Moderna, we stratified the analysis on the basis of whether the vaccination occurred <3 months or ≥3 months before the COVID-19 test (Figure 4).
      Figure 4
      Figure 4Comparison of effectiveness between Pfizer-BioNTech and Moderna vaccines. (A) VE against SARS-CoV-2 infection and (B) VE against the composite outcome of severe COVID-19 disease are shown for individuals being fully vaccinated at any time (top), within 3 months (center), or over 3 months (bottom) before being tested for COVID-19.
      VE, vaccine effectiveness.
      In the overall cohort, we observed lower VE against infection for Pfizer-BioNTech than for Moderna (82.9% [80.7%, 84.9%] vs 88.1% [85.5%, 90.2%]), a trend that was consistent across all the 4 quarters of 2021, although the difference was lowest in Q2 (89.6% [87.6%, 91.3%] vs 91.7% [89.3%, 93.5%]) (Figures 4 and Appendix Figure 4, available online). Contrarily, VE against severe outcome was higher for Pfizer-BioNTech (87.1% [80.3%, 91.6%]) than for Moderna (84.9% [76.2%, 90.5%]) in the overall cohort, consistent in the first half of 2021 but reversed in the last half of 2021. Of note, the VE against severe outcome in Q4 was higher for Moderna (93.4% [5.3%, 99.6%]) than for Pfizer-BioNTech (68.3% [23.6%, 87.2%]), with overlapping CIs (Appendix Figure 5, available online).

      Factors Affecting Vaccine Effectiveness

      Overall, we found that VE against infection ranged between 57.1 and 65.2% across all strata but was lower for people with a documented past COVID-19 infection (47.6% [24.5%, 63.9%]; p-interaction [pInt.]=0.065) (Figure 5). Only 182 individuals had a past infection, and only 48 of these were vaccinated (Appendix Table 8, available online). In Figure 5, we found that compared to the trend among those aged <50 years, VE shows a decreasing trend among those aged >65 years against both infection and severe outcomes, but the differences were not statistically significant.
      Figure 5
      Figure 5VE against SARS-CoV-2 infection (infection) and VE against the composite outcome of severe COVID-19 disease (severe outcome) across various covariate strata. Analyses were adjusted for the following set of covariates (after excluding the variable of interest): age, gender, race/ethnicity, Elixhauser score, persons per square mile, neighborhood disadvantage index, past COVID-19 infection, and healthcare worker status.
      VE, vaccine effectiveness.
      For HCWs, we observed a higher VE against infection than for others (HCW: 70.4% [62.8%, 76.5%] vs non-HCW: 61.7% [59.9%, 63.9%]; pInt.=0.031).
      In addition, we observed decreased VE against infection in individuals living in neighborhoods with increasing NDI quartiles compared with that in those in the lowest NDI quartile (pInt.≤0.023).
      For VE against severe outcome, we observed effective protection against severe COVID-19 across all strata (VE against severe outcome≥66.6%), noting the estimated reduced effectiveness for individuals with a past COVID-19 infection (VE against severe outcome Past COVID-19 Infection=45.3% [0%, 77.1%]; pInt.=0.133). Owing to the lower counts of individuals with severe COVID-19, CIs were wider (Figure 5).

      DISCUSSION

      In a study of 170,487 adults tested for COVID-19 within Michigan, we found strong protection provided by Pfizer-BioNTech and Moderna vaccines against infection and severe disease. Booster doses increased effectiveness against both infection and severe disease. In our study, fully vaccinated individuals were less likely to test SARS-CoV-2 positive and were less likely to have the severe disease than unvaccinated individuals. We observed declines in estimated VE over time, indicating potential waning effectiveness, consequences of spreading of the more immune-evasive SARS-CoV-2 B.1.617.2 (Delta) and the B.1.1.159 (Omicron) variants.
      • Lopez Bernal J
      • Andrews N
      • Gower C
      • et al.
      Effectiveness of Covid-19 vaccines against the B.1.617.2 (delta) variant.
      • Chemaitelly H
      • Tang P
      • Hasan MR
      • et al.
      Waning of BNT162b2 vaccine protection against SARS-CoV-2 infection in Qatar.
      • Torjesen I.
      Covid-19: Omicron may be more transmissible than other variantsand partly resistant to existing vaccines, scientists fear.
      Because uptake in booster shots increased in Q4, both infection and severe disease rates were lowest among boosted individuals. This evidence supports a booster's protection against infections and severe disease in the face of potential waning of VE and emerging virus variants.
      Among individuals who received the vaccine within the past 3 months, VE against infection tended to be slightly higher for those who received the Moderna vaccine, and VE against severe outcomes was moderately high for both vaccines. However, the overlapping CIs suggest similar vaccine protection or the need for a larger sample size to tease out the vaccines’ performance.
      Our results are largely consistent with previously published findings, although differences among studies make direct comparison challenging. For example, some studies adjusted differently for potential confounders (age only, additional patient characteristics, or time since vaccination),
      • Dagan N
      • Barda N
      • Kepten E
      • et al.
      BNT162b2 mRNA Covid-19 vaccine in a nationwide mass vaccination setting.
      • Thompson MG
      • Stenehjem E
      • Grannis S
      • et al.
      Effectiveness of COVID-19 vaccines in ambulatory and inpatient care settings.
      • Bruxvoort KJ
      • Sy LS
      • Qian L
      • et al.
      Effectiveness of mRNA-1273 against delta, mu, and other emerging variants of SARS-CoV-2: test negative case-control study.
      ,
      • Rosenberg ES
      • Holtgrave DR
      • Dorabawila V
      • et al.
      New COVID-19 cases and hospitalizations among adults, by vaccination status - New York, May 3–July 25, 2021.
      ,
      • Suthar AB
      • Wang J
      • Seffren V
      • Wiegand RE
      • Griffing S
      • Zell E.
      Public health impact of covid-19 vaccines in the U.S.: observational study.
      whereas others characterized the variant associated with each case through whole-genome sequencing.
      • Tenforde MW
      • Self WH
      • Naioti EA
      • et al.
      Sustained effectiveness of Pfizer-BioNTech and Moderna vaccines against COVID-19 associated hospitalizations among adults - United States, March–July 2021.
      Our study spans 2021 and thus likely includes data on multiple variants (including the Omicron variant) and on boosters that have become more common. From other studies conducted from late 2020 to early 2021, VE estimates against hospitalization ranged around 87% (95% CI=55, 100)
      • Dagan N
      • Barda N
      • Kepten E
      • et al.
      BNT162b2 mRNA Covid-19 vaccine in a nationwide mass vaccination setting.
      and 89% (95% CI=87, 91)
      • Thompson MG
      • Stenehjem E
      • Grannis S
      • et al.
      Effectiveness of COVID-19 vaccines in ambulatory and inpatient care settings.
      against infection. Studies during mid-2021 reported VE in the ranges of 75.0%, 91.8%,
      • Rosenberg ES
      • Holtgrave DR
      • Dorabawila V
      • et al.
      New COVID-19 cases and hospitalizations among adults, by vaccination status - New York, May 3–July 25, 2021.
      79.9%, and 98.4%
      • Bruxvoort KJ
      • Sy LS
      • Qian L
      • et al.
      Effectiveness of mRNA-1273 against delta, mu, and other emerging variants of SARS-CoV-2: test negative case-control study.
      against infection with different variants, including Delta, and between 89.5% and 95.1%
      • Rosenberg ES
      • Holtgrave DR
      • Dorabawila V
      • et al.
      New COVID-19 cases and hospitalizations among adults, by vaccination status - New York, May 3–July 25, 2021.
      and 86% (95% CI=82, 88)
      • Tenforde MW
      • Self WH
      • Naioti EA
      • et al.
      Sustained effectiveness of Pfizer-BioNTech and Moderna vaccines against COVID-19 associated hospitalizations among adults - United States, March–July 2021.
      against hospitalization across different variants. A recent study assessing VE from late 2020 through Q3 of 2021 found VE rates for mRNA vaccines to be the highest 2 weeks after a 2-dose regimen (94.5%; 95% CI=94.1, 94.9), lining up with our estimates of VE in Q2 and lowest at 66.6% (95% CI=65.2, 67.8) after 7 months,
      • Lin DY
      • Gu Y
      • Wheeler B
      • et al.
      Effectiveness of COVID-19 vaccinesover a 9-month period in North Carolina.
      which corresponds with our VE estimates in Q3. Taken together, these results highlight the need for appropriate adjustment for patient covariates, timing of vaccination, booster status, and replication across different regions and time periods to more fully understand VE and potential modifying factors. Ongoing studies should also be estimating the effectiveness of the vaccine against newer variants and how the previous infection modifies the effectiveness of primary or booster doses of vaccination.
      Michigan offers an important location to study COVID-19 VE for several reasons. The southeast corner of the state, where this study is focused, has extensive socioeconomic disparities across its urban, suburban, and rural locations. Notably, Detroit, the most populous Black-majority city in the U.S., was severely affected by COVID-19 outbreaks early in the pandemic. Within the city, there were still substantial differences in COVID-19 vaccination coverage between Black (56%) and White (82%) populations in mid-2021.
      • Wagner AL
      • Wileden L
      • Shanks TR
      • Goold SD
      • Morenoff JD
      • Sheinfeld Gorin SN
      Mediators of racial differences in COVID-19 vaccine acceptance and uptake: a cohort study in Detroit, MI.
      Because there is substantial evidence that how effective the vaccine is could influence acceptance of the vaccine,
      • Wagner AL
      • Sheinfeld Gorin S
      • Boulton ML
      • Glover BA
      • Morenoff JD.
      Effect of vaccine effectiveness and safety on COVID-19 vaccine acceptance in Detroit, Michigan, July 2020.
      this study, reporting relatively high VE in this area, could be influential. However, we also note that VE, although not significantly different by race/ethnicity, might be lower among individuals living in neighborhoods with higher NDI. This suggests that more effort is needed to study protection from vaccines in more disadvantaged areas. Past studies of other respiratory infections have posited that differences in risk of infection across physical space could include housing overcrowding, limited access to health care, and higher rates of contact with others who are infected.
      • Noppert GA
      • Malosh RE
      • Moran EB
      • Ahuja SD
      • Zelner J.
      Contemporary social disparities in TB infection and disease in the USA: a review.
      Future studies could sample within the community and not just among those with healthcare access.
      • Dean NE
      • Hogan JW
      • Schnitzer ME.
      Covid-19 vaccine effectiveness and the test-negative design.
      Strengths of this study are the availability of detailed EHR data for study subjects, COVID-19 test and vaccination data from all the 4 quarters of 2021, and the large number of participants from a single academic medical center in Michigan. This allows us to adjust for potential confounding by patient characteristics, including socioeconomic and demographic characteristics, and relevant medical conditions, including previous COVID-19 testing results and receipt of a booster dose. Furthermore, we considered multiple outcomes of interest and assessed variation in VE by different demographic, clinical, and other factors. The availability of potential confounding patient characteristics allowed us to evaluate the potential modification of VE by patient subgroups. After correcting for multiple testing, none of the stratified analyses were significant. Differences before multiple testing corrections potentially indicate a stricter adherence to the vaccination series or additional protective measures about HCW.
      • Yassi A
      • Grant JM
      • Lockhart K
      • et al.
      Infection control, occupational and public health measures including mRNA-based vaccination against SARS-CoV-2 infections to protect healthcare workers from variants of concern: a 14-month observational study using surveillance data.
      ,
      • Simione L
      • Gnagnarella C.
      Differences between health workers and general population in risk perception, behaviors, and psychological distress related to COVID-19 spread in Italy.
      There are some discrepancies in the literature on the effectiveness, with some literature suggesting that unvaccinated individuals who acquired immunity through a previous COVID-19 infection may benefit less from vaccination than people who were not previously infected.
      • Kojima N
      • Klausner JD.
      Protective immunity after recovery from SARS-CoV-2 infection.
      ,

      Shrestha NK, Burke PC, Nowacki AS, Terpeluk P, Gordon SM. Necessity of COVID-19 vaccination in persons who have already had COVID-19. Clin Infect Dis. In press. Online January 13, 2022. https://doi.org/10.1093/cid/ciac022.

      However, there is a growing body of literature suggesting that COVID-19 vaccination is the best protection against infection and severe disease
      • Dye C.
      The benefits of large scale covid-19 vaccination.
      whether or not one has already had COVID-19.

      Shrestha NK, Burke PC, Nowacki AS, Terpeluk P, Gordon SM. Necessity of COVID-19 vaccination in persons who have already had COVID-19. Clin Infect Dis. In press. Online January 13, 2022. https://doi.org/10.1093/cid/ciac022.

      ,
      • Cavanaugh AM
      • Spicer KB
      • Thoroughman D
      • Glick C
      • Winter K.
      Reduced risk of reinfection with SARS-CoV-2 after COVID-19 vaccination - Kentucky, May–June 2021.
      • Hall V
      • Foulkes S
      • Insalata F
      • et al.
      Protection against SARS-CoV-2 after Covid-19 vaccination and previous infection.
      • Hammerman A
      • Sergienko R
      • Friger M
      • et al.
      Effectiveness of the BNT162b2 vaccine after recovery from Covid-19.
      • Patel MM
      • Jackson ML
      • Ferdinands J.
      Postlicensure evaluation of COVID-19 vaccines.

      Limitations

      A potential weakness is any missing or incomplete information from the EHR, for example, variant sequencing, undocumented test results, past infections, vaccinations, hospitalizations, or mortality outcomes. Although we cannot test the exact completeness of the EHR vaccination record, for example, it is possible, although unlikely, that they may have been vaccinated elsewhere and that these records were not available, our sensitivity analysis examining MM primary care patients suggests the possibility that VE estimates in the full cohort might be influenced by incomplete vaccination documentation in the EHR. In this scenario, categorizing individuals into an unknown or unvaccinated status tends to produce conservative VE estimates (Appendix Figure 3, available online). Another explanation for these findings regards potential differences in health-seeking behaviors and therefore differential VE between patient groups. Results from the test-negative design are generalizable to individuals tested for COVID-19 and more straightforward with tests having similar sensitivity and specificity properties.
      • Thompson MG
      • Stenehjem E
      • Grannis S
      • et al.
      Effectiveness of COVID-19 vaccines in ambulatory and inpatient care settings.
      ,
      • Dean NE
      • Hogan JW
      • Schnitzer ME.
      Covid-19 vaccine effectiveness and the test-negative design.
      ,
      • Patel MM
      • Jackson ML
      • Ferdinands J.
      Postlicensure evaluation of COVID-19 vaccines.
      Still, these studies provide insight into the long-term effectiveness and durability of vaccine protection.
      • Patel MM
      • Jackson ML
      • Ferdinands J.
      Postlicensure evaluation of COVID-19 vaccines.
      How to combine the RT-PCR test and the rapid antigen tests for VE studies in a statistically valid manner remains a question of interest in the future.

      CONCLUSIONS

      The study provides robust evidence for the ability of COVID-19 vaccines to protect against infection and severe disease. As new variants arise, continued observational studies will be necessary to assess the effectiveness of currently available vaccine regimens. Overall, these results provide evidence of the effectiveness of the vaccines over time and may encourage those who have not been vaccinated or not received a booster to consider doing so to prevent severe outcomes, including mortality.

      ACKNOWLEDGMENTS

      Any opinions, findings, conclusions, or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.
      This study was supported by the University of Michigan Precision Health Initiative, the University of Michigan Rogel Cancer Center , and the Michigan Institute of Data Science . BM's research was supported by Grant Number NSF DMS 1712933 from the National Science Foundation , and LGF's research was supported by Grant Number CA 046592 from the National Cancer Institute , NIH. Determination for exemption was made by the University of Michigan Medical School IRB (study identification HUM00180294).
      Declarations of interest: none

      CRediT AUTHOR STATEMENT

      Emily K. Roberts: Investigation, Methodology, Writing - original draft, Writing – review and editing. Tian Gu: Methodology, Writing - original draft, Writing – review and editing. Abram L. Wagner: Writing – review and editing. Bhramar Mukherjee: Conceptualization, Funding acquisition, Investigation, Methodology, Writing – review and editing. Lars G. Fritsche: Data Curation, Formal analysis, Investigation, Methodology, Visualization, Writing - original draft, Writing – review and editing.

      Appendix. Supplementary materials

      REFERENCES

        • Baden LR
        • El Sahly HM
        • Essink B
        • et al.
        Efficacy and safety of the mRNA-1273 SARS-CoV-2 vaccine.
        N Engl J Med. 2021; 384: 403-416https://doi.org/10.1056/NEJMoa2035389
        • Polack FP
        • Thomas SJ
        • Kitchin N
        • et al.
        Safety and efficacy of the BNT162b2 mRNA Covid-19 vaccine.
        N Engl J Med. 2020; 383: 2603-2615https://doi.org/10.1056/NEJMoa2034577
        • Sadoff J
        • Gray G
        • Vandebosch A
        • et al.
        Safety and efficacy of single-dose Ad26.COV2.S vaccine against COVID-19.
        N Engl J Med. 2021; 384: 2187-2201https://doi.org/10.1056/NEJMoa2101544
      1. COVID-19 vaccinations in the United States. Centers for Disease Control and Prevention. https://covid.cdc.gov/covid-data-tracker/#vaccinations_vacc-people-onedose-pop-12yr. Updated July 29, 2022. Accessed November 18, 2021.

        • Bar-On YM
        • Goldberg Y
        • Mandel M
        • et al.
        Protection of BNT162b2 vaccine booster against COVID-19 in Israel.
        N Engl J Med. 2021; 385: 1393-1400https://doi.org/10.1056/NEJMoa2114255
        • Arbel R
        • Hammerman A
        • Sergienko R
        • et al.
        BNT162b2 vaccine booster and mortality due to Covid-19.
        N Engl J Med. 2021; 385: 2413-2420https://doi.org/10.1056/NEJMoa2115624
      2. Our world in data. Coronavirus (COVID-19) vaccinations; 2022 2022/02/10; https://ourworldindata.org/covid-vaccinations.

        • Dagan N
        • Barda N
        • Kepten E
        • et al.
        BNT162b2 mRNA Covid-19 vaccine in a nationwide mass vaccination setting.
        N Engl J Med. 2021; 384: 1412-1423https://doi.org/10.1056/NEJMoa2101765
        • Thompson MG
        • Stenehjem E
        • Grannis S
        • et al.
        Effectiveness of COVID-19 vaccines in ambulatory and inpatient care settings.
        N Engl J Med. 2021; 385: 1355-1371https://doi.org/10.1056/NEJMoa2110362
        • Bruxvoort KJ
        • Sy LS
        • Qian L
        • et al.
        Effectiveness of mRNA-1273 against delta, mu, and other emerging variants of SARS-CoV-2: test negative case-control study.
        BMJ. 2021; 375e068848https://doi.org/10.1136/bmj-2021-068848
        • Tenforde MW
        • Self WH
        • Naioti EA
        • et al.
        Sustained effectiveness of Pfizer-BioNTech and Moderna vaccines against COVID-19 associated hospitalizations among adults - United States, March–July 2021.
        MMWR Morb Mortal Wkly Rep. 2021; 70: 1156-1162https://doi.org/10.15585/mmwr.mm7034e2
        • Bajema KL
        • Dahl RM
        • Prill MM
        • et al.
        Effectiveness of COVID-19 mRNA vaccines against COVID-19-associated hospitalization - five Veterans Affairs Medical Centers, United States, February 1–August 6, 2021.
        MMWR Morb Mortal Wkly Rep. 2021; 70: 1294-1299https://doi.org/10.15585/mmwr.mm7037e3
        • Rosenberg ES
        • Holtgrave DR
        • Dorabawila V
        • et al.
        New COVID-19 cases and hospitalizations among adults, by vaccination status - New York, May 3–July 25, 2021.
        MMWR Morb Mortal Wkly Rep. 2021; 70: 1150-1155https://doi.org/10.15585/mmwr.mm7034e1
        • Lin DY
        • Gu Y
        • Wheeler B
        • et al.
        Effectiveness of COVID-19 vaccinesover a 9-month period in North Carolina.
        N Engl J Med. 2022; 386: 933-941https://doi.org/10.1056/NEJMoa2117128
        • Andrews N
        • Tessier E
        • Stowe J
        • et al.
        Duration of protection against mildand severe disease by Covid-19 vaccines.
        N Engl J Med. 2022; 386: 340-350https://doi.org/10.1056/NEJMoa2115481
      3. Results of COVID-19 vaccine effectiveness studies: an ongoing systematic review. International Vaccine Access Center, Johns Hopkins Bloomberg School of Public Health, World Health Organization.https://view-hub.org/sites/default/files/2022-03/COVID19%20Vaccine%20Effectiveness%20Transmission%20Studies%20-%20Summary%20Tables_20220310.pdf. Updated March 10, 2022. Accessed March 19, 2022.

        • Moline HL
        • Whitaker M
        • Deng L
        • et al.
        Effectiveness of COVID-19 vaccines in preventing hospitalization among adults aged ≥65 Years – COVID-NET, 13 States, February-April 2021.
        MMWR Morb Mortal Wkly Rep. 2021; 70: 1088-1093https://doi.org/10.15585/mmwr.mm7032e3
        • Tenforde MW
        • Olson SM
        • Self WH
        • et al.
        Effectiveness of Pfizer-BioNTech and moderna vaccines against COVID-19 among hospitalized adults aged ≥65 Years – United States, January-March 2021.
        MMWR Morb Mortal Wkly Rep. 2021; 70: 674-679https://doi.org/10.15585/mmwr.mm7018e1
      4. Do I qualify for a COVID-19 vaccine booster and which one? United States Food and Drug Administration.https://www.fda.gov/emergency-preparedness-and-response/coronavirus-disease-2019-covid-19/do-i-qualify-covid-19-vaccine-booster-and-which-one. Updated May 18, 2022. Accessed August 10, 2022.

        • Gu T
        • Mack JA
        • Salvatore M
        • et al.
        Characteristics associated with racial/ethnic disparities in COVID-19 outcomes in an academic health care system.
        JAMA Netw Open. 2020; 3e2025197https://doi.org/10.1001/jamanetworkopen.2020.25197
      5. State of Michigan. COVID-19 vaccine dashboard. 2022. https://www.michigan.gov/coronavirus/resources/covid-19-vaccine/covid-19-dashboard. Accessed February 10, 2022.

      6. QuickFacts Michigan. United States Census Bureau. https://www.census.gov/quickfacts/MI. Updated July 1, 2021. Accessed January 3, 2022.

      7. How to be a COVID-19 immunizing provider. Michigan Care Improvement Registry.https://mcir.org/2021/02/16/how-to-be-a-covid-19-immunizing-provider/. Updated February 16, 2021. Accessed May 9, 2022.

      8. Interim clinical considerations for use of COVID-19 vaccines currently approved or authorized in the United States. Centers for Disease Control and Prevention.https://www.cdc.gov/vaccines/covid-19/clinical-considerations/interim-considerations-us.html. Updated April 21, 2022. Accessed April 29, 2022.

      9. COVID-19 vaccines for moderately or severely immunocompromised people. Who is moderately or severely immunocompromised? Centers for Disease Control and Prevention.https://www.cdc.gov/coronavirus/2019-ncov/vaccines/recommendations/immuno.html. Updated February 17, 2022. Accessed March 19, 2022.

      10. Clarke P, Melendez R. National neighborhood data archive (NaNDA): neighborhood socioeconomic and demographic characteristics by tract, United States, 2000–2010. Ann Arbor, MI: Inter-university Consortium for Political and Social Research. https://www.openicpsr.org/openicpsr/project/111107/. Published August 28, 2019. Accessed March 19, 2022.

      11. Melendez R, Clarke P, Khan A, Gomez-Lopez I, Li M, Chenoweth M. National Neighborhood Data Archive (NaNDA): Socioeconomic Status and Demographic Characteristics of ZIP Code Tabulation Areas, United States, 2008–2017. Ann Arbor, MI: Inter-university Consortium for Political and Social Research.https://www.openicpsr.org/openicpsr/project/119451/view. Published December 14, 2020. Accessed March 19, 2022.

        • Wu P
        • Gifford A
        • Meng X
        • et al.
        Mapping ICD-10 and ICD-10-CM codes to phecodes: workflow development and initial evaluation.
        JMIR Med Inform. 2019; 7: e14325https://doi.org/10.2196/14325
        • Gasparini A.
        comorbidity: an R package for computing comorbidity scores.
        J Open Source Softw. 2018; 3: 648https://doi.org/10.21105/joss.00648
        • Elixhauser A
        • Steiner C
        • Harris DR
        • Coffey RM.
        Comorbidity measures for use with administrative data.
        Med Care. 1998; 36: 8-27https://doi.org/10.1097/00005650-199801000-00004
        • Venmani A.
        Comparison of regression models on estimation of vaccine efficacy in anti-leprosy vaccination trial-a large prospective vaccination trial.
        AIP Conf Proc. 2019; 2112020148
        • Zhang J
        • Yu KF.
        What's the relative risk? A method of correcting the odds ratio in cohort studies of common outcomes.
        JAMA. 1998; 280: 1690-1691https://doi.org/10.1001/jama.280.19.1690
      12. United States Michigan: overview. Johns Hopkins University of Medicine. https://coronavirus.jhu.edu/region/us/michigan. Updated December 20, 2021. Accessed January 3, 2022.

        • R Core Team
        R: a language and environment for statistical computing.
        R Foundation for Statistical Computing, Vienna, Austria2022
        https://www.R-project.org/
        Date accessed: November 1, 2021
        • Hadfield J
        • Megill C
        • Bell SM
        • et al.
        Nextstrain: real-time tracking of pathogen evolution.
        Bioinformatics. 2018; 34: 4121-4123https://doi.org/10.1093/bioinformatics/bty407
      13. Genomic epidemiology of SARS-CoV-2 with global subsampling focus globally over the past 6 months. Nextstrain. https://nextstrain.org/ncov/open/global. Updated April 29, 2022. Accessed April 29, 2022.

        • Lopez Bernal J
        • Andrews N
        • Gower C
        • et al.
        Effectiveness of Covid-19 vaccines against the B.1.617.2 (delta) variant.
        N Engl J Med. 2021; 385: 585-594https://doi.org/10.1056/NEJMoa2108891
        • Chemaitelly H
        • Tang P
        • Hasan MR
        • et al.
        Waning of BNT162b2 vaccine protection against SARS-CoV-2 infection in Qatar.
        N Engl J Med. 2021; 385: e83https://doi.org/10.1056/NEJMoa2114114
        • Torjesen I.
        Covid-19: Omicron may be more transmissible than other variantsand partly resistant to existing vaccines, scientists fear.
        BMJ. 2021; 375: n2943https://doi.org/10.1136/bmj.n2943
        • Suthar AB
        • Wang J
        • Seffren V
        • Wiegand RE
        • Griffing S
        • Zell E.
        Public health impact of covid-19 vaccines in the U.S.: observational study.
        BMJ. 2022; 377e069317https://doi.org/10.1136/bmj-2021-069317
        • Wagner AL
        • Wileden L
        • Shanks TR
        • Goold SD
        • Morenoff JD
        • Sheinfeld Gorin SN
        Mediators of racial differences in COVID-19 vaccine acceptance and uptake: a cohort study in Detroit, MI.
        Vaccines (Basel). 2021; 10: 36https://doi.org/10.3390/vaccines10010036
        • Wagner AL
        • Sheinfeld Gorin S
        • Boulton ML
        • Glover BA
        • Morenoff JD.
        Effect of vaccine effectiveness and safety on COVID-19 vaccine acceptance in Detroit, Michigan, July 2020.
        Hum Vaccin Immunother. 2021; 17: 2940-2945https://doi.org/10.1080/21645515.2021.1917233
        • Noppert GA
        • Malosh RE
        • Moran EB
        • Ahuja SD
        • Zelner J.
        Contemporary social disparities in TB infection and disease in the USA: a review.
        Curr Epidemiol Rep. 2018; 5: 442-449https://doi.org/10.1007/s40471-018-0171-y
        • Dean NE
        • Hogan JW
        • Schnitzer ME.
        Covid-19 vaccine effectiveness and the test-negative design.
        N Engl J Med. 2021; 385: 1431-1433https://doi.org/10.1056/NEJMe2113151
        • Yassi A
        • Grant JM
        • Lockhart K
        • et al.
        Infection control, occupational and public health measures including mRNA-based vaccination against SARS-CoV-2 infections to protect healthcare workers from variants of concern: a 14-month observational study using surveillance data.
        PLoS One. 2021; 16e0254920https://doi.org/10.1371/journal.pone.0254920
        • Simione L
        • Gnagnarella C.
        Differences between health workers and general population in risk perception, behaviors, and psychological distress related to COVID-19 spread in Italy.
        Front Psychol. 2020; 11: 2166https://doi.org/10.3389/fpsyg.2020.02166
        • Kojima N
        • Klausner JD.
        Protective immunity after recovery from SARS-CoV-2 infection.
        Lancet Infect Dis. 2022; 22: 12-14https://doi.org/10.1016/S1473-3099(21)00676-9
      14. Shrestha NK, Burke PC, Nowacki AS, Terpeluk P, Gordon SM. Necessity of COVID-19 vaccination in persons who have already had COVID-19. Clin Infect Dis. In press. Online January 13, 2022. https://doi.org/10.1093/cid/ciac022.

        • Dye C.
        The benefits of large scale covid-19 vaccination.
        BMJ. 2022; 377: o867https://doi.org/10.1136/bmj.o867
        • Cavanaugh AM
        • Spicer KB
        • Thoroughman D
        • Glick C
        • Winter K.
        Reduced risk of reinfection with SARS-CoV-2 after COVID-19 vaccination - Kentucky, May–June 2021.
        MMWR Morb Mortal Wkly Rep. 2021; 70: 1081-1083https://doi.org/10.15585/mmwr.mm7032e1
        • Hall V
        • Foulkes S
        • Insalata F
        • et al.
        Protection against SARS-CoV-2 after Covid-19 vaccination and previous infection.
        N Engl J Med. 2022; 386: 1207-1220https://doi.org/10.1056/NEJMoa2118691
        • Hammerman A
        • Sergienko R
        • Friger M
        • et al.
        Effectiveness of the BNT162b2 vaccine after recovery from Covid-19.
        N Engl J Med. 2022; 386: 1221-1229https://doi.org/10.1056/NEJMoa2119497
        • Patel MM
        • Jackson ML
        • Ferdinands J.
        Postlicensure evaluation of COVID-19 vaccines.
        JAMA. 2020; 324: 1939-1940https://doi.org/10.1001/jama.2020.19328