Advertisement

Trends in Elective Deliveries in California and New Jersey

Open AccessPublished:November 23, 2022DOI:https://doi.org/10.1016/j.focus.2022.100052

      Highlights

      • Elective deliveries increased sharply until the mid-2000s in 2 U.S. states.
      • Rates of elective deliveries were higher for more educated and White women.
      • Trends in elective deliveries were similar across socioeconomic and racial groups.
      • Among low-risk early-term births 7–10 years ago, about 20% had elective interventions.

      ABSTRACT

      Introduction

      : Caesarean section (C-section) deliveries in the U.S. increased from 5% of births in 1970 to 32% in 2020. Little is known about trends in C-sections and inductions in low-risk pregnancies (i.e., those for which interventions would not be medically necessary). This study addresses the following questions: (1) What is the prevalence of elective deliveries at the population-level? (2) How has that changed over time? (3) To what extent do rates of elective deliveries vary across the population?

      Methods

      : We first documented long-term trends in C-sections in the U.S., California (CA), and New Jersey (NJ). We then used linked birth and hospital discharge records and an algorithm based on Joint Commission guidelines to identify low-risk pregnancies and document trends in C-sections and inductions in low-risk pregnancies in CA and NJ over a recent 2-decade period, overall and by maternal characteristics and gestational age.

      Results

      : In low-risk pregnancies in CA and NJ, C-sections and inductions increased sharply from the early 1990s through the mid-2000s, peaked at 33% in CA and 41% in NJ in 2007, and then declined somewhat, and the proportions of inductions that were followed by C-sections increased from fewer than 1 in 5 to about 1 in 4. More education, non-Hispanic White race-ethnicity, U.S.-born status, and non-Medicaid were associated with higher rates of interventions. Trends were similar across all socioeconomic groups, but differences have been narrowing in CA. Among early term (37-38 weeks gestational age) births in low-risk pregnancies, the rates of elective deliveries increased substantially in both states until the mid/late-2000s, peaked at about 35% in CA and over 40% in NJ, and then decreased in both states to about 20%.

      Conclusions

      : Given established health risks of non-medically necessary C-sections, that a non-trivial share of induced deliveries in low-risk pregnancies result in C-sections, and that interventions in low-risk pregnancies have not substantially declined since their peak in the mid-2000s, the trends documented in this paper suggest that sustained, even increased, public health attention is needed to address the still-too-high rates of C-sections and inductions in the U.S.

      Graphical Abstract

      Keywords

      INTRODUCTION

      Rates of caesarean section (C-section) delivery in the U.S. increased from 5% of births in 1970 to a peak of 33% of births in 2009 and then declined only minimally to 32% in 2020.1-3 Induced deliveries increased by 223% between 1990 and 2020, to 31% of U.S. births (national data on inductions are not available before 1990).3,4 When medically necessary, C-sections can save lives; for example, increased rates of preterm C-sections in the U.S. between 1990 and 2004 were associated with perinatal survival, mostly because of decreases in stillbirths.5 When not medically necessary, C-sections increase health risks to newborns,6 and children,7,8 and pose surgical risks to mothers. Induced vaginal deliveries do not appear to be associated with such risks, but inductions of labor are often followed by C-section deliveries.6,8
      Since the mid 2000s, professional guidelines and public health campaigns have encouraged pregnant women and health-care providers to wait until 39 weeks of gestation before intervening and appear to have attenuated the sharp increases in C-sections and inductions in the U.S.9 However, the rates remain close to historic highs, and while increases in C-sections before 2000 were accompanied by reductions in infant and fetal mortality, increases afterward coincided with no such improvements,10,11 and maternal mortality increased by 19% between 2000 and 2017 as part of a decades-long upward trend.12
      Little is known about trends in C-sections and inductions in low-risk pregnancies (i.e., those for which interventions would not be medically necessary). In this study, we document trends in elective C-sections and inductions in two populous and diverse U.S. states, California (CA) and New Jersey (NJ), for which we were able to obtain the necessary data. We address the following questions: (1) What is the prevalence of elective deliveries at the population-level? (2) How has that changed over time? (3) To what extent do rates of elective deliveries vary across the population?

      METHODS

      The study was approved by the Institutional Review Boards of Columbia University, UCLA, Rutgers University, CA Health and Human Services Agency, and NJ Department of Health.

      Study Sample

      We used linked birth and hospital discharge records from the states of CA and NJ (linked birth and hospital discharge data are not available at the national level). For CA, we used CA's birth master files (BMF) of all state-registered births in 1992–2012 (> 11.3 million births), 96% of which were linked to maternal hospital discharge records by the CA Office of Statewide Health Planning and Development (OSHPD). For NJ, we linked records from NJ's Electronic Birth Certificate & Perinatal Database (EBC) of all births in the state in 1997–2015 (> 2.1 million births) to maternal hospital discharge records; the linkage rate was 93%. In both states, the linkages were performed by probabilistic matching using identifying variables including the mother's name, dates of birth of mother and child, and the child's birth weight. The birth records included method of delivery, gestational age (GA), and demographic factors. The discharge records included maternal hospital discharge diagnoses and procedures codes from the International Classification of Diseases, Ninth Revision (ICD-9). The observation period for CA was 1992–2012 based on the linked data made available to us by the CA OSHPD. The observation period for NJ was from 1997, the first full year of data available from the NJ EBC, through 2015. We had access to more recent years of data in NJ but found that the transition to ICD-10 codes after 2015 resulted in irregularities in trend lines that no doubt reflected changes in coding patterns rather than changes in obstetric practice. The 2003 revision of the U.S. Standard Certificate of Live Birth resulted in inconsistent measures of race-ethnicity across states between 2003 and 2016. CA transitioned to the 2003 revised form early in the observation period and the data were harmonized across years. NJ began transitioning to the new form in July 2014; observations based on the revised form were dropped from all analyses (10% of births in 2014 and 50% of births in 2015).

      Measures

      Following recent studies that investigated effects of elective deliveries on neonatal and child outcomes,6,8 we created low-risk samples for each state using Joint Commission (JC) guidelines for the perinatal core outcomes of “decreasing the rate of elective deliveries” and “decreasing the cesarean delivery rate in nulliparous women with a term, singleton baby in a vertex position.” From all births, we excluded those that had any of the ICD-9 codes on the JC lists for “Conditions Possibly Justifying Elective Delivery Prior to 39 Weeks Gestation” or “Contraindications to Vaginal Delivery” in the maternal delivery discharge record (Appendix Table 1). The codes in the lists, which were compiled by a panel of perinatal experts and implemented in 2014 as part of an effort to monitor perinatal outcomes for the purposes of hospital accreditation, cover pregnancy and birth complications (including preeclampsia, diabetes, breech/malposition, fetal abnormalities, fetal distress/abnormal heart rate, and prolonged labor), multiple and preterm/post-term births, and most small-for-GA births.
      Table 1Obstetric interventions and maternal characteristics of births in the U.S. (2002), CA (1992-2012), and NJ (1997-2015)
      United States
      Figures for the U.S. are from Martin JA, Hamilton BE, Sutton PD, Ventura SJ, Menacker F, Munson ML. Births: final data for 2002. National vital statistics reports. 2003 Dec 17;52(10):1-13 and Curtin SC, Osterman MJ, Uddin SF, Sutton SR, Reed PR. Source of payment for the delivery: births in a 33-state and District of Columbia reporting area, 2010. National Vital Statistics Reports, updated 2013. Figures are for 2002 only because the slow rollout across states of the 2003 revision of the U.S. Standard Certificate of Live Birth resulted in inconsistent measures of education and race-ethnicity across states between 2003 and 2016, precluding us from providing national characteristics of births over the entire observation period.
      California
      Figures for CA were compiled from our own data. CA transitioned to the 2003 revision of the U.S. Standard Certificate of Live Birth early in the observation period and the data were harmonized across years.
      New Jersey
      Figures for NJ were compiled from our own data. NJ began transitioning to the 2003 revision of the U.S. Standard Certificate of Live Birth in July 2014; observations based on the revised form were dropped (10% of births in 2014 and 50% of births in 2015).
      All Births

      (2002)
      All Births

      19922012
      Low-Risk Births

      19922012
      All Births

      19972015
      Low-Risk Births

      19972015
      Obstetric interventions
      C-sections2627133419
      Inductions2115122420
      Education
      ≤ 12 years5258534138
      13-15 years2121212121
      16 + years2622253841
      Race-ethnicity
      Non-Hispanic White5732325050
      Asian51113911
      Hispanic2249472424
      Non-Hispanic Black14661614
      Nativity
      U.S.-born7755586766
      Foreign-born2345423334
      Insurance status
      Medicaid3645442524
      Non-Medicaid6455567576
      N4,021,72611,300,2571,654,8881,869,635270,192
      Notes: Figures are column percentages.
      a Figures for the U.S. are from Martin JA, Hamilton BE, Sutton PD, Ventura SJ, Menacker F, Munson ML. Births: final data for 2002. National vital statistics reports. 2003 Dec 17;52(10):1-13 and Curtin SC, Osterman MJ, Uddin SF, Sutton SR, Reed PR. Source of payment for the delivery: births in a 33-state and District of Columbia reporting area, 2010. National Vital Statistics Reports, updated 2013. Figures are for 2002 only because the slow rollout across states of the 2003 revision of the U.S. Standard Certificate of Live Birth resulted in inconsistent measures of education and race-ethnicity across states between 2003 and 2016, precluding us from providing national characteristics of births over the entire observation period.
      b Figures for CA were compiled from our own data. CA transitioned to the 2003 revision of the U.S. Standard Certificate of Live Birth early in the observation period and the data were harmonized across years.
      c Figures for NJ were compiled from our own data. NJ began transitioning to the 2003 revision of the U.S. Standard Certificate of Live Birth in July 2014; observations based on the revised form were dropped (10% of births in 2014 and 50% of births in 2015).
      We further limited the sample to first births and infants with GA between 37 and 40 completed weeks. The first restriction is consistent with the JC's focus on nulliparous women; having a CS for one delivery increases the likelihood that subsequent children will be delivered by CS, making first-time mothers a vital focus.13 We excluded births with GA of 41 weeks to ensure that they were far from the margin for medically indicated inductions.14 GA was calculated as the number of completed weeks between the date of the first day of the mother's last menstrual period and the child's birthday.
      We considered C-sections and inductions in low-risk pregnancies to be non-medically necessary, or elective, interventions.

      ANALYSIS

      Statistical analysis was performed in 2021-2022. First, we compiled and documented long-term trends in rates of C-sections and inductions in the U.S., CA, and NJ from the 1960–70s through 2020 from various published sources and our own data, to place the trends for CA and NJ in context. Then we documented trends over time in: (1) Rates of low-risk births (37-40 weeks GA and not having any of the conditions listed by the JC as justifying interventions) among all singleton first births in CA and NJ during the observation period of our study. (2) Rates of C-sections and inductions and maternal sociodemographic characteristics of births in the U.S., CA, and NJ, for the observation period of our study for CA and NJ and for the year 2002 for the U.S. 2002 was the last year before the rollout of the 2003 revision of the U.S. Standard Certificate of Live Birth, which resulted in inconsistent measures of many characteristics across states between 2003 and 2016 and precluded us from providing national characteristics of births for the entire observation period. We present characteristics both overall (for the U.S., CA, and NJ) and for the sub-samples of low-risk births in CA and NJ. Sociodemographic characteristics were maternal education (< high school, high school graduate or some college, college education or more), race-ethnicity (non-Hispanic White, non-Hispanic Black, Hispanic, Asian), nativity (foreign-born, U.S.-born), and Medicaid coverage for the birth (MediCal in CA). (3) Trends in rates of vaginal deliveries following inductions, C-sections, and either intervention in low-risk pregnancies in CA (N = 1,654,888) and NJ (N = 270,192). (4) Trends in any elective intervention in low-risk pregnancies in each state by maternal sociodemographic characteristics listed above, as well as by gestational age (37-38, 39-40 weeks).
      We also conducted: (5) Sensitivity analyses that did not consider ICD-9 codes with large increases over time in both states compared to the other JC risk factors as criteria that would eliminate cases from the low-risk samples. (6) Supplementary analyses that used a clinical measure of GA to define the sample, which was available for NJ but not CA, to compare trends in NJ using the LMP-based and clinical measures of GA to define the samples.

      RESULTS

      Over the past 5+ decades, C-section rates in CA were very similar to those in the U.S. (Appendix Figure 1). Rates in NJ were very similar to those in the U.S. until the 1990s, but afterward exceeded the national rates.
      Figure 1:
      Figure 1Percentage of low-risk births with inductions and C-sections, California and New Jersey, 1990s through 2010s
      The proportion of singleton first births characterized as low risk using our JC-based algorithm was stable during the observation period (1992–2012) in CA and from 1997 to 2012 in NJ (Appendix Figure 2). The rate decreased somewhat in NJ between 2012 and 2015, the last three years of our observation period for that state. Overall, there is no indication that the large increases in C-section rates in CA and NJ in the 1990s and 2000s reflected increases in obstetric risk factors that would justify C-sections according to the JC criteria (which are listed in Appendix Table 1), as these have not changed in the aggregate among all singleton first births (including high risk) over time (as shown in Appendix Figure 2), and the decrease in low-risk births in NJ in the later years occurred at a time when C-sections were declining, not increasing.
      Figure 2:
      Figure 2Percentages of low-risk births with C-sections following inductions, California and New Jersey
      The overall rates of C-sections and inductions were higher in NJ (34% and 24%, respectively) than in CA (27% and 15%, respectively) and the U.S. (26% and 21%, respectively, in 2002) (Table 1). Rates of interventions were also higher in NJ than in CA among low-risk births. Maternal characteristics differed across states; e.g., mothers in NJ were more educated and less likely to be covered by Medicaid than those in CA and those in the U.S., and the percentages of mothers that were Hispanic and foreign-born were higher in CA. However, within each state, maternal characteristics of the overall and low-risk samples were similar.
      Rates of elective deliveries (i.e., interventions in low-risk pregnancies) demonstrated similar trends as those for C-sections overall (from Appendix Figure 1), almost doubling in both states from the beginning of the observation period until 2007, from 17 to 33% in CA and from 22 to 41% in NJ (Figure 1). Increases in inductions and C-sections contributed equally to the increase in elective deliveries in CA. In NJ, the rate of C-sections increased considerably more than that of inductions.
      The trends shown in Figure 1 reflect the final delivery method, not the initial attempted delivery method. Throughout the observation period, a non-trivial proportion of inductions in low-risk pregnancies ended in C-section deliveries (which could have been because of failure to progress, delivery complications following induction, or decisions to move on to C-section after brief attempts at induction) and the proportion increased over time, from less than 1 in 5 to about 1 in 4 (Figure 2). In fact, most of the “elective” deliveries in both states involved induction of labor.
      In both states, maternal socioeconomic advantage (more education, non-Hispanic White race-ethnicity, U.S.-born status, and non-Medicaid status) was associated with higher rates of elective deliveries, but the trends in elective deliveries over time were similar across all groups (Figure 3). In CA, most socioeconomic differences have decreased since the mid-2000s, when professional guidelines and public health campaigns started encouraging pregnant women and health-care providers to wait until 39 weeks to intervene. For example, at the beginning of the observation period, the rate of elective deliveries was almost 50% higher among college educated women than among women with less than a high school education, but by the end of the period rates of elective deliveries were only about 10% higher among the college educated group. Racial-ethnic differences likewise narrowed during this period and the nativity and poverty differences were virtually eliminated by the end of the period. Overall, the convergence across socioeconomic status in CA after the mid-2000s reflected larger declines in rates of elective deliveries among more economically advantaged groups. Socioeconomic differences in rates of elective deliveries have narrowed much less in NJ.
      Figure 3:
      Figure 3Percentages of low-risk births that were elective (C-sections or inductions) by maternal characteristics, California and New Jersey
      Trends in interventions at early term (37-38 weeks gestational age) were similar to those at full term (39-40 weeks) until the mid-2000s, when guidelines to delay until 39 weeks when possible became widespread; after that inflection point, interventions at early term declined substantially but remained at approximately 20% of all elective deliveries in both states.
      Sensitivity analyses that re-classified fetal distress (ICD9 code 65631) and abnormal heart rate (ICD9 code 6597)—the only two JC conditions in Appendix Table 1 that increased substantially over time in both states—as low-risk conditions revealed trends in elective interventions virtually identical to those presented in Fig. 1 and 3 (not shown). Trends for NJ using clinical GA to define the sample were very similar to those for NJ in Fig. 1 and 3, which used LMP-based GA to define the samples in both states. Clinical GA was not available in the CA data.

      DISCUSSION

      This study found that rates of C-sections and inductions in-low risk pregnancies increased sharply in NJ and CA from the early 1990s to the mid-2000s, peaking at 33% in CA and 41% in NJ in 2007, when efforts by the American College of Obstetrics and Gynecology, March of Dimes, and other organizations to wait until 39 weeks to intervene were implemented, and then decreased somewhat, particularly at early-term. We also found that since the mid-2000s, socioeconomic differences in rates of elective deliveries have been narrowing in CA but not NJ, and that overall, the large increases in elective deliveries observed in the earlier period were not confined to groups with higher socioeconomic status, suggesting that maternal demand-side factors (e.g., C-sections upon request) did not play a major role. Rather, it is more likely that for most of the period, institutional (e.g., medical practice) factors were driving up the rates. It is noteworthy that among early term births, the rates of elective deliveries increased substantially in both states until the mid/late-2000s, peaked at about 35% in CA and over 40% in NJ, and then decreased in both states to about 20%, which according to recommendations at the time was still too high.
      The narrowing of socioeconomic differences in rates of elective deliveries in CA since the mid-2000s is noteworthy, as that state accounts or 12% of the U.S. population,15 and has been at the forefront of initiatives to decrease rates of cesarean delivery.16 CA could even be heading toward a point at which socioeconomically advantaged women are less, rather than more, likely to have elective deliveries. This potential crossover in risk reduction by socioeconomic status following increases in medical knowledge would be consistent with prior observations,17 and concerning from a public health equity perspective.
      As noted earlier, past research has found that associations between C-sections and neonatal/child morbidities are larger than those between induced deliveries and adverse neonatal/child morbidities. However, we found that an increasing fraction of inductions in low-risk pregnancies (now around 25% in both NJ and CA) have led to C-section deliveries and other studies found that the risks to children in those cases are the same as those of planned elective C-sections.6,8 In other words, while elective induced vaginal deliveries do not appear to be detrimental to children's health, elective inductions are often followed by C-sections (for any possible number of reasons), which do confer health risks to children.
      Strengths of our study including the use of linked administrative data on all births in two populous and diverse states, the focus on a time period with large changes in delivery methods in the U.S., and the documentation of trends at the population level, which provides important context for clinicians and practitioners.
      Limitations include our inability to document trends in elective deliveries in the U.S. as a whole owing to lack of available data and the use of administrative data instead of clinical reviews of cases. Another limitation is the possibility that changes in practice and recording of data confounded the trends. However, in supplementary analyses we found that prevalence of each individual high-risk condition was stable over time, with the exception of fetal distress (ICD9 code 65631) and abnormal heart rate (ICD9 code 6597), and when we re-classified those conditions as low risk the trends in elective deliveries did not change.

      CONCLUSIONS

      Given evidence that elective C-sections increase risks for children and their mothers, that a non-trivial share of induced deliveries result in C-sections, and that rates of elective deliveries have not substantially declined since their peak in the early 2000s, the trends documented in this paper suggest that sustained, or even increased, public health attention is needed to address the still-too-high rates of elective C-sections and inductions in the U.S.

      ACKNOWLEDGEMENTS

      The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH, HHS/HRSA, or the Robert Wood Johnson Foundation. This work was funded by the Eunice Kennedy Shriver National Institute of Child Health and Human Development of the National Institutes of Health (awards R01HD090119 and P2CHD058486). The research was also supported by the National Center for Advancing Translational Sciences, a component of NIH under award number UL1TR003017; the HHS/Health Resources and Service Administration (HRSA) under award number U3DMD32755; and the Robert Wood Johnson Foundation through its support of the Child Health Institute of New Jersey (Grant 74260).
      Julien Teitler: Supervision, Conceptualization, Methodology, Funding acquisition, Writing – Original Draft preparation. Valentina Chegwin: Writing - Review & Editing, Data curation. Linda Li: Writing - Review & Editing, Data curation. Kayuet Liu: Supervision, Conceptualization, Methodology, Funding acquisition, Formal analysis, Writing – Original Draft preparation. Peter Bearman: Conceptualization, Funding acquisition, Writing - Review & Editing. Marilyn Gorney-Daley: Writing - Review & Editing. Nancy E. Reichman: Supervision, Conceptualization, Methodology, Funding acquisition, Writing – Original Draft preparation.
      No financial disclosures were reported by the authors of this paper.

      Declaration of interests

      The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

      REFERENCES

      • 1
        Placek PJ, Taffel SM. Trends in cesarean section rates for the United States, 1970–78. Public Health Rep. 1980;95(6):540-548.
      • 2
        Martin JA, Hamilton BE, Osterman MJ, Driscoll AK, Mathews T. Births: final data for 2015. Natl Vital Stat Rep. 2017;66(1):1.
      • 3
        Osterman MJ, Hamilton BE, Martin JA, Driscoll AK, Valenzuela CP. Births: Final Data for 2020. Natl Vital Stat Rep. 2021;70(17):1-50.
      • 4
        Osterman MJ, Martin JA. Recent declines in induction of labor by gestational age. NCHS Data Brief. 2014;(155):1-8.
      • 5
        Ananth CV, Vintzileos AM. Trends in cesarean delivery at preterm gestation and association with perinatal mortality. Am J Obstet Gynecol. 2011;204(6):505.e1-505.e5058. doi:10.1016/j.ajog.2011.01.062.
      • 6
        Teitler JO, Plaza R, Hegyi T, Kruse L, Reichman NE. Elective deliveries and neonatal outcomes in full-term pregnancies. Am J Epidemiol. 2019;188(4):674-683. doi:10.1093/aje/kwz014.
      • 7
        Black M, Bhattacharya S, Philip S, Norman JE, McLernon DJ. Planned cesarean delivery at term and adverse outcomes in childhood health. JAMA. 2015;314(21):2271-2279. doi:10.1001/jama.2015.16176.
      • 8
        Liu K, Teitler JO, Rajananda S, Chegwin V, Bearman P, Hegyi T, Reichman NE. Elective deliveries and the risk of autism. Am J Prev Med. 2022;S0749-3797(22)00108-8. doi: 10.1016/j.amepre.2022.01.024.
      • 9
        Buckles K, Guldi M. Worth the wait? The effect of early term birth on maternal and infant health. J Policy Anal Manage. 2017;36(4):748-72. doi:10.1002/pam.22014.
      • 10
        March of Dimes. Maternal, Infant, and Child Health in the United States: March of Dimes Data Book for Policy Makers. Washington, D.C: March of Dimes; 2016. 8–14.
      • 11
        Kung HC, Hoyert DL, Xu J, Murphy SL. Deaths: final data for 2005. Natl Vital Stat Rep. 2008;56(10):1-120.
      • 12
        CDC Pregnancy Mortality Surveillance System. Centers for Disease Control and Prevention Web site. cdc.gov/reproductivehealth/maternal-mortality/pregnancy-mortality-surveillance-system.htm. Updated 2022.
      • 13
        Spong CY, Berghella V, Wenstrom KD, Mercer BM, Saade GR. Preventing the first cesarean delivery: summary of a joint Eunice Kennedy Shriver National Institute of Child Health and Human Development, Society for Maternal-Fetal Medicine, and American College of Obstetricians and Gynecologists Workshop. Obstet Gynecol. 2012;120(5):1181‒1193. https://doi.org/10.1097/AOG.0b013e3182704880.
      • 14
        Mozurkewich E, Chilimigras J, Koepke E, Keeton K, King VJ. Indications for induction of labour: a best-evidence review. BJOG. 2009;116(5):626‒636. doi:10.1111/j.1471-0528.2008.02065.x.
      • 15
        U.S. Census Bureau, Population Division, 2021. U.S. Census Bureau Web Site. https://www.census.gov/quickfacts/fact/table/CA,US/PST045221. Updated 2021.
      • 16
        Rosenstein MG, Chang SC, Sakowski C, et al. Hospital quality improvement interventions, statewide policy initiatives, and rates of cesarean delivery for nulliparous, term, singleton, vertex births in California. JAMA. 2021;325(16):1631-1639. doi:10.1001/jama.2021.3816.
      • 17
        Link B, Phelan J, Tehranifar P. Social conditions as fundamental causes of health inequalities: theory, evidence, and policy implications. J Health Soc Behav. 2010;51 Suppl:S28-S40. doi:10.1177/0022146510383498.

      Appendix. Supplementary materials