Advertisement

Evaluation of a Contact Tracing Training Program and Field Experience

Open AccessPublished:July 28, 2022DOI:https://doi.org/10.1016/j.focus.2022.100017

      HIGHLIGHTS

      • Training sessions improved volunteers’ knowledge and self-efficacy to contact trace.
      • Health professions students developed structural competency through contact tracing work.
      • Support systems are needed to mitigate moral injury in COVID-19 contact tracers.
      • Contact tracing work has potential career benefits for health professions trainees.
      • These findings should inform more robust and sustainable public health workforces.

      Introduction

      The study objective was to evaluate a contact tracing training program and the role of contact tracing on volunteers’ professional development.

      Methods

      A COVID-19 contact tracing program was conducted at an urban academic medical center, in collaboration with the local health department, between March 2020 and May 2021. Contact tracers, most of whom were health professions students, completed pretraining and post-training surveys to assess knowledge and self-efficacy to conduct contact tracing, plus an 18-month follow-up survey regarding career impacts.

      Results

      We observed statistically significant post-training increases in knowledge and self-efficacy to conduct contact tracing. Contact tracers described benefiting from training regarding cultural humility, empathy, and trauma-informed interviewing. They also expressed a deeper understanding of COVID-19 inequities and their structural causes and reported that the work was emotionally demanding.

      Conclusions

      Key to pandemic preparedness is having a trained and supported workforce. This study showed how contact tracing training and field experience strengthened students’ education in the health professions by sharpening interpersonal skills and structural competency and by generating insights regarding current gaps in both public health infrastructure and support for vulnerable populations.

      Graphical Abstract

      Keywords

      INTRODUCTION

      Contact tracing is a longstanding epidemic-mitigation approach that has been used successfully to control outbreaks of many infectious diseases, including Ebola,
      • Swanson KC
      • Altare C
      • Wesseh CS
      • et al.
      Contact tracing performance during the Ebola epidemic in Liberia, 2014-2015.
      smallpox,
      • Deria A
      • Jezek Z
      • Markvart K
      • Carrasco P
      • Weisfeld J.
      The world's last endemic case of smallpox: surveillance and containment measures.
      and measles.
      • Carlson A
      • Riethman M
      • Gastañaduy P
      • et al.
      Notes from the field: community outbreak of measles – Clark County, Washington, 2018–2019.
      One study of a large-scale natural experiment in which 20% of nationwide coronavirus disease 2019 (COVID-19)-positive cases in England were accidentally not interviewed in a timely manner revealed that contact tracing reduced COVID-19 infections by 63% and deaths by 66%.
      • Fetzer T
      • Graeber T.
      Measuring the scientific effectiveness of contact tracing: evidence from a natural experiment.
      In a financial model from October 2020 evaluating the U.S. costs of the COVID-19 crisis because of reduced economic output and health loss (morbidity and mortality), the benefit of increased investment in testing and contact tracing was estimated to be 30 times greater than the cost of the investment itself.
      • Cutler DM
      • Summers LH.
      The COVID-19 pandemic and the $16 trillion virus.
      Nonetheless, the U.S. entered the COVID-19 pandemic with sparse infrastructure to conduct contact tracing, reflecting decades of declining investment in public health. Nationwide, only 2,200 contact tracers were employed at state and local health departments (LHDs) at the beginning of 2020.
      • Ruebush E
      • Fraser MR
      • Poulin A
      • Allen M
      • Lane JT
      • Blumenstock JS.
      COVID-19 case investigation and contact tracing: early lessons learned and future opportunities.
      This workforce fell short of the estimated 100,000 contact tracers required to address COVID-19, which would have required an additional $3.6 billion in emergency funding from Congress.

      Watson C, Cicero A, Blumenstock J, Fraser M. A national plan to enable comprehensive COVID-19 case finding and contact tracing in the U.S. Baltimore, MD: Johns Hopkins Bloomberg School of Public Health Center for Health Security.https://www.centerforhealthsecurity.org/our-work/pubs_archive/pubs-pdfs/2020/200410-national-plan-to-contact-tracing.pdf. Published April 10, 2020. Accesssed February 2, 2022.

      By the end of 2020, the U.S. contact tracing workforce had increased nearly 23-fold to an estimated 50,000 workers, but this still fell short of the 30 contact tracers per 100,000 people models suggested would be needed to control viral transmission.
      • Lewis D.
      Why many countries failed at COVID contact-tracing — but some got it right.
      When the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic reached the U.S., LHDs mobilized to build contact tracing capacity quickly. Communities variably reassigned public health employees, activated National Guard troops, partnered with private firms and academic institutions, relied on volunteers, or implemented a combination of approaches.
      • Niccolai L
      • Shelby T
      • Weeks B
      • et al.
      Community trace: rapid establishment of a volunteer contact tracing program for COVID-19.
      NC Department of Health and Human Services
      NCDHHS selects first vendors to expand testing and contact tracing for COVID-19.
      • Venteicher W.
      State workers: volunteer or be reassigned to virus response.
      • Dreher A.
      Washington health department enlists private firms to aid state's contact-tracing efforts.
      State approaches to contact tracing during the COVID-19 pandemic.
      Further complicating early contact tracing efforts was the lack of a national standard for training or skills and preparation required to conduct contact tracing,
      • Beech BM
      • Woodard L.
      Contact tracing: a clarion call for national training standards.
      along with the absence of a unified platform or protocol for data collection and sharing across programs and jurisdictions. Each program thus adopted its own training, standards, protocols, and processes, resulting in a heterogeneous array of programs all called contact tracing. To optimize future contact tracing efforts, the public health field must review and critique the varied programmatic responses to COVID-19, including the training received by contact tracers.
      Between March 2020 and May 2021, we collaborated with the Philadelphia Department of Public Health to establish a volunteer-led contact tracing initiative at an urban academic medical center. We report an evaluation of a virtual training program for volunteer contact tracers, most of whom were students of the health professions. These findings should inform the building of a more robust and sustainable public health workforce, better prepared to address epidemic threats.

      METHODS

      Study Sample

      The Contact Tracing Program

      In response to the first wave of COVID-19 in Philadelphia, we built a contact tracing effort in partnership with the Philadelphia Department of Public Health. Volunteers were trained to investigate COVID-19 cases newly identified among health system patients, ascertain close contacts, and deliver isolation and infection control guidance. Contacts were then called to deliver quarantine guidance and answer testing questions. Contact tracing volunteers worked in tandem with a health system Social Needs Response Team (SNRT) to connect patients with supports (e.g., food and medication delivery, transport, primary care referral). These SNRT needs were identified through an optional set of SNRT screener questions at the end of each interview. Optional demographic questions, including race, ethnicity, gender identity, and age, were also included.
      Volunteers were recruited through university listservs reaching students in medicine, nursing, public health, and social-work programs, for whom fieldwork credit was arranged where relevant. From March 28 to June 1, 2020, information from potential volunteers was collected through an electronic survey administered through Google Forms survey (Alphabet Inc., Mountain View, CA), including desired roles, time commitment, and a distress thermometer, to roughly approximate candidate resilience (Appendix A, available online). Select candidates were then interviewed to screen for responsibility, maturity, resilience, and empathy (Appendix B, available online). During the program, a total of 160 volunteers were trained: 130 contact tracing volunteers and 30 program operations volunteers. The 30 program operations volunteers are not discussed in this paper, given that they were not making contact tracing calls.
      Thus, this program relied on essential academic resources. These resouces included access to academic and health system professionals, trainees, legal and bioethics consultants, and data management tools (e.g., REDCap).
      This research study was conducted under a Quality Assurance protocol approved by the IRB of the University of Pennsylvania.

      Measures

      Contact Tracer Trainings

      Training sessions were developed and ready for implementation within 2 weeks of the LHD's request for volunteers, with input from the U.S. and international public health organizations. These training sessions were developed to prepare volunteers to navigate the workflow, interview cases or contacts, and communicate infection control guidance, with the goals of increased knowledge and self-efficacy to conduct contact tracing. In April and May 2020, 109 contact tracers participated in virtual synchronous training sessions administered through Zoom (Zoom Video Communications, San Jose, CA) in 2 separate groups (Figure 1). Details regarding the structure and composition of the training sessions are available in Table 1 and Appendix C (available online). All volunteers role-played before initiating calls and completed asynchronous training on the protection of human subjects. A shared Google folder (Alphabet Inc., Mountain View, CA) included a regularly updated frequently asked questions guide and sample scripts.
      Figure 1
      Figure 1Visual timeline of contact tracing training sessions, events, and surveys. Note that the timeline is not to scale.
      #, number; Dec, December; Jan, January; Jun, June; Nov, November; Oct, October.
      Table 1Summary of Contact Tracing Training Sessions Structure and Composition
      Training sessionTraining topicTopic detailsTopic goals and benchmarks
      Initial training (April/May 2020)Introduction: who are we, and what are we doing?
      • Mission
      • Objectives
      • Organizational structure
      • Provide contact tracers with knowledge regarding how they fit into the broad program's goals and structure
      COVID-19 Science and CDC Guidelines
      • Virus versus disease
      • Symptoms, infectious period, incubation period, treatments (or lack thereof at the time)
      • Importance of flattening the curve
      • Current rates in Philadelphia
      • Ensure that contact tracers understand key scientific background and implications related to the work, to help them make decisions on the job and interact with interviewees
      Case investigation and contact tracing processes
      • What is contact tracing?
      • Definition of “case” and “contact”
      • Operational workflow
      • Ensure that contact tracers have foundational knowledge of the work they are doing, as well as how and why the information is being collected
      Contact tracing operational workflow
      • How to use REDCap data collection system
      • How to use other management tools (e.g., Slack, Google Drive, Doximity)
      • Ensure that contact tracers have the knowledge and self-efficacy to use the essential tools on the job
      Consent and interviewing techniques
      • Important considerations before, during, and after the interview
      • Role-play script before making calls
      • Role-play challenging situations (See Appendix C, available online, for example scenarios)
      • Ensure contact tracers have the knowledge, self-efficacy, and interpersonal skills to navigate all aspects of the interview process, including potential difficult situations
      Trauma-informed interview techniques training (June 16, 2020)Trauma-informed interviewing approaches
      • Disproportionate impact of COVID-19 on Black communities
      • Social needs among cases and contacts within our system
      • SAMHSA's concept of trauma
      • Six key principles of a trauma-informed approach
      • Crisis intervention theory: “Starting where the patient is at”
      • Principles of empathetic inquiry: motivational interviewing, communication skill building, cultural humility
      • When a case needs to be escalated/referred to a specialist and how to conduct escalation
      • Provide contact tracers with a contextual understanding of the pandemic- and systemic racism-induced stress experienced within local communities to possess the interpersonal skills required show respect, empathy, and cultural humility toward all persons being interviewed
      • Note: this training was not evaluated on the pre- and post-training surveys
      For each training session (either the initial training session in April/May 2020 or the Trauma-Informed Interview Techniques training session on June 16, 2020), the training topic, topic details, and topic goals and benchmarks is listed.
      CDC, Centers for Disease Control and Prevention; SAMHSA, Substance Abuse and Mental Health Services Administration.
      Although 62 contact tracers from the second training group completed the pretraining survey, only 56 of them went on to work with us after training. Owing to the anonymous nature of the survey, it is not possible to exclude the 6 pretraining survey responses from participants who did not work with us after training.
      Recognizing stresses caused by both the pandemic and systemic racism in local communities, a trauma-informed interview techniques training was added in June 2020 (Table 1). All contact tracers making calls were required to attend this training.
      Team leaders held optional twice-weekly office hours to virtually answer questions and monitor volunteer wellbeing. Team managers and volunteers communicated frequently by e-mail and Slack (Salesforce, San Francisco, CA), with at least 1 manager always on call to address time-sensitive concerns.

      Pre-training, Post-training, and 18-Month Follow-Up Surveys

      The contact tracing initiative described in this study was launched as an emergency response to a public health crisis. The focus, especially initially, was on establishing the foundation of a contact tracing workforce. After rapidly implementing and designing the program, we realized, through discussions regarding continuous quality improvement, the need for evaluation data regarding the training program. Consequently, only the second group of contact tracers trained was invited to complete the pretraining and post-training surveys (Figure 1) by e-mail and in the training itself.
      The pretraining and post-training surveys were administered through the REDCap secure data collection platform. These surveys assessed knowledge and self-efficacy to conduct contact tracing, measured on a 5-point Likert scale ranging from completely agree to completely disagree. In November 2021, an 18-month follow-up survey was deployed through REDCap to assess the personal and career impacts of working as a contact tracer and to solicit training feedback (Figure 1). Both the first and second training groups of contact tracers were invited to participate. Because the third group was trained asynchronously (owing to rolling recruitment and onboarding), contact tracers in this group were not invited to participate in the evaluation surveys. Figure 1 summarizes the timeline of events, and Table 2 summarizes the breakdown of volunteers across the 3 groups.
      Table 2Summary of Self-Reported and REDCap-Calculated Participant Characteristics (N = 62, Unless Otherwise Noted)
      CharacteristicsStatistics
      Contact Tracing Program
       Total contact tracers130 contact tracers
       Total calls assigned9,191 calls
       Pretraining survey participants62 contact tracers
       Post-training survey participants52 contact tracers
       18-month follow-up survey participants29 contact tracers
      Age
       Mean27.7 years
       Median26.5 years
       Range18–68 years
       Standard deviation6.8 years
      Call number per volunteer (n = 130)
      Statistics were calculated using data from REDCap records, so all 130 volunteers are included in this calculation.
       Mean70 calls
       Median43 calls
       Range1–487 calls
       Standard deviation79.8 calls
      Estimated Elapsed Time per Volunteer (n = 122)
      Statistics was calculated using data from REDCap, but an estimated end date could not be determined for 8 of the 130 total volunteers, so n = 122.
       Mean172 days
       Median162 days
       Range17–369 days
       Standard deviation117.6 days
      n (%)
      Race
      Survey participants could select more than one response, so total will be >100%.
       White52 (83.9)
       Asian6 (9.7)
       Other4 (6.5)
       Black or African American2 (3.2)
       Decline to answer2 (3.2)
       American Indian or Alaskan native0 (0)
       Native Hawaiian or Pacific Islander0 (0)
      Gender identity
       Female57 (91.9)
       Male5 (8.1)
       Another identity0 (0)
       Decline to answer0 (0)
      Student/retiree status
       Currently a student52 (83.9)
       Currently retired1 (1.6)
      Current field of work
      Survey participants could select more than one response, so total will be >100%.
       Nursing36 (58.1)
       Research Scientist13 (21.0)
       Other12 (19.4)
       Medicine3 (4.8)
       Social work1 (1.6)
      Language(s) spoken
      Survey participants could select more than one response, so total will be >100%.
       English61 (98.4)
       Spanish5 (8.1)
       French3 (4.8)
       Portuguese2 (3.2)
       Hebrew1 (1.6)
       Creole1 (1.6)
       Russian1 (1.6)
       Telugu1 (1.6)
       Arabic1 (1.6)
      Volunteer location during contact tracing work
      Survey participants could enter multiple ZIP codes. CA, California; MA, Massachusetts; MD, Maryland; MN, Minnesota; NJ, New Jersey; NY, New York; PA, Pennsylvania; TN, Tennessee; TX, Texas.
       PA65 (81.3)
       NJ6 (7.5)
       NY2 (2.5)
       MA2 (2.5)
       CA1 (1.3)
       MD1 (1.3)
       MN1 (1.3)
       TN1 (1.3)
       TX1 (1.3)
      Group distribution (n = 130)
      Statistics were calculated using data from REDCap records, so all 130 volunteers are included in this calculation.
       Group One53 (40.8)
       Group Two56 (43.1)
       Group Three21 (16.2)
      These data come from the responses provided by the 62 contact tracers who completed the pretraining survey.
      a Statistics were calculated using data from REDCap records, so all 130 volunteers are included in this calculation.
      b Statistics was calculated using data from REDCap, but an estimated end date could not be determined for 8 of the 130 total volunteers, so n = 122.
      c Survey participants could select more than one response, so total will be >100%.
      d Survey participants could enter multiple ZIP codes.CA, California; MA, Massachusetts; MD, Maryland; MN, Minnesota; NJ, New Jersey; NY, New York; PA, Pennsylvania; TN, Tennessee; TX, Texas.
      The pretraining survey was completed before the volunteer's training session on either May 11 or 12 and included 44 questions on demographics, COVID-19 experiences, and knowledge/self-efficacy to conduct contact tracing. The post-training survey was completed immediately after the training session (before any contact tracing experience) and included 21 questions on knowledge/self-efficacy to conduct contact tracing. The 18-month follow-up survey was completed in November 2021 (regardless of when the contact tracer began/finished their work) and included 40 questions on COVID-19 experiences, impacts of contact tracing work, and knowledge/self-efficacy to conduct contact tracing. All surveys were provided through a link to an online REDCap form to be completed on the contact tracer's own time. All surveys are available in Appendix D (available online).

      Statistical Analysis

      All statistical tests were performed in R, Version 4.0.2. Bar graphs were generated in PRISM, Version 9.3.1. To calculate the approximate number of calls and duration of volunteering for each volunteer, timestamp and call assignment data from REDCap data collection projects were used.
      Survey responses were dichotomized for both knowledge (i.e., correct, incorrect) and self-efficacy (“Agree” or “Completely Agree” versus “Unsure,” “Disagree,” or “Completely Disagree”) questions. To assess within-subject changes between the pre- and post-training surveys on individual knowledge questions, McNemar's test was performed. To assess changes for each self-efficacy question, Wilcoxon's signed rank test was performed, which analyzed changes in proportion of respondents who endorsed the specific self-efficacy question before and after training. All open-response questions were coded for content by 2 investigators, who iteratively reviewed and discussed the data to agree on classification (Appendix E, available online).

      RESULTS

      Descriptive statistics for the 62 participants who completed the pretraining survey (out of approximately 100 invitees) are reported. Most contact tracers were current students aged in their late 20s, white, female, and specializing in a health-related field (Table 2).
      A total of 9 languages were spoken among the contact tracers (Table 2). Most contact tracers conducted calls from Pennsylvania, and 15 (19%) reported working from other states including New Jersey, New York, Texas, and Californai. Across all 130 volunteers, the contact tracers volunteered an average of 172 days, and each was assigned an average of 70 calls, with a range of 1–487 calls (Table 2).
      Pre- and post-training surveys were administered to volunteers to assess their knowledge and self-efficacy for conducting contact tracing (Figure 2). Of the 62 volunteers who completed the pretraining survey, 52 (83.9%) also completed post-surveys.
      Figure 2
      Figure 2Evaluation of pretraining and post-training survey results reveals overall efficacy of training. Top: A significant improvement between the pre- and post-training surveys was observed on 8 of the 10 self-efficacy questions. Wilcoxon signed rank tests, N = 52. Bottom: A significant improvement between the pre- and post-training surveys was observed for 3 of the 8 knowledge questions. McNemar tests, N = 52.
      Volunteers’ responses varied on the pretraining survey, and responses significantly improved on 9 of 10 measures of self-efficacy after training (p<0.05). (Figure 2, top). Participants also demonstrated significant improvements for 3 of 8 knowledge questions (Figure 2, bottom). These questions related to answering interviewees’ clinical questions (p<0.001), the difference between SARS-CoV-2 and COVID-19 (p<0.01), and the definition of a case (p<0.01).
      In both pre- and post-training surveys, contact tracers described via open response what they anticipated to be the most challenging aspect(s) of their forthcoming contact tracing work. Anticipated challenges included supporting the interviewee, e.g., when the interviewee was emotionally distressed. After completing the training, a smaller proportion of contact tracers reported concern regarding how to support interviewees (p<0.01). Other anticipated challenges included logistic (e.g., data entry, script familiarity), personal aspects of the work (e.g., emotional/mental toll, personal time management), and eliciting information from reluctant interviewees.
      In the 18-month follow-up survey, open-ended questions were asked regarding what ultimately the most challenging aspect of the contact tracing work was. One unanticipated challenge was establishing trust with the interviewee. One contact tracer described how “it was challenging getting people to divulge their contacts… particularly for marginalized patients who do not have good relationships with the healthcare system.” Another challenge was striking the balance between collecting necessary information, meeting the interviewee's needs, and maintaining boundaries. One contact tracer wrote, “A big part of contact tracing is the emotional side of it. It was often difficult to provide comfort to people on the other side of the phone… to balance being supportive but not getting too emotionally invested.” Volunteers described the personal impacts as the most difficult part of the work (e.g., the “emotional” or “psychological” impacts). Two contact tracers reported that “not knowing follow-up on social needs - whether loops were closed [for the interviewee],” was challenging, especially because it was “taxing to talk to people who are scared and/or in need of food, shelter, or medical assistance without being able to offer an immediate solution [beyond a referral to the Social Needs Response Team].”
      In the 18-month follow-up survey participants were asked to describe their most difficult experiences as contact tracers, which included delivering COVID-19–related news when someone was at work or calling when the interviewee had been hospitalized or passed away. Other difficult experiences included working with people who had extreme social needs, such as elderly individuals experiencing disrupted services and individuals experiencing homelessness. One contact tracer remarked: “I spoke to the mother of a homeless man who was unable to locate her son [after] he had been discharged from the hospital.”
      Open responses on the 18-month follow-up surveys provided additional insight into the training evaluation (Table 3). Nearly 1 in 3 volunteers recalled finding aspects of training related to cultural humility, empathy, and trauma-informed interviewing most helpful in their contact tracing work (Table 3). Eight (40%) of the survey participants indicated that they would have benefited from training in additional areas that were not covered, such as contact tracing-specific language training (e.g., in Spanish) and how to tailor broad infection control guidance to individuals’ specific circumstances (Table 3). Three (15%) reported that the training was sufficient (Table 3).
      Table 3Contact Tracer Experiences Retrospectively Reveal Areas in Which the Training Was Both Successful and Could Be Improved
      Retrospective training efficacy questionsn (%)
      What part of the training was most helpful to you in your work as a contact tracer? (n = 21)
      Survey participants could provide more than one response to the question, so total will be >100%.
       Cultural humility/empathy/trauma training6 (28.6)
       Interview techniques4 (19)
       Continuing communication and education4 (19)
       Practice calls4 (19)
       Script/REDCap walkthrough4 (19)
       Organizational structure2 (9.5)
       Other2 (9.5)
      What do you wish you had more training on before starting your work as a contact tracer? (n = 20)
      Survey participants could provide more than one response to the question, so total will be >100%.
       Additional trainings (e.g., contact tracing-specific language training in Spanish, how to tailor broad infection control guidance to individuals’ specific circumstances)8 (40)
       Training was sufficient3 (15)
       Handling difficult situations3 (15)
       Cultural humility/empathy/trauma training2 (10)
       Practice calls2 (10)
       Eliciting information2 (10)
       Organizational structure1 (5)
      Did your work as a contact tracer influence your perspectives on the COVID-19 pandemic? If so, how? (n = 16)
      Survey participants could provide more than one response to the question, so total will be >100%.
       COVID-19 disparities7 (43.8)
       Importance of contact tracing5 (31.3)
       Expanded worldview4 (25)
       Fast viral spread2 (12.5)
       Guidance adherence2 (12.5)
       Difficulty of contact tracing2 (12.5)
       Broken public health infrastructure1 (6.3)
       Connectedness1 (6.3)
       No1 (6.3)
      Can you please describe your most uplifting or positive experience while working as a contact tracer? (n = 16)
      Survey participants could provide more than one response to the question, so total will be >100%.
       Gratitude7 (43.8)
       Connectedness5 (31.3)
       Listening to stories4 (25)
       Providing guidance4 (25)
       Addressing social needs3 (18.8)
       Speaking with elderly persons3 (18.8)
       Helping someone get to a hospital1 (6.3)
       Interviewee wanted to help1 (6.3)
      Note: These data come from the 18-month follow-up survey.
      a Survey participants could provide more than one response to the question, so total will be >100%.
      Seven contact tracers (43.8%) described in open-ended responses how the work showed them the disproportionate impact of COVID-19 on certain communities (COVID-19 Disparities; Table 3). In addition, 1 in 4 contact tracers noted that their work provided an expanded understanding of other peoples’ experiences or perspectives regarding the pandemic (Expanded Worldview; Table 3). As one volunteer said, “[Contact tracing] broadened my perspective on how the pandemic was impacting low-income and minority communities particularly. A large number of the people being called needed additional resources such as food, water, and medication during their quarantine period.” Other volunteers reported developing an appreciation for the importance (31.3%) and difficulty (12.5%) of contact tracing, while some described a new understanding of how quickly the virus was spreading through the community (12.5%) or their own increased likelihood of adhering to public health guidelines (12.5%) (Table 3). One contact tracer noted “how important it was to pursue public health interventions beyond clinical medicine,” and another said the work “showed me just how broken our public health infrastructure is.”
      Nearly half (43.8%) of survey participants reported that their most uplifting or positive experience was receiving gratitude from the interviewee (Table 3). One volunteer wrote, “I once spoke with [someone] who sounded exhausted but had much gratitude to express. Paraphrasing her sentiments… the calls you are doing may be a thankless job… I couldn't emotionally do what you are doing. Thank you for caring about the strangers and seeing the stories behind the statistics.” Other uplifting or positive experiences included a feeling of connectedness with the interviewee, listening to the interviewee's stories, and providing guidance to the interviewee in an empowering way (Table 3). Addressing interviewees’ social needs was also rewarding, as one volunteer described how “being able to link people with rent, food, and clinical support was really meaningful.”
      Given that most (83.9%) contact tracers were current students (Table 2), this study explored how volunteering impacted their career development and/or trajectory. In the pretraining survey, participants reported their motivation for becoming a contact tracer, both in multiple choice and open response formats. Fifty-six (90.3%), 45 (72%), and 36 (58%) of the respondents indicated that they wanted to help mitigate the pandemic, had an applicable skillset, or were receiving academic credit for volunteering (Table 4). Ten motivation themes were identified when analyzing the open responses. In addition to the previously mentioned motivations, contact tracers reported that their interest in volunteering stemmed from being unable to partake in normal work/school and having time (Table 4).
      Table 4Contact Tracing Work Was Applicable to and Impacted Contact Tracers’ Careers
      Career impact and applicability questionsn (%)
      What is your motivation for becoming a contact tracer? (multiple choice) (n = 62)
      Survey participants could provide more than one response to the question, so total will be >100%.
       I want to help mitigate the pandemic56 (90.3)
       I have a skillset that applies to this work45 (73)
       I am receiving credit in some way for participating (e.g., clinical hours, field work)36 (58)
       Other0 (0)
      What is your motivation for becoming a contact tracer? (open response) (n =62)
      Survey participants could provide more than one response to the question, so total will be >100%.
       Desire to help52 (83.9)
       Applicable to field of interest16 (25.8)
       Use applicable skills15 (24.2)
       Have time15 (24.2)
       Receive credit8 (12.9)
       Recognition of contact tracing importance6 (9.7)
       Develop applicable skills6 (9.7)
       Work from home5 (8.1)
       Feeling helpless2 (3.2)
       COVID-19 experience1 (1.6)
      How has your work as a contact tracer impacted or been applicable to your career? (n = 16)
      Survey participants could provide more than one response to the question, so total will be >100%.
       Current or future healthcare/public health worker11 (68.8)
       Develop skills6 (37.5)
       Expanded worldview3 (18.8)
       Influenced job/career path3 (18.8)
       COVID-19 knowledge2 (12.5)
       Similar in nature to current work2 (12.5)
       Importance of public health/intersection with medicine1 (6.3)
       Balance urgency with support1 (6.3)
      Note: These data come from the pretraining survey and the 18-month follow-up survey.
      a Survey participants could provide more than one response to the question, so total will be >100%.
      Most (68.8%) of the respondents indicated that contact tracing was applicable to their current or future work in healthcare or public health (Table 4). One volunteer described, “Hearing about the journeys of patients leading up to showing symptoms and their experience with the virus has been very helpful for personalizing how this virus has impacted patients I might work with.” Finally, while only 9.7% of the survey participants initially described skill development as a motivating factor for volunteering, this proportion increased to 37.5% in the 18-month follow-up survey when additional participants reported that this was one of the ways in which the work ultimately impacted their career (Table 4). These skills included expressing empathy, balancing information-seeking and infection control with provision of support, and rapid relationship building.

      DISCUSSION

      This article describes how a volunteer contact tracing program contributed to pandemic response and health profession career development through training and field experience. These results may inform contact tracing and pandemic responses in 3 key ways.
      First, contact tracing programs require both infrastructure and skilled staff, and a volunteer-based system is not a sustainable pandemic preparedness model. University faculty and staff managed this program alongside their normal job responsibilities, and all contact tracers were volunteers. The program also relied on academic health system professionals, trainees, legal and bioethics consultants, and data management tools (e.g., REDCap). These human and institutional resources supported rapid program development and implementation. However, these resources were tapped as an emergency adaptation to insufficient health department resources and cannot substitute for durable public health infrastructure, especially because of the nature of volunteerism. This program ultimately ended because both students and staff needed to return to their prepandemic responsibilities related to studying and paid work. The phenomenon of abandoning volunteer positions for paid work has been identified beyond the context of COVID-19.
      • Paull M
      • Holmes K
      • Omari M.
      • et al.
      Myths and misconceptions about university student volunteering: development and perpetuation.
      Furthermore, it has been proposed that enlisting unpaid volunteers to conduct long-term, significantly beneficial work integral to an organization is problematic.
      • Overgaard C.
      Rethinking volunteering as a form of unpaid work.
      Consequently, this suggests that a robust, effectively maintained contact tracing program requires well-resourced public health infrastructure including funding for contact tracers to be compensated, rather than serving as volunteers. The study findings also demonstrated that effective contact tracing required refinement of interpersonal skills (e.g., empathy, communication under stress, ability to respectfully probe for additional details or guide the conversation during an interview) that should be selected for in recruitment and reinforced through training of contact tracers (Table 1 and Figure 2, top).
      Second, contact tracers developed structural competency, defined as an understanding of how a patient's clinical presentation represents not only that person's individual decisions and actions, but also the consequence of upstream systemic health inequities embedded within the infrastructure of our society.
      • Metzl JM
      • Hansen H.
      Structural competency: theorizing a new medical engagement with stigma and inequality.
      The call for improved structural competency in healthcare settings and health professions education resounds.
      • Hansen H
      • Metzl J.
      Structural competency in the U.S. Healthcare crisis: putting social and policy interventions into clinical practice.
      • Downey MM
      • Neff J
      • Dube K.
      Don't just call the social worker: training in structural competency to enhance collaboration between healthcare social work and medicine.
      • Metzl JM
      • Roberts DE.
      Structural competency meets structural racism: race, politics, and the structure of medical knowledge.
      • Willging C
      • Gunderson L
      • Shattuck D
      • Sturm R
      • Lawyer A
      • Crandall C.
      Structural competency in emergency medicine services for transgender and gender non-conforming patients.
      • Knight KR.
      Structural factors that affect life contexts of pregnant people with opioid use disorders: the role of structural racism and the need for structural competency.
      • Waite R
      • Hassouneh D.
      Structural competency in mental health nursing: understanding and applying key concepts.
      In the post-training survey, there was a significant improvement in volunteers’ confidence in their ability to be culturally sensitive when speaking to interviewees (p < 0.001). Furthermore, although the development of structural competency was not an explicit goal of this program, many contact tracers gave feedback consistent with an evolved understanding regarding structural determinants of health (provided as COVID-19 Disparities and Expanded Worldview in Table 3, and the codebook in Appendix E, available online). Volunteers described how interviewees’ structural constraints and lack of social safety net put them at-risk for COVID-19 exposure, thus demonstrating structural competency. These study findings suggest the educational value of engagement and conversation with individuals regarding their pandemic lived experiences. Although this program was similar to other institution-based student-volunteer COVID-19 contact tracing programs, this training evaluation is the first among recent evaluations to identify the development of structural competency among volunteers, which has potential career benefits for health professions students.
      • Shelby T
      • Hennein R
      • Schenck C
      • et al.
      Implementation of a volunteer contact tracing program for COVID-19 in the United States: a qualitative focus group study.
      • Chengane S
      • Cheney A
      • Garth S
      • Medcalf S.
      The COVID-19 response in Nebraska: how students answered the call.
      • Koetter P
      • Pelton M
      • Gonzalo J
      • et al.
      Implementation and process of a COVID-19 contact tracing initiative: leveraging health professional students to extend the workforce during a pandemic.
      Finally, this work reinforced that the COVID-19 pandemic has been a collective trauma,
      • Silver RC
      • Holman EA
      • Garfin DR.
      Coping with cascading collective traumas in the United States.
      • Kalsched D.
      Intersections of personal vs. collective trauma during the COVID-19 pandemic: the hijacking of the human imagination.
      • Taylor M.
      Collective trauma and the relational field.
      with a disproportionate impact on people with marginalized social positions and other vulnerabilities.
      • Silver RC
      • Holman EA
      • Garfin DR.
      Coping with cascading collective traumas in the United States.
      ,
      • Watson MF
      • Bacigalupe G
      • Daneshpour M
      • Han WJ
      • Parra-Cardona R.
      COVID-19 interconnectedness: health inequity, the climate crisis, and collective trauma.
      ,
      University of Illinois Chicago
      COVID-19: the disproportionate impact on marginalized populations.
      The contact tracers in this study observed that this was true for the community members with whom they interacted. They also described the emotional and mental impact of contact tracing as the most challenging aspect of the work. They recognized their limited personal and programmatic capacity to address COVID-19 disparities. While connecting interviewees to resources through the Social Needs Response Team referrals mitigated short-term difficulties, many interviewees were vulnerable in other systemic ways that our contact tracers could not address—sometimes leading to frustration and hopelessness. This recognition is related to the concept of moral injury, or the accumulation of distress individuals feel when external factors prevent them from accomplishing what they believe is right.
      • Roycroft M
      • Wilkes D
      • Pattani S
      • Fleming S
      • Olsson-Brown A.
      Limiting moral injury in healthcare professionals during the COVID-19 pandemic.
      These data suggest that contact tracers, like other frontline workers, are vulnerable to moral injury, and consequently need training and support.

      Litam SDA, Balkin RS. Moral injury in health-care workers during COVID-19 pandemic. Traumatology. 2021;27(1):14–19. https://doi.org/10.1037/trm0000290.

      • Gaitens J
      • Condon M
      • Fernandes E
      • McDiarmid M.
      COVID-19 and essential workers: a narrative review of health outcomes and moral injury.
      • Lesley M.
      Psychoanalytic perspectives on moral injury in nurses on the frontlines of the COVID-19 pandemic.
      Support structures were provided in this program, such as team office hours, consultation with social work-trained team members, and trauma-informed training. Roycroft et al.
      • Roycroft M
      • Wilkes D
      • Pattani S
      • Fleming S
      • Olsson-Brown A.
      Limiting moral injury in healthcare professionals during the COVID-19 pandemic.
      (2020) suggest additional approaches may be beneficial, such as consistent teams to foster an environment of reliable support and guidance, as well as the provision of time and space for workers to rest and process their experiences.
      In response to the COVID-19 pandemic, academic institutions partnered with LHDs to aid in contact tracing initiatives.
      • Niccolai L
      • Shelby T
      • Weeks B
      • et al.
      Community trace: rapid establishment of a volunteer contact tracing program for COVID-19.
      ,
      • Chengane S
      • Cheney A
      • Garth S
      • Medcalf S.
      The COVID-19 response in Nebraska: how students answered the call.
      ,
      • Koetter P
      • Pelton M
      • Gonzalo J
      • et al.
      Implementation and process of a COVID-19 contact tracing initiative: leveraging health professional students to extend the workforce during a pandemic.
      However, to the best of our knowledge only one of these groups conducted a follow-up study investigating contact tracers’ perspectives on the initiative. Shelby and colleagues
      • Shelby T
      • Hennein R
      • Schenck C
      • et al.
      Implementation of a volunteer contact tracing program for COVID-19 in the United States: a qualitative focus group study.
      (2021) described a qualitative focus group study in which 36 volunteer contact tracers provided insight regarding “facilitators, barriers, and potential solutions for improving implementation of COVID-19 contact tracing.” Our findings corroborate many of those described in Shelby et al.,
      • Shelby T
      • Hennein R
      • Schenck C
      • et al.
      Implementation of a volunteer contact tracing program for COVID-19 in the United States: a qualitative focus group study.
      including contact tracers’ motivations to volunteer (e.g., desire to help mitigate the pandemic and use an applicable skillset, as described in Table 4), difficulty in reaching/establishing trust with the interviewee, and appreciation for the provided interview script. Both studies also report sustainability barriers because of the volunteer nature of the work. However, our study contributes novel findings in terms of our detailed training session evaluation, exploration of health professions career benefits, the development of structural competency among student volunteers, and the need to provide supports to contact tracers to mitigate moral injury. These studies are complementary, with findings corroborated despite differing methodologic approaches (focus group conversations versus surveys) and different study goals (improving the program structure based on volunteer feedback versus evaluating training sessions and professional development impacts).

      Limitations

      The contact tracing initiative described here was launched as an emergency response to a public health crisis, and thus our primary responsibility was to rapidly establish a robust contact tracing workforce to stop the chain of SARS-CoV-2 transmission in Philadelphia. Consequently, the findings presented here are the result of a secondary analysis of the training sessions and evaluations that were designed for the field as opposed to research. This context explains key limitations of our work, including incomplete data regarding comprehensive contact tracer demographics, small survey sample sizes, and lack of a control group. The generalizability of these findings is consequently impacted. For example, this study cannot examine differences in experiences, knowledge, self-efficacy, or outcomes between responders and non-responders.
      Contact tracer demographics could have also impacted responses on all 3 surveys, resulting in a lack of generalizability. Furthermore, both recall bias and differences in experience level (e.g., call assignments ranging from 1 to 487 and elapsed time ranging from 17 to 369 days; Table 2) among the contact tracers could have impacted survey responses on the 18-month follow-up survey. The results from this survey are also limited by low response rate, which could be because of the time delay between when the contact tracing program ended (May 2021) and when the survey was administered (November 2021). Consequently, there was a lack of current contact information for all contact tracers, and the delay itself could have disincentivized contact tracers from completing the survey. This low response rate also impacts the generalizability of these findings. Finally, another limitation of this program is the low retention of volunteers over time, a common obstacle encountered by volunteer-led programs.
      Corporation for National and Community Service, Office of Research and Policy Development
      Volunteering in America: 2007 city trends and rankings.
      • Netting FE
      • O'Connor MK
      • Thomas ML
      • Yancey G
      Mixing and phasing of roles among volunteers, staff, and participants in faith-based programs.
      • Garner JT
      • Garner LT.
      Volunteering an opinion: organizational voice and volunteer retention in nonprofit organizations.
      Despite these limitations, this work provides valuable foundational insights into the training materials necessary to prepare volunteers for conducting contact tracing. A key strength of this work is the insights provided from the evaluation of both training and field experience for a large volunteer effort that assigned more than 9,000 calls between April 2020 and May 2021.

      CONCLUSIONS

      Through training sessions and field experience, our volunteer contact tracing program contributed to health profession education and career development for trainees. Our evaluation also reveals how preparation and support systems for contact tracers can be improved for future contact tracing programs.

      ACKNOWLEDGMENTS

      The authors thank our contact tracers and patients, as well as colleagues from the DrPH Coalition (Boston, MA), the Health Service Executive Ireland (Dublin, Ireland), University of Pennsylvania Campus Health (Philadelphia, PA), and the Washington State Department of Health (Olympia, WA), as well as Dr. Josh Sharfstein and Dr. Emily Gurley, for their insights in creating the contact tracing training sessions. The authors thank the WW Smith Foundation for their support of this work.
      This work was supported by the WW Smith Foundation.
      The authors have no conflicts of interest to report. Most staff effort was donated by volunteers, with philanthropic support from the WW Smith Foundation to partially offset salaries of 2 staff members who led the program (RFS and CCC). The university provided in-kind support, such as online tools for data collection. All research team members contributed to the writing of the manuscript. No disclosures were reported.

      CRediT AUTHOR STATEMENT

      Katherine M Strelau: Conceptualization, Methodology, Software, Formal Analysis, Investigation, Data Curation, Visualization, Writing – Original Draft, Writing – Review & Editing.  Nawar Naseer: Methodology, Validation, Formal Analysis, Writing – Original Draft, Writing – Review & Editing.  Rachel Feuerstein-Simon: Conceptualization, Methodology, Data Curation, Project Administration, Writing – Original Draft, Writing – Review & Editing.  Kierstyn Claycomb: Investigation, Writing – Review & Editing.  Heather Klusaritz: Conceptualization, Resources, Writing – Review & Editing.  Hillary Nelson: Conceptualization, Resources, Writing – Review & Editing.  Carolyn Cannuscio: Conceptualization, Resources, Supervision, Funding Acquisition, Writing – Original Draft, Writing – Review & Editing.

      REFERENCES

        • Swanson KC
        • Altare C
        • Wesseh CS
        • et al.
        Contact tracing performance during the Ebola epidemic in Liberia, 2014-2015.
        PLoS Negl Trop Dis. 2018; 12e0006762https://doi.org/10.1371/journal.pntd.0006762
        • Deria A
        • Jezek Z
        • Markvart K
        • Carrasco P
        • Weisfeld J.
        The world's last endemic case of smallpox: surveillance and containment measures.
        Bull World Health Organ. 1980; 58: 279-283
        • Carlson A
        • Riethman M
        • Gastañaduy P
        • et al.
        Notes from the field: community outbreak of measles – Clark County, Washington, 2018–2019.
        MMWR Morb Mortal Wkly Rep. 2019; 68: 446-447https://doi.org/10.15585/mmwr.mm6819a5
        • Fetzer T
        • Graeber T.
        Measuring the scientific effectiveness of contact tracing: evidence from a natural experiment.
        Proc Natl Acad Sci U S A. 2021; 118e2100814118https://doi.org/10.1073/pnas.2100814118
        • Cutler DM
        • Summers LH.
        The COVID-19 pandemic and the $16 trillion virus.
        JAMA. 2020; 324: 1495-1496https://doi.org/10.1001/jama.2020.19759
        • Ruebush E
        • Fraser MR
        • Poulin A
        • Allen M
        • Lane JT
        • Blumenstock JS.
        COVID-19 case investigation and contact tracing: early lessons learned and future opportunities.
        J Public Health Manag Pract. 2021; 27: S87-S97https://doi.org/10.1097/PHH.0000000000001290
      1. Watson C, Cicero A, Blumenstock J, Fraser M. A national plan to enable comprehensive COVID-19 case finding and contact tracing in the U.S. Baltimore, MD: Johns Hopkins Bloomberg School of Public Health Center for Health Security.https://www.centerforhealthsecurity.org/our-work/pubs_archive/pubs-pdfs/2020/200410-national-plan-to-contact-tracing.pdf. Published April 10, 2020. Accesssed February 2, 2022.

        • Lewis D.
        Why many countries failed at COVID contact-tracing — but some got it right.
        Nature. 2020; 588: 384-387https://doi.org/10.1038/d41586-020-03518-4
        • Niccolai L
        • Shelby T
        • Weeks B
        • et al.
        Community trace: rapid establishment of a volunteer contact tracing program for COVID-19.
        Am J Public Health. 2021; 111: 54-57https://doi.org/10.2105/ajph.2020.305959
        • NC Department of Health and Human Services
        NCDHHS selects first vendors to expand testing and contact tracing for COVID-19.
        NC Department of Health and Human Services, Raleigh NCPublished June 19, 2020 (Accessed March 22, 2022)
        • Venteicher W.
        State workers: volunteer or be reassigned to virus response.
        Governing. May 27, 2020; (Accessed March 22, 2022)
        • Dreher A.
        Washington health department enlists private firms to aid state's contact-tracing efforts.
        The Spokesman Review. August 27, 2020; (Accessed March 22, 2022)
      2. State approaches to contact tracing during the COVID-19 pandemic.
        The National Academy for State Health Policy. 2022; (Accessed March 22, 2022)
        • Beech BM
        • Woodard L.
        Contact tracing: a clarion call for national training standards.
        Ethn Dis. 2020; 30: 437-440https://doi.org/10.18865/ed.30.3.437
        • Paull M
        • Holmes K
        • Omari M.
        • et al.
        Myths and misconceptions about university student volunteering: development and perpetuation.
        Voluntas. 2022; (In press. Online January 3)https://doi.org/10.1007/s11266-021-00437-4
        • Overgaard C.
        Rethinking volunteering as a form of unpaid work.
        Nonprofit Volunt Sect Q. 2019; 48: 128-145https://doi.org/10.1177/0899764018809419
        • Metzl JM
        • Hansen H.
        Structural competency: theorizing a new medical engagement with stigma and inequality.
        Soc Sci Med. 2014; 103: 126-133https://doi.org/10.1016/j.socscimed.2013.06.032
        • Hansen H
        • Metzl J.
        Structural competency in the U.S. Healthcare crisis: putting social and policy interventions into clinical practice.
        J Bioeth Inq. 2016; 13: 179-183https://doi.org/10.1007/s11673-016-9719-z
        • Downey MM
        • Neff J
        • Dube K.
        Don't just call the social worker: training in structural competency to enhance collaboration between healthcare social work and medicine.
        J Sociol Soc Welf. 2019; 46: 77-96
        https://scholarworks.wmich.edu/jssw/vol46/iss4/6
        Date accessed: February 21, 2022
        • Metzl JM
        • Roberts DE.
        Structural competency meets structural racism: race, politics, and the structure of medical knowledge.
        AMA J Ethics. 2014; 16: 674-690https://doi.org/10.1001/virtualmentor.2014.16.09.spec1-1409
        • Willging C
        • Gunderson L
        • Shattuck D
        • Sturm R
        • Lawyer A
        • Crandall C.
        Structural competency in emergency medicine services for transgender and gender non-conforming patients.
        Soc Sci Med. 2019; 222: 67-75https://doi.org/10.1016/j.socscimed.2018.12.031
        • Knight KR.
        Structural factors that affect life contexts of pregnant people with opioid use disorders: the role of structural racism and the need for structural competency.
        Womens Reprod Health. 2020; 7: 164-171https://doi.org/10.1080/23293691.2020.1780400
        • Waite R
        • Hassouneh D.
        Structural competency in mental health nursing: understanding and applying key concepts.
        Arch Psychiatr Nurs. 2021; 35: 73-79https://doi.org/10.1016/j.apnu.2020.09.013
        • Shelby T
        • Hennein R
        • Schenck C
        • et al.
        Implementation of a volunteer contact tracing program for COVID-19 in the United States: a qualitative focus group study.
        PLoS One. 2021; 16e0251033https://doi.org/10.1371/journal.pone.0251033
        • Chengane S
        • Cheney A
        • Garth S
        • Medcalf S.
        The COVID-19 response in Nebraska: how students answered the call.
        Prev Chronic Dis. 2020; 17: E81https://doi.org/10.5888/pcd17.200269
        • Koetter P
        • Pelton M
        • Gonzalo J
        • et al.
        Implementation and process of a COVID-19 contact tracing initiative: leveraging health professional students to extend the workforce during a pandemic.
        Am J Infect Control. 2020; 48: 1451-1456https://doi.org/10.1016/j.ajic.2020.08.012
        • Silver RC
        • Holman EA
        • Garfin DR.
        Coping with cascading collective traumas in the United States.
        Nat Hum Behav. 2021; 5: 4-6https://doi.org/10.1038/s41562-020-00981-x
        • Kalsched D.
        Intersections of personal vs. collective trauma during the COVID-19 pandemic: the hijacking of the human imagination.
        J Anal Psychol. 2021; 66: 443-462https://doi.org/10.1111/1468-5922.12697
        • Taylor M.
        Collective trauma and the relational field.
        Humanist Psychol. 2020; 48: 382-388https://doi.org/10.1037/hum0000215
        • Watson MF
        • Bacigalupe G
        • Daneshpour M
        • Han WJ
        • Parra-Cardona R.
        COVID-19 interconnectedness: health inequity, the climate crisis, and collective trauma.
        Fam Process. 2020; 59: 832-846https://doi.org/10.1111/famp.12572
        • University of Illinois Chicago
        COVID-19: the disproportionate impact on marginalized populations.
        Jane Addams college of social work, University of Illinois Chicago, Chicago, ILPublished April 29, 2020 (Accessed March 22, 2022)
        • Roycroft M
        • Wilkes D
        • Pattani S
        • Fleming S
        • Olsson-Brown A.
        Limiting moral injury in healthcare professionals during the COVID-19 pandemic.
        Occup Med (Lond). 2020; 70: 312-314https://doi.org/10.1093/occmed/kqaa087
      3. Litam SDA, Balkin RS. Moral injury in health-care workers during COVID-19 pandemic. Traumatology. 2021;27(1):14–19. https://doi.org/10.1037/trm0000290.

        • Gaitens J
        • Condon M
        • Fernandes E
        • McDiarmid M.
        COVID-19 and essential workers: a narrative review of health outcomes and moral injury.
        Int J Environ Res Public Health. 2021; 18: 1446https://doi.org/10.3390/ijerph18041446
        • Lesley M.
        Psychoanalytic perspectives on moral injury in nurses on the frontlines of the COVID-19 pandemic.
        J Am Psychiatr Nurs Assoc. 2021; 27: 72-76https://doi.org/10.1177/1078390320960535
        • Corporation for National and Community Service, Office of Research and Policy Development
        Volunteering in America: 2007 city trends and rankings.
        Corporation for National and Community Service, Office of Research and Policy Development, Washington, DCPublished July 2007
        • Netting FE
        • O'Connor MK
        • Thomas ML
        • Yancey G
        Mixing and phasing of roles among volunteers, staff, and participants in faith-based programs.
        Nonprofit Volunt Sect Q. 2005; 34: 179-205https://doi.org/10.1177/0899764005275204
        • Garner JT
        • Garner LT.
        Volunteering an opinion: organizational voice and volunteer retention in nonprofit organizations.
        Nonprofit Volunt Sect Q. 2011; 40: 813-828https://doi.org/10.1177/0899764010366181