Introduction

The unmet need for emergency care in resource limited settings has been well established [1, 2]. The World Health Assembly has called for universal access to safe, high-quality, needs-based emergency care [3]. Implementation of the Basic Emergency Care (BEC) course, developed by the World Health Organization (WHO) and the International Committee for the Red Cross (ICRC) in 2018, is one mechanism to achieve this goal [4]. This course is designed to train front-line health providers in the assessment and treatment of acute and emergent conditions, particularly in resource limited settings. It has been implemented in several countries, with high-performing participants being further trained to become instructors of future classes with a Training-of-Trainers (ToT) course [5, 6, 7, 8].

Scaling the BEC course requires significant logistic, personnel and equipment input into both the BEC and ToT courses. These challenges include the cost of holding and attending courses, the time required for attending a course, and travel logistics. Some regions of the world that could most benefit from BEC training are remote or have security challenges, which also creates a barrier to in-person training. Prior BEC implementations often limited ToT courses to small groups immediately preceding or following the BEC course itself. In an attempt to address the need for additional BEC trainers and more efficient training of trainers, a large ToT was held at the annual meeting of the American College of Emergency Physicians (ACEP) in 2019 in Denver, Colorado, with 121 participants from 22 countries. While this large ToT expanded the number of personnel trained at one time, most participants were from high income countries and per BEC delivery best practice recommendations, the new trainers would need to partner with local stakeholders at BEC implementation sites.

Furthermore, when the COVID-19 pandemic restricted travel and large gatherings globally in the spring of 2020, many prior BEC participants were forced to cancel plans to attend in-person BEC ToT courses. As a result, many BEC courses were paused or canceled. Many educational programs have successfully transitioned to a virtual format in response to COVID-19, including the ICRC Health Emergencies in Large Population (HELP) course, and a number of the London School of Hygiene and Tropical Medicine short training courses [9, 10, 11]. Computer based simulations have been studied and have shown to be effective in remote training of emergency care providers [12]. New training programs like Project Hope’s Virtual ToT on COVID-19 preparedness have also shown to be highly successful in knowledge gain [13].

Adjuncts to support asynchronous and remote BEC training have already been created, including a mobile application and practice cases [14, 15]. In response to COVID-19 and the additional pre-existing barriers to BEC implementation, the BEC ToT course was modified to a virtual format, incorporating these existing adjuncts. A needs assessment was performed amongst current trainers who had directed both BEC and ToT courses and an adapted curriculum was created and taught virtually. Here, we report on the effectiveness, feasibility, and acceptability of a virtual BEC ToT course through a mixed methods approach as compared to a recent in-person ToT course.

Methods

A needs assessment involving five BEC trainers and past participants reviewed the existing ToT curriculum and feedback from prior in-person ToT courses. Each curricular component was reviewed for inclusion in the virtual course and categorized as keep, modify, or drop. Of the 16 components, 14 were kept, none were modified, and two were dropped. The standard in-person curriculum and final virtual curriculum are shown in Figure 1. The detailed schedules from the in-person and the virtual ToT courses can be found in Appendices 1 and 2, respectively. The two-day course was taught in English, and Zoom (Zoom Video Communications, Inc., San Jose, CA) was used as the virtual platform. Moderated breakout rooms were used for small group and skill sessions. Facilitators were asked to join a WhatsApp (Facebook, Menlo Park, CA) group to allow for seamless communication throughout the day. The virtual course was taught by 13 facilitators, whose competencies are summarized in Figure 2.

De-identified participant data was prospectively collected during the virtual course and then compared to a retrospective analysis of de-identified data from an in-person ToT course that had been held in Denver, Colorado in October 2019 at the ACEP conference.

Participants for both courses were recruited via email advertisements, internet advertisements, and social media. There was no requirement for years of emergency medicine experience or prior BEC course participation. Eligible international and U.S. participants included emergency medicine physicians, residents, fellows, advanced practitioners, and nurses practicing in an acute emergent setting for at least two years. Due to the lack of uniformity of emergency medicine training outside of the United States, international participants had slightly different eligibility criteria. They were required to have had prior BEC training and to be working in the emergency setting.

Participants for both courses took the same 20 question multiple choice pre- and post-tests and completed a course feedback form upon completion of the course. Test questions were developed by the course publisher and include content on course logistics and teaching theory. In updating the content for the virtual session, content for one post-test question was not covered in the virtual course and was removed from analysis, making the virtual course post-test 19 questions. The tests and feedback forms were the standard versions developed as part of the BEC ToT training package.

The course feedback form contained seven questions structured on a 5-point Likert scale with responses ranging from 1 (least favorable) to 5 (most favorable). One question on optimal course length was structured differently with 3 being the optimal course length, and less than or more than 3 representing too short or too long of course length respectively. Participants in the virtual course had an additional three open-ended questions about opportunities for improvement, areas of success, and plans for future use of the virtual training course.

Study data were collected and managed using REDCap electronic data capture tools. Pre- and post-test scores and the course feedback form Likert scale questions were compared using a paired two-sample Student’s t-test, two-tailed (Excel version 16.0, Microsoft, Redmond, WA, and MATLAB version R2015(a), The Mathworks Inc, Natick, MA). A qualitative content analysis was performed on participant responses to the open-ended questions on the course feedback forms form each course. A content analysis was done on all responses and coded by three authors (MR, JD, SK). Inductive coding was used for each open-ended question to identify key ideas which were descriptively summarized. We used the SQUIRE checklist when writing our report [16].

Data from the in-person course was determined to be non-human subjects research by Colorado Multiple Institutional Review Board (COMIRB) protocol #21-2778. Data from the virtual course was prospectively collected and informed consent was obtained prior to participant assessment. The virtual course data plan was reviewed and approved by the Mass General Brigham IRB, protocol # 2020P003386.

Results

Discussion

The virtual format of the BEC ToT course was effective in improving knowledge as evidenced by the significant increase in test scores among participants. No difference in effectiveness was detected between the virtual ToT format and the in-person ToT format as measured by improvement in test scores and course feedback. Together, these findings suggest that a virtual ToT could be used to overcome the many barriers to in-person training and to more rapidly scale up BEC training.

Teaching the BEC ToT course virtually was feasible as evidenced by the engagement and completion rate of the course. While no major connectivity issues were encountered by participants or instructors, some individuals had to briefly restart a device or internet connection. Most curriculum components were preserved from the in-person format. The Zoom platform was useful in facilitating breakout rooms which are integrated into the software platform and allow for smaller group discussions. Breakout rooms were frequently utilized throughout the course for small group discussions, to practice teaching skills, and for skill simulations. Participants were asked to keep their video cameras turned-on throughout the entire course to ensure participant presence. Attendance was taken during each session and facilitators tracked participant presence during each session to verify participants met the requirements for course completion. The WhatsApp groups allowed for real time coordination and problem solving between facilitators and for clear and accurate directions to participants.

Responses in the course feedback forms indicate that the virtual ToT format was acceptable to participants. No significant difference was detected in responses related to the quality of the course except for with regards to course length. While both in-person and virtual course participants felt the course was the optimal length, in-person participant responses significantly trended to a feeling that the course was too long. Both courses were held over two days; the in-person ToT course was 12.5 hours long and the virtual course was 13.5 hours long. However, the in-person course was split to have only 2.5 hours on the first day and 10 hours on the second day while the virtual course was a more even split of 7 and 6.5 hours, which may account for this difference in responses. Participants felt that early access to course materials would have improved the experience. They also commented on the opening of the virtual course, which is likely related to the initial difficulties some had joining the online session. While these were resolved quickly and did not appear to affect the overall delivery of the course, future virtual courses should emphasize the initial logon experience. The breakout rooms and facilitator variety were seen as strengths, which supports the need for a low student to facilitator ratio and the importance of small group sessions.

There are pros and cons to both the in-person and the virtual formats. Positives for the in-person course include interactive teaching and learning while being in-person, ability to perform skills fully hands-on with standardized equipment, and the networking benefits of down time. These are balanced by the negatives of an in-person course, such as the financial costs of attendance in travel and conference fees and the need to schedule additional clinical time off.

The financial cost of travel, housing, and meals for in-person trainings can easily exceed $500 to $1500 for domestic and international conferences, respectively [17, 18]. These costs do not include conference and training fees and the financial burden of time away from work. While many academic institutions provide their employees with reimbursement for conference attendance, in-person conferences and trainings inevitably exclude participants who cannot afford these costs. In-person courses may disproportionately exclude potential participants from low- and middle-income countries, whom the BEC ToT seeks to engage [19].

A virtual course allows for lower costs as participants and facilitators do not need to travel, nor are classrooms or catering required. Scheduling is more flexible as participants can take less time off than would be required for domestic or international travel. Additionally, the course does not need to be attached to a large conference as was previously done to facilitate attendance. While the in-person course discussed here was particularly diverse, this was largely due to one-time scholarship funding for many of the international participants, and it is unlikely other in-person courses could maintain such diversity without ongoing funding. Virtual training can allow participants from regions where travel is logistically difficult, such as areas in a humanitarian crisis or with ongoing conflict, to attend.

In addition, virtual courses limit the impact of travel on the environment, and, in the time of the coronavirus pandemic, concerns of disease transmission. Virtual courses allow for safe continued training during the coronavirus pandemic, which can translate to future public health events that may require social distancing. By providing a virtual training format, this course can overcome many existing barriers, increase opportunities for the targeted providers to attend, and expedite scaling of the course globally.

There are downsides to virtual training that we identified, including time zone challenges for a live course delivered to people around the world and internet connectivity issues. The virtual ToT was delivered to participants in eight countries on four continents. Future virtual courses can mitigate some of the time zone differences through recorded sessions of selected content. Participants are required to obtain their own equipment to practice skills in a virtual course which places an additional burden on them and can make corrective action by facilitators more challenging. Networking, which is a benefit to attending courses such as this, can be more challenging when virtual but improved through the use of breakout room activities and ice-breaking sessions.

Conclusion

The findings here suggest that the virtual ToT course is at least as effective as the in-person format with the added benefit of overcoming barriers to training. Despite the limitations and challenges of conducting a virtual training, the overall success of the course supports plans for future virtual courses. Virtual training can rapidly increase the number of qualified BEC course instructors which will exponentially increase the number of health care providers who can be reached by future BEC courses. This will hopefully translate into improved access to care and better patient outcomes around the world. Future efforts should be aimed at evaluating higher-level educational outcomes from virtual training and to determine if the BEC course can also be delivered in a virtual or hybrid format.

Additional Files

The additional files for this article can be found as follows: