Training advisory committee

We convened a training advisory committee (TAC) made up of five individuals with experience in supporting efforts in strengthening mathematical modeling and analytical capacities in Africa. The TAC was set up to provide oversight to the training team. Their role included advising on the curriculum and course structure, recruitment strategies, and facilitating dissemination of deliverables from the training.

Training goals

Our overall goals were to strengthen the mathematical modeling capacity to enhance their use in decision-making and effective communication of modeling outputs to policymakers. Specifically, we hoped for trainees to:

• – Understand the fundamental concepts of infectious disease dynamics and mathematical modeling,
• – Know the types of models and their use in public health for different infectious diseases,
• – Identify research questions suitable for mathematical modeling,
• – Critically review infectious disease models and develop them to assess the effectiveness of pharmaceutical and nonpharmaceutical interventions,
• – Build mathematical models for assessing the effectiveness of pharmaceutical and nonpharmaceutical interventions, and
• – Prepare policy briefs and abstracts for conference submissions as main deliverables. We did not include manuscripts/publications as a primary output given the limited time for the training and the focus of the training on using mathematical models to support policymaking.

Course structure and curriculum overview

We provided four weeks of in-person training at the UGHE campus in Northern Rwanda. These sessions were divided into two two-week sessions with three months of practicum in between sessions. The first in-person session occurred March 13–24, 2023, and the second in-person session occurred July 24–August 2, 2023. This course was longer than the three weeks of in-person training for the IORT because the content was more challenging.

The course was divided into four weeks: a) Week One: Introduction to Mathematical Modeling b) Week Two: Building a Mathematical Model, c) Week Three: Incorporating Complexity into Mathematical Models, and d) Week Four: Disseminating and Communicating Model Findings (Table 2).

The in-person sessions included lectures focusing on concepts of infectious disease and modeling methods. We included several activities each day for trainees to apply the concepts to their projects to reinforce the knowledge and skills gained. Group and panel discussions and plenary sessions offered trainees the opportunity to learn from and engage with other experts in the field. We encouraged peer learning by including several slots for teams to present their project’s progress to their peers and receive feedback. During practicum periods, participants received hands-on and intensive mentorship from the training team.

Application process

The call for applications was advertised within professional networks and social media handles like LinkedIn, Twitter, and Facebook. A complete application included questions about background in infectious diseases, research areas of interest, the relevance of training to their career, a letter of support from a supervisor, and curriculum vitae (see appendix). Applicants were encouraged to apply in pairs, with one applicant from a quantitative background and the other with a program/policy background. Following the advice from the TAC, individual applications were welcomed, and those who applied as individuals were eventually paired based on common research interests and backgrounds. The application docket required applicants to select two topics from a table of suggested infectious disease and intervention areas based on recommendations from the TAC and the research expertise of the core training team.

Participant selection process

Eligible trainees were public health professionals working in infectious disease programs or research in Africa, with an interest in using mathematical models and a good command of the English language. We targeted this group to provide professional development, foster inter-country learning, and improve quantitative skills in public health practice. As this was the first cohort of the training, and with few facilitators, the training slots were limited to ten individuals divided into five teams.

The selection committee, including the core training team and the TAC, reviewed applications in three stages. We first assessed the completeness based on application instructions and the timing of submission. The second stage included assessing eligibility based on the quality of the application and professional background to identify 20 finalists. The TAC was consulted to select ten successful applicants based on the strength of the application and organizational priorities and made final decisions also ensuring a range of representation from different countries and by gender.

Background of applicants and selected trainees

We received 875 applications, 650 (74%) of which were complete and screened. Most of the complete applications were from Kenya (n = 116, 18%), Nigeria (n = 103, 16%), and Uganda (n = 75, 12%) (Figure 1). Ten participants— four females and six males—were selected. Six trainees were from Rwanda, two from Uganda, and two from Kenya (Table 3). Most trainees (7/10) had master’s degrees, and 9/10 were from government institutions. Although our goal was to have each team have a person with a quantitative background and the other with a policy/program background, one team had both members with the same background. However, this did not alter the dynamics of the team, as one member had more quantitative training.

Projects of trainees

The five projects the trainees worked on were: i) the impact of vaccination and surveillance on Ebola in Rwanda, ii) the impact of preventive therapy on tuberculosis incidence in Nakuru county, Kenya, iii) the role of enhanced surveillance and vaccination on Ebola outcomes in Mubende District, Uganda, iv) impact of Paxlovid, an antiviral drug, on long COVID and COVID-19 mortality in Rwanda, and v) impact of direct-acting antivirals on Hepatitis C elimination targets in Rwanda.

Training evaluation

The application packet included questions about research background, how skills obtained from the training would impact their career, and commitment to training. Trainees’ expectations and experience with the training and feedback on the content were assessed using a survey administered at the course’s beginning and at the end of each week to allow trainers to modify subsequent sessions. We also solicited feedback on the structure and logistics at the end of the course using a survey and trainer-trainee meetings with each team. The surveys were semi-structured and included multiple-choice, linear scales, and open-ended questions (see appendix). We calculated the frequency of responses and median scores of linear scales, generated bar charts, and open-ended questions were analyzed thematically.

Five trainees (50%) had previously taken a course in infectious disease epidemiology, but none had ever taken a course in mathematical modeling. When asked about the relevance of the training to their field of work, 70% deemed the training to be “extremely relevant” (Table 3). We assessed the extent to which mathematical modeling competency was attained using the indicators in Figure 2. Six trainees agreed they could confidently review and critique mathematical modeling studies and five strongly agreed they could confidently design their compartmental models for future projects at the end of the training. Moreover, four strongly agreed that they could translate mathematical modeling findings into policy. For the overall quality of the training, on a scale of 1–10 (1 = very poor, 10 = excellent), the median score was 9 (see Appendix). All trainees completed the course. All trainees indicated they would recommend the course to their colleagues.

Trainees were asked to rate the clarity, pace, and content of lectures at the end of each training week (Figure 3). Six participants indicated the lectures were very clear in weeks one, three, and four. The highest number of participants (n = 5) indicated that the pace of the lectures was very fast in week four. None of the trainees indicated that the pace was slow or very slow in the four weeks of training. Most trainees rated the content of lectures as very complex in week 1 (n = 7), followed by week 3 (n = 6). The quality of class activities was also rated on a similar scale (Figure 4). Six trainees rated the clarity of activities as “very clear” in weeks one and three. The rating of the time allocated to class activities being short was observed in weeks one (n = 7) and two (n = 4). Three trainees rated the content of activities as very complex in weeks one, three, and four, while the majority (7/10) assigned a rating of complex in week two.

There are several limitations to our training evaluation. First, we have not yet conducted a longitudinal follow-up to assess the application of the skills gained in their professional roles and if policy briefs impacted decision-making, but we anticipate doing so in the future. Second, while these surveys were anonymous, given the small number of participants, the responses may be subject to desirability bias. For that reason, we have paid attention both to the scaled responses and the open-ended replies. Third, in the goal of keeping the surveys brief, not all possible areas were assessed. For example, we did not ask about the trainees’ confidence that the skills gained would be useful in their work settings.

Dissemination

The training deliverables were policy briefs and abstracts distributed among relevant stakeholders. Trainees presented their projects to guests, including policymakers, program managers, and researchers, at the graduation ceremony, where trainees were presented with certificates. Four teams submitted their abstracts to the 2023 edition of the Conference on Public Health in Africa, and two projects are being supported for publication.

Course cost

All training-related cost was covered for all sessions. The scholarship included transportation to the training location, meals, accommodation, and flights for non-Rwanda trainees. Trainees also received a stipend of 100 USD per person at the end of each session to help cover costs incurred for the training, including public transport to airports for non-Rwanda trainees and transport to pick-up locations. Trainee teams whose abstracts are accepted for a conference presentation would receive financial support to attend and present their projects. Berkeley Madonna team also supported the training in kind by providing all trainees, two of the UGHE-affiliated training advisors with perpetual licenses, and four of the trainers with one-year licenses. The trainees also received approval from their supervisors to offer their time in-kind to participate in the training.

Strengths and facilitators of success

A variety of factors contributed to the success and strengths of the training, where success is defined as achieving the deliverables by the course end date and 100% course completion rate. A core feature of this program, and critical for trainees’ success, is the intensive mentorship provided, the hands-on approach, and the regular communication between trainees and trainers, especially during the practicum. There were many practical sessions, which provided trainees the space to implement the concepts learned and to receive feedback. We created a WhatsApp group to facilitate easier communication between trainers and trainees, as well as knowledge sharing and networking among trainees. Additionally, during in-person sessions, trainers were available for consultations at the end of each training day. Trainees used these consultations to seek clarity on concepts taught or on project-specific challenges encountered. Selecting trainees from three countries offered cross-country networking opportunities and greatly enriched the diversity of perspectives in class discussions, helping trainees to learn from others’ systems. Moreover, the variety in the infectious disease expertise of trainees (tuberculosis, COVID-19, Ebola, and Hepatitis C) further contributed to the diversity of perspectives.

In addition, instrumental support from trainees’ institutions was important. The application packet required applicants to confirm that their supervisors supported their applications and to submit a letter of support from the employers. Applicants’ eligibility therefore depended on supervisors and employers providing approval for trainees to attend the training. Requiring trainees to develop their models for their institutions’ priority questions served as a motivation for employers to approve trainees’ time away to attend training. We also received institutional support from the Rwanda Biomedical Center, which was the home institution for five trainees.

Another facilitator of success was the trainees’ commitment, demonstrated by completing milestones typically on time and earning certificates of completion amidst competing commitments from their full-time work. To ensure fewer distractions for trainees, especially those based in Rwanda, UGHE’s Butaro campus was chosen as the training location, as the campus is located in a rural community 80 miles north of Rwanda’s capital Kigali. Trainees rated their satisfaction with the location of the training with a median score of 6.5, on a scale of 1–10 (1 = not satisfied, 10 = very satisfied). The score may be due to transportation to the location, which is remote and accessed through unpaved roads, as transportation also received a median score of 6.5 (See Appendix).

The pre-training orientation, held two months before the first training session, was also integral for success as it helped prepare trainees for the intensive training period by providing foundational information on the course structure, expected outcomes and software tools to be employed. The orientation was also an opportunity to propose topics of interest and receive feedback on topic feasibility from trainers. Finally, the weekly evaluation of the clarity, pace, and content of the training, helped improve the training program as it was implemented, continually structuring delivery to better suit trainees’ needs.

Challenges

Here we highlight some challenges in the course’s planning and implementation to inform other mathematical modeling training programs. First, the demand for the course, demonstrated through the volume of applications (875 applications), far exceeded the number anticipated by the training team and the number of slots available. Consequently, processing these applications took longer than planned and required substantial effort from two trainers. Additional support for program coordination might have alleviated this issue. Second, given that most of the content was developed originally for this training, the time and effort required to prepare the course content was substantial. We were able to draw from previous teaching experience and courses of our training team, but we still spent considerable time developing content appropriate for these trainees, that had practical sessions integrated, and that applied concepts to the trainees’ project. One benefit is that we can leverage this first offering’s materials for future courses. However, even these materials need to be tailored to specific trainee populations and projects and doing so is non-trivial.

Furthermore, even with planning and dedicated trainer resources, providing intensive mentorship to meet trainees’ interests and demands was challenging. Mentorship required mentors to be flexible with their schedules to ensure they were readily available to respond to participants’ questions and troubleshoot errors in their codes. The training could also have benefited from full-time in-country mentorship support, especially during the practicum. We attempted to address the issue of in-country support by having training mentors based in Rwanda a week before and after each session to provide in-person mentorship. However, the non-Rwanda trainees did not benefit from these in-Rwanda resources.

Third, not all five teams could be supported to translate their findings to publication, given the limited bandwidth of the trainers and the limited availability of some trainees to invest the time required to realize a full research article. For these reasons, only two projects are currently being developed for publication. Lastly, using the 2-2 training model meant many new concepts were introduced to trainees over a short period and trainees had limited time to apply these concepts to their projects during the training, making the training very intensive. Trainees expressed a preference for a 2-2-2 training model, adding a third two-week session to be able to spread out the concepts over a longer period (see Appendix). This would offer many advantages but with the obvious consequence of needing additional resources to implement and the bigger commitment for trainees to be absent from their work and family obligations.

Recommendations

Based on our experience with planning and implementing the UGHE/Harvard Mathematical Modeling for Infectious Diseases Planning Course, we recommend the following as avenues to improve and scale infectious disease modeling training in Africa.

Funding

The high demand for infectious disease modeling training should be matched with more funding opportunities to support this kind of training. Notably, there has recently been an increasing number of calls from international non-governmental organizations to fund these kinds of training endeavors, although these funding calls are spontaneous and irregular. Making training more sustainable demands that funding streams are continuous and established, such as that which may come from the national/local government budget or a national/local health agency [18, 19]. When trainings are locally sponsored, it is more likely that training goals will serve to answer infectious disease questions of local priority and thus would be more relevant. We acknowledge that in low- or middle-income countries, national/local budgets may be limited [20, 21]. We, therefore, encourage funding partnerships between national/local governments and non-governmental organizations to achieve the goal of a continuous funding stream while ensuring that funded training tackles questions of mutual interest [22]. We also advise funders to consider funding programs that are smaller, longer, and with more intensive mentorship over models, seemingly prioritized by funders, that train more people on superficial levels.

In-country expertise

Improving the sustainability of infectious disease modeling training also requires more country-based faculty training experts [13]. This will reduce the cost of sponsoring international trainers and ease collaborations between national health policymakers and modeling researchers, given that locally-based researchers have a better context and understanding of local infectious disease issues. In-country faculty, ideally in a local university or research institute, will also provide more opportunities for longer-term training, such as at the master’s or PhD level. Moreover, some of the trainees indicated their interest in further training in infectious disease modeling. We acknowledge however that Africa-based faculty-level expertise in infectious disease modeling is currently limited; however, if trainings such as the one we have organized are more frequent, the situation can be expected to improve in a few years [11].

Geography

We recommend that as much as possible, future trainings of this kind target individuals from across Africa [23]. We limited our participant pool to East Africa, mainly because of resource limitations; however, the interest (Figure 1) and need [11] for infectious disease modeling are not limited to a particular geographical region of the continent.

Orientation

We recommend that future trainings include an orientation session to offer trainees conversance with course structure and expectations before the training begins [24]. There should be enough time between the orientation and the start of the training to allow trainees to adjust to training expectations and to allow trainers to adapt content to trainees’ needs as may be identified during the orientation.

Training time

We recommend that future trainings consider the need to balance the time allocated to teaching concepts and having hands-on sessions where trainees apply the concepts learned. Future training should consider an extended model, for example, having three in-person sessions with longer practicums in between.