Implementation of Online Research Training and Mentorship for Sub-Saharan African Family Physicians

Background: To improve the delivery and reach of primary health care, a robust scientific foundation driven by research is needed. However, few family physicians conduct research, especially in sub-Saharan Africa. Early-career and trainee family physicians are a key part of the primary care research pipeline and have an expressed need for research training and mentorship. Objective: AfriWon Research Collaborative (ARC) was an online research training and mentorship pilot program whose objective was to increase research activity among participants from AfriWon Renaissance, the family physician young doctors’ movement of sub-Saharan Africa. Methods: ARC utilized a 10-module online curriculum, supported by peer and faculty e-mentorship, to guide participants through writing a research protocol. The feasibility, acceptability, and scalability of this program was evaluated via a mixed-methods RE-AIM-guided process evaluation using descriptive statistics and inductive/deductive thematic analysis. Findings: The pilot reached participants from Botswana, Democratic Republic of the Congo, Ghana, Nigeria and Sierra Leone and was adopted by mentors from 11 countries across three continents. Four of the 10 pilot participants completed a full research protocol by the end of the six-month core program. Seven out of the 10 participants, and nine out of the 15 mentors, planned to continue their mentorship relationships beyond the core program. The program helped instill a positive research culture in active participants. Some participants’ and mentors’ engagement with the ARC program was limited by confusion over mentorship structure and role, poor network connectivity, and personal life challenges. Conclusions: Online research training and mentorship for trainee and early-career family physicians in sub-Saharan Africa is feasible and acceptable to participants and mentors. Similar programs must pay careful attention to mentorship training and provide a flexible yet clearly organized structure for mentee-mentor engagement. Additional work is needed to determine optimal implementation strategies and ability to scale.


INTRODUCTION
For primary health care to achieve its promise of improving health, reducing cost, and decreasing inequity [1], primary care research capacity must be strengthened globally [2]. Primary care researchers provide needed information about planning, monitoring, and evaluating a country's health system [3]. Clinician-researchers in primary care, especially generalist Family Physicians (FPs), conduct research that is highly relevant to people, families, and communities [3,4].
Around the world, primary health care research is limited by a lack of skilled researchers, training opportunities, and resources, with these challenges being further amplified in sub-Saharan Africa [4,5]. Recommendations for increasing FP research activity include increasing enrollment in doctoral research programs, ensuring all family medicine departments set a research agenda, and building a stronger research culture by integrating service, learning, and research [3,4]. In sub-Saharan Africa, FP postgraduate training programs require a research project or thesis [6]. Exposure to research during training could build a foundation for implementing the recommendations above. However, many sub-Saharan African FP training programs have limited available local mentorship and research expertise [4,7]. Without proper training, mentorship, and sufficient allocated time, the research thesis can become a burdensome obligation that could contribute to already elevated levels of burnout and discourage further research activity [4,8,9].
Peer research mentorship has successfully filled gaps in mentorship within academic institutions where the small number of research faculty are over-burdened [10]. Health research training through distance learning is increasingly used as a way to expand access in low resource settings [11]. Research mentorship is also moving online, since e-mentorship can help overcome distance and time constraints [12]. AfriWon Renaissance, or AfriWon, is the young doctors movement within the African region of the World Organization of Family Doctors (WONCA) and is comprised of current postgraduate FP trainees and early-career FPs within five years of graduation [13]. Collaborative research and opportunities for mentorship are among the top reasons young FPs join groups like AfriWon [14].
The feasibility, acceptability, and scalability of an online research training program that provides peer and faculty e-mentorship to early-career FPs in sub-Saharan Africa, where internet connectivity and research mentorship experience tend to be limited, is unknown. In response to this need, members of this study partnered with AfriWon to develop and pilot test the AfriWon Research Collaborative (ARC) research training and mentorship program. This is the first of a series of papers on the ARC program. This paper describes the program components and reports on key implementation indicators. out of the 15 mentors, planned to continue their mentorship relationships beyond the core program. The program helped instill a positive research culture in active participants. Some participants' and mentors' engagement with the ARC program was limited by confusion over mentorship structure and role, poor network connectivity, and personal life challenges.

Conclusions:
Online research training and mentorship for trainee and early-career family physicians in sub-Saharan Africa is feasible and acceptable to participants and mentors. Similar programs must pay careful attention to mentorship training and provide a flexible yet clearly organized structure for mentee-mentor engagement. Additional work is needed to determine optimal implementation strategies and ability to scale.

PROGRAM DESCRIPTION
The ARC pilot program was designed to increase participants' research activity through an online curriculum (Figure 1), supported by an e-mentorship team (Figure 2). The six-month pilot ran from September 2019 to March 2020 and was limited to 10 participants. ARC's program development was guided by two logic models that mapped the development of needed inputs, activities, outputs, and anticipated outcomes (Supporting Figures S1.1, S1.2). Participants and mentors received a certificate of completion for meeting a priori criteria for engagement in the program (Figure 2).

Mentorship
The suggested ARC mentorship structure was tiered in order to support mentorship skill development. Peer research mentors provided direct mentorship to pilot participants. Faculty research mentors provided direct mentorship to the peer mentors and indirectly to the participants (see mentorship criteria in Box 1). Small groups of participants, peer mentors, and faculty mentors formed mentorship teams and were given guidelines for communication frequency (Figure 2).
WhatsApp (Facebook, Inc., Menlo Park, California, USA) groups for each of the five teams were created for this study. Additional WhatsApp groups were created for participants and also for peer mentors to provide the opportunity to support each other horizontally across the program. Peer mentors, who were expected to check-in with their participants at least once per module, were given a professional Zoom account to host mentorship meetings (Zoom Video Communications, San Jose, California, USA). During the program orientation, both peer and faculty mentors were trained in the "spirit" of motivational interviewing [16] as the foundation of ARC mentorship approach. In addition to providing research guidance, an explicit objective of ARC mentors was to build mentee's resiliency and combat burnout. Participants were encouraged to communicate with local research supervisors and other local collaborators where applicable.

Organization and management
The ARC pilot program was developed, implemented, and evaluated by the ARC working group, which met bi-weekly and included co-authors CMM, BBF, HK, KY, PA, LVW, and ER. One working group member served as a liaison to each ARC mentorship team to support implementation and evaluation at the team level. An ARC advisory group, composed of stakeholders from WONCA, AfriWon, FP training program faculty within sub-Saharan Africa, and Boston University advisors, including co-authors NAS and BWJ, met quarterly to provide expert guidance to the working group.

RECRUITMENT, SELECTION, AND MATCHING
Participants were recruited from the membership of AfriWon Renaissance and from the FP training program in Botswana.   email. Interested individuals first submitted a brief electronic form that evaluated inclusion criteria. Those who met inclusion criteria were sent an electronic application. The working group conducted a blinded rating of completed applications to select the participants. All peer mentors who completed the application were accepted. Faculty mentors who completed the application were accepted on a rolling basis until all positions were filled.
To match the mentorship teams, participants were provided with brief profiles of prospective peer and faculty mentors and asked to rank their choices. The working group created the mentorship teams based on a combination of participant preferences, research interests, research methods expertise, and location. Each team had at least one mentor from within sub-Saharan Africa to ensure participants were exposed to successful family medicine researchers from contexts similar to their own.

PROCESS EVALUATION
The ARC pilot was evaluated by a mixed-methods, prospective process evaluation that focused on monitoring and evaluation to allow for real-time adaptation. Individual, team, and program level implementation indicators were assessed to understand acceptability, feasibility, and scalability of the program.

Evaluation framework
The RE-AIM implementation framework was selected to guide the evaluation because it offers a systematic way to assess key design and implementation processes [17]. This process evaluation paper focuses on three of the RE-AIM domains: reach (R), adoption (A), and implementation (I).
Box 2 depicts how these domains have been adapted to the ARC pilot and the indicators used to measure each.

Reach
The number, demographic characteristics, and geographic spread of the PPs within AfriWon Renaissance; reasons for participating; linked to target logic model construct "Working group recruits PPs."

Adoption
The number, demographic characteristics, and geographic spread of mentors within regions of WONCA; reasons for adoption; linked to the nontarget logic model construct, "Working group recruits FRMs and PRMs"

Inclusion Criteria for Peer Research Mentors:
1) Be a trainee or early-career family physician who is within five years of qualification 2) Have at least one first-author, peer-reviewed research publication

Inclusion Criteria for Faculty Research Mentors:
1) Be a family physician who is at least five years post-qualification 2) Have at least five first-author, peer-reviewed publications Exclusion Criteria for All Groups (Self-determined): 1) Lack of proficiency in reading, writing and/or speaking the English language; or 2) Insufficient access to the internet to participate in training and/or mentorship activities  (Figure 2) was captured. Midpoint and final anonymous online surveys invited feedback from all participants and mentors. Program logs tracked recruitment, onboarding, program activities, adaptations, needed resources (financial, human, and time), and program observations by the working group. Data were extracted from program records and directly collected from participants, peer mentors, and faculty mentors. A table that outlines each data type, variables, and management strategy is provided as Supporting Table S2.

Quantitative data analysis
Descriptive statistics of demographic data, including medians, frequencies, and ranges were calculated using RStudio (Version 1.2.1335) [18]. Missing values were omitted from calculations. To approximate participant and mentor representativeness, their geographic locations were compared to the regions represented within AfriWon and WONCA organizations, respectively. Deliverable completion and mentorship check-ins were analyzed descriptively at the participant, team, and program levels and compared to the requirements to receive an ARC program completion certificate.

Qualitative data analysis
All qualitative data collected via open-ended survey responses, as summary documents of program logs, as well as from meeting minutes and recordings from feedback group discussions were uploaded into NVivo (Version 12.6.0) for analysis [19]. Data were analyzed by thematic analysis using a mixed inductive-deductive approach [20]. All data were read and checked for errors by the coding team, CM, LVW, BF and HK, during the familiarization phase. An a priori codebook was developed using program implementation indicators and relevant RE-AIM domains [17]. All coders conducted independent open-coding to refine codebook definitions and identify new codes; the codebook was finalized with agreement by all. All qualitative data were then independently coded by two coders, reviewed by team members to identify common themes, and findings were mapped to the RE-AIM framework. These themes were then reviewed by the full working group to finalize the core findings. Memoing was used by coders throughout the qualitative analysis process [21]. This allowed for reflexivity regarding how the evaluators' role in the program might influence its evaluation [22]. Triangulation of qualitative and quantitative data within and across each RE-AIM domain was used to increase validity of findings [23]. Additionally, member checking provided all participants and mentors the opportunity to comment on the findings [24].

ETHICAL CONSIDERATIONS
Ethical approval was obtained from the Boston University Medical Campus Institutional Review Board in the USA (Protocol H-38521) and Federal Medical Center Keffi Health Research Ethics Committee in Nigeria (Reference FMC/KF/HREC/299/19). ARC orientation sessions included a verbal description of evaluation procedures, risks, and benefits. After orientation, all participants and mentors signed an online consent form prior to initiating the program.

RESULTS
Quantitative followed by qualitative findings within each domain are presented here. Implementation domain results are reported at the individual, team, and program levels. Qualitative excerpts are shown in italics; those with quotation marks are direct quotes. Quotes are followed by the either the source of the aggregate data, or the code of the respondent, with pilot participants coded by "PP;" peer research mentors by "PRM;" and faculty research mentors by "FRM;" followed by a letter indicating the team, and a number indicating the unique individual.

REACH
Twenty-one prospective participants were recruited, among whom eighteen met inclusion criteria and were sent applications. Twelve completed the application process and ten participants were selected. Nine of the ten participants were in-training and half were female ( Table 1). The baseline self-reported burnout level varied, with one participant reporting "no burnout," six experiencing occasional stress and/or low energy and three reporting they are "definitely burning out." The ARC pilot reached participants from five countries across Western, Central, and Southern Africa, representing three of the four regions that make up AfriWon Renaissance.
The desire to increase one's research capacity and access research mentorship were among the most commonly reported reasons participants applied to ARC:

ADOPTION
Eleven individuals were interested in becoming peer mentors; five completed the full application and onboarding process and were accepted. The peer mentors were a median of 4 years out of training (range 4 to 10) and had a median of 3 prior first-author, peer-reviewed publications (range 1 to 34). Thirteen individuals were interested in becoming faculty mentors; 11 completed the full application and 10 were accepted. The faculty mentors were a median of 13.5 years out of training (range 6 to 32) and had a median of 8.5 prior first-author, peer-reviewed publications (range 2 to 47). The 15 peer and faculty mentors came from eleven different countries, representing adoption of the ARC intervention by FP mentors from four of the seven regional groups within WONCA (Table 1) [25].
Both peer and faculty mentors reported joining ARC to support FP research capacity in Africa: "I would like to join the ARC program to promote capacity-building for research in Africa -to support more research conducted and published by Africans in Africa." (FRMH1) Peer mentors reported wanting to collaborate globally on research, improve mentorship skills, and to support other earlycareer FPs and general practitioners (GPs): "As an early career GP researcher, I'm aware of the many barriers that exist that make it difficult for GPs to start doing research (…) I want to bring young researchers closer together and share the knowledge and experience that I have." (PRMH1) Faculty mentors noted wanting to "give back" through mentorship. Two faculty mentors specifically highlighted an interest in distance mentorship: "I feel I can help other students around the world with research activities since I have done this before remotely" (FRML1); "I'm interested in joining the ARC program (…) to gain experience in distance mentorship." (FRMX1)

Individual level
Four out of 10 pilot participants submitted the final deliverable, a full research protocol draft. One participant got as far as the ninth deliverable, selection of a target conference and journals. In all, eight participants submitted at least one deliverable; two submitted no deliverables. Research topics explored in these deliverables ranged from patient-centered research on hypertension and postpartum depression to health systems research on the use of electronic medical records. Participants reported a wide range of time spent on ARC activities, most falling between three to six hours per week.

Team level
Mentorship implementation by team was mixed. Figure 3 shows reported check-ins on each team. One team had high levels of communication between all members of the mentorship team. Other teams had varying levels of communication between participants and peer mentors, which typically mirrored participants' deliverable completion rates. Significantly less than expected communication between peer mentors and faculty mentors was observed.
The team with highest mentorship communication rates also reported implementing the suggested tiered mentorship structure (Figure 2) and three full team meetings. Another team implemented a tiered structure initially, but when the peer mentor had to stop at Module 8 for personal reasons, one of the faculty mentors became the primary mentor to the two participants. A third team reported using a more direct 1:1 mentoring relationship between each mentor and the participant. Two teams had little or no faculty mentor involvement in the mentorship structure.
As the key themes shown in Figure 3 demonstrate, the mentorship relationships that formed during ARC were supportive to both participants and peer mentors alike. One peer mentor reported supporting his participant through a loss: "It was a down moment for everyone in the hospital. It puts him off psychologically for some time. Eventually with motivational interviewing, he was  able to pick it up." (PRMU1) While another peer mentor reported being guided in his mentorship approach by his faculty mentor:

"[The faculty mentor] was able to answer any question I had. Like if you have two different participants, and one is behind, do you give more effort to the one who is behind or to both? I had a lot of questions like that (…) he gave me some really good advice." (PRMH1)
The lack of involvement of the faculty mentors on some of the mentorship teams was noted by participants and faculty mentors alike: "Interaction with faculty mentors was too limited, positive/ negative feedback would be appreciated with each deliverable." (Anonymous PP, Final Evaluation); "I commented on a few proposals of my mentee and phoned with him and the mentor in [country] and with the coordinator. But I feel very insufficient over my contribution." (Anonymous FRM, Midway Feedback)

Program level
Across the entire program, participants completed an average of 5.5 of 10 curriculum deliverables and checked in with their peer mentors an average of 5.8 times over the 10 module time periods. Overall, four out of 10 participants, three out of five peer mentors, and three of 10 faculty mentors met the completion certificate requirements over the six months. Seven participants, four peer mentors, and eight faculty mentors expressed interest in continuing their mentorship relationships beyond the core program.
There was limited cross-program engagement amongst participants or peer mentors despite setting up the dedicated participant and peer mentor WhatsApp groups and encouraging the use of the "comments" function in Google Classroom. During the program feedback group discussion, two peer mentors noted that they enjoyed hearing about others' experiences and wished they had connected sooner. One participant commented that she was nervous to share her work and ask questions in the participant group forum:

"One of the modules we had to write down our research questions [in the WhatsApp group]. I was very nervous (…) [about] the idea of having everybody reading and being like: 'What was she thinking?' (…) I think about three or so of us only posted our research questions, but I was very grateful because I was corrected (…) going forward, I think, as participants we do need to discuss and share ideas." (PPH3)
Adaptations were made during program implementation. Mentorship teams were originally designed to include two participants, peer mentors, and faculty mentors each, however, only five peer mentors joined the program and thus the mentorship structure was modified to include just one peer mentor per team (Figure 2). The initial anticipation was that the program would be completed in four months, with a mix of one and two-week modules. However, after module three, received feedback showed that many participants were struggling with the pace of the program. Therefore, "catch-up" periods seen in Figure 1 were added and all remaining modules were extended to two weeks, increasing the total program length to six months. It was initially a completion certificate requirement to participate in mentorship full team meetings. However, some teams experienced communication and scheduling difficulties since members came from numerous time-zones and some had inconsistent internet connectivity. Thus, this requirement was removed and teams were encouraged to meet in smaller groups or use email and/or WhatsApp for asynchronous communication as needed. Adaptability was highlighted as a strength of the pilot program: The program was adaptable which helped a lot; (it) was important to give all the participants leeway. (FRMH1) Peer mentors reported investing an average of one to two hours per week in ARC activities. Two faculty mentors who were active in the program reported spending an average of one hour per week. Both felt this was appropriate, with FRMH1 noting faculty might "run-away scared" if asked to do more. Direct costs to provide an e-book to all participants and peer mentors totaled $450 USD. Out-of-pocket costs incurred by participants and mentors were not directly captured. However, network connectivity was a commonly cited program barrier and one individual reported purchasing a new modem to improve internet access. An eight-person working group, including two master of public health practicum students who dedicated over 250 hours each, provided the human resources needed to develop, implement, and evaluate the ARC program.

KEY FINDINGS
Results demonstrate that an online research training and mentorship program for early-career FPs in sub-Saharan Africa is both feasible and acceptable. Participants completed more than half of the total curriculum deliverables and four completed full research protocols during the six-month core program. Mentorship structures were established on all five teams with varying degrees and strategies of implementation. The most active mentors reported the time-commitment was still manageable and the overall financial cost of the pilot was low. Findings from the reach and adoption domains are promising for scalability. ARC reached participants across five African countries who were mostly postgraduate FP trainees. Participants joined primarily due to the desire to access research mentorship that is limited in their setting, which prior reports suggest is a widespread concern for FP trainees [8,9]. Adoption of this pilot by mentors from 11 countries across three continents demonstrates that FP researchers worldwide are willing and able to engage in distance mentorship to support trainees. These findings are especially relevant in the context of the covid-19 pandemic, which is pushing medical and postgraduate education online like never before [26,27].

RESEARCH CULTURE DEVELOPMENT
One of the most promising successes of the pilot was the evidence of a developing primary health care research culture among active participants and the commitment to continue research activity and mentorship beyond the core program:

"Every day in clinical practice, all of a sudden I'm finding myself asking questions: What can I do to try and improve this? And for me it's a sign that going forward, research will definitely be part of what I do." (PPH3)
A program completion celebration was held to virtually present ARC completion certificates and acknowledge the hard work by all, responsive to suggestions for building research culture by Mash, et al [4]. A six-month ARC post-program period, comprised of monthly online research work-inprogress meetings is ongoing and participants and mentors who reach requirements over that period will also earn a certificate.

RECRUITMENT CHALLENGES
It was difficult to recruit the intended number of peer mentors for an equal ratio of mentors and mentees; it was also a challenge to accurately assign mentors as a peer mentor or faculty mentor. One possible reason for low peer mentor adoption is the time-pressure experienced by early career FP researchers. A near-peer mentorship program in Uganda, in which masters students mentored undergraduate students, was limited by the near-peers' heavy workload and conflicting priorities [28]. One peer mentor (anonymous) in our program noted: "I didn't feel competent in my mentorship skills during the first period of the program." A lack of confidence in mentoring others may have been another barrier to peer mentor recruitment. In addition, one of the mentors was mis-assigned as a peer mentor when she met the faculty mentor criteria by both training completion year and number of first-author publications. Also, two of the faculty mentors did not reach the firstauthor publication criteria. These oversights occurred because a method to systematically request and receive CVs from all mentors at the beginning of the program to cross-check self-report on application surveys was not in place. The resulting range of research experience among those assigned to the peer and faculty mentor role may have impacted team mentorship dynamics.

MENTORSHIP STRUCTURE
The tiered mentorship team structure was designed to capitalize on peer-mentorships, which can flatten hierarchies [29] and extend limited faculty research capacity [10]. Peer relationships [30], and specifically peer-to-peer mentoring [31], have been cited as strategies to protect against burnout, which was an explicit goal of this program. This structure worked well when peer mentors were active in outreach to not only their peer mentees, but also their faculty mentors. However, if the peer mentor did not initiate and maintain the relationship with the faculty mentors, some faculty mentors ended up being left out of the team mentorship process entirely. On some teams this was partly due to confusion over responsibility for outreach: "There should be an indication for who calls the meeting between the peer mentors and faculty mentors." (Anonymous PRM, Midway Evaluation).
Additionally, the participants were the target of the ARC program and thus the primary focus of peer mentor and participant relationship was clear: to check in each module and support the participant's research protocol development. The primary focus of the peer mentor and faculty mentor relationship, however, was a broader goal of improving peer mentor's mentorship skills. While peer mentors were encouraged at orientation to set learning goals and complete a mentorship reflection tool [32], it was not required. Mentorship contracts, which can clarify expectation and objectives [33], were also suggested but not required. Cole et al. highlight the importance of balancing structure and flexibility in global research mentorship programs [34]. The ARC pilot was lauded for its flexibility in terms of curriculum timeline, especially given participants and mentors alike experienced personal and professional challenges that impacted available time and energy. However, the mentorship portion required more structure and clarity, especially between peer mentors and faculty mentors: Suggestions for improvement in the future include (…) having more defined guidelines for team functioning, when to reach out, how much to reach out, how much faculty mentor feedback is desired or "enough." (FRMU2) Participants and mentors largely limited mentorship interactions to their teams and did not utilize platforms to engage horizontally across the program. An evaluation of an online learning platform for postgraduate trainees in Germany reported that a perceived lack of involvement by other users can become a negative feedback loop [35]. Normalizing the utilization of cross-program structures for communication should be established at the beginning of the program. Creating opportunities for early and ongoing rapport-building both within and across teams may help participants and mentors feel these are safe spaces [36] where it is "okay to demonstrate that you don't know something." (FRMH1)

FUTURE DIRECTIONS
The pilot study suggests a number of strategies that may improve upon reach, adoption, and implementation of online research training and mentorship programs for early career physicians in low-resource settings. Directly linking such programs to doctoral training or junior faculty development programs may not only provide increased opportunities for peer mentor recruitment, but also enhance the pipeline effect of these programs. The mentees are provided with an example to follow if they decide to pursue further training, and the early-career peer mentor gains skills in mentorship, which is equally crucial to building the pipeline [36]. The ARC pilot only provided mentorship training during orientation, however, results suggest that providing additional formal mentorship training sessions may have been of benefit. These results affirm calls for a greater focus on dedicated research mentorship training [7,37]. The curriculum itself was generally well received, however some participants requested more focus on research methodology. Programs such as ours should consider providing funds to offset costs to participants with limited internet access and offering training materials and mentors proficient in languages other than English to avoid exacerbating existing inequities in access to research training opportunities [38].

LIMITATIONS
The small size of this pilot, which was restricted by available resources, limits interpretation of our findings. Our recruitment methods did not allow for analysis of non-participants and non-adopters, a key component of the reach and adoption domains [17]. Our mentorship team evaluation relied largely on reports by the peer mentors, and may have missed faculty mentor mentorship activity as a result. Additionally, since the authors were involved in development, implementation, and evaluation of the program, the findings may be subject to both researcher bias and response bias from participants and mentors. Using mixed-methods and multiple sources of data, some of which were anonymous, to triangulate our findings helps to mitigate these limitations. Furthermore, authors' use of reflexive memoing, regular debriefing, and the use of member checking of results help to ensure the validity of our findings.

CONCLUSIONS
It is both feasible and acceptable to conduct online research training and mentorship for trainee and early career FPs in sub-Saharan Africa. Similar programs must pay careful attention to mentorship training and provide a flexible yet clearly organized structure for mentee-mentor engagement. Future work is needed to determine optimal implementation strategies and to scale up efforts to develop primary health care research capacity in sub-Saharan Africa.  x Non-target refers to programmatic inputs, including ARC mentors.