Participants characteristics
In the initial cohort of the course, 88 study coordinators, 56 (64%) females, hailing from 19 countries across Africa, were enrolled and completed the pre-course assessment questionnaire. Regarding their clinical trials experience, 85 (97%) of the participants had prior experience in clinical trial roles, serving as study coordinators, co-investigators, or research assistants. Moreover, 57 (65%) were actively employed as study coordinators during the training period. Out of the total number of participants, 71 (81%) completed the course, with 69 (97%) of them also finishing the post-course assessment.
Findings from pre-post test evaluations
The Wilcoxon Signed-Ranks test revealed significant post-training improvements in participants' scores across the four fundamental areas of competence for clinical trials. For conducting clinical trials, post-training scores (mean rank = 31.78) significantly exceeded pre-training scores (mean rank = 8.0) (Z = -6.682, p < 0.001). Similarly, in managing clinical trials, post-training scores (mean rank = 36.89) surpassed pre-training scores (mean rank = 9.83) (Z = -6.813, p <0.001). In designing clinical trials, post-training scores (mean rank = 36.27) were notably higher than pre-training scores (mean rank = 12.20) (Z = 6.799, p <0.001). Likewise, for reporting outputs of clinical trials, post-training scores (mean rank = 16.94) exceeded pre-training scores (mean rank = 3) (Z = -4.897, p < 0.001) (Table 1).
Table 1: Difference in the score of the participants in the measures of the four dimensions of competence (conduct, manage, design & report) in clinical trials before and after the training
Dimension of clinical trial competency
|
Rank
|
n
|
Mean rank
|
Sum of ranks
|
Z-score
|
P-value
|
Conduct CT, post
Conduct CT, pre
|
Negative
|
2
|
8.00
|
16
|
-6.682
|
.001
|
Positive
|
59
|
31.8
|
1875
|
Ties
|
8
|
|
|
Total
|
69
|
|
|
Manage CT, post
Manage CT, pre
|
Negative
|
6
|
9.8
|
59
|
-6.813
|
.001
|
Positive
|
62
|
36.9
|
2287
|
Ties
|
1
|
|
|
Total
|
69
|
|
|
Design CT, post
Design CT, pre
|
Negative
|
5
|
12.2
|
61
|
-6.799
|
.001
|
Positive
|
63
|
36.3
|
2285
|
Ties
|
1
|
|
|
Total
|
69
|
|
|
Report CT, post
Report CT, pre
|
Negative
|
1
|
3
|
3
|
-4.897
|
.001
|
Positive
|
31
|
16.9
|
525
|
Ties
|
7
|
|
|
Total
|
39
|
|
|
Post: post-training; pre: pre-training; CT: clinical trial; significance at p < 0.05
Findings from end-course qualitative survey
Participants’ responses to the four open-ended questions are summarized as described below.
The most beneficial aspects of the course
The responses to this question are summarized in four themes:
- Courses content
In this context, participants noted that the course comprehensively addressed nearly all essential aspects of a clinical trial. They found the study materials to be abundant, offering detailed information on clinical trials along with supplementary readings and references. The content was deemed highly informative, covering everything needed from the inception to the conclusion of a clinical trial. Participants felt it provided a robust foundation to build upon. The objectives were well-defined, and the organization and relevance of the course content were praised for their direct applicability to improving their work.
- Course design
Participants expressed appreciation for various aspects of the training design. They found the distribution of course content over time to be appropriate and accessible, with resources related to clinical trials readily available without restrictions. The shared documents and templates used were directly applicable to clinical trial practices, enhancing the realism of the training. Reading materials, checklists, and risk management plans were particularly lauded for their practicality. VoiceThreads explanations, questions, and discussions were deemed helpful.
Moreover, the availability of study materials and direct communication with tutors, coupled with flexible timing, enabled participants to progress at their own pace without sacrificing quality. The flexibility to undertake the training at their convenience was highly valued. The online format, featuring virtual interaction with both the audience and tutors, was described as remarkable.
Additionally, collaborative activities such as risk management group work and tutorial sessions provided opportunities for active participation and clarification of doubts. Discussions with foreign partners, group forums, and exposure to new methods like Moodle were cited as beneficial experiences. The interactive and practical nature of the sessions allowed participants to express themselves and learn from individuals with diverse backgrounds, facilitating the sharing of ideas. The learning platform's accessibility from anywhere and at any time was commended, as was the ability to listen to recordings offline and meet deadlines. Despite being virtual, the sessions fostered an interactive environment where constructive feedback was provided, enhancing the overall learning experience.
- Trainers competency
In general, trainers were regarded as highly knowledgeable about the course, drawing from significant lived experience in clinical trials. This expertise greatly enriched the learning experience for trainees. Additionally, the tutors were recognized for their subject matter expertise, injecting vitality into the course content beyond what textbooks could offer. They were noted for their approachability and accessibility, with their lectures skillfully presenting course material. Their receptiveness, follow-up, and encouragement were appreciated, as they often shared practical experiences and cases to enhance understanding. Some guest speakers were also praised for their contributions.
The availability and effective communication of trainers during virtual training sessions were consistently commended. Coordinators were also lauded for their deep understanding of the subject matter, enabling them to provide insightful examples.
- Practicality
Participants expressed appreciation for the practicality of the course in relation to their clinical trials. They found the content to be directly relevant to their everyday work duties and activities. The course was deemed practical, offering real-life experiences and insights into handling and managing clinical trials. Regarding its practicality, one participant articulated:
“It was timely on my part, as I am considering a new career path into research monitoring and CRO management. It encompassed project management, research skills, etc. Further to that, it has helped a lot in my day-to-day execution of study coordination roles at the facility especially on the risk assessment and prioritization (PM52).”
The satisfaction with the entire approach, encompassing content, trainers, and methodology, is aptly captured in the following quote from another participant:
“The best thing about this course for me was the easy accessibility to study materials and availability of experts in the field to explain them further during tutorials and in comment sections on Voice threads” (YN70).”
Another participant stated:
“The best thing about the course was the discussion forums where we had to discuss the topics. This part enables me to express my thoughts and understanding about the topics under discussion. It also afforded me the opportunity to know what my colleagues’ thoughts were on issues being discussed. lastly, it allows me to see how diverse people's views are regarding a given matter.”
As a concluding remark, the overall evaluation of the training can be encapsulated most effectively with the following quote from one of the participants:
“The best thing about this course was that not only content was good but also moderators and other participants were committed to sharing experiences, and much more. Being a part of this course was a great opportunity for me and I would love to get another chance if there is another opportunity (PAB34).”
Application in participants' current roles
Participants' responses for this section are organized into three themes: improved understanding, improved practice, and improved skill of project management.
- Improved understanding
Participants highlighted that the training facilitated their acquisition of techniques in clinical trials, leading to revisions and enhancements of their existing documents and manuals. Moreover, it aided in refining their ability to teach clinical research methods, particularly focusing on clinical trials. Some participants noted a deepened understanding of specific topics such as pharmacovigilance and safety reporting. They also mentioned an improved comprehension of the organizational aspects of clinical trials. Specifically, participants cited increased knowledge in operations, conducting, and reporting clinical trials, as well as in preparing Investigator ISF. Overall, these improvements contributed to enhancing confidence in their clinical trial-related roles.
- Improved clinical practice
Several participants noted that the training facilitated improvements in their daily clinical trial responsibilities. It provided valuable input for coordinating ongoing studies and proved particularly beneficial for those in roles such as clinical coordinator, allowing them to integrate performance indicators into their work. This enabled them to prepare comprehensive checklists, including quality management checklists, prior to clinical trials, enhancing performance monitoring through key performance indicators. Additionally, the training assisted in the development of SOPs. Participants highlighted an increase in their adherence to clinical trial procedures and improvement in team management. Furthermore, they found that the training equipped them with informed knowledge to support customary practices within clinical trials.
- Improved project management skills
Several participants found the training beneficial for enhancing quality management, data management, and regulatory activities within their clinical trial work. They noted that the training aided in improving documentation practices and streamlining work processes and scheduling. Moreover, participants mentioned that the training equipped them with skills in financial management and risk management, including identifying and implementing mitigating measures for potential risks. Additionally, they learned how to effectively close out a project. Other skills gained included improved preparation for audits and inspections, as well as fostering and maintaining positive relationships with both internal and external partners.
Dissemination of course contents within their institutions
This question reveals two distinct themes, which are identified and presented as follows:
- To teach, mentor, or share experience
Some participants utilized the course to instruct clinical research methods and ethics to resident students, while others employed it to mentor doctors and junior colleagues in coordinating clinical trials. They provided guidance to staff on maintaining quality standards in clinical trial conduct and shared their course experiences with their team members. In specific instances, they trained peers and colleagues on risk management plans. Furthermore, they shared course documents with individuals engaged in similar practices and distributed modules and templates to colleagues. Participants also indicated using the course material as a training tool for other members. Additionally, some organized their own capacity-building training sessions, which included retraining lab staff, onboarding new personnel, and instructing fellow workers involved in trials they coordinated. Many participants mentioned recommending the course to others.
- Practical support
In addition to teaching and training, participants also emphasized providing direct support in practice. For instance, they assisted their sites in reporting, updating, and distributing data, as well as re-evaluating their site key performance indicators. They trained their assistants in managing ISFs and implemented changes in documentation and reporting within their own organizations. Some participants actively worked to combat fraud, fabrication, and falsification that could compromise the quality of data collection. They also played a role in correcting mistakes made by others and supported colleagues in designing risk management plans for their sites, resulting in significant improvements in planning and performance.
Potential modifications to the course to better meet their needs
The responses to this question are categorized into three themes:
- Proceed as it is
Some participants expressed their appreciation for the training and conveyed a desire for it to continue without any changes. They indicated that they had no suggestions for improvement. For instance, three participants specifically mentioned that they believe the training is already perfect and should remain unchanged.
- Satisfaction Expression
Rather than providing specific suggestions for improving the training as requested, most participants chose to express their satisfaction by applauding and offering praise such as: informative, well-organized, perfect for current position, effective, simply perfect, meticulously planned, superb content, immediately relevant, comprehensive material, highly engaging, well-crafted, tailored to needs, and so forth.
- Specific improvement suggestions
Although many respondents expressed satisfaction without specific comments and praised the training's quality, there were also constructive suggestions for enhancement. In this section, these suggestions are categorized into four areas.
Timing
Several participants suggested extending the duration of the course from 10 weeks to 14 weeks to allow for more in-depth discussions of the course material. They recommended extending the one-hour tutorial sessions to provide attendees with additional time for exchanging views. Additionally, some participants found the timing of the live tutorials challenging as they occurred during working hours, highlighting the need for greater time flexibility. There was a consensus on the need to allocate more time to live classes. The current schedule was deemed too tight for busy professionals, with suggestions to break down the program into more manageable modules. Participants proposed allocating two weeks for submission of group work and extending the duration of each lesson to two weeks to accommodate these changes.
Group work management
One participant identified group work as a significant challenge within an otherwise excellent course. They recommended improving coordination of group projects and enhancing participant interaction, potentially through open forum discussions. The approach to team activities, particularly regarding the risk management plan, was perceived as lacking smoothness. It was suggested that senior management should oversee group work, and if feasible, teams should be organized by country to facilitate communication. Additionally, tutorials and forum discussions should be enhanced. Measures should be implemented for non-responsive participants during group assignments, with coordinators working closely with team leads to clarify requirements. Furthermore, there was a suggestion for more practice and less emphasis on group work, reflecting concerns about its management. Finally, one participant proposed that group meetings should be financially supported by the institution offering the course.
Accessibility
Another suggestion pertained to accessibility, particularly concerning VoiceThreads. One participant noted difficulty accessing VoiceThreads via phone and recommended improvement in this area. They proposed sharing VoiceThreads in the form of PowerPoint presentations to facilitate easier access. Participants found it challenging to complete VoiceThreads before live tutorials and suggested making presentations readily available for download in PowerPoint format. Furthermore, they recommended incorporating more interactive virtual sessions with colleagues through platforms like Zoom or Microsoft Teams. Additionally, there was a suggestion for a smartphone interface to enhance accessibility. Some participants recommended including more real-life scenarios in assessments to better appreciate practical applications, as well as providing opportunities for participants to observe simulations for real-life applications.
A unique suggestion related to access, albeit not directly tied to the current course provision, was the need for regular offering of the course in the future due to its perceived perfection. Participants advocated for the course to be organized annually, with former participants allowed to participate again. Another unique suggestion was coordinating alumni of the course for grant writing. Additionally, two participants suggested that the 10-week duration of the course is too lengthy for a certificate and proposed issuing a diploma instead, as certificates are typically awarded for shorter training durations.
Findings from the monitoring and evaluation
The routine information collected throughout the various stages of course development, implementation, and evaluation, as part of the program's monitoring and evaluation framework, have documented various opportunities, challenges, and prospects associated with the ClinOps program. These data have been summarized in alignment with the six best practices outlined in the Framework and Best Practices for Training Initiatives of Health Professionals in Low-income Countries (Table 2).
Table 2: Summary of monitoring and evaluation findings based on the six outlined best practices in the Framework and Best Practices for Training Initiatives of Health Professionals in Low-income Countries.
Anticipated best practices
|
Findings
|
Alignment with local priorities
|
- Participants promptly implemented the new-found knowledge within their professional capacities, effectively serving the objectives of their respective local institutions.
- Participants exchanged personal insights and problem-solving approaches, with the learning activities tailored to address authentic challenges encountered in their fields.
|
Country ownership
|
- Transfer of the program to Addis Ababa University from FCD facilitated country ownership
|
Pedagogic innovation
|
- Downloadable content with offline access enabled participants to work offline, uploading their work to forums upon reconnecting.
- Moodle LMS provided convenient access for participants to course materials, interactions, and submissions.
- VoiceThread enhanced dynamic teaching, enabling interactive sessions and real-time addressing of academic challenges.
- Zoom tutorials generally satisfactory, minor audio/connection issues reported.
- Assessment strategies included forum participation, quizzes, and group assignments, promoting active participation and engagement.
- Course leaders' proactive follow-up boosted participant confidence, ensuring continued engagement.
|
Institutional capacity building
|
- Training local faculty on the design and management of the course as part of knowledge transfer component enhanced local capacity.
- Utilization of local institution-hosted digital course delivery and management platforms enhanced capacity managing training locally.
|
Sustainability strategy
|
- The local partner institution becomes a sustainable clinical trial training hub for Africa
.
|
Long-lasting partnerships
|
- Involving academic and product development partners in the program team, such as FCD, FIND, IAVI, MMV, PATH, TB Alliance, and TDR, fostered enduring partnerships.
- The program cultivated formal and informal partnerships among participants, likely persisting post-course.
- Top students invited to join a two-year MSc in Clinical Trials program at CDT-Africa Addis Ababa University.
|
The transfer of the ClinOps training program from a high-income to a lower-middle-income country—specifically from FDC to Addis Ababa University—spanning administrative, technical, and online teaching functions through knowledge transfer training for local faculty and research staff facilitated the alignment of the program with local priorities. The transfer of the program to Addis Ababa University CDT-Africa, facilitated by FCD and PDPs, ensured sustainability. The local institution gained capacity to sustain and develop new programs, serving as a regional asset for LMIC capacity building.
Participants from various countries initially expressed concerns about potential disruptions to their participation in the program due to poor internet connectivity. However, the implementation of innovative teaching and learning platforms in this course helped alleviate these anticipated challenges. By utilizing downloadable content, including a "downloads folder" in each lesson for participants to access all materials at once, individuals were able to work offline and subsequently upload their work to discussion forums upon connecting to the internet. While participants were encouraged to post on forums, and no time limits were imposed, offering maximum flexibility for those encountering internet or utility-related issues.
Utilizing Moodle as the learning management system, where participants access course materials, upload their work, and interact with peers and faculty, facilitated convenient access to course resources. Incorporating VoiceThread into the teaching-learning process enabled dynamic lesson presentations, facilitating flipped classrooms and fostering interactive sessions. Participants were able to add comments or questions to any part of the presentation, allowing instructors to address academic challenges or misconceptions in real-time. The use of Zoom for live tutorials has generally been satisfactory, with few participants reporting issues with audio speed or connection. When issues arise, they are often attributed to participants relying on slow, local WiFi. To optimize connection quality for all, participant mics and cameras were typically switched off during tutorials, with participants typing comments or questions to the class instead. The recordings of the live tutorials were also made available immediately after the session. Each tutorial included at least one faculty member tasked with monitoring and engaging with the chat, bringing key points to the presenter's attention as needed.
Some challenges reported by participants during the course stemmed from personal issues. Health and family reasons, such as malaria or dengue fever, led some to lag on some specific courses. Other challenges included government internet shutdowns, bereavement, divorce, and pregnancy-related issues. Many faced obstacles but persevered to complete the course, despite these challenges. Issues were addressed through mentor and instructor discussions. The initial three weeks were crucial; participants falling behind during this period tended to withdraw. Vigilance during this period and proactive engagement helped resolve issues. Weekly mentor engagement and peer support fostered success within a positive learning environment. Participants appreciated the course leaders' proactive approach of following up through one-to-one emails and, if necessary, Zoom calls when engagement was lacking. This practice instilled confidence and encouraged continued participation.
Initially, the teamwork aspect of the course assessment posed challenges for most groups, particularly in navigating effective remote collaboration across different time zones. It was frustrating when some team members did not fully engage. However, considering time zone differences during team formation helped minimize these issues. With approximately twelve participants and one faculty mentor per team, each group had a private shared space on the Learning Management System for project discussions and file sharing. This setup fostered friendships and trust among team members, facilitating their working dynamics and fostering informal networks that may persist beyond the course.
Although proficiency in English was required for all participants, it was not the native language for some. Initially, a few lacked confidence to engage publicly on forums. However, through a positive learning environment and continuous support, their language confidence and verbal skills improved. The tutors clarified that participants didn't need fluent English proficiency. It was more important to communicate effectively, even if it meant making grammatical or spelling errors, rather than refraining from asking questions or sharing experiences.