In the GCP Renovation, including ICH E8(R1) and ICH E6(R3), the focus on identifying and evaluating risks and planning risk reduction activities has shifted from the critical data and process to the critical to quality factors. Since the time ICH E6(R2) was issued, various organizations such as pharmaceutical companies, academia, and contract research organizations have been investigating methods for risk identification and evaluation. However, there has been no consensus, and it is difficult to determine the methods that should be used when planning RBA. This study aims to help in the selection of an appropriate method for identifying and evaluating risks and planning risk reduction activities.
The amount of error data per subject visit was lower in the RAF group than in the RAT group, and the number of protocol deviations per subject visit was also lower in the RAF group than in the RAT group. The actual number of protocol deviations was also lower in the RAF group than in the RAT group. We identified fewer risks using RAF than RAT. RAF can effectively mitigate risk with fewer categories. These results imply that the RAF method is effective in risk identification and evaluation and risk mitigation planning by identifying specific risks in a small number of abstract categories. While the amount of error data per subject visit in the high-risk category was comparable between the two groups, the number of errors in the medium-risk category in the RAT group was slightly more than twice that of the RAF group. This implies that there is no difference between the two methods in terms of ensuring the reliability of the trial results.
The number of error data and protocol deviations per person-time taken to implement risk identification and evaluation and planning risk reduction activities were both lower in the RAF group than in the RAT group. These may be due to differences in the risk identification method. RAF was used to identify risks in 12 abstract categories. RAT was used to select relevant risks from a list of 164 concrete risks. Among the categories in RAF, “Liability” and “Intellectual Proper” in “Organizational Hazards” are not common or similar to any of the categories in RAT. As no risk reduction activity was identified in these categories, the difference in the risk categories might not affect the results. In addition, as RAF has a five-point evaluation for “Impact” and “Likelihood” and RAT has a three-point evaluation for “Severity” and ”Probability,” the two risk evaluation methods are similar, although the number of evaluation points is different. Therefore, the difference in the risk evaluation method might also not affect the results.
DMs, CRAs, statisticians from the Clinical Research Data Center in the Clinical Research Innovation and Education Center Tohoku University Hospital, and RBA specialists conducted risk identification and evaluation and planning risk reduction activities in this study. Unlike the members of the research, people who are not familiar with clinical research or people who have no experience in risk identification and evaluation and planning risk reduction activities might be given more time in identifying and evaluating risks and planning risk reduction activities.
This might have a significant impact on RAF as it is used to identify risks from 12 abstract categories, and the derivation of risks from abstract questions requires knowledge of clinical research and experience in risk identification, risk evaluation, and planning of risk reduction activities in clinical research. Therefore, risk identification from abstract categories or questions should be employed by those who are familiar with clinical research or who have experience in risk identification, risk evaluation, and risk reduction activities. Otherwise, it may be better to use the method to select relevant risks from a list of concrete risks.
The risk reduction activity was implemented, in which the deadline for data entry was described in the CRF completion guideline in both groups for the risk of missing data entry. In the RAF group, an additional risk reduction activity was implemented, in which the precautions and training were conveyed in video. However, the number of missing data entry per subject visit was lower in the RAT group (RAF group: 0.25, RAT group: 0.07). Similarly, the risk reduction activity was implemented, in which the data entry of protocol deviation was described in the CRF completion guideline in both groups for the risk of missing data entry of the deviation. In the RAT group, an additional risk reduction activity was implemented, in which the precautions and training were conveyed in video. The number of missing data entry of protocol deviation per subject visit was lower in the RAF group (RAF group: 0.03, RAT group: 0.07). These results may be due to the difficulty of consciously maintaining the precaution of data entry as the timeliness of data entry is generally trained in clinical research and is not a clinical research-specific precaution for investigators. In addition, although no entry of protocol deviation was considered the clinical research-specific risk, it is general in terms of data entry. Missing data entry may not be effective through risk reduction activities, such as precautions and training.
The risk reduction activities were implemented, in which the precaution and training were conveyed in video and the alert text was set to appear on the EDC screen in the RAF group for the missing risk of blood concentration measurement of ponatinib. The number of deviations related to this measurement was lower in the RAF group (RAF group: 0.01, RAT group: 0.04). The risk reduction activity of SAE reporting procedures and the deadline for data entry of SAE were implemented in both groups, and the precautions and training were conveyed in the video. In the RAT group, an additional risk reduction activity was implemented, in which the alert text was set to appear on the EDC screen. The number of deviations related to this measurement was lower in the RAT group (RAF group: 0.03, RAT group: 0.01). All investigators were trained on the contents of the risk reduction activities during the investigator meeting prior to the main clinical study. Therefore, they may be effective in reducing risks in one way communication of intention in the study process, such as by watching movies again as precautions and training and by presenting alert texts on the EDC screen.
Our study reveals that protocol deviations are prevented by setting the alert text to appear on the EDC screen, which can be done in addition to training, which is commonly used as a risk-mitigating measure in EMA Reflection Paper,1 TransCelerate Position paper,8 ECRIN,9 and JPMA publications.10–11 In clinical research where CRAs make it difficult to mitigate risks or take preventive actions, setting the alert text to appear on the EDC screen may provide low-cost quality control. In clinical trials conducted by pharmaceutical companies, the CRF design has become more standardized, and such efforts have not been implemented, although they may be effective for critical data.
A limitation of our study is that the main clinical research was conducted with a small sample size. Therefore, the results cannot be generalized. In addition, risk identification, risk evaluation, and planning risk reduction activities were carried out by the same members. Preceding planning in RAT by planning in RAF may have affected the results of risk identification, risk evaluation, and planning risk reduction activities.
Because 100% SDV and SDR were not conducted in the main clinical research, it may not have collected all the errors and protocol deviations. However, because the scope of monitoring and data review covered all data, it ensured the comparability of RAF and RAT.
Our study reveals that it is effective to use RBA to identify risks in a small number of abstract categories. Risk identification, risk evaluation, and planning risk reduction activities are commonly practiced in clinical trials. However, to the best of our knowledge, this is the first study to evaluate the relationship between risk identification and evaluation and risk reduction activities. As critical to quality factors can be considered abstract categories, this evidence is consistent with the policy of risk identification in the GCP Renovation.