We conducted a comparative analysis of risk and TAT between SBT and NGS for HLA typing. Our findings revealed that the total RPN score of NGS was higher than SBT (1,171 vs. 465 RPN), primarily due to the increased number of processing steps (44 vs. 25 steps). The mean RPN score in NGS was also higher than SBT (26 vs. 19 RPN). NGS showed higher mean severity and detectability scores than SBT (both 5 vs. 4). Despite the elevated risk of NGS compared to SBT, it remained acceptable according to the RAM rating scale. The total TAT, total hands-on time and hands-on time/step of NGS were longer than SBT (26:47:20 vs. 12:32:08, 03:59:35 vs. 00:47:39, 00:05:13 vs. 00:01:54 hh:mm:ss, respectively).
Recently, there has been a growing interest in assessing laboratory efficiency in clinical laboratories to improve productivity and patient safety. The assessments involved the evaluation of risk assessment, TAT and costs. FMEA, initially used in the industrial field, has recently found application in laboratory risk assessment. In a previous study by Nam M et al. 19, a risk assessment comparison using FMEA was conducted between the tube test and automated column agglutination technology for anti-A/B isoagglutinin titers. The automated column agglutination technology demonstrated lower RPN scores than the tube test (337 vs. 1,843 RPN), with the manual serial dilution step of the tube test accompanying the highest RPN score of 1,080. This finding highlighted that RPN scores significantly increased when a processing step involved manual procedures, required skilled techniques, or included repetitive and numerous steps. NGS involved numerous complex steps, such as DNA fragmentation, adapter ligation, library pooling, size selection, and library quantification. The critical steps in NGS required more manual procedures and skilled techniques compared to SBT, leading to increased mean severity, mean detectability, mean RPN, and total RPN score. RAM, another indicator of laboratory risk management suggested by CLSI EP23-A, was also evaluated in this study. Although the risk of NGS was higher than SBT, NGS was still deemed acceptable for HLA typing. Since there is currently no research applying the RAM in laboratory efficiency, further research is needed.
Our novel findings involved a comparison between FMEA and RAM. FMEA demonstrated significant advantages over RAM in comparing two or more items by quantifying the risk through scores. However, a disadvantage of FMEA was its inability to establish a clear standard for evaluating the true acceptability of each processing step. There was no standardized or validated cutoff for RPN scores. Nevertheless, Han et al. 18 suggested careful review of steps exceeding a score of 300 RPN. In this study, none of the processing steps in SBT and NGS exceeded 300 RPN. In regarding to RAM, the rating scale appeared to be a more suitable method than FMEA for performing conformity assessments at each step. However, RAM was not suitable for performing comparisons between different methods. In this study, a risk comparison between SBT and NGS could not be performed using RAM due to the same proportion of acceptable steps. Each laboratory should consider the appropriate criteria for conducting a risk assessment based on its purpose.
TAT, another critical indicator of laboratory efficiency, has been actively used in previous research. Kim et al. 20 measured mean, 99th percentile, and CV% TAT to assess laboratory efficiency of total laboratory automation. The mean TAT, representing the timeliness of result reporting, decreased by 6.1%, the 99th percentile TAT, representing the outlier rate, decreased by 13.3%, and the CV% TAT, representing predictability, decreased by 70.0%. Despite the effectiveness of TAT in assessing laboratory efficiency, there is no clear consensus on which period should be incorporated when establishing TAT for a specific test21,22. In this study, we did not analyze the pre-analytical phase and DNA extraction step due to performing the same process in SBT and NGS. TATs were assessed after DNA extraction until the final reporting. Similar to risk assessment, TAT increased when the test involved manual procedures, required skilled techniques, or included repetitive and numerous steps. Due to the complex and numerous steps involved in NGS, NGS indicated longer total TAT, total hands-on time, and particularly hands-on time/step.
According to several reports, an increase in hands-on times has been correlated with increasing medical costs. Shin et al. 23 reported labor costs into the total expense, considering the average technician salaries and the hands-on time required for testing a single sample. In the case of unexpected antibody screening, the automated analyzer significantly reduced labor costs compared to manual methods (539 vs. 2,961 Korean won), subsequently reducing the total expense (5,920 vs. 7,157 Korean won). In this study, the increased hands-on time in NGS could potentially elevate labor costs, consequently contributing to an overall increase in total expense. However, we did not comprehensively compare costs in this study; further research is needed to compare direct and indirect costs between NGS and SBT.
Recent studies have shown controversial results regarding the impact of laboratory automation in terms of laboratory efficiency. Chung et al. 16 revealed that the automated crossmatching method has a lower RPN score (229 vs. 1,435 RPN), shorter TAT (19.1 vs. 23.3 min), shorter hands-on time (1.1 vs. 5.3 min), and lower costs/test (1.44 vs. 2.70 $) than manual crossmatching method. Nam et al. 17 found that automated column agglutination technology showed lower RPN scores (33,700 vs. 184,300 RPN) than manual tube tests for anti-A/B isoagglutinin titers. However, TAT and cost were similar in automated and manual methods (TAT, 15:23:00 vs. 14:26:40; cost, 1377.4 vs. 1312.4, respectively). Nam et al. 17 revealed that automated white blood cell counting in leukopenic samples showed even longer TAT than manual counting. In this study, NGS showed increased risk and TAT due to the increase in the number and complexity of each step. Despite the controversial results on laboratory efficiency for automation, we expect that the automation of several NGS processes can improve laboratory efficiency. In particular, the DNA clean-up step revealed the highest RPN score, and the DNA quantification step revealed remarkably longer hands-on time. Automation of these steps can contribute to improving the laboratory efficiency of NGS.
This study has some limitations. It involved the comparison of risk and TAT for a single sample without the evaluation of multiple samples. Given that NGS utilized a flow cell for analyzing multiple samples simultaneously, there is potential for a significant reduction in risk and TAT. In addition, TAT might differ between qualified and unskilled technicians. Therefore, it will be necessary to minimize the possibility of variation by conducting multiple measurements of the performance of a qualified and skilled technician.
In conclusion, this is the first study to compare risk and TAT between NGS and SBT for HLA typing. NGS required more processing steps and a more significant manual workload than SBT, resulting in higher RPN scores and longer TATs. These results suggest that it is necessary to consider the high risk and long TAT in NGS when contemplating the transition from SBT to NGS for HLA typing. Furthermore, companies involved in NGS development need to optimize the testing workflow, reduce manual workload, and increase the automation process.