A retrospective study of using a three-dimensional printed kidney model to evaluate R.E.N.A.L. nephrometry score: an educational tool to improve the diagnostic accuracy of urology residents

doi:10.21203/rs.3.rs-35180/v1

Download PDF

Research article

A retrospective study of using a three-dimensional printed kidney model to evaluate R.E.N.A.L. nephrometry score: an educational tool to improve the diagnostic accuracy of urology residents

https://doi.org/10.21203/rs.3.rs-35180/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

Background

The R.E.N.A.L. nephrometry scoring system is used to evaluate the complexity of renal tumors; however, inconsistency among evaluators, especially junior physicians, is an issue. The objective of the study was to evaluate whether it is useful for junior physicians to use a three-dimensional (3D) kidney model when evaluating the R.E.N.A.L. nephrometry score.

Methods

An expert and four urology residents retrospectively evaluated the R.E.N.A.L. nephrometry scores of 64 renal tumors (62 patients) which underwent robot-assisted partial nephrectomy at our hospital. The expert evaluated 64 R.E.N.A.L. nephrometry scores with computed tomography (CT) scan imaging, while four residents evaluated 32 cases using only CT and the other 32 cases using CT and 3D kidney model. Consistency between the expert and residents was assessed by Cohen’s kappa score. Patient-specific 3D kidney models were created in a gird style using a 3D printer based on the CT or magnetic resonance imaging of the patient.

Results

For all four residents, the accuracy of the overall R.E.N.A.L. nephrometry score was significantly higher by 3D model and CT than CT only (p < 0.001). Regarding individual components of the R.E.N.A.L. nephrometry score, the accuracy of “E,” “N,” “A,” and “L” scores was higher by 3D model and CT than by CT only (p = 0.020–0.089).

Conclusion

Patient-specific 3-D printed kidney model would improve the resident’s understanding of renal tumor complexity and could be an important educational tool for residents.

Urology & Nephrology

R.E.N.A.L. nephrometry score

three-dimensional printed model

kappa score

resident

education

The R.E.N.A.L. nephrometry score was developed in 2009 as an index of renal tumor complexity [1]. Several studies have shown that the R.E.N.A.L. nephrometry score reflects the degree of surgical difficulty of partial nephrectomy and perioperative/postoperative complications [2, 3]. Thus, measuring the R.E.N.A.L. nephrometry score before a partial nephrectomy is important for urologists.

However, the R.E.N.A.L. nephrometry scores vary between individuals performing the assessment. The consistency rate of the R.E.N.A.L. nephrometry score was reported to be low with the urologists assessment compared with the radiologists assessment or collaborative assessment by the tumor board including urologists, medical oncologists, radiation oncologists, pathologists, and radiologists [4].

Owing to recent advances in three-dimensional (3D) printing technologies that allow easy preparation at low costs, models have been widely used to improve the level of understanding of anatomy, pre-operative planning, and education [5–7]. However, as far as we know, using 3D kidney models to determine the R.E.N.A.L. nephrometry score by urologists has not been reported previously. Therefore, urology residents from our institution determined the R.E.N.A.L. nephrometry score to assess whether the use of 3D kidney models led to more accurate scores.

Survey design

Using a 3D kidney model, we retrospectively investigated the R.E.N.A.L. nephrometry scores of patients who underwent robot-assisted partial nephrectomy (RAPN) at our institution between March 2016 and September 2019. This study received the approval of the Institutional Review Board from our institution with an opt-out system (A-18-210).

The study included an analysis of 64 tumors from 62 patients. Tumors were divided into groups A and B in a time series (n = 32 each). In other words, the first 32 tumors were included in group A and the remaining in group B. The R.E.N.A.L. nephrometry score was determined by one expert and four urology residents. The expert had 25 years of experience in urology and had conducted approximately 60 RAPN procedures while being part of the Department of Urology. Among the four residents, two had 1 year of experience in urology, one had 2 years of experience in urology, and one had 3 years of experience in urology. Four residents had previously scored 5–15 cases of renal tumors using the R.E.N.A.L nephromtery score prior to the study.

The expert determined the R.E.N.A.L. nephrometry score by using contrast-enhanced computed tomography (CT) images in all 64 cases. The four residents were divided into two groups. First, residents 1 and 2 assessed the cases in group A by using only contrast-enhanced CT images. Second, they assessed the cases in group B by using both CT and 3D kidney models. On the other hand, residents 3 and 4 assessed the cases in group A by firstly using contrast-enhanced CT and 3D kidney models. Second, they assessed the cases in group B by using only contrast-enhanced CT. Groups were formed to ensure the grades were not biased.

Contrast-enhanced CT was performed at a slice thickness of 0.75–1.0 mm. Axial/coronal/sagittal images were assessed. For the preparation of the 3D kidney models, we used a contrast-enhanced CT in 63 cases and plain magnetic resonance imaging (MRI) in one case. Residents received orientation using a study by Kutikov et al. [1], and they evaluated 64 tumors. They were allowed to use all CT views as the expert surgeon.

Development Of The 3D Model

We used a UP plus2 (Beijing Tiertime Technology Co., Ltd., Beijing, China) 3D printer, which is a Fused Deposition Modeling type of 3D printer that utilizes acrylonitrile butadiene styrene resin to make the 3D kidney models as described in previous studies [8, 9]. The Standard Triangulated Language (STL) data from contrast-enhanced CT scans or plain MRI were prepared by a radiology technologist. The 3D models were made from STL data with the 3D printer. The 3D models were life-size kidneys based on measurements taken from the patient using the square-block type model (Fig. 1).

Statistical Analyses

Each component (R.E.N.A.L.) and the overall nephrometry score were evaluated by the expert and the residents. The consistency rate between the assessments of the expert and the residents was assessed using Cohen’s kappa score. Pearson’s chi-square test was used to evaluate case variability in groups A and B.

Differences in the rate of consistency of R.E.N.A.L. nephrometry score between those evaluated using only CT and those evaluated using a 3D model and CT were assessed using a paired t-test. All statistical analyses were conducted using JMP v.14.0 (SAS Institute Inc., Cary, NC, USA).

Among the 62 analyzed patients (64 tumors), the mean (standard deviation [SD]) age was 59 (12.5) years. Of the 62 patients analyzed, 37 were men and 25 were women. In 33 cases, renal tumors were located in the right kidney and 31 cases were located in the left kidney. The mean tumor size was 24 (7.7) mm. RAPN was performed in 49 cases. Histopathological evaluation showed that the tumor was clear cell carcinoma in 49 cases, papillary cell carcinoma in three cases, AML in seven cases, and other pathology types in five cases.

The R.E.N.A.L. nephrometry scores for groups A and B evaluated by the expert are shown in Table 1. In groups A and B, there were no differences in score variation in each category.

Table 1

R.E.N.A.L. nephrometry scores of the 64 cases evaluated by an expert
	Group A (n = 32)	Group B (n = 32)	p-value
R: 1/2/3	32/0/0	29/3/0	0.0760
E: 1/2/3	11/13/8	14/15/3	0.2886
N: 1/2/3	10/6/16	15/7/10	0.2921
A: a/p/x	15/10/7	14/15/3	0.2679
L: 1/2/3	13/7/12	17/8/7	0.3837
Cases were divided into groups A and B in a time series.

For all four residents, the resident-expert kappa score of overall R.E.N.A.L. nephrometry score was higher in the 3D model and CT series compared with the CT only series. (0.110 vs 0.3051, difference = + 0.1951, p < 0.001; Table 2).

Table 2

Comparison of Cohen’s kappa score of overall R.E.N.A.L. nephrometry score between the expert and residents
	Cohen’s kappa score of R.E.N.A.L. nephrometry scores
	CT	CT + three-dimensional model
Resident 1	0.1368	0.3163
Resident 2	0.1031	0.2845
Resident 3	0.0409	0.2304
Resident 4	0.1595	0.3895

Regarding individual “E,” “N,” “A,” and “L” categories, a higher kappa score was observed in the 3D model series (p-value =, 0.02 to 0.08; Table 3). The kappa score in the “R” category was not available because almost all cases were cT1a tumors.

Table 3

Mean Cohen’s kappa scores for each component of the R.E.N.A.L. nephrometry scoring system
	CT	CT + 3D model	p-value
Cohen’s kappa (mean)
“E” score	0.5245	0.7777	0.0202
“N” score	0.346	0.5342	0.0894
“A” score	0.3914	0.6361	0.0671
“L” score	0.3907	0.5865	0.0713
“RENAL” score	0.11	0.3051	< 0.001
Paired t-test for CT and CT + 3D model

To the best of our knowledge, no studies have investigated whether 3D kidney models allow for a more accurate assessment of the R.E.N.A.L. nephrometry score among urology residents. Our study had two main findings. First, the rate of overall R.E.N.A.L. nephrometry score consistency between residents and the expert improved with the use of the 3D kidney models. Second, the “E” score significantly improved in each category.

This study showed that using the 3D kidney models the R.E.N.A.L. nephrometry scores determined by residents were similar to those determined by the expert. Although the rate of R.E.N.A.L. nephrometry score consistency was reportedly low in physicians with lesser clinical experience [10], we observed that the use of the 3D kidney models resulted in an increase in the rate of consistency among residents. The effectiveness of 3D model for residents has been reported in various departments [11, 12]; however, this study found that it is also useful for renal cell carcinoma.

A previous study reported that when students used 3D kidney models, the rate of R.E.N.A.L. nephrometry score consistency between students and physicians improved [13]. Those cases were first assessed by CT, followed by a re-assessment using 3D kidney models; thus, assessing the same cases twice was a drawback. Our study differed from the aforementioned study because assessments of different cases were separately performed using CT only and CT and 3D models. As frequent assessments improve the accuracy of the R.E.N.A.L. nephrometry score for residents, we divided the cases into one group where the R.E.N.A.L. nephrometry score was first determined by CT only followed by CT and 3D kidney models, and another group where the R.E.N.A.L. nephrometry score was first determined by CT and 3D kidney models followed by CT only. This resulted in a study design in which a learning curve was taken into account; thus, we assumed that this would be helpful in reducing bias.

Based on the study results, we concluded that in professionals with few years of experience, the level of understanding of anatomy and ability to read CT images could be lower as compared with senior physicians; thus, 3D kidney models can be useful in improving the level of understanding of anatomy and can play important roles in education for residents.

Furthermore, our study showed that in the R.E.N.A.L. nephrometry score determined by residents, only the “E” score of each item significantly improved by using 3D kidney models. A higher rate of consistency in the C index than the R.E.N.A.L. nephrometry score and PADUA score between individuals was reported previously [10]. Measuring only the direct distance is easier and may reduce errors. In contrast, “E” requires space perception; thus, a 3D model, which is easy to understand in terms of spatial recognition, would be useful, particularly in less-experienced doctors.

Conversely, some studies reported a high rate of consistency of R.E.N.A.L. nephrometry scoring [14–17]. One of the reasons may be variation in the cases. In a study evaluating the rate of consistency between radiologists, though the rate of consistency was high (0.88), more than 50% of the cases were of high complexity [14].

In a study showing low rates of consistency, there were a few tumor cases with high complexity, and most cases had low or moderate complexity [4]. In our cases, most tumors were cT1a (95%) and only a few cases (6%). had high complexity. Accordingly, the difficulty in achieving consistency would vary depending on the case. The 3D kidney models were considered useful, at least for junior physicians, to determine the R.E.N.A.L. nephrometry score, particularly in cases in which the evaluation was difficult. Whether a similar result would be produced with 3D virtual reality images rather than 3D models is a question that remains unanswered. Because 3D models were created using STL data of 3DCT, a 3DVR could be similarly effective as the 3D model. An assessment of the R.E.N.A.L. nephrometry score conducted using 3DVR was shown to predict postoperative complications efficiently [18]; thus, assessment using 3DVR and 3D models are likely to become relevant in the near future.

In our study, 3D models were considered to improve junior resident’s anatomy understanding and improve their diagnostic ability. 3D kidney models have been reported to be effective for student education and patient education [13, 19]; however, it was also found to be effective for educating junior physicians. Due to the widespread use of robotic surgery, junior physicians have performed robot-assisted radical prostatectomy [20]. RAPN is also likely to be performed by junior physicians in the near future, and it is important to improve the ability to read CT images.

This study had some limitations. First, as most cases were cT1a high-risk cases, larger tumors could not be assessed. Further research is needed to verify in which cases the model is useful by increasing the number of renal tumors with high complexity.

Second, although it was better that there was CT and 3D model of the consistency rate of all the residents, because the evaluation of the expert was not validated by other experts, it is unknown whether the evaluation of the expert was accurate. The possibility remains that the expert could have made an incorrect assessment.

The 3D kidney models could be useful in educating residents. Whether the use of 3D models by experts will have a positive effect and will affect outcome of RAPN is a subject of future studies.

Three-dimensional

Computed tomography

MRI

Magnetic resonance imaging

RAPN

Robot-assisted partial nephrectomy

STL

Standard Triangulated Language

Virtual reality

Ethics approval and consent to participate:

The study protocol was reviewed and approved by the Institutional Review Board at Jichi Medical University in accordance with Helsinki Declaration and Act on the Protection of Personal Information in Japan (A-18-210) and obtaining additional consent from patients was not required by the ethics committee for this retrospective study with an opt-out system.

Consent for publication:

Not applicable

Availability of data and materials:

The datasets used and analyzed during the current study are available from the corresponding author on reasonable request.

Competing interests:

The authors declare that they have no competing interests.

Funding:

None

Authors’ contributions:

MY, JK, TS, and TT conceptualized and designed the study. MY, TT, SM, KY, and SK performed data acquisition. MY, MK, AF, HM, and SA performed statistical analysis. TK provided technical support. TF supervised the study.

MY wrote the manuscript.

All authors read and approved the final manuscript.

Acknowledgement:

Not applicable

Kutikov A, Uzzo RG. The R.E.N.A.L. nephrometry score: a comprehensive standardized system for quantitating renal tumor size, location and depth. J Urol. 2009;182:844–53.
Schiavina R, Novara G, Borghesi M, Ficarra V, Ahlawat R, Moon DA, et al. PADUA and R.E.N.A.L. nephrometry scores correlate with perioperative outcomes of robot-assisted partial nephrectomy: analysis of the Vattikuti Global Quality Initiative in Robotic Urologic Surgery (GQI-RUS) database. BJU Int. 2017;119:456–63.
Borgmann H, Reiss AK, Kurosch M, Filmann N, Frees S, Mager R, et al. R.E.N.A.L. score outperforms PADUA Score, C-Index and DAP Score for outcome prediction of nephron sparing surgery in a selected cohort. J Urol. 2016;196:664–71.
Monn MF, Gellhaus PT, Masterson TA, Patel AA, Tann M, Cregar DM, et al. R.E.N.A.L. Nephrometry scoring: how well correlated are urologist, radiologist, and collaborator scores? J Endourol. 2014;28:1006–10.
Porpiglia F, Bertolo R, Checcucci E, Amparore D, Autorino R, Dasgupta P, et al. Development and validation of 3D printed virtual models for robot-assisted radical prostatectomy and partial nephrectomy: urologists’ and patients’ perception. World J Urol. 2018;36:201–7.
Bernhard JC, Isotani S, Matsugasumi T, Duddalwar V, Hung AJ, Suer E, et al. Personalized 3D printed model of kidney and tumor anatomy: a useful tool for patient education. World J Urol. 2016;34:337–45.
Wake N, Rude T, Kang SK, Stifelman MD, Borin JF, Sodickson DK, et al. 3D printed renal cancer models derived from MRI data: application in pre-surgical planning. Abdom Radiol (NY). 2017;42:1501–9.
Yamazaki M, Takayama T, Fujisaki A, Kamimura T, Mashiko T, Fujimura T. Robot-assisted partial nephrectomy of initial cases using a 3D square-block type kidney model. Transl Androl Urol. 2020;9:494.
Mashiko T, Otani K, Kawano R, Konno T, Kaneko N, Ito Y, et al. Development of three-dimensional hollow elastic model for cerebral aneurysm clipping simulation enabling rapid and low cost prototyping. World Neurosurg. 2015;83:351–61.
Spaliviero M, Poon BY, Aras O, Di Paolo PL, Guglielmetti GB, Coleman CZ, et al. Interobserver variability of R.E.N.A.L., PADUA, and centrality index nephrometry score systems. World J Urol. 2015;33:853–8.
Lim PK, Stephenson GS, Keown TW, Byrne C, Lin CC, Marecek GS, et al. Use of 3D printed models in resident education for the classification of acetabulum fractures. J Surg Educ. 2018;75:1679–84.
Montgomery SJ, Kooner SS, Ludwig TE, Schneider PS. Impact of 3D printed calcaneal models on fracture understanding and confidence in orthopedic surgery residents. J Surg Educ. 2020;77:472–8.
Knoedler M, Feibus AH, Lange A, Maddox MM, Ledet E, Thomas R, et al. individualized physical 3-dimensional kidney tumor models constructed from 3-dimensional printers result in improved trainee anatomic understanding. Urology. 2015;85:1257–61.
Vilaseca RM, Westphalen AC, Reis HF, Zogbi OS, Silva GE, Reis RB, et al. Reproducibility and interobserver agreement of the R.E.N.A.L. nephrometry score: focus on imaging features. Radiol Bras. 2017;50:7–12.
Nunez-Mora C, Garcia-Mediero JM, Cabrera-Castillo PM, Perez-Utrilla M, Angulo-Cuesta J. Results of simplified urethropexy in the recovery of continence after radical laparoscopic prostatectomy. J Endourol. 2011;25:1759–62.
Kolla SB, Spiess PE, Sexton WJ. Interobserver reliability of the RENAL nephrometry scoring system. Urology. 2011;78:592–4.
Weight CJ, Atwell TD, Fazzio RT, Kim SP, Kenny M, Lohse CM, et al. A multidisciplinary evaluation of inter-reviewer agreement of the nephrometry score and the prediction of long-term outcomes. J Urol. 2011;186:1223–8.
Porpiglia F, Amparore D, Checcucci E, Manfredi M, Stura I, Migliaretti G, et al. Three-dimensional virtual imaging of renal tumours: a new tool to improve the accuracy of nephrometry scores. BJU Int. 2019;124:945–54.
Teishima J, Takayama Y, Iwaguro S, Hayashi T, Inoue S, Hieda K, et al. Usefulness of personalized three-dimensional printed model on the satisfaction of preoperative education for patients undergoing robot-assisted partial nephrectomy and their families. Int Urol Nephrol. 2018;50:1061–6.
Davis JW, Kamat A, Munsell M, Pettaway C, Pisters L, Matin S. Initial experience of teaching robot-assisted radical prostatectomy to surgeons-in-training: can training be evaluated and standardized? BJU Int. 2010;105:1148–54.

Download PDF

Version 1

posted

You are reading this latest preprint version

A retrospective study of using a three-dimensional printed kidney model to evaluate R.E.N.A.L. nephrometry score: an educational tool to improve the diagnostic accuracy of urology residents

Status:

Version 1

Abstract

Background

Methods

Results

Conclusion

Figures

Background

Methods

Survey design

Development Of The 3D Model

Statistical Analyses

Results

Discussion

Conclusions

Abbreviations

Declarations

Ethics approval and consent to participate:

Funding:

Authors’ contributions:

Acknowledgement:

References

Status:

Version 1