Electronic Patient-Reported Outcomes and Machine Learning in Predicting Immune-Related Adverse Events of Immune Checkpoint Inhibitor Therapies

doi:10.21203/rs.3.rs-128917/v1

Download PDF

Research Article

Electronic Patient-Reported Outcomes and Machine Learning in Predicting Immune-Related Adverse Events of Immune Checkpoint Inhibitor Therapies

https://doi.org/10.21203/rs.3.rs-128917/v1

This work is licensed under a CC BY 4.0 License

You are reading this latest preprint version

Background: Immune-checkpoint inhibitors (ICIs) have introduced novel immune-related adverse events (irAEs), arising from various organ systems without strong timely dependency on therapy dosing. Early detection of irAEs could result in improved toxicity profile and quality of life (QoL). Symptom data collected by electronic (e) patient-reported outcomes (PRO) could be used as an input for machine learning (ML) based prediction models for early detection of irAEs.

Methods: The utilized dataset consisted of two data sources. The first dataset (n=16540) comprised symptoms from 34 ICI-treated cancer patients, including 18 monitored symptoms collected using the Kaiku Health digital platform. The second dataset included prospectively collected irAE data, Common Terminology Criteria for Adverse Events (CTCAE) class, and severity of 26 irAEs. The ML models were built using extreme gradient boosting algorithms. The first model was trained to detect the presence and the second model the onset of irAEs.

Results: The model trained to predict the presence of irAEs had an excellent performance based on four metrics: accuracy score 0.97, Area Under the Curve (AUC) value 0.99, F1-score 0.94 and Matthew´s correlation coefficient (MCC) 0.92. The prediction of the irAE onset was more difficult with accuracy score 0.96, AUC value 0.93, F1-score 0.66 and MCC 0.64 but the model performance was still at a good level.

Conclusion: Current study suggests that ML based prediction models, using ePRO data as an input, can predict the presence and onset of irAEs with high accuracy, indicating that ePRO follow-up with ML algorithms could facilitate the detection of irAEs in ICI-treated cancer patients. The results should be validated with a larger dataset.

Medical Informatics

ePRO

machine learning

prediction model

irAE

immune checkpoint inhibitor

prognosis

In recent years, there has been a remarkable development in cancer immune checkpoint inhibitor (ICI) therapies. ICIs have become the first line treatments in several malignancies ^1–12. However, ICIs are associated with unique immune related adverse events (irAEs). These toxicities can arise from various organ systems, and at any time point without temporal connection to the therapy which makes these more unpredictable than AEs with traditional cancer therapies. irAEs may also persist or appear in a similar manner after ICI discontinuation while immune-mediated toxicity seems to be independent of dose and duration of the given anti-PD-(L)1 treatment ^13–15. Even though irAEs can be severe and even life threatening, if caught and treated early, most of them are reversible ¹⁶. Thus, early detection of irAEs could result in an improved safety of the treatment and better quality of life.

Due to the unique nature of irAEs, there is a need for all-comprising assessment, grading, and long-term surveillance of patients’ symptoms. Patient reported outcomes (PROs) consist of health-related questionnaires filled by the patients, which can capture symptoms and signs. Evolving data suggests that electronic patient reported outcome (ePRO) tools could be one solution for optimizing patient surveillance during and after ICI therapies ^17,18. To be able to better predict the onset of irAEs, ePROs combined with other clinical data could be used to develop machine learning (ML) based prediction models. The Kaiku Health digital platform has been used in a real-world setting to capture symptom data from cancer patients receiving ICIs ^{18, 19}. In this study, anonymized and aggregated ePRO data collected with the Kaiku Health ePRO tool was used to train and tune prediction models built using an open source Python library XGBoost (extreme gradient boosting algorithm) for detection of the presence and onset of irAEs ^20–22.

Patients

The study subjects (n = 34) consisted of patients recruited to the prospective KISS trial investigating ePRO follow-up in cancer patients receiving ICIs ¹⁹. In brief, the trial included patients with advanced cancers treated with anti-PD-(L)1 s in outpatient settings with availability of internet access and email. At the initiation of the treatment phase (within 0–2 weeks from the 1st anti-PD-(L)1 infusion), patients received an email notification to complete the baseline electronic symptom questionnaire of 18 symptoms and weekly thereafter until treatment discontinuation or six months of follow-up. Data on the irAEs (nature of AE, date of onset and resolving, dates of change in AE severity, and the highest grade based on CTCAE classification) was prospectively collected in the trial.

The study was approved by Pohjois-Pohjanmaan sairaanhoitopiiri (PPSHP) ethics committee (number 9/2017), Valvira (number 361), and details of the study are publicly available at clinicaltrials.gov (NCT3928938). The study was conducted in accordance with the Declaration of Helsinki and Good Clinical Practice guidelines.

Prediction models

The aim of this study was to create a model for predicting the presence (is the predicted irAE truly an irAE) and onset (is an irAE developing) of irAEs based on evolving patient-reported symptoms collected digitally in prospective manner from cancer patients receiving ICI therapies. For both modelling cases, the output of the prediction model is a continuous value [0–1] depicting the probability for the positive event, i.e., presence or onset of irAEs. With a classification threshold (0.5 was used with both models), the continuous probabilities were converted into binary outcomes, i.e., when the predicted probability for the positive event is larger than 0.5, prediction is labeled positive (irAE onsetting or present), and if less than 0.5, then negative (irAE not onsetting or present). Hence, the modelling methodology used in this study follows a general framework of binary classification in machine learning (ML). The first dataset included 16 540 reported symptoms from 34 ICI treated cancer patients in outpatient settings, comprising 18 monitored symptoms collected using the Kaiku Health digital platform. The second dataset included physician-confirmed prospectively collected irAE data from those 34 patients recorded in the eCRFs of the trial, containing initiation and end dates, CTCAE class, and severity of 26 irAEs. The timelines of ePROs and irAEs were synchronized according to dates. Of note, some patients might have experienced multiple irAEs, thus, the incidence of irAEs in this patient cohort was ~ 40% (n = 14). Multiple observations across same patients were used to create a timeline of irAEs, however, in every time point analyzed, the parameters differ comprising a new sample. Furthermore, the gradient boosting trees-algorithm used can handle intercorrelated observations or features.

Two ML models were built using an open source Python library XGBoost, which offers a widely used high-performance implementation of gradient boosting, an established algorithm suitable for classification problems. Gradient boosting is an ensemble-learning algorithm, i.e., it is an ensemble of many, usually tens or hundreds of decision trees. These decision trees, i.e., classification trees, are weak learners, but when combined using gradient boosting approach, they form a strong learner capable of capturing complex relationships in the training data. By combining the ePRO data and the clinical data, the first model was trained to detect the presence and the second model the onset (0–21 days prior to the diagnosis) of irAEs. The dataset was split into training (70 % of the data) and test sets (30 % of the data) by random allocation at patient-observation level. The test set was left out from the model training and tuning and was used only to evaluate the model performance. The hyperparameter tuning for both prediction models was done using grid search with repeated, stratified 5-fold cross-validation with five repeats. The model features included patient information including age, sex, and time from the treatment initiation, and ePRO data from 18 monitored symptoms. From the symptom data, three past values, linearly scaled based on the time difference from the latest report, and the latest change in symptom severity were included as features for each symptom. This yielded 75 features in total for both models.

The prediction performance of the models was evaluated using four commonly used metrics: accuracy, Area Under the Curve (AUC), F1-score and Mathew´s correlation coefficient (MCC), which are described shortly next. Accuracy describes how many predictions were correct as percentages, and 100% indicates a perfect classification. AUC is a performance metric for binary classification ranging from 0 to 1. F1-score is the weighted average of precision (how many of the cases predicted as positive are actually positive) and recall (how many of the positive cases are detected) which gets values between 0 and 1. Matthew’s correlation coefficient (MCC) summarizes all possible cases for binary predictions: true and false positives, and true and false negatives. MCC is also suitable for analyzing imbalanced datasets, where other class is much rarer than the other. MCC can be considered as a correlation coefficient between the observed and the predicted classifications and it gets values between − 1 and 1, where 1 is a perfect classification, 0 is random guessing and − 1 indicates a completely contradictory classification.

Temporal association to the treatment and the duration of irAEs alternate greatly. In this patient group, the longest irAE lasted for 799 days, and the shortest two days while median duration was 61 days. The model trained to predict the presence of irAEs had an excellent performance with the test dataset. According to the accuracy score, both models were performing at a very good level. The accuracy score for the prediction of the presence of irAEs was 0.97, and 0.96 for the onset of irAEs. The AUC values (0.99 for the presence of irAEs and 0.93 for the onset of irAEs) suggested a good quality level of model performance. The F1-score (0.94) indicates that the model was accurate in the prediction of the presence of irAEs. However, F1-score for the prediction of the onset of irAEs was somewhat lower, 0.66. According to the MCC values, the model performed well in predicting the presence of irAEs (0.92) while the accuracy in predicting the onset of irAEs was lower, 0.63. The performance metrics for the ML models are presented in Table 1.

Table 1. Performance metrics for the prediction models for the presence and onset of irAES

	Presence of irAEs	Onset of irAEs
Accuracy	0.97	0.96
AUC	0.99	0.93
F1-score	0.94	0.67
MCC	0.92	0.64

Figure 1 presents the confusion matrix for predicting the presence of irAEs and Figure 2 for predicting the onset of irAEs. As is evident from Figure 1, the prediction performance for the presence of irAEs was excellent as there were only two false negative (the lower left corner) and six false positive (the upper right corner) predictions with the test dataset. The prediction performance for the onset of irAEs was much lower as can be seen in Figure 2. As illustrated, there were only 17 positive samples in the test dataset, of which 11 were classified correctly as positive and six were classified as negative, thus, being false negatives. There were also five false positives present with the test dataset.

Figure 3 presents the feature importance plot for the model predicting the presence of irAEs and Figure 4 for the model predicting the onset of irAEs. The presented feature importances display the relative average improvement in prediction accuracy across all trees in the model where the feature in question is used. The feature importances are relative, i.e., they display how much features contribute to the final prediction relative to each other. A higher value indicates that the feature is more important for generating the prediction compared to a feature with a lower value.

As is presented in Figure 3, the most important features for predicting the presence of irAEs were related to diarrhea, pain in joints, dizziness, age and time from the treatment initiation (weeks in treatment). According to Figure 4, for predicting the onset of irAEs, the most important features were related to fever, chest pain, stomach pain, dizziness, nausea, and diarrhea. Furthermore, Figure 3 reveals that roughly, two thirds of the features contributed to the predictions and one third was not used. The features, which do not contribute to the predictions, could be removed from the model using feature selection, but it is not mandatory, or does not impact the model performance due to the tree structure of the used algorithm. Approximately similar number of features were utilized in the prediction of the onset of irAEs neither, as presented in Figure 4.

In this study, ePRO and irAE data related to advanced cancer patients treated with ICIs were investigated to better understand their potential correlation. The aim was to use prospectively collected patient-derived data and clinical data from treating physicians during ICI treatments to build ML based tools to improve the early detection of irAEs, and ultimately, enhance patient care, QoL and cost-effectiveness of ICIs. The results show that ML models based on the ePRO and structured electronic health care record (EHR) data could accurately predict the presence of irAEs. The ML models had a good level of discrimination in predicting the onset and continuation of irAEs.

Of the reported symptoms, the feature importance analysis revealed that diarrhea and pain in the joints, known irAEs with predictive nature, had the highest weighted value in the building of the model when predicting the presence of irAE. This could be interpreted that if ICI treated cancer patients report these symptoms; they very often reflect the presence of irAE. In contrast, in predicting the future onset of irAE, fever, chest, and stomach pain were the predominant features in the model. All these symptoms demand further, often acute, assessment, and could imply an onset of a fatal irAE such as colon perforation, pneumonitis, or myocarditis. In addition, these symptoms often indicate progressive disease in lung cancer patients or patients with lung metastases, which underlines the importance of prompt diagnostic measures to exclude pseudo-progression with concomitant irAE demanding rapid initiation of an immunosuppressive medication ¹⁴.

There are several limitations to this analysis. The most relevant limitation is the low number of study subjects. However, the total amount of reported symptoms is close to 20 000 which increases the value of the data collected. Furthermore, our cohort consisted of ICI-monotherapy treated patients and may not be generalized to patients treated with combination therapies. According to registry data, however, the toxicity profile of ICI-ICI or ICI-chemotherapy compared to ICI-monotherapy differs mainly in the overall incidence rather than in the variety of symptoms ^{14, 23–27}. In addition, our study provides a proof-of-concept for building ML based prediction models on ePRO data and clinical data and can be further exploited also to create models for combination therapies of ICIs. The F1-score and the MCC value for the prediction of the onset of irAE were somewhat low. The lower level of performance of the model in predicting the onset of irAEs probably reflects the rather low incidence of irAEs in ICI-monotherapy treated cancer patients. Nevertheless, modelling methods were chosen to overcome imbalanced data sets, and intercorrelated parameters to minimize such bias. On the other hand, the generalizability of the model inevitably requires testing in another, preferably larger cohort of similar data available.

Artificial intelligence (AI) based analytics have gained growing interest in the field of cancer care. Deep learning systems have shown promising results especially in cancer diagnostics ²⁸. AI based methods can be used to analyze vast data pools to create predictive and prognostic analytics for generating value-based healthcare assets. In addition, recent data shows that ML algorithms could identify patients with cancer who are at risk of short-term mortality ²⁹. The possibility of creating truly individual prognostic assessments could facilitate more timely conversations between patients and physicians regarding goals and values.

Evolving data show that EHR based predictive algorithms may improve clinicians´ prognostication and decision-making ^30,31. Oncology-specific ML algorithms based on EHR data have been shown to accurately predict short-term mortality among patients starting chemotherapy ^{32, 33}. However, utilization of electronic patient-derived symptom data, ePROs, related to ICI therapy toxicity in creating ML algorithms is a novel approach. Furthermore, ePROs have many advantages compared to paper questionnaires such as reducing timely and locational limitations and offering continuous collection of symptoms in a cost-effective manner ^34–36. Thus, based on the results of the study, it is feasible to use ePROs in the development of ML based approaches, such as symptom prediction models to enable earlier detection of toxicities.

In cancer care, adaptive follow-up of patients is crucial and probably more cost-effective. AI models offer a possibility for individually evolving follow-up which predictive nature improves constantly. ICIs used in cancer care require long-term follow-up because of the potentially fatal toxicity, that is, however, highly treatable. We argue that ePRO follow-up combined to ML models for ICI toxicity prediction, share these important features and would optimize the clinical impact of the therapy. Future research will focus on combining other clinical data such as laboratory values to the model to create a criterion-standard prognostic assessment tool to predict ICI related toxicity. Whether this could be enhanced by combining the symptom reports of a patient to other eHealth apps sensing for example metabolic or physiologic changes is another fascinating possibility ³⁷.

The current study suggests that ML based prediction models using ePRO and EHR data as an input can predict the presence and onset of irAEs with a high accuracy. Thus, it indicates that digital symptom monitoring combined with ML could facilitate the detection of irAEs in ICI-treated cancer patients.

ICIs Immune-checkpoint inhibitors

irAEs immune-related adverse events

QoL Quality of Life

PRO patient-reported outcome

ML machine learning

CTCAE Common Terminology Criteria for Adverse Events

AUC Area under curve

MCC Mathew`s correlation coefficient

EHR Electronic health care record

AI Artificial intelligence

XGBoost Extreme gradient boosting algorithm

Ethics approval and consent to participate

The study was approved by Pohjois-Pohjanmaan sairaanhoitopiiri (PPSHP) Ethics Committee (9/2017) and registered at the Clinical Trials Register with identifier code NCT03928938. All study subjects signed an informed consent to participate in the trial.

Availability of data and materials

The datasets generated and/or analyzed during the current study are not publicly available but are available from the corresponding author on reasonable request.

Competing interest

SI declares no competing interests. JE, HV, and VK are employees of Kaiku Health. JPK is an advisor for Kaiku Health.

Funding

Study was funded by Oulu University and Finnish Cancer Society. Kaiku Health employees were involved in the data acquiring and analysis.

Authors’ contributions

SI, JE, HV, VK, and JPK contributed to the conception and design of the study; SI and JE acquired the data; SI and JE analyzed the data and SI, JE, and JPK interpreted the data. SI, JE and JPK contributed to the writing of the manuscript. All authors read and approved the final manuscript.

Consent for publication

Not appilicable.

Acknowledgement

Not applicable.

Topalian SL, Hodi FS, Brahmer JR, et al. Safety, activity, and immune correlates of anti-PD-1 antibody in cancer. N Engl J Med 2012; 366(26): 2443–54. doi:10.1056/NEJMoa1200690
Schachter J, Ribas A, Long GV, et al. Pembrolizumab versus ipilimumab for advanced melanoma: final overall survival results of a multicentre, randomised, open-label phase 3 study (KEYNOTE-006). Lancet 2017; 390(10105): 1853–62.
Robert C, Schachter J, Long GV, et al. Pembrolizumab versus Ipilimumab in Advanced Melanoma. N Engl J Med 2015; 372(26): 2521–32.
Weber JS, Hodi FS, Wolchok JD, et al. Safety Profile of Nivolumab Monotherapy: A Pooled Analysis of Patients With Advanced Melanoma. J Clin Oncol 2017; 35(7): 785–92.
Borghaei H, Paz-Ares L, Horn L, et al. Nivolumab versus Docetaxel in Advanced Nonsquamous Non-Small-Cell Lung Cancer. N Engl J Med 2015; 373(17): 1627–39.
Brahmer J, Reckamp KL, Baas P, et al. Nivolumab versus Docetaxel in Advanced Squamous-Cell Non-Small-Cell Lung Cancer. N Engl J Med 2015; 373(2): 123–35.
Herbst RS, Baas P, Kim DW, et al. Pembrolizumab versus docetaxel for previously treated, PD-L1-positive, advanced non-small-cell lung cancer (KEYNOTE-010): a randomised controlled trial. Lancet 2016; 387(10027): 1540–50.
Reck M, Rodriguez-Abreu D, Robinson AG, et al. Pembrolizumab versus Chemotherapy for PD-L1-Positive Non-Small-Cell Lung Cancer. N Engl J Med 2016; 375(19): 1823–33.
Rittmeyer A, Barlesi F, Waterkamp D, et al. Atezolizumab versus docetaxel in patients with previously treated non-small-cell lung cancer (OAK): a phase 3, open-label, multicentre randomised controlled trial. Lancet 2017; 389(10066): 255–65.
Bellmunt J, de Wit R, Vaughn DJ, et al. Pembrolizumab as Second-Line Therapy for Advanced Urothelial Carcinoma. N Engl J Med 2017; 376(11): 1015–26.
Motzer RJ, Escudier B, McDermott DF, et al. Nivolumab versus Everolimus in Advanced Renal-Cell Carcinoma. N Engl J Med 2015; 373(19): 1803–13.
Balar AV, Castellano D, O'Donnell PH, et al. First-line pembrolizumab in cisplatin-ineligible patients with locally advanced and unresectable or metastatic urothelial cancer (KEYNOTE-052): a multicentre, single-arm, phase 2 study. Lancet Oncol 2017; 18(11): 1483–92.
Spain L, Diem S, Larkin J. Management of toxicities of immune checkpoint inhibitors. Cancer Treat Rev 2016; 44: 51–60.
Haanen, J. B. A. G., Carbonnel F, Robert C, et al. Management of toxicities from immunotherapy: ESMO Clinical Practice Guidelines for diagnosis, treatment and follow-up. Ann Oncol 2017.
Nathan P, Ascierto PA, Haanen J, et al. Safety and efficacy of nivolumab in patients with rare melanoma subtypes who progressed on or after ipilimumab treatment: a single-arm, open-label, phase II study (CheckMate 172). Eur J Cancer 2019; 119: 168–78. doi:10.1016/j.ejca.2019.07.010.
Wang DY, Salem JE, Cohen JV, et al. Fatal Toxic Effects Associated With Immune Checkpoint Inhibitors: A Systematic Review and Meta-analysis. JAMA Oncol 2018; 4(12): 1721–8. doi:10.1001/jamaoncol.2018.3923.
Denis F, Koontz BF, Letellier C. Application and Benefits of Web-Mediated Symptom Reporting for Patients Undergoing Immunotherapy: A Clinical Example. Case Rep Oncol 2018; 11: 763–8.
Iivanainen S, Alanko T, Peltola K, Konkola T, Ekstrom J, Virtanen H, Koivunen JP. ePROs in the follow-up of cancer patients treated with immune checkpoint inhibitors: a retrospective study. J Cancer Res Clin Oncol 2019; 145(3): 765–74. doi:10.1007/s00432-018-02835-6.
Iivanainen S, Alanko T, Vihinen P, Konkola T, Ekstrom J, Virtanen H and Koivunen J. Follow-Up of Cancer Patients Receiving Anti-PD-(L)1 Therapy Using an Electronic Patient-Reported Outcomes Tool (KISS): Prospective Feasibility Cohort Study. JMIR Form Res 2020; 4(10):e17898. doi:10.2196/17898
XGBoost: Scalable, Portable and Distributed Gradient Boosting (GBDT, GBRT or GBM) Library.
Chen T, Guestrin C. A Scalable Tree Boosting System: In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (San Francisco, CA: ACM)
Taninaga J, Nishiyama Y, Fujibayashi K, Gunji T, Sasabe N, Iijima K, Naito T. Prediction of future gastric cancer risk using a machine learning algorithm and comprehensive medical check-up data: A case-control study. Sci Rep 2019; 9(1): 12384-y. doi:10.1038/s41598-019-48769-y.
Larkin J, Chiarion-Sileni V, Gonzalez R, et al. Combined Nivolumab and Ipilimumab or Monotherapy in Untreated Melanoma. N Engl J Med 2015; 373(1): 23–34. doi:10.1056/NEJMoa1504030.
Hellmann MD, Paz-Ares L, Bernabe Caro R, et al. Nivolumab plus Ipilimumab in Advanced Non-Small-Cell Lung Cancer. N Engl J Med 2019; 381(21): 2020–31. doi:10.1056/NEJMoa1910231.
Paz-Ares L, Luft A, Vicente D, et al. Pembrolizumab plus Chemotherapy for Squamous Non-Small-Cell Lung Cancer. N Engl J Med 2018; 379(21): 2040–51. doi:10.1056/NEJMoa1810865.
Gandhi L, Rodriguez-Abreu D, Gadgeel S, et al. Pembrolizumab plus Chemotherapy in Metastatic Non-Small-Cell Lung Cancer. N Engl J Med 2018; 378(22): 2078–92. doi:10.1056/NEJMoa1801005.
Schmid P, Adams S, Rugo HS, et al. Atezolizumab and Nab-Paclitaxel in Advanced Triple-Negative Breast Cancer. N Engl J Med 2018; 379(22): 2108–21. doi:10.1056/NEJMoa1809615.
McKinney SM, Sieniek M, Godbole V, et al. International evaluation of an AI system for breast cancer screening. Nature 2020; 577(7788): 89–94. doi:10.1038/s41586-019-1799-6.
Parikh RB, Manz C, Chivers C, et al. Machine Learning Approaches to Predict 6-Month Mortality Among Patients With Cancer. JAMA Netw Open 2019; 2(10): e1915997. doi:10.1001/jamanetworkopen.2019.15997.
Amarasingham R, Audet AM, Bates DW, et al. Consensus Statement on Electronic Health Predictive Analytics: A Guiding Framework to Address Challenges. EGEMS (Wash DC) 2016; 4(1): 1163,9214.1163. eCollection 2016. doi:10.13063/2327-9214.1163.
Bates DW, Saria S, Ohno-Machado L, Shah A, Escobar G. Big data in health care: using analytics to identify and manage high-risk and high-cost patients. Health Aff (Millwood) 2014; 33(7): 1123–31. doi:10.1377/hlthaff.2014.0041.
Elfiky AA, Pany MJ, Parikh RB, Obermeyer Z. Development and Application of a Machine Learning Approach to Assess Short-term Mortality Risk Among Patients With Cancer Starting Chemotherapy. JAMA Netw Open 2018; 1(3): e180926. doi:10.1001/jamanetworkopen.2018.0926.
Bertsimas D, Dunn J, Pawlowski C, et al. Applied Informatics Decision Support Tool for Mortality Predictions in Patients With Cancer. JCO Clin Cancer Inform 2018; 2: 1–11. doi:10.1200/CCI.18.00003.
Holch P, Warrington L, Bamforth LCA, et al. Development of an integrated electronic platform for patient self-report and management of adverse events during cancer treatment. Ann Oncol 2017; 28(9): 2305–11.
Kotronoulas G, Kearney N, Maguire R, Harrow A, Di Domenico D, Croy S, MacGillivray S. What is the value of the routine use of patient-reported outcome measures toward improvement of patient outcomes, processes of care, and health service outcomes in cancer care? A systematic review of controlled trials. J Clin Oncol 2014; 32(14): 1480–501.
Lizee T, Basch E, Tremolieres P, et al. Cost-Effectiveness of Web-Based Patient-Reported Outcome Surveillance in Patients With Lung Cancer. J Thorac Oncol 2019; 14(6): 1012–20. doi:S1556-0864(19)30113-3.
Abbasi J. "Electronic Nose" Predicts Immunotherapy Response. JAMA 2019; 322(18): 1756. doi:10.1001/jama.2019.18225.

Download PDF

Editorial decision: Major revision
24 May, 2021
Reviews received at journal
25 Apr, 2021
Reviewers agreed at journal
23 Apr, 2021
Reviewers agreed at journal
08 Feb, 2021
Reviewers agreed at journal
01 Feb, 2021
Reviewers invited by journal
17 Jan, 2021
Editor assigned by journal
11 Jan, 2021
Editor invited by journal
08 Jan, 2021
Submission checks completed at journal
08 Jan, 2021
First submitted to journal
15 Dec, 2020

You are reading this latest preprint version

Electronic Patient-Reported Outcomes and Machine Learning in Predicting Immune-Related Adverse Events of Immune Checkpoint Inhibitor Therapies

Status:

Version 1

Abstract

Figures

Background

Methods

Results

Discussion

Conclusions

Abbreviations

Declarations

References

Status:

Version 1