The establishment of a multi-source dataset of images on common oral lesions

doi:10.21203/rs.3.rs-496890/v1

Download PDF

Research Article

The establishment of a multi-source dataset of images on common oral lesions

https://doi.org/10.21203/rs.3.rs-496890/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

Purpose

To establish an oral lesion image database that could accelerate the development of artificial intelligence systems for lesion recognition and referral decision.

Materials and Methods

We describe the establishment of a multi-sourced image dataset through the development of a platform for the collection and annotation of images. Further, we developed a used-friendly tool (MeMoSA® ANNOTATE) for systematic annotation to collect a rich dataset associated with the images. We evaluated the sensitivities comparing referral decisions through the annotation process with the clinical diagnosis of the lesions to identify lesions that are challenging to identify through images alone.

Results

The image repository hosts 2474 images of oral lesions consisting of oral cancer, oral potentially malignant disorders, benign lesions, normal anatomical variants and normal mucosa that were collected through our platform, MeMoSA® UPLOAD. Over 800 images were annotated by seven oral medicine specialists on MeMoSA®ANNOTATE, to mark the lesion and to collect clinical labels. The sensitivity in referral decision for all lesions that required a referral for cancer management/surveillance was moderate to high depending on the type of lesion (64.3–100%).

Conclusion

This is the first description of a database with well-annotated oral lesions. This database has already been used for the development of AI algorithm for classifying oral lesions. Further expansion of this database could accelerate the improvement in AI algorithms that can facilitate the early detection of oral potentially malignant disorders and oral cancer.

Cancer Biology

Oncology

Theoretical Computer Science

Dentistry

Oral Cancer

Oral Oncology

Annotation Tool

Access to Care

Artificial Intelligence

Oral Diagnosis

Oral cancer (OC) was diagnosed in more than 350,000 individuals globally, with over 177,000 deaths in 2018 (GLOBOCAN 2018). OC disproportionately affects low- and middle-income countries (LMICs), and the majority are detected late resulting in poor patient survival (Warnakulasuriya 2009). Late diagnosis is mainly due to poor oral health awareness and inaccessibility to healthcare due to large rural populations that are exacerbated by remote geographical locations and insufficient healthcare resources (Nagao and Warnakulasuriya 2020). OC is often preceded by oral potentially malignant disorders (OPMD), affording the opportunity to detect these lesions for management before the development of OC. Early detection of OPMD and OC requires trained healthcare practitioners who can differentiate high-risk lesions that are malignant or likely to become malignant from the many types of oral lesions that do not have any risk for malignant transformation and enabling the appropriate management of oral lesions (Güneri and Epstein 2014). Indeed, a lack of dental specialists has been associated with increased rates of delay in the detection of oral cancer (Crossman et al. 2016; Onizawa et al. 2003).

Applications of artificial intelligence (AI) techniques for disease detection, prognostication and prediction by medical image analysis are now being embraced for healthcare decision-making in medicine and dentistry (Joda et al. 2020; Lindsell et al. 2020). A study by Esteva et al. (2017) has successfully used a form of AI called deep convolutional neural networks (CNNs) to classify skin lesions, identifying those that are malignant and most deadly. The trained CNN performance was on par with board-certified dermatologists, demonstrating that AI algorithms could reach a level of competence comparable to trained experts (Esteva et al. 2017). Such automated classification systems could be applied to address the limited number of dental specialists which is often a bottleneck for clinical diagnosis LMICs.The development of AI to automate the identification of OPMD and OC through clinical images is still at its infancy mainly because of the lack of systematically collected well-annotated images of oral lesions that are required to develop such systems. Large training datasets with their corresponding ground truth (GT) and annotated labels are necessary to achieve good performance through efficient learning and to prevent overfitting (Krig 2014; Mendonça et al. 2013; Yamashita et al. 2018).

We have initiated a repository for the collection of oral lesion images through selected members of the recently established Asia-Pacific Oral Cancer Network (APOCNET) (Syed Mohd Sobri et al. 2020) and other clinical collaborators. Here, we report on the development of a platform for secure transfer of images to the repository and a customized annotation tool for a uniform collection of clinically relevant information of images of the oral cavity that have hitherto not been available. This resource has resulted in the development of AI systems for the classification of oral lesions and will continue to improve the performance of AI. When incorporated into a mobile phone application, such as MeMoSA® (Haron et al. 2020), early detection of OPMD and OC at point-of-care can be facilitated.

Images utilised in this study were collected from clinical collaborators and Open Access sources. Ethical approvals were obtained from the Institutional Review Board (IRB) of each centre for the use of the anonymised images. Open Access images were downloaded systematically using specific keywords through Google Images with pre-determined exclusion criteria (Supplementary Fig. 1). To build a repository of well-annotated images, a platform to collect, store and annotate images securely was developed (Fig. 1). Two components make up the platform; MeMoSA® UPLOAD and MeMoSA® ANNOTATE, which are linked to the MeMoSA® Data Vault (an image repository) through individual workbenches. All components are located on a secure server in the Data Intensive Computing Centre (DICC) at the University of Malaya, Malaysia. Authorised users with unique passwords can access the DICC network via an encrypted Secure Socket Layer (SSL)/HTTPS connection. The server is maintained by an in-house system administrator.

MeMoSA® UPLOAD

MeMoSA® UPLOAD is a customised web interface, hosted on a web server used by clinical contributors to transfer images securely to the MeMoSA® Data Vault. Collaborators access their accounts to upload images with the associated metadata, which includes patient demographics and risk habits for each corresponding image. Uploaded data is transferred via a secure DICC internal network communication and stored in specific folders on the DICC server.

MeMoSA® ANNOTATE

MeMoSA® ANNOTATE is a customised annotation tool built on top of the open-source tool, ImageTagger (Fiedler et al. 2018). MeMoSA® ANNOTATE was used for systematic annotation of the images in the MeMoSA® Data Vault. The images were chosen from the repository by a research team member to include a variety of lesions and of normal mucosa, and uploaded to MeMoSA® ANNOTATE with its metadata. Users perform the annotation on their workbenches by accessing the DICC network via an encrypted (SSL)/HTTPS connection.

MeMoSA® ANNOTATE enables the systematic annotation of images with labels describing the appearance of the lesion and captures referral recommendations made by the annotators. Guided by three board-certified oral medicine specialists, a decision tree that describes the flow of the tool was first developed. Seven main types of lesions were identified as significant clinical descriptors to describe an oral lesion: ulcer; white lesion, red lesion, mixed white and red lesion, swelling, pigmented lesion, and erosion (Scully 2012). Lesions that did not fit into these descriptions such as oral submucous fibrosis (OSF) were categorized as “not applicable”. Descriptions of the appearances including site, colour, presence of ulceration or swelling, texture, number of lesions, borders, and shape (Scully 2012) were incorporated into the tool under the seven main types of lesions. Finally, the annotation process culminated in a referral decision and a classification of the lesion into a disease type (Table 1). Data on the image quality of each image was also collected.

Table 1

List of referable and non-referrable lesions and the number of images collected through MeMoSA® UPLOAD
Disease Type	Number of images
Referrable lesions – cancer/high risk OPMD
Oral Cancer/Suspicious of Oral Cancer	298
Non-homogeneous leukoplakia	47
Erythroplakia	18
Oral submucous fibrosis	482
Verrucous Hyperplasia	2
Referrable lesions – low risk OPMD
Homogeneous leukoplakia	90
Lichenoid lesion/Lichen planus	539
Discoid lupus erythematosus (other than lip)	29
Referrable lesions - for other reasons/no referral needed
Benign	1041
Developmental abnormalities	23
No referral needed
Normal anatomical variant	73
Normal mucosa	61
Total	2703

Four board-certified oral medicine specialists validated the annotation tool by using a test image dataset consisting of eight OPMD, ten non-OPMD and two oral cancers. A questionnaire was administered to collect feedback on the accuracy of clinical descriptors, the feasibility of the annotation process and the user interface experience. The tool was modified based on this feedback. The tool was further tested by seven board-certified oral medicine specialists using 400 Open Access oral lesion images where each image was annotated by at least three specialists. These images included a variety of oral lesions and normal variants that fall within the following broad categories of OC, OPMD, non-OPMDs (benign lesions and developmental abnormalities (DA)) and normal mucosa/normal anatomical variants (NAV). A virtual workshop was conducted amongst all the annotators to discuss discrepancies in the clinical diagnosis, to reach a consensus on annotation labels. Annotations included bounding boxes and labels, such as size, margin, site, outline were captured for each image in SQL format and stored in the PostgreSQL database in the MeMoSA® Data Vault.

The web server provides an application programming interface (API) for secure outward transfer of information from the MeMoSA® Data Vault (Fig. 1). For example, to facilitate AI algorithm training using the annotated images, between members of our group, data can be downloaded as described above. The server is password protected and only accessible to an authorized user. The use of these images in the development of AI for the automated detection and classification of oral lesions have been published (Welikala et al. 2020a; 2020b).

Sensitivity of Annotations to Ground Truth

To identify lesions that were difficult to diagnose and make referral decisions from visual images alone which could indicate similar difficulties for the AI algorithm to classify lesions, we calculated the sensitivity of referral recommendations made by the annotators through MeMoSA® ANNOTATE compared to the clinical diagnosis (ground truth; GT). We also computed accuracy, positive predictive value (PPV) and F1 scores for each referral category compared to GT. For this analysis, 400 images were analysed, consisting of 61 images of OC, 90 of OPMD, 159 of benign lesions, 20 of DA and 70 of normal mucosa/NAV.

OC and OPMD are classified as referable lesions, benign lesions and DA could be either ‘refer for other reasons’ or ‘no referral’ depending on the clinical attention required, whilst mucosa with NAV and those that do not have any changes would not require any referral and therefore categorized as ‘no referral’. Sensitivity between the referral decision from the data collected through the annotation tool versus the GT was calculated for each disease category (OC, OPMD, benign, DA and normal mucosa/NAV) and for individual disease type for the OPMD category. Statistical analysis was conducted using SAS V9.4 (SAS Institutes, Cary, NC, USA).

MeMoSA® UPLOAD

Two thousand seven hundred and three (2703) images were uploaded using MeMoSA® UPLOAD between June 2019 to September 2020, comprising of a variety of oral lesions as detailed in Table 1. The largest number of images collected was of benign lesions with 1041 images. This was followed by 539 images representing oral lichen planus (OLP), 482 images OSF and 298 images of OC. Images were assigned a unique sequential ID by the clinical collaborators as they were submitted through MeMoSA® UPLOAD. Manual assessment of the quality of uploaded images was done to i) determine image quality ii) identify images with potentially identifiable content and iii) remove duplicated images. Of the 2703 images, 105 images were excluded because they were out of focus, 105 images had extraoral content that could potentially identify a subject and another 19 images were excluded because they were duplicates. A subset of these images was randomly chosen to be annotated as described below.

MeMoSA®ANNOTATE

Images to be annotated are preloaded by the administrator and displayed as shown in Fig. 2a. A selected image will appear along with its corresponding metadata (Fig. 2b & c). Annotators will first annotate if a lesion is present on the image (Fig. 2d). If present, a bounding box is placed around the lesion using their cursor, marking the location of the lesion (Fig. 2e). The lesion type and its associated clinical descriptors to describe the appearance of the lesion is determined (Fig. 2d) and a referral decision is decided (Fig. 2f), followed by naming the lesion (Fig. 2g). Finally, the image quality is rated (Fig. 2h) before the annotations are saved (Fig. 2i). Once submitted, the annotation is filed (Fig. 2j). More than 800 images were annotated by seven oral medicine specialists.

Sensitivity of Annotations to Ground Truth

Of the 800 images, 400 images with GT resulting in 2800 annotations were included in our analysis. Overall, the sensitivities in referral decision for all lesions that required a referral for cancer management or surveillance was moderate to high, ranging from 64.3% for erythroplakia and OSF up to 100% for discoid lupus erythematosus (DLE). For the lesions falling into the category of “refer-high risk” the overall sensitivity between the annotated decision and the GT to refer a lesion was 86.7%, and for those with cancer it was 90.6%. In this category, the lowest sensitivity was for erythroplakia and OSF at 64.3% for each of these categories. About 10.4% of the “refer-high risk” lesions were annotated to be “refer-low risk” where the majority of these fell into the non-homogenous leukoplakia disease type (30.2%) (Table 2). For lesions in the “refer-low risk” category, the overall sensitivity to refer a lesion was 85.7% with DLE having the highest sensitivity at 100% and homogenous leukoplakia at 80.2% being the lowest. Lesions in the “refer-low risk” category that were annotated as “refer-high risk” were mainly homogenous leukoplakia (16.5%). Overall, 13.3% of “refer-high risk” and 14.2% of “refer-low risk” lesions, were annotated as “refer for other reasons” or “no referral needed”. The majority of these were OSF, erythroplakia and homogenous leukoplakia. Accuracy of 90.5% and 92.0% were achieved for “refer-high risk” and “refer-low risk” categories respectively. The PPV was 80.7% for “refer-high risk” and 71.5% for “refer-low risk”, with F1 scores of 78.5% and 74.2% respectively (Supplementary Table 1). Benign lesions and developmental abnormalities also were correctly referred for other reasons/no referral with accuracies of 88.8% and 93.6% respectively. Similarly, normal mucosa or mucosa with NAV were correctly identified as requiring no referral with a sensitivity of 83.7%, with only 5.7% annotated for referrals associated with cancer management or surveillance (Table 2). Accuracy of 88.6% and 96.0% were achieved for “refer-other reasons” and “no referral” categories respectively. The positive predictive value was 85.8% for “refer-other reasons” and 93.0% for “no referral”, with F1 scores of 87.5% and 88.1% respectively (Supplementary Table 1).

Table 2

Sensitivity of referral decisions made following annotations on MeMoSA® ANNOTATE compared to the Ground Truth
Referral Decision	Disease type based on ground truth	Sensitivity of annotations in MeMoSA®ANNOTATE compared to Ground Truth, % (n refers to number of annotations)
Referral Decision	Disease type based on ground truth	Refer* (%)	(n)	Refer-high risk (%)	(n)	Refer-low risk (%)	(n)	Refer-other reasons /No referral (%)	(n)	No referral (normal mucosa) (%)	(n)
Refer-high risk	Overall (n = 637)	86.7	552	76.3	486	10.4	66	11.6	74	1.7	11
	Oral Cancer (n = 427)	90.6	387	87.1	372	3.5	15	9.1	39	0.2	1
	Non-homogenous leukoplakia, NHL (n = 126)	88.1	111	57.9	73	30.2	38	11.9	15	0	0
	Erythroplakia (n = 14)	64.3	9	57.1	8	7.1	1	35.7	5	0	0
	Oral submucous fibrosis, OSF (n = 70)	64.3	45	47.1	33	17.1	12	21.4	15	14.3	10
Refer-low risk	Overall (n = 420)	85.7	360	8.6	36	77.1	324	14	59	0.2	1
	Homogenous leukoplakia, HL (n = 91)	80.2	73	16.5	15	63.7	58	18.7	17	1.1	1
	Lichenoid lesion / Oral lichen planus, OLP (n = 322)	87	280	6.5	21	80.4	259	13.1	42	0	0
	Discoid lupus erythematous, DLE (n = 7)	100	7	0	0	100	7	0	0	0	0
Refer-other reasons / no referral needed	Overall (n = 1253)	9.2	115	5.6	70	3.6	45	89.3	1119	1.5	19
	Benign (n = 1113)	10.1	112	6.3	70	3.8	42	88.8	988	1.2	13
	Developmental anomalies, DA (n = 140)	2.1	3	0	0	2.1	3	93.6	131	4.3	6
No referral (normal mucosa)	Normal anatomical variant/no lesion (n = 490)	5.7	28	2	10	3.7	18	10.6	52	83.7	410
*Refer: Both refer -high and -low risk

Whilst emerging evidence demonstrates that AI could be used in classifying oral lesions (Uthoff et al. 2018), progress has been slow due to the lack of a clinically labelled, well-annotated training dataset on oral lesions. MeMoSA® UPLOAD provides a standardised system to collect and uniformly transfer images and data, facilitating collaborations across many countries where oral cancer incidence is high. The large number of OC images collected over a short time represents the high burden of OC in these countries (GLOBOCAN 2018). The images collected were representative of the most prevalent oral mucosal lesions in South and South-East Asia. These included OLP, which is prevalent in Malaysia (Zain et al. 1997), while OSF is prevalent in Sri Lanka, Nepal and India (Amarasinghe et al. 2010; Warnakulasuriya et al. 1984). Similar efforts to establish image datasets have been seen in dermatology, in the HAM10000 (10,015 images) (Tschandl et al. 2018) and PH2 (200 images) databases (Mendonça et al. 2013). Both these databases have facilitated machine learning for automated classification of skin lesions and accelerating the development in this field (Esteva et al. 2017). The Age-Related Eye Disease Study Research Group (2001) have also collected over 130,000 colour fundus images from 4,613 patients as a result of a 12-year longitudinal study for a better understanding of disease progression and risk factors behind macular degeneration. We believe that a large global network would accelerate the collection of a substantial number of diverse images of oral lesions that will cover all important diagnosis of oral lesions. MeMoSA® UPLOAD will enable the expansion to other clinical collaborators with ease to continue building on the database.

MeMoSA® ANNOTATE enables images to be annotated in a way that mimics the observations that a clinical specialist makes when examining a lesion. Tools similar to MeMoSA®ANNOTATE have been described in the literature. ‘DerMat’, used to annotate images in the PH2 database (Mendonça et al. 2013) allows users to draw and focus on a region of interest (Ferreira et al. 2012). In the work of Age-Related Eye Disease Study Research Group (2001), colour fundus images were graded by-hand by technicians and were used in several studies to train and validate deep learning algorithms (Burlina et al. 2017a; Burlina et al. 2019; Burlina et al. 2017b). The difference between MeMoSA®ANNOTATE and the tools mentioned is that it was built with a decision tree to capture or mimic a clinical oral examination by a specialist, and customised to annotate features of oral lesions. Specialists manually annotated the images and referred to the metadata to arrive at a referral recommendation based on the risk of malignant transformation of the suspected lesion. Bounding boxes and clinical descriptions were collected to build a rich set of labels for each lesion. Some of these annotations have been proven to improve the performance of the AI algorithm for image detection and classification by employing attention mechanisms in the training by guiding the AI to regions of interest within the image (Welikala et al., manuscript in preparation). Further, the detailed annotated descriptions could improve AI through multiheaded training. Besides, this information could be used in the future to generate text rich reports for each referral recommendation by the AI. This could give clinicians more confidence in the type of information that has been used by the AI to arrive at a referral decision. This database containing these annotated images has been used to guide the training of a deep learning algorithm, as described in recent publications (Welikala et al. 2020a; 2020b).

We observed that sensitivity was high in providing the correct referral decision for cancer surveillance compared to the GT, with more than 85% for all disease types except erythroplakia and OSF. The high sensitivity shows that the data input in the form of images, and information on patient demographics and risk habits are reliable in making a referral decision and should be reliable information for AI training. However, further refinements can be made. Some lesions have been identified to be difficult to make decisions based on images alone, such as early OSF, which is particularly difficult to diagnose even during a clinical oral examination, since it may appear as blanching of the oral mucosa or loss of pigmentation which only appears in some South Asian populations (Warnakulasuriya 2018). Clinical diagnosis is usually reached after examination of mucosa and evaluating clinical history and information on symptoms such as burning sensation, dry mouth or limited mouth opening along with the presence of palpable fibrous bands. Therefore, such should be collected as metadata in anticipation that these would be also useful in the training of the AI algorithm. As for erythroplakia, 35.7% were annotated as “refer for other reasons”, by identifying it as a benign lesion. Clinically, erythroplakia could be mistaken as several red-appearing benign conditions, including erythematous candidiasis or erythema migrans (Warnakulasuriya 2018). However, the review of a single image hampered the ability of the specialist to distinguish erythroplakia from these benign lesions. We will need to consider displaying multiple images of the oral cavity for each patient to present a comprehensive view to obtaining more accurate annotations, or collect clinical information and medical history based on questions that enable the capture of these information. As for lesions that did not require a referral, the high positive predictive values and F1 scores for “refer for other reasons” and “no referral needed” categories indicate these lesions were correctly identified and not referred, meaning the AI could reduce the burden to the healthcare system due to inaccurate referral should it be implemented.

Whilst patient demographics and risk habit information are available, further questions to collect more comprehensive clinical information that could help annotators reach a more accurate decision should be considered. This is particularly so for the OPMD lesions such as erythroplakia and leukoplakia which are by definition diagnosed based on the exclusion of other diagnoses. Therefore, in the next developmental stage of MeMoSA®ANNOTATE, avenues for richer data collection possibly including patient history such as chief complaint, history of chief complaint, medical history and oral hygiene products used would be incorporated. We could also benefit from a standardisation of images captured and an image capturing protocol is being developed.

We described the establishment of a systematic and secure platform for the development of an oral lesion image repository that has hitherto not been available. Given that clinical oral examination can be conducted relatively easily by a primary healthcare practitioner, automated detection and classification algorithm would go a long way in helping clinicians distinguish the multitude of types of oral lesions and NAV that occur in the oral cavity. Our work as described here could accelerate the progress in developing such an AI system.

Acknowledgements

We would like to thank Prof Mahima V Guledgud of JSS Dental College & Hospital and Dr Anand Ramanathan of the University of Malaya for initial discussions on the project. The authors are grateful to Dr Felix Anthony Wiranata Wong of Trisakti University, Indonesia for technical assistance in collecting images.

Funding source

This work was supported by the Medical Research Council [grant number MR/S013865/1]. The funders had no involvement in the conduct of the research or preparation of the manuscript.

Conflicts of Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Availability of data and material

The datasets are not publicly available due to privacy reasons but are available from the corresponding author on reasonable request.

Code availability

Not applicable

Author Contributions

Rajendran, S contributed to design, data acquisition, analysis and interpretation, drafted and critically revised the manuscript; Lim, JH, Yogalingam, K and Lim, YZ contributed to data acquisition and drafted manuscript; Kallarakkal, T, Jayasinghe, R, Rimal, J, Amtha, R, Patil, K and Welikala, R contributed to data acquisition and critically revised the manuscript; Zain, R contributed to conception and design, data acquisition and critically revised the manuscript; Kerr, A contributed to design, data acquisition and critically revised the manuscript; Remagnino, P, Gibson, J, Tilakaratne, W, Barman, S and Chan, CS contributed to conception and critically revised the manuscript; Liew,CS contributed to design, data acquisition and critically revised the manuscript; Yang,YH contributed to data analysis and interpretation, and critically revised the manuscript; Cheong,SC contributed to conception and design, data acquisition, analysis and interpretation, drafted and critically revised the manuscript. All authors gave their final approval and agree to be accountable for all aspects of the work.

Ethics Approval

This study was performed in line with the principles of the Declaration of Helsinki. Ethical approval for the transfer and use of images for this study was granted from all provider institutions in Sri Lanka (ERC/FDS/UOP/I/2019/4), Indonesia (249/S3/KEPK/FKG/3/2019), Malaysia (UM ethics – DF OS1902/0011(L)/1970) and Nepal (ref no. 1430) and Kingston University (project no. 083.1).

Consent to participate

Informed consent was obtained from all individual participants included in the study.

Consent to publication

No identifying information was included in this article. However, all participants provided informed consent to publish their anonymised images.

Age-Related Eye Disease Study Research Group. 2001. The age-related eye disease study system for classifying age-related macular degeneration from stereoscopic color fundus photographs: The age-related eye disease study report number 6. American Journal of Ophthalmology. 132(5):668-681.

Amarasinghe H, Johnson N, Lalloo R, Kumaraarachchi M, Warnakulasuriya S. 2010. Derivation and validation of a risk-factor model for detection of oral potentially malignant disorders in populations with high prevalence. British Journal of Cancer. 103(3):303-309.

Burlina P, Pacheco KD, Joshi N, Freund DE, Bressler NM. 2017a. Comparing humans and deep learning performance for grading amd: A study in using universal deep features and transfer learning for automated amd analysis. Computers in Biology and Medicine. (82):80-86.

Burlina PM, Joshi N, Pacheco KD, Liu TA, Bressler NM. 2019. Assessment of deep generative models for high-resolution synthetic retinal image generation of age-related macular degeneration. JAMA Ophthalmology. 137(3):258-264.

Burlina PM, Joshi N, Pekala M, Pacheco KD, Freund DE, Bressler NM. 2017b. Automated grading of age-related macular degeneration from color fundus images using deep convolutional neural networks. JAMA Ophthalmology. 135(11):1170-1176.

Crossman T, Warburton F, Richards MA, Smith H, Ramirez A, Forbes LJ. 2016. Role of general practice in the diagnosis of oral cancer. British Journal of Oral and Maxillofacial Surgery. 54(2):208-212.

Esteva A, Kuprel B, Novoa RA, Ko J, Swetter SM, Blau HM, Thrun S. 2017. Dermatologist-level classification of skin cancer with deep neural networks. Nature. 542(7639):115-118.

Ferreira PM, Mendonça T, Rozeira J, Rocha P. 2012. An annotation tool for dermoscopic image segmentation. Proceedings of the 1st International Workshop on Visual Interfaces for Ground Truth Collection in Computer Vision Applications; 2012.

Fiedler N, Bestmann M, Hendrich N. 2018. Imagetagger: An open source online platform for collaborative image labeling. Robot World Cup; 2018: Springer.

Globocan: Estimated cancer incidence, mortality and prevalance worldwide in 2018. 2018. Lyon:World Health Organisation,; [accessed 2020 3 April]. https://gco.iarc.fr/today/fact-sheets-populations.

Güneri P, Epstein JB. 2014. Late stage diagnosis of oral cancer: Components and possible solutions. Oral Oncology. 50(12):1131-1136.

Haron N, Zain RB, Ramanathan A, Abraham MT, Liew CS, Ng KG, Cheng LC, Husin RB, Chong SMY, Thangavalu LAP. 2020. M-health for early detection of oral cancer in low-and middle-income countries. Telemedicine and e-Health. 26(3):278-285.

Joda T, Yeung A, Hung K, Zitzmann N, Bornstein M. 2020. Disruptive innovation in dentistry: What it is and what could be next. Journal of Dental Research. 16:22034520978774.

Krig S. 2014. Ground truth data, content, metrics, and analysis. Computer vision metrics: Survey, taxonomy, and analysis. Berkeley, CA: Apress. p. 283-311.

Lindsell CJ, Stead WW, Johnson KB. 2020. Action-informed artificial intelligence—matching the algorithm to the problem. JAMA. 323(21):2141-2142.

Mendonça T, Ferreira PM, Marques JS, Marcal AR, Rozeira J. Ph 2-a dermoscopic image database for research and benchmarking. 2013. 35th annual international conference of the IEEE engineering in medicine and biology society (EMBC); 2013: IEEE.

Nagao T, Warnakulasuriya S. 2020. Screening for oral cancer: Future prospects, research and policy development for asia. Oral Oncology. 105:104632.

Onizawa K, Nishihara K, Yamagata K, Yusa H, Yanagawa T, Yoshida H. 2003. Factors associated with diagnostic delay of oral squamous cell carcinoma. Oral Oncology. 39(8):781-788.

Scully C. 2012. Oral and maxillofacial medicine-e-book: The basis of diagnosis and treatment. 2nd ed. Edinburgh: Churchill Livingstone/Elsevier.

Syed Mohd Sobri SNS, Kanapathy J, Liew CS, Cheong SC. 2020. The establishment of the asia-pacific oral cancer network (APOCNET)-inaugural stakeholders' meeting. Oral Diseases. 26(5):1094-1097.

Tschandl P, Rosendahl C, Kittler H. 2018. The ham10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Scientific data. 5:180161.

Uthoff RD, Song B, Sunny S, Patrick S, Suresh A, Kolur T, Keerthi G, Spires O, Anbarani A, Wilder-Smith P. 2018. Point-of-care, smartphone-based, dual-modality, dual-view, oral cancer screening device with neural network classification for low-resource communities. PloS one. 13(12):e0207493.

Warnakulasuriya K, Ekanayake A, Sivayoham S, Stjernswärd J, Pindborg J, Sobin L, Perera K. 1984. Utilization of primary health care workers for early detection of oral cancer and precancer cases in sri lanka. Bulletin of the World Health Organization. 62(2):243.

Warnakulasuriya S. 2009. Global epidemiology of oral and oropharyngeal cancer. Oral Oncology. 45(4-5):309-316.

Warnakulasuriya S. 2018. Clinical features and presentation of oral potentially malignant disorders. Oral surgery, oral medicine, oral pathology and oral radiology. 125(6):582-590.

Welikala RA, Remagnino P, Lim JH, Chan CS, Rajendran S, Kallarakkal TG, Zain RB, Jayasinghe RD, Rimal J, Kerr AR et al. 2020a. Automated detection and classification of oral lesions using deep learning for early detection of oral cancer. IEEE Access. 8:132677-132693.

Welikala RA, Remagnino P, Lim JH, Chan CS, Rajendran S, Kallarakkal TG, Zain RB, Jayasinghe RD, Rimal J, Kerr AR et al. Fine-tuning deep learning architectures for early detection of oral cancer. 2020b. Mathematical and Computational Oncology: Second International Symposium, ISMCO 2020, San Diego, CA, USA, October 8–10, 2020, Proceedings; 2020b: Springer Nature.

Yamashita R, Nishio M, Do RKG, Togashi K. 2018. Convolutional neural networks: An overview and application in radiology. Insights into imaging. 9(4):611-629.

Zain RB, Ikeda N, Razak IA, Axéll T, Majid ZA, Gupta PC, Yaacob M. 1997. A national epidemiological survey of oral mucosal lesions in malaysia. Community dentistry and oral epidemiology. 25(5):377-383.

Download PDF

Version 1

posted

You are reading this latest preprint version

The establishment of a multi-source dataset of images on common oral lesions

Status:

Version 1

Abstract

Purpose

Materials and Methods

Results

Conclusion

Figures

Introduction

Materials And Methods

MeMoSA® UPLOAD

MeMoSA® ANNOTATE

Sensitivity of Annotations to Ground Truth

Results

MeMoSA® UPLOAD

MeMoSA®ANNOTATE

Sensitivity of Annotations to Ground Truth

Discussion

Limitations

Conclusion

Declarations

References

Status:

Version 1