AI Gum Health Evaluation with Smartphone

I. Introduction The most prevalent dental diseases are tooth decay (caries) and gum diseases (gingivitis and periodontitis). It is evidence that these diseases are caused by dental plaque (bacterial biofilm) \[1\]-\[3\]. Although most patients brush their teeth every day, they cannot keep all their teeth clean. Areas in the mouth that are difficult to access, such as crowded areas, posterior teeth or interdental areas, are usually affected (site-specific) \[4\]. After thorough professional tooth cleaning, plaque will begin to accumulate on the surface of the teeth near the edge of the gum within few days. Clinical studies indicating that regular disruption to plaque are needed and can prevent and arrest gum disease \[5\]. However, dental diseases may take years to develop, the patient usually does not have any pain symptoms \[6\] unless the disease has progressed to the advanced stage. Significant amount of resources and clinical time have been used to motivate and instruct patients to keep their mouth clean and yet the results are not satisfactory. It is desirable to adopt an automated technique for monitoring oral health daily so participants can seek for treatment when it is needed. Patients' response to plaque accumulated at the gum margin is by inflammation which brings more blood cells to the site to fight against the bacterial invasion \[7\]. Inflammation of gum is manifested as increased in redness (colour), increase in volume (oedema), and loss of surface characteristics (stippling; gum fibre attachment) \[8\]. These diseased sites can be identified by visual examination of dentists. Moreover, these inflammatory changes of gum can also be recognized by intraoral photography. The objective of this research is to apply artificial intelligent (AI) techniques to detect gum inflammation from intraoral photographs. As the target inflammation site is at gum margin with varied shape and size, semantic segmentation at pixel level is needed. For this research, the investigators have done some preliminary research works. DeepLabv3+ \[9\] encoder-decoder network with a lightweight backbone MobileNetV2 \[10\] was adopted to perform pixelwise semantic segmentation of the gingival inflammation from the intraoral photographs. The photographs are indexed by a dental specialist with more than 15 years clinical experience to obtain the index category images for the network training. II. Work done by others The first attempt to automated segmentation of gingival diseases from intraoral images with deep learning approach is proposed in \[11\]. It adopts an autoencoder network architecture with deep convolution neural network. The dataset used comprises 405 color-augmented intraoral biomarker images from 150 individuals. Areas of gingival inflammation were labelled by dental professional and the trained network can predict the inflammation with area under the receiver operating characteristic curve (AUC) 0.746. The precision and recall values are 0.347 and 0.621, respectively. The network was trained with the labeling on diseased gum. Some calculus on teeth was also predicted as diseased gum as its yellowish color is technically close to that of diseased gum. Moreover, some parts of uninterested gingival area were also predicted as diseased gum. Overall segmentation is not satisfactory. III. Our preliminary works The intraoral photographs of patients from the Faculty of Dentistry, The University of Hong Kong (HKU), which underwent periodontic treatment, were collected for the preliminary study. The study was approved by the Institutional review board of HKU (UW20-230). Total 110 standard intraoral photographs with different resolutions were collected. They are manually cropped into different smaller images, and the target labels occupy the largest possible image, which is very beneficial for training. The size of the cropped image is unified to 512×512. The completed dataset is divided into two sets, respectively 337 images for training, and 110 images for validation. Considering that there are multiple images corresponding to one patient, so when dividing the dataset, the image of the same patient will not appear in the two divided datasets. They are labeled into four health status levels (healthy, questionable healthy, questionable diseased and diseased) and verified by a dental specialist with more than 15 years clinical experience. The proposed semantic segmentation architecture is based on the DeepLabv3+ network with Xception and MobileNetV2 as the backbone. Experimental results show the effectiveness of the proposed system, which shows possible application on dental self check-up using mobile app particularly during the disease pandemic where visit to dentists are difficult or even impossible. The proposed network model can predict the contour of the interested gingival area. Experiment results show that the proposed segmentation model can accurately divide most of the gum inflammation area into four categories. The mIoU is 0.3514. It is believed that by expanding the dataset and optimizing the network structure, the performance can be further improved. The research work about the preliminary was published in \[16\]. IV. Key issues and Research gap A. There is no multiple level site-specific semantic segmentation neural network model for automatic gingival inflammation detection from intraoral photograph. B. There is no well-labeled training dataset for the application of automatic gingival inflammation detection from intraoral photograph. Reference: 1. J.K. Clarke, "On the bacterial factor in the aetiology of dental caries," British journal of experimental pathology, vol. 5, no. 3, pp. 141-147, 1924. 2. S.S. Socransky, A.D. Haffajee, "The bacterial etiology of destructive periodontal disease: current concepts," Journal of periodontology, vol. 63, pp. 322-331, 1992. 3. W.F. Liljemark, C. Bloomquist, "Human oral microbial ecology and dental caries and periodontal diseases," Critical Reviews in Oral Biology \& Medicine, vol. 7, no. 2, pp. 180-198, 1996. 4. H.J. Breen, N.W. Johnson, P.A. Rogers, "Site-specific attachment level change detected by physical probing in untreated chronic adult periodontitis: review of studies 1982超1997," Journal of periodontology, vol. 70, no. 3, pp. 312-328, 1999. 5. H. Löe, "Oral hygiene in the prevention of caries and periodontal disease," International dental journal, vol. 50, no. 3, pp. 129-139, 2000. 6. L. Croxson, "Periodontal awareness: the key to periodontal health," International dental journal, vol. 43, (2 Suppl 1) pp. 167-177, 1993. 7. R. Genco, J. Slots, "Host Responses Host Responses in Periodontal Diseases," Journal of dental research, vol. 63, no. 3, pp. 441-451, 1984. 8. G.C. Armitage, "Clinical evaluation of periodontal diseases," Periodontology 2000, vol. 7, no. 1, pp. 39-53, 1995. 9. L.C. Chen, Y. Zhu, G. Papandreou, F. Schroff and H. Adam, "Encoder-decoder with atrous separable convolution for semantic image segmentation," in Proceedings of the European conference on computer vision (ECCV), 2018, pp. 801-818. 10. M. Sandler, A. Howard, M. Zhu, A. Zhmoginov and L. Chen, "MobileNetV2: Inverted Residuals and Linear Bottlenecks," in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 2018, pp. 4510-4520. 11. A. Rana, G. Yauney, L.C. Wong, O. Gupta, A. Muftu and P. Shah, "Automated segmentation of gingival diseases from oral images," in 2017 IEEE Healthcare Innovations and Point of Care Technologies (HI-POCT), Bethesda, MD, USA, 2017, pp. 144-147. 12. O. Ronneberger, P. Fischer, T. Brox, "U-Net: Convolutional Networks for Biomedical Image Segmentation," in 18th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), 2015, pp. 234-241. 13. B. Zoph, V. Vasudevan, J. Shlens and Q.V. Le, "Learning Transferable Architectures for Scalable Image Recognition," in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 2018, pp. 8697-8710. 14. F. Chollet, "Xception: Deep Learning with Depthwise Separable Convolutions," in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 2017, pp. 1800-1807. 15. M.X. Tan and Q.V. Le, "EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks," in Proceedings of the 36th International Conference on Machine Learning, PMLR, 2019, vol. 97, pp. 6105-6114. 16. G.-H. Li, T.-C. Hsung, B. W.-K. Ling, W. Y.-H. Lam, G. Pelekos, and C. McGrath, "Automatic Site-Specific multiple level gum disease detection based on deep neural network," in 2021 15th International Symposium on Medical Information and Communication Technology (ISMICT) (ISMICT2021), Xiamen, China, Apr. 2021.