Augmented Reality-assisted Localization of Solitary Pulmonary Nodules for Precise Sublobar Lung Resection

Overview

With the popularization of CT screening, the detection rate of small lung nodules has greatly increased. Therefore, the clinical thoracoscopic lung nodule biopsy and sub-lobectomy for radical resection of lung cancer are greatly required. Accurate resection of lung nodule depends on precise localization of pulmonary nodules. However, preoperative CT-guided Hook-wire positioning under local anesthesia, which is the current primary localization method, requires high equipment and expense, and may cause physical and mental trauma to the patient. Augmented reality (AR) is an innovative technology that superimpose a virtual scene into the real environment by fusing images, videos, or computer-generated models with patients during surgical operations. It can visually display the anatomical structures of organs or lesions, which significantly improves surgical efficiency. This project intends to use AR technology to localize the solitary pulmonary nodule (SPN) before surgery, compared with CT-guided Hook-wire localization. Compared with the localization of SPNs under CT guidance, AR-assisted localization technology apparently is less time-consuming and can be performed immediately before surgery under general anesthesia, lessening pain, reduce costs of time and equipment, increase the success rate of sub-lobectomy, and improve the overall efficiency of surgical treatment of pulmonary nodules.

1. Overall objective: 1. Research and development of puncturing system of preoperative rapid positioning of lung nodules under the guidance of augmented reality images with hook-wire. 2. Accurate sublobectomy of early-stage lung cancer assisted by thoracoscopic positioning needle and virtual imaging to relieve the pain caused by localization of pulmonary nodule, reduce the risk of puncturing. 2. Innovation points: 1. Establish an accurate fitting database of 3D virtual lung model reconstructed from CT scans. Because the patient's posture may be different when in CT scan and under general anesthesia, and also affected by breathing at the same time, it may cause difference between the virtual lung and the real one. In addition, during the operation of the thoracoscopy, lungs are in a collapsed state. Therefore, in order to achieve an accurate fit between virtual and reality lung, it is necessary to generate a virtual lung model that can be automatically adjusted according to body surface markers and lung anatomical markers. This technology has not yet been reported in the field of digital reconstruction of lung. 2. Through augmented reality technology, puncturing localization of lung nodule will not necessarily depend on multiple CT scan. Puncture can be performed under general anesthesia pre-operatively, which can relieve the pain of the patient, reduce the cost of puncture and the harm of complications, and reduce the radiation exposure of the patient and the doctor. This technical concept has not been reported. 3. Detailed description 20 patients with solitary pulmonary nodule were selected and prepared for VATS sublobectomy. For this clinical study, after fixing the enrolled patients on the operating bed, they were general anesthetized to reduce oral and airway secretions. The patients were intubated, the ventilator was connected, and vital signs were monitored. Markers will be set on their chest three days before surgery and CT scan will be performed for everyone. The digital imaging and communications (DICOM) image data obtained from the scan were stored. After the first CT scan for each patient, a 3D Mask of patient was constructed and cropped using the Crop Mask function and the Erase curve erasing tool in Edit Masks; using the Calculate Part tool to calculate the 3D model of each part; STL format files of the model were exported separately to Autodesk FBX format for the subsequent development and study; Autodesk FBX model and Vuforia Image Target database were imported into Unity to create a suitable AR scene. After compiling the data, the C# solution was generated and deployed into Microsoft HoloLens before starting the AR surgical system. Hook-wire was used for percutaneous puncture according to the AR localization of the SPNs. 4. Data collection During the procedure, the angles between the planned and actual puncture paths, the distance between the planned entry point and the punctured point, and the distance between the planned and actual puncture depths were all recorded and calculated. Another CT scan was took before the calculation for each patient to evaluate the puncture result. The three-dimensional position of the puncture needle was reconstructed by JinSe MIDIVI Intelligent Cloud Platform (JinSe Medical Co., Ltd. Shanghai, China). Then the three-dimensional puncture needle was compared with the preset puncture path. The main parameters for comparison are the angle between the two lines, the difference in the spatial position of the vertices of the two lines, and the distance between the actual entry point and the plan one after the puncture needle enters the body surface. Then these parameters are automatically calculated by the software Materialise 3-matic (Materialise Inc., Belgium). The hit rate of hook-wire localization was also statistically significant.