Artificial intelligence (AI) is a computational approach to problem-solving commonly associated with human intelligence. Recently, AI has experienced a surge applied to many fields, including finance,1 transportation,2 and health care.3,4 Many studies have explored the application of AI in radiology.5-11 With AI’s growing capabilities to analyze images, review large data sets, and constantly learn from new data, it has potential to transform interventional oncology (IO). The purpose of this article is to review the applications of AI in IO. Some of these applications have been tested and presented at meetings or published; other potential applications are inferred from application of AI in other fields.

DEFINITION OF ARTIFICIAL INTELLIGENCE

AI uses computing power to analyze data and perform complex tasks involving pattern recognition or problem-solving. A subset of AI is machine learning (ML), where a computer “learns” a task from data by automatically improving through repeated experiences.12 In contrast to other forms of AI, the ML system independently learns without explicit hard-coded “if-then” instructions.

Artificial neural networks (ANNs) are a specific type of ML that automatically learn relationships using “artificial neuron” layers. Deep learning involves the use of ANNs that have a large number of artificial neuron layers to learn more complex relationships. In the case of convolutional neural networks (CNNs), “convolutional” layers perform image analysis in a manner somewhat similar to the connectivity pattern of the retina and visual cortex. AI can be applied along the entire clinical continuum of IO, from research/outcome analysis to clinical applications for diagnosis, patient selection, order scheduling/workflow, and intraprocedural guidance.

RESEARCH WITH ARTIFICIAL INTELLIGENCE: BIG DATA AND OUTCOME REPOSITORIES

AI has potential to impact outcomes research in IO. The traditional paradigm for clinical research involves starting with a retrospective study, conducting prospective clinical trials that span several years, and forming clinical management guidelines (combining existing data from these trials with expert opinion and consensus). A limitation of this approach is that these efforts are often independent, resulting in the release of several conflicting management algorithms.13-17 Moreover, the current clinical research process is incremental, usually involving the manipulation of one clinical variable. In reality, several clinical variables may affect a particular outcome, and relationships may not be linear. Because the current process is time-consuming, expensive, and cumbersome for one variable alone,18,19 it is not usually feasible to simultaneously evaluate multiple variables.

Consider a different paradigm in which a standardized organized effort is made to collect multi-institutional pre-, intra-, and postprocedural data.20 This information can then be stored in centralized databases that can be continuously analyzed by ML systems. Clinical questions can be translated into ML prediction algorithms, which can eventually act as a substitute for retrospective studies. These trained models can be validated with prospective data, with a continuous cycle of prospective validation and model optimization. Unlike the incremental approach of single-variable studies, the entire clinical picture can be explored with inclusion of an unlimited number of variables. Management guidelines can then become more personalized, combining preset robust clinical variables prospectively validated by these ML systems and the output of a patient’s entire clinical picture fed through ML models.

POTENTIAL CLINICAL APPLICATIONS IN INTERVENTIONAL ONCOLOGY

In addition to impact on research, AI has potential to improve clinical care in IO. ML may be utilized to review preprocedural images for improved lesion detection and characterization and make assessments to predict the success of various IO procedures.

Lesion Detection

Earlier identification of tumors is important for improving care of IO patients, allowing for more curative-intent IO therapies (eg, ablation) to be utilized. In this manner, ML techniques can assist in diagnosing malignant lesions earlier. A recent study utilized CNNs to distinguish between different classes of liver masses on CT examinations.21 The study utilized select two-dimensional slices of three phases of contrast-enhanced CT as input to train a CNN model. “Ground truth” output consisted of labels for five categories of lesions, ranging from classic hepatocellular carcinoma to hemangiomas and cysts. Median lesion classification accuracy was 84% with an area under the receiver operating characteristic curve of 0.92 for malignant and benign/determinate lesion classification. Similar to this research, ML has also been used for breast cancer imaging and prostate cancer MRI.22-27

Lesion Characterization/Outcome Prediction and Radiomics

Predicting tumor response to IO treatment options is critical to the future success of IO techniques. A recent study investigated the use of ML to predict tumor response to transarterial chemoembolization (TACE) using retrospective data from hepatocellular carcinoma patients.28 The goal was to predict responders versus nonresponders from baseline clinical, laboratory, demographic, and imaging data. Input data were prescreened by filtering out features demonstrating low variance and low contributions to the outcome. Logistic regression and random forest techniques were utilized for the ML model. Although this was a feasibility study in a small patient cohort with simple ML techniques, building upon this approach may eventually result in tools that optimize patient selection for TACE.

Another approach for outcome prediction is use of radiomics, which is the assessment of tumor biology aspects not readily discernible to the human eye on imaging. Radiomics analysis consists of extracting subvisual quantitative imaging features and assessing the correlation to patient prognosis and treatment response. Two small abstracts presented at the 2018 Society of Interventional Radiology (SIR) meeting showed that preablation radiomics improves survival prediction in patients with adrenal metastasis who underwent ablation.29,30 Further studies can utilize ML techniques to analyze preablation images and predict treatment success based on texture analysis correlated to follow-up imaging.

Tumor Board Recommendations

In the realm of interdisciplinary decision-making, ML can also be used to improve tumor board recommendations and help in triaging oncology patients to the appropriate treatments. A study presented at the 2017 SIR meeting utilized an ML approach to identify the relative importance of clinical and imaging features contributing to tumor board recommendations.31 Data involving a combination of clinical and imaging characteristics were collected from 76 training cases and presented to a multidisciplinary team consisting of specialists from interventional radiology, radiation oncology, medical oncology, surgical oncology, and transplant. Input data included the number of enhancing lesions, largest lesion size, Organ Procurement and Transplantation Network (OPTN) classification, model for end-stage liver disease score, and Child-Pugh score. The output for each data set was the tumor board treatment recommendation. A random forest algorithm model was utilized. The model found a set of highly predicted features, including lesion size, segments involved, enhancing lesions, patient age, and number of OPTN 5 lesions.

Order Scheduling/Workflow

Critical to a successful practice is the smooth transition of a patient through an interventional radiology department. ML tools are currently being used to enhance clinical decision support.32 They are also being applied for intelligent scheduling, with the goal of reducing missed patient appointments.11 Similarly, ML and predictive analytics are being used to identify patients at high risk for missing their radiology care appointments.33

Intraprocedural Planning

Most directly relevant to the interventional oncologist is the application of AI to intraprocedural guidance—for both embolic and ablative therapies.34 Computational methods can be used to fuse images with registration algorithms, allowing the combination of detailed high-resolution preprocedural images with real-time intraprocedural images. Because these algorithms can predict an output based on a certain input, data related to catheter position, therapeutic effect, and patient outcomes can theoretically be fed through AI systems to predict patient outcome from intraprocedural catheter position. For ablation, therapeutic effect can be optimized by algorithms that guide probe placement, estimate ablation margins, select energy settings for thermal energy deposition, and minimize collateral damage to nearby structures.

Limitations and Challenges

Although the aforementioned studies and ideas lay the groundwork for integrating ML into IO, several important elements are necessary to bridge the gap between proof of concept and successful clinical translation. Due to limitations in data availability at single institutions, most currently published studies utilize small subsets of data that may not accurately simulate a realistic clinical practice environment. On the other hand, formation of large multicenter databases can allow for development of more robust algorithms that more closely simulate reality. In this setting, more heterogeneous data sets can be used to help models learn from cases with higher variability. Additionally, larger data sets allow for models that can technically accommodate multiple modalities, parameters, and time points as input, providing a more complete input set for decision-making.

In addition to employing large multicenter databases, a few other possible approaches can improve clinical applicability of AI systems. Although initial proof-of-concept algorithms could focus on including “ideal” data input, these models may break down when analyzing de novo information that does not follow simple classical patterns. Thus, it is important to eventually incorporate more “challenging” cases into training data sets, including atypical data and cases with artifacts or incomplete data.

Additionally, well-annotated data sets are crucial to the validity of any AI system. This can be accomplished by having multiple physicians assign labels, allowing for interreader variability to be compared to produce higher-quality “ground truths.” Moreover, techniques that utilize only one set of input data (eg, imaging data) may not provide a balanced global perspective; therefore, ML models can be optimized for clinical relevance by incorporating multiple input sources, including imaging, textual, and other clinical information. The creation of large data sets and defining standards that meet clinical practice guidelines are critical to the success of this approach. The National Cancer Institute’s Annotation and Image Markup model is an example used to standardize images.35

From a more technical perspective, ML systems that function as a “black box” (ie, provide a prediction without explanations) are less likely to gain clinical acceptance, because they do not provide a mechanism for predicting technical mistakes. This can be remedied by developing algorithms that provide evidence for predictions and are capable of quantifying uncertainty. Finally, such models should be flexible and dynamic, easily adapting to the latest clinical guidelines and research findings for continuous improvement.

CONCLUSION

Despite the recent surge of AI in medicine, the application to IO is still in its infancy. An expanding body of data are increasingly demonstrating its potential to revolutionize the field. As described, AI has the potential to transform the entire clinical continuum of IO care. With an established pioneering history of blending technology with clinical care, interventional radiologists are ideally suited to lead these developmental and clinical translation efforts. In this manner, interventional radiologists can carve out a custom path for AI in IO, incorporating AI as their own tool to provide patients with optimal minimally invasive clinical care.

1. de Prado ML. Advances in Financial Machine Learning. Hoboken, NJ: John Wiley & Sons; 2018.

2. Rossetti RJF, Liu R. Advances in Artificial Transportation Systems and Simulation. San Diego: Elsevier; 2015.

3. Kononenko I. Machine learning for medical diagnosis: history, state of the art and perspective. Artif Intell Med. 2001;23:89-109.

4. Deo RC. Machine learning in medicine. Circulation. 2015;132:1920-1930.

5. Wernick M, Yang Y, Brankov J, et al. Machine learning in medical imaging. IEEE Signal Process Mag. 2010;27:25-38.

6. Wang S, Summers RM. Machine learning and radiology. Med Image Anal. 2012;16:933-951.

7. van Ginneken B. Fifty years of computer analysis in chest imaging: rule-based, machine learning, deep learning. Radiol Phys Technol. 2017;10:23-32.

8. Erickson BJ, Korfiatis P, Akkus Z, Kline TL. Machine learning for medical imaging. Radiographics. 2017;37:505-515.

9. Shen D, Wu G, Suk HI. Deep learning in medical image analysis. Annu Rev Biomed Eng. 2017;19:221-248.

10. Kohli M, Prevedello LM, Filice RW, Geis JR. Implementing machine learning in radiology practice and research. AJR AM J Roentgenol. 2017;208:754-760.

11. Choy G, Khalilzadeh O, Michalski M, et al. Current applications and future impact of machine learning in radiology. Radiology. 2018;288:318-328.

12. Neapolitan RE. Contemporary Artificial Intelligence. Boca Raton, FL: Chapman and Hall/CRC; 2013.

13. Li JW, Goh BG, Chang PE, Tan C-K. Barcelona Clinic Liver Cancer outperforms Hong Kong Liver Cancer staging of hepatocellular carcinoma in multiethnic Asians: real-world perspective. World J Gastroenterol. 2017;23:4054-4063.

14. Liu PH, Hsu CY, Hsia CY, et al. Prognosis of hepatocellular carcinoma: assessment of eleven staging systems. J Hepatol. 2016;64:601-608.

15. Kolly P, Reeves H, Sangro B, et al. Assessment of the Hong Kong Liver Cancer staging system in Europe. Liver Int. 2016;36:911-917.

16. Sohn JH, Duran R, Zhao Y, et al. Validation of the Hong Kong Liver Cancer staging system in determining prognosis of the North American patients following intra-arterial therapy. Clin Gastroenterol Hepatol. 2017;15:746-755.e4.

17. Yau T, Tang VY, Yao TJ, et al. Development of Hong Kong Liver Cancer staging system with treatment stratification for patients with hepatocellular carcinoma. Gastroenterology. 2014;146:1691-1700.e3.

18. Sharma A, Harrington RA, McClellan MB, et al. Using digital health technology to better generate evidence and deliver evidence-based care. J Am Coll Cardiol. 2018;71:2680-2690.

19. Fogel DB. Factors associated with clinical trials that fail and opportunities for improving the likelihood of success: a review. Contemp Clin Trials Commun. 2018;11:156-164.

20. Kim HS, Chapiro J, Geschwind JH. From the guest editor: interventional oncology: the fourth pillar of oncology. Cancer J. 2016;22:363-364.

21. Yasaka K, Akai H, Abe O, Kiryu S. Deep learning with convolutional neural network for differentiation of liver masses at dynamic contrast-enhanced CT: a preliminary study. Radiology. 2018;286:887-896.

22. Huynh BQ, Li H, Giger ML. Digital mammographic tumor classification using transfer learning from deep convolutional neural networks. J Med Imaging (Bellingham). 2016;3:034501.

23. Gu P, Lee WM, Roubidoux MA, et al. Automated 3D ultrasound image segmentation to aid breast cancer image interpretation. Ultrasonics. 2016;65:51-58.

24. Bickelhaupt S, Paech D, Kickingereder P, et al. Prediction of malignancy by a radiomic signature from contrast agent-free diffusion MRI in suspicious breast lesions found on screening mammography. J Magn Reson Imaging. 2017;46:604-616.

25. Le MH, Chen J, Wang L, et al. Automated diagnosis of prostate cancer in multi-parametric MRI based on multimodal convolutional neural networks. Phys Med Biol. 2017;62:6497-6514.

26. Turkbey B, Fotin SV, Huang RJ, et al. Fully automated prostate segmentation on MRI: comparison with manual segmentation methods and specimen volumes. AJR Am J Roentgenol. 2013;201:W720-W729.

27. Kwak JT, Xu S, Wood BJ, et al. Automated prostate cancer detection using T2-weighted and high-b-value diffusion-weighted magnetic resonance imaging. Med Phys. 2015;42:2368-2378.

28. Arndt S, Sandow T, Milburn J, et al. Machine learning and machine vision image analysis can predict treatment response from preprocedural imaging alone for Y90 radioembolization and DEB-TACE in hepatocellular carcinoma [abstract]. J Vasc Interv Radiol. 2018;29:S79-S80.

29. Daye D, Tabari A, Caton M, et al. Tumor heterogeneity features as predictors of post-ablation local progression in patients undergoing ablation for adrenal metastases [abstract]. J Vasc Interv Radiol. 2018;29:S142-S143.

30. Daye D, Tabari A, Caton M, et al. Machine learning–based radiomics improve post-ablation prognostication in patients with adrenal metastases [abstract]. J Vasc Interv Radiol. 2018;29:S144.

31. Valentin L, McCarthy C, Do S, et al. Predicting multidisciplinary tumor board recommendations: Initial experience with machine learning in interventional oncology. J Vasc Interv Radiol. 2017;28:S19-S20.

32. Bal M, Amasyali MF, Sever H, et al. Performance evaluation of the machine learning algorithms used in inference mechanism of a medical decision support system. ScientificWorldJournal. 2014;2014:137896.

33. Glover M 4th, Daye D, Khalilzadeh O, et al. Socioeconomic and demographic predictors of missed opportunities to provide advanced imaging services. J Am Coll Radiol. 2017;14:1403-1411.

34. Society of Interventional Radiology. SIO central: conference program. http://www.sio-central.org/page/conference-program. Accessed September 9, 2018.

35. Mongkolwat P, Kleper V, Talbot S, Rubin D. The National Cancer Informatics Program (NCIP) Annotation and Image Markup (AIM) Foundation model. J Digit Imaging. 2014;27:692-701.

Brian Letzen, MD, MS
Department of Radiology & Biomedical Imaging
Yale School of Medicine
New Haven, Connecticut
brian.letzen@yale.edu
Disclosures: None.

Raul N. Uppot, MD
Division of Interventional Radiology
Massachusetts General Hospital
Harvard Medical School
Boston, Massachusetts
uppot.raul@mgh.harvard.edu
Disclosures: Received a grant from CRICO Controlled Risk Insurance Corporation.