Lung cancer is one of the most challenging diseases, making early diagnosis crucial for effective treatment. Fortunately, advancements in artificial intelligence (AI) are transforming lung cancer screening, improving both accuracy and efficiency. While current screening methods like low-dose CT help confirm suspicions of lung cancers, they often suffer from high false-positive rates and variability in reporting incidental yet critical findings, such as those pertaining to cardiovascular diseases. Additionally, the screening rate for low-dose CT remains low (<10%), due to a global shortage of radiologists.

A new study published in Nature Communications introduces a multimodal multitask foundation model that significantly enhances the capabilities of low-dose CT. This AI model improves the prediction of lung cancer risk by 20% and cardiovascular risk by 10%. Developed and tested by an interdisciplinary team from Rensselaer Polytechnic Institute (RPI), Wake Forest University (WFU), and Massachusetts General Hospital (MGH), this model is the first of its kind to simultaneously address more than a dozen related tasks, incorporating data from multiple sources including CT scans, radiology reports, patient risk factors, and key clinical findings.  

The first author of the study is Chuang Niu, Ph.D., research scientist at RPI. The corresponding authors include Ge Wang, Ph.D., Clark-Crossan Chaired Professor and director of the Biomedical Imaging Center at RPI, Christopher T. Whitlow, M.D./Ph.D., professor at WFU, Mannudeep K. Kalra, M.D., professor at MGH. Key collaborators at RPI include Pingkun Yan, Ph.D., and Christopher D. Carothers, Ph.D., as well as other important coauthors.  

The potential clinical impact of this work is immense. By integrating CT images with text information, the model significantly improves the detection and prediction of lung cancer, a critical factor in improving patient outcomes. Also, one of the major benefits of using foundation models in medicine is that when trained with large-scale screening CT scans and other data types, these models can boost the model performance in related new tasks. For instance, this model can improve performance in fields such as oncology, where task-specific data is often limited.  

“This work has been significantly accelerated using RPI’s high-performance computing facility,” said Wang. “Now, our multi-institutional team is further enhancing our foundation model on an increasing size of multimodal data, using both our own GPUs and New York State’s Empire AI high-performance computing facility. The collaboration across leading institutions underscores the growing synergy between artificial intelligence and medical research, with the potential to revolutionize how diseases are detected and treated.”  

“Dr. Wang and his team are making important strides toward improving human health by combining the power of medical imaging, AI, and high-performance computing.  RPI has always been at the forefront of computational sciences and engineering, providing faculty and students access to the world’s best computational infrastructure to accelerate development and translation of transformative ideas. We are excited about what this work means for the future of early detection of diseases and look forward to seeing further advances,” said Shekhar Garde, Ph.D., the Thomas R. Farino Jr. ’67 and Patricia E. Farino Dean of the School of Engineering at RPI.  

Read the full paper.