NLM DIR Seminar Schedule

AI Pipeline for Characterization of the Tumor Microenvironment

Karla Paniagua, Yufang Jin, Mario Flores

We present an innovative artificial intelligence framework, TG-ME, designed to analyze the tumor microenvironment (TME) using high-resolution spatial transcriptomics datasets. The TME is a complex ecosystem composed of tumor cells, immune cells, fibroblasts, and the extracellular matrix. These elements interact dynamically in both spatial and molecular dimensions, profoundly influencing cancer progression, metastasis, and therapeutic resistance. TG-ME integrates two advanced deep learning models. First, a Transformer model is employed to analyze gene expression data, with a focus on uncovering gene–gene interactions and complex molecular dependencies. Second, a Graph Variational Autoencoder (GraphVAE) model incorporates spatial relationships within the TME, capturing how different cell types and structures are organized and interact within the tumor environment. Together, these models enable TG-ME to provide a comprehensive, multidimensional view of the TME by integrating gene expression, morphology, spatial organization, and cell-type composition.

Applied to non-small cell lung cancer (NSCLC) samples, TG-ME uncovered distinct spatial niches which are specific regions within the TME where unique biological processes are active. These niches were enriched in key cancer-related pathways such as Hypoxia, Epithelial-Mesenchymal Transition (EMT), and Interferon Signaling. Importantly, these spatial features correlated with disease severity and therapeutic resistance, suggesting their potential use as biomarkers for assessing tumor aggressiveness and predicting treatment response. TG-ME also delineated tumor–stroma borders, which play a central role in metastasis and therapy resistance, underscoring the framework’s ability to capture clinically relevant features of the TME.

Beyond NSCLC, the TG-ME framework holds broad potential for other diseases where the microenvironment plays a critical role. Many pathologies, including autoimmune diseases, chronic inflammatory conditions like Diabetic Foor Ulcer (DFU), neurodegenerative disorders, and cardiovascular disease, involve complex interactions between cells and their surrounding microenvironment that influence disease initiation, progression, and therapeutic outcomes. For example, the immune microenvironment in DFU, the glial–neuronal interactions in Alzheimer’s disease, or the stromal remodeling in fibrotic disorders could all be studied using TG-ME’s integrative approach. By capturing spatial and molecular heterogeneity across these contexts, TG-ME offers a powerful framework not only for cancer biology but also for understanding and treating a wide spectrum of diseases where the cellular microenvironment is a key determinant of pathology.

TG-ME provides a novel and versatile platform for microenvironmental analysis. Its ability to uncover spatially organized molecular programs, resolve critical tissue boundaries, and highlight prognostic features demonstrates its value in advancing precision medicine strategies across oncology and beyond.

Heterogeneous Graph Re-ranking for CLIP-based Medical Cross-modal Retrieval

Cross-modal retrieval of medical radiographs is a critical component of clinical decision support, cohort discovery, and large-scale data reuse. While CLIP-based vision–language models enable effective zero-shot retrieval, ranking based solely on embedding similarity does not explicitly capture higher-order relationships among images, reports, and clinical semantics. We propose a heterogeneous graph re-ranking framework that augments CLIP-based retrieval with structured relational reasoning while keeping the backbone representation model frozen. Starting from an initial CLIP ranking, the method constructs a heterogeneous k-nearest-neighbor graph over image and report embeddings and applies relation-aware message passing to refine candidate rankings.
We instantiate the framework using three representative graph neural network layer variants (GraphSAGE, GCN, and GAT), and evaluate it on chest radiograph retrieval using the OpenI-CXR and MIMIC-CXR datasets under both within-dataset validation and cross-dataset transfer. On the smaller OpenI dataset, heterogeneous graph re-ranking yields substantial improvements, with GraphSAGE increasing Strong MRR by 47.7%, Precision@10 by 58.2%, and mAP@10 by 45.3%, alongside consistent gains in nDCG. Text-to-image retrieval benefits most, with MRR improving from 0.254 to 0.384 (50.8%). On the larger MIMIC-CXR dataset, gains are more moderate but consistent: GAT improves Strong Precision@10 by 8.5% and mAP@20 by 4.9%, while GraphSAGE enhances weak retrieval performance and normal CXR screening accuracy by up to 3.1%. Cross-dataset experiments further show that heterogeneous graph re-ranking improves robustness relative to embedding-only retrieval, with attention-based models providing the most stable transfer performance.
Overall, these results demonstrate that heterogeneous graph re-ranking is an effective and practical extension to CLIP-based medical cross-modal retrieval, improving ranking quality, clinically relevant screening performance, and generalization without modifying the underlying vision–language encoder.

FastSpel: A simple peptide spectrum predictor that achieves deep learning-level performance at a fraction of the computational cost

Mass spectrometry–based proteomics enables the identification and quantification of peptides and proteins by matching observed fragmentation spectra to candidate peptides from a protein database. Incorporating computationally predicted spectra into this process can substantially improve peptide identification. While recently proposed deep learning–based spectrum prediction methods achieve high performance, they are computationally expensive and thus unsuitable for some applications. In this talk, I introduce FastSpel, a simple, accurate, and efficient peptide spectrum prediction method that achieves performance comparable to state-of-the-art deep learning–based approaches at a fraction of the computational cost. FastSpel is therefore well suited for applications that require, or benefit from, on-the-fly predictions. Moreover, unlike deep learning-based methods, FastSpel includes easily interpretable parameters and thus may provide new insights into the peptide fragmentation process.