Brain UK study ref: 25/003,

Lay summary,

Project status: Active

VLeMoN: a Vision-Language Model for Neuropathology

Prof. Jakob Kather, Else Kröner Fresenius Center for Digital Health

Diagnosing brain tumors is a complex and time-consuming process that involves analysing tissue samples under a microscope and conducting detailed laboratory tests. These steps are essential for determining the appropriate treatment and predicting patient outcomes. However, the current diagnostic workflows often take days or weeks as multiple different tests are required. This project aims to use advanced artificial intelligence (AI) technologies to transform the diagnostic process for brain tumours. 

The goal of this project is to create a foundation model (a very large AI model) and an interactive AI chatbot (an AI model that you can talk to, similar to ChatGPT) but specialised for neuropathology (brain tumours). This chatbot will provide detailed descriptions and potential diagnoses from neuropathology images, to help pathologists and other healthcare professionals more quickly and easily diagnose brain tumours. Our model will also have visual question answering (VQA) capabilities, so pathologists will be able to interact with the chatbot by asking image-related questions and receiving accurate responses in real-time. 

By developing this innovative AI-powered platform, the project aims to streamline the diagnostic workflow, accelerate identification of brain tumor subtypes, and enable precise prediction of molecular markers (changes in the DNA that tell us about the tumour). This will ultimately lead to faster, more accurate diagnoses, improve clinical decision-making, and enhance outcomes for patients with brain tumors.