Improving Medical Image Segmentation through Human Feedback and Reward Modeling

Medical image segmentation is a fundamental task in AI diagnosis and clinical image analysis. It consists of identifying and delineating regions of interest within medical images, such as tumors, lesions, or specific anatomical structures. Accurate segmentation enables clinicians and AI systems to better characterize findings based on properties such as shape, size, texture, and spatial organization, while also supporting downstream tasks including disease detection, treatment planning, quantitative analysis, and quality control. However, the creation of high-quality segmentation annotations remains a major bottleneck, as it requires substantial domain expertise and is both time-consuming and expensive.

Recent advances in deep learning have led to highly effective segmentation architectures, including models such as U-Net, DeepLabv3+, and other transformer-based approaches, which achieve strong performance across multiple imaging domains. Nevertheless, medical imaging continues to present unique challenges due to high inter-observer variability, noisy annotations, domain shifts across institutions, and the need for extremely precise boundaries in clinically relevant regions.

This project proposes the development of reward-based learning methods for medical image segmentation using Reinforcement Learning from Human Feedback (RLHF). The central idea is to incorporate expert preferences directly into the training process through reward models that learn to evaluate the quality of segmentation masks based on human feedback. These reward models may consist of lightweight neural networks trained to distinguish preferred segmentations from suboptimal ones, enabling the segmentation system to iteratively improve according to expert-defined criteria such as boundary precision, anatomical consistency, or clinical usefulness.

The initial focus of the project will be on histopathology datasets based on Hematoxylin and Eosin (H&E) stained tissue images, where fine-grained segmentation plays a crucial role in cancer analysis and tissue characterization. The proposed framework, however, will be designed to generalize to other medical imaging modalities and applications.

The expected outcomes include the development of novel RLHF methodologies for medical image segmentation and improved human-in-the-loop annotation workflows.