Self-Supervised Feature Extraction in Breast Cancer: Comparing Weakly Versus Fully Supervised Approaches

Pathological complete response (pCR) to neoadjuvant chemotherapy (NAC) is a key early indicator of long-term treatment success in breast cancer, yet accurately predicting pCR remains challenging. This thesis dives into a deep learning approach using self-supervised feature extractors—specifically, DINOv2, a vision transformer, and ResNet-50, a convolutional neural network—to analyze baseline T0 Dynamic Contrast-Enhanced MRI (DCE-MRI) scans from the I-SPY 2 trial. Instead of relying on manual metrics, the study aims to see how well these learned features can spot key tumor traits linked to NAC response. Two methods are put to the test: 1) traditional supervised classification, using logistic regression and SVM on pooled image embeddings, and 2) a weakly supervised method that uses attention-based Multiple Instance Learning (MIL). In this latter approach, entire DCE volumes act as bags while individual slices are treated as instances. The results show that DINOv2 features generally outperform ResNet-50 for pCR prediction, often achieving higher AUC scores of around 0.60–0.65 in certain scenarios. Additionally, the MIL method tends to bridge the gap between analyzing cropped tumors versus full volumes, suggesting the model can learn to concentrate on relevant tumor slices without needing manual labels. While there are some hurdles, like class imbalance, a small dataset, and pretrained models that are based on natural images, the findings show that a self-supervised, weakly supervised approach can be a practical way to assess early treatment responses. Moving forward, it’ll be important to test larger, more balanced groups, enhance the pretraining for DINOv2 and ResNet-50, and fine-tune MIL architectures. With these improvements, fully automated NAC response predictions from baseline DCE-MRI could become a powerful tool in tailoring breast cancer treatments