A leakage-aware and reliability-focused evaluation framework for multiclass Alzheimer’s Disease staging using MRI slices

Abstract

Two-dimensional magnetic resonance imaging (MRI) studies of Alzheimer’s disease (AD) can overestimate performance when slices from the same subject appear in both training and assessment sets. This thesis evaluated a leakage-aware framework for multiclass AD staging based on subject-independent partitioning, subject-level prediction, calibration, and external validation. Three model families were compared on 347 OASIS subjects using subject-wise crossvalidation: a convolutional neural network (CNN), a Vision Transformer (ViT), and a Hybrid CNN–ViT architecture. A matched slice-random ablation quantified leakageassociated inflation. Supporting analyses examined imbalance-aware objectives and subject-level reliability. Generalizability was tested on 502 ADNI subjects after OASIS model-selection decisions were fixed. Slice-random evaluation contaminated 98.3% of assessment subjects and inflated macro- F1 by 0.252–0.288. Under subject-wise evaluation, the Hybrid achieved the highest OASIS macro-F1 (0.711). On ADNI, the ViT achieved the highest external macro-F1 (0.718), reversing the internal ranking. These findings show that evaluation protocol and independent cohort testing can materially change conclusions about model quality. Keywords: Alzheimer’s disease, ADNI, data leakage, MRI, subject-wise cross-