Honors Thesis Presentation by CS Senior

Join us Wednesday, November 18 at 3:30 PM in Alumni Hall 291! CS Senior Annie Luo is going to have her honors thesis presentation on topic titled Evaluating Vision-Language Models for Wild Palm Detection Verification in UAV Imagery advised by Dr. Fan Yang.

Abstract

Accurate interpretation of object detections in UAV imagery is challenging, particularly when visual ambiguity and incomplete annotations complicate evaluation. This thesis investigates the behavior of vision-language models (VLMs) as second-stage verifiers of wild palm detections generated by a pretrained YOLO detector. Using 5,747 candidate detections from 870 UAV image patches, the study evaluates multiple VLM checkpoints under five input conditions that vary visual presentation and accompanying detection information. Model decisions are analyzed in terms of classification behavior, sensitivity to input conditions, abstention, cross-model agreement, and alignment with existing annotations. Results reveal substantial model-dependent variation in verification decisions, including differences in uncertainty handling and sensitivity to input presentation. A human review of 638 detections unmatched to LabelMe annotations identified 619 palms, 19 ambiguous cases, and no confirmed non-palms, demonstrating that annotation mismatch does not necessarily indicate an incorrect detection. Consequently, annotation-based evaluation metrics cannot be interpreted directly as measures of semantic verification accuracy. These findings highlight important limitations in evaluating VLM-based verification and motivate more careful consideration of input design, model behavior, and ground-truth validity in UAV object detection workflows.

All are welcome.