Project
Title
Erforschung und Erweiterung von Computer Vision Foundation Models (ExploRing and Expanding the FrontierRs of FoundAtion ModEls)
Acronym
REFRAME
Start
01.10.2024
End
30.09.2027
Coordinating Institute
Fraunhofer Heinrich Hertz Institut
Contact
Partner
Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V.
Bergische Universität Wuppertal
Summary
In the field of computer vision, Vision Foundation Models (VFM) have set new standards in visual tasks for understanding visual data, e.g. for image and object recognition, segmentation and classification, e.g. CLIP, SAM, SEEM. Despite the progress of VFM, crucial questions remain about the trustworthiness of these models. It is unclear when a model leaves the domain of its training It is unclear when a model leaves the range of its training data and how the accuracy behaves across different domains or how the accuracy is changed by adaptation and fine-tuning. REFRAME addresses these open questions. The overall goal is to achieve a sustainable, robust, flexible and efficient use of VFM in specific tasks. To this end, it is essential 1) to develop methods to investigate their limitations and identify uncertainties in the predictions and to provide them to users and 2) to increase the trustworthiness and explainability through appropriate methods, e.g. to recognise and reduce biases, and 3) to develop resilient and efficient methodologies to adapt VFM to specific domains and tasks even with little data. We focus on some important aspects and the associated method development that will help to strengthen confidence and reliability in the predictions of VFM and the models based on them.
T
Funding
Bundesministerium für Bildung und Forschung (BMBF)
Funding agency
Deutsches Zentrum für Luft- und Raumfahrt (DLR)
Grant agreement number
01IS24073B
Funding framework
Richtlinie zur Förderung von Forschungsprojekten zum Thema „Flexible, resiliente und effiziente Machine-Learning-Modelle“, Bundesanzeiger vom 07.09.2023