The aim of the REFRAME research project is to investigate the possibilities and limitations of so-called foundation models in computer-aided image recognition and to develop new methods for improving these models. Foundation models form the basis for many AI applications in image recognition. They are trained using billions of data sets from the Internet, for example with images and associated descriptions, and can therefore perform a wide range of tasks.

This gives these models a broad basic knowledge and enables them to develop new skills. "However, they have been trained with so much data that we don't know where their limits lie - in other words, where they stop working," explains Dr Matthias Rottmann, junior research group leader for reliable and efficient artificial intelligence at the University of Wuppertal. This becomes a problem above all when image recognition is used where reliability is required, for example in accident avoidance systems.

New methods for recognising uncertainties in foundation models

The researchers are therefore addressing the following questions: How reliable are these models, how well can they be explained and how can they be customised to meet specific requirements? The team at the University of Wuppertal, led by Rottmann, is researching new methods for recognising uncertainties in the models. The researchers are also investigating the extent to which the knowledge of the models can be generalised.

The research results are particularly important for safety-critical applications such as in medicine, robotics or automated driving. "The project thus makes an important contribution to the further development of artificial intelligence and supports the current goals of the German government's AI strategy," says Matthias Rottmann.