Detecting novel objects with only few examples

PreciBake is a company based in Munich, New York and Mumbai, developing AI solutions for the food-tech and baking industry. Our AI team is continuously working on developing and improving our machine learning algorithms for tasks such as image classification, object detection and tracking.

Real time inventory monitoring is an application that can hugely benefit from recent advances in object detection. Typical inventories might however contain hundreds or thousands of unique items which makes building high quality datasets tedious and difficult. What makes it even more challenging is that new items or new packaging could be introduced which would occasionally require additional relabeling and retraining. 

Would it not be convenient to be able to have our model automatically adapt to new object categories by simply having an updated description of the objects that change? This is a trivial task for a human, but still very complicated for most modern object detection models. 

Instead of training a model to detect and classify specific objects, a model could be trained to detect specific feature descriptions of any given objects in an image. For example a croissant could be described by its specific texture, color, shape and size. Given the feature description or an example image of the object the model should be able to find examples of croissants in a dataset matching the description, even if it was not specifically trained for this task.

The goal of this project is to develop a universal object detection model that could detect any type of objects in an image dataset at test time, given just a feature description or a few example images of the object of interest. 

Accepted students to this project should attend (unless they have proven knowledge) online workshops at the LRZ from 19.04. - 22.04.22. More information will be provided to students accepted to this project.