Active Scene Understanding for Video Labeling
This project took place in winter term 2020, you CAN NOT apply to this project anymore!
Results of this project are explained in detail in the final documentation and presentation.
- Sponsored by: Design AI
- DI Incubator: Startup Design AI
- Loop of knowledge: Keesiu Wong, Lisa-Marie Bernhardt
- Project Lead: Dr. Ricardo Acevedo Cabra
- Scientific Lead: M.Sc. Frederik Mattwich, M.Sc. Keesiu Wong and M.Sc. Lisa-Marie Bernhardt
- TUM Co-Mentor: PhD candidate Marija Tepegjozova
- Term: Winter semester 2020
Design AI GmbH is a Munich based AI start-up supported by the TUM, UnternehmerTUM, and NVIDIA. We develop cutting edge machine learning / deep learning solutions by combining Design Thinking and Artificial Intelligence.
We want to develop an AI-based video labeling tool, that supports humans to efficiently annotate video scenes with high-level semantic information. Therefore, we want to combine state-of-the-art techniques in online scene understanding (e.g. action detection or scene graph generation) with an active learning framework. The goal of this project is to do the necessary initial research of state-of-the-art techniques for online scene understanding, development of a working prototype and evaluation on public datasets like Youtube-8M, KINETICS-600 or Moments in Time dataset. Furthermore, if time allows, this online computer vision model should be integrated into an effective and scalable active learning pipeline.
Your tasks would include
- the initial research & identification the most promising deep learning approaches for online scene understanding,
- prototyping and evaluation of the online scene understanding module with public datasets and
- the development and integration into an active learning framework.
You should have
- strong foundations in Deep Learning, e.g. respective TUM course I2DL
- first research experiences, favourably in Computer Vision, e.g. via ADL4CV,
- fluency in either PyTorch or Tensorflow,
- know about the idea behind Online and/or Active Learning and
- have the ability to work independently and to think and act entrepreneurially.
In return, you will
- learn how to build a globally innovative AI product based on state-of-the-art technology,
- experiment as much as you want – up to 10.000 $ worth of AWS credits for your free usage and last but not least
- work in an growing, TUM-native start-up and feel the spirit of entrepreneurship.