Data-Science from Whiteboard to Production: End-to-End Image Captioning

Generating short precise textual descriptions of images is called Image Captioning. Unfortunately, most images do not have such a precise description preventing an automatic information retrieval. Novel Image Captioning models are frequently presented in the scientific literature, each employing different ideas and technologies. However, developing an Image Captioning system from a prototypical version to an intelligent service usable by thousands of end-users is a non trivial task requiring Data-Science, Data-Engineering and Software-Engineering skills. As Full-Stack Data-Science and Data-Engineering consultants we are dealing with these kind of challenging and fascinating problems on a daily basis!

In this project, we address the statistical modeling, data preparation and storage, model training and refinement, and the scalable serving of an Image Captioning service in order to handle thousands of request, simultaneously. We will make use of publicly available labeled data sets, and start with simple, already published models in order to create a first Minimum Viable Product (MVP).

We will further develop the Image Captioning system in agile iterations. Hence, every new predictive model version that is likely to be more complex than its predecessor has to prove that its increased complexity also significantly increases the accuracy. In alignment with Occam's Razor and also with machine learning theory, this way we seek to not overcomplicating things. Complex models like deep neural nets have the potential to learn extraordinary complex relations. However, this ability comes with a high risk of overfitting, instability, and outliers. Building a productive, fully automated system you better take care of these risks.