With the tremendous increase of video recording devices and the resulting abundance of digital video, finding a particular video sequence in ever-growing collections is a major challenge. Existing approaches to retrieve videos mostly still rely on text-based retrieval techniques to find desired sequences. With vitrivr, we present a fully working system for indexing and retrieving multimedia data based on its content.
The following image shows the overall architecture of vitrivr. vitrivr has three components: a user interface (vitrivr-ng), a retrieval engine (Cineast) and a custom database (Cottontail-DB). Cineast is our retrieval engine responsible for performing shot segmentation, extracting features from multimedia data and generating database queries based on user queries. These queries are then processed by Cottontail-DB which is able to return the k nearest neighbors to a query in a very efficient way. Cineast then combines the results from various queries to one result set.
Cineast, Cottontail-DB and vitrivr-ng are open source and actively being worked on.
The browser-based vitrivr UI offers multiple query modes to facilitate retrieval.
Cineast is a multi-modal multimedia content retrieval engine. It is capable of retrieving multimedia data based on a diverse range of user input such as sketches, text, and 3D Models.
Cottontail-DB is a database system to store and retrieve multimedia data. It provides Boolean retrieval and similarity search and makes use of a various index structures for efficient retrieval.
vitrivr has participated to Google Summer of Code 2016 and 2018. While vitrivr has not participated in last year’s GSoC, we still have a page with project ideas if you want to contribute to vitrivr.
We have set up a mailing list for the vitrivr project. You can use the mailing list for questions regarding the code, bugs, etc. Note that the mailing list is publicly visible.