With the tremendous increase of video recording devices and the resulting abundance of digital video, finding a particular video sequence in ever-growing collections is a major challenge. Existing approaches to retrieve videos mostly still rely on text-based retrieval techniques to find desired sequences. With vitrivr, we present a fully working system for indexing and retrieving multimedia data based on its content.
Architecture
The following image shows the overall architecture of vitrivr. vitrivr has three components: a user interface (vitrivr-ng), a retrieval engine (Cineast) and a custom database (Cottontail-DB). Cineast is our retrieval engine responsible for performing shot segmentation, extracting features from multimedia data and generating database queries based on user queries. These queries are then processed by Cottontail-DB which is able to return the k nearest neighbors to a query in a very efficient way. Cineast then combines the results from various queries to one result set.
Components
Cineast, Cottontail-DB and vitrivr-ng are open source and actively being worked on.
vitrivr-ng
The browser-based vitrivr UI offers multiple query modes to facilitate retrieval.
Cineast
Cineast is a multi-modal multimedia content retrieval engine. It is capable of retrieving multimedia data based on a diverse range of user input such as sketches, text, and 3D Models.
Cottontail-DB
Cottontail-DB is a database system to store and retrieve multimedia data. It provides Boolean retrieval and similarity search and makes use of a various index structures for efficient retrieval.
Awards
- Winner Best Demo Award at the International Conference on Multimedia Retrieval 2019
- Winner Lifelog Search Challenge 2019
- Winner Video Browser Showdown 2019
- Winner Video Browser Showdown 2017 (iMotion)
- Winner Best Demo Award at the International Conference on Multimedia
- Fritz Kutter Award for Industry Related Thesis in Computer Science
In The Media
chevron_rightFull List of Media Coverage
Google Summer of Code
vitrivr has participated to Google Summer of Code 2016 and 2018. While vitrivr has not participated in last year’s GSoC, we still have a page with project ideas if you want to contribute to vitrivr.
Contact Us
We have set up a mailing list for the vitrivr project. You can use the mailing list for questions regarding the code, bugs, etc. Note that the mailing list is publicly visible.