With the tremendous increase of video recording devices and the resulting abundance of digital video, finding a particular video sequence in ever-growing collections is a major challenge. Existing, currently deployed approaches to retrieve videos still mostly rely on metadata-based retrieval techniques to find desired sequences. With vitrivr, we present a fully working system for indexing and retrieving multimedia data based on its content.
The following image shows the overall architecture of vitrivr. vitrivr has three components: a user interface (vitrivr-ng and vitrivr-VR), a retrieval engine (Cineast) and a custom database (Cottontail DB). Cineast is our retrieval engine responsible for performing shot segmentation, extracting features from multimedia data and generating database queries based on user queries. These queries are then processed by Cottontail DB which is able to return the k nearest neighbors to a query in a very efficient way. Cineast then combines the results from various queries to one result set.
Cineast, Cottontail DB, vitrivr-ng and vitrivr-VR are open source and actively being worked on.
The browser-based vitrivr UI offers multiple query modes to facilitate retrieval.
Cineast is a multi-modal multimedia content retrieval engine. It is capable of retrieving multimedia data based on a diverse range of user input such as sketches, text, and 3D Models.
Cottontail DB is a database system to store and retrieve multimedia data. It provides Boolean retrieval and similarity search and makes use of various index structures for efficient retrieval.
The virtual reality-based vitrivr UI offers novel interaction modes in virtual space.
vitrivr has participated in Google Summer of Code 2016, 2018 and 2021. If you want to contribute to vitrivr, you can use the project descriptions remaining from previous GSoC participations as inspiration.
The best way to contact us is on GitHub through issues, pull requests, and discussions on the different repositories.