The vitrivr stack consists of three components: ADAM/ADAMpro (the database), Cineast (the retrieval engine) and the front-end which is browser-based. The following describes the steps needed to set up all components of the stack and populate it with some exemplary data.
ADAMpro is the persistent polystore (based on Apache Spark) for all data required for retrieval. For experimenting with ADAMpro, a version of ADAMpro without any data, can be easily run as a Docker image. For this, run
which will pull the Docker image from Docker Hub and run it with the corresponding ports open. You can access the ADAMpro UI by navigating to http://localhost:9099.
Furthermore, we provide a pre-built Docker image which not only contains ADAMpro but is already populated with the data extracted from the Open Short Video Collection. Simply download the archive and run:
You can find a more extended documentation on ADAMpro, the code structure and deployment possibilities, in the repository on github.
Cineast is the retrieval engine in charge of feature extraction and query processing. To build and run Cineast you need Java JDK 8. Clone the Cineast github repository and run:
This will create the cineast.jar file which can be found in build/libs.
Edit the cineast.json file and point it to the location where you set up ADAMpro in Step 1:
If you used the prebuilt osvc images, you need to change the database name from cineast to osvc and the user and the password for adam need to be changed to docker as both user and password.
The freshly created cineast jar file can then be started with:
The front end is web-based and therefore requires a web-server. Any web-server is fine as long as it runs php. Once you have one up and running clone the github repo into a folder accessible by the web-server. If you chose to use our pre-populated ADAM image, also download the videos and thumbnails which go together with the meta-data in the database. Place them somewhere the web-server can find them.
Next you need to configure the front end. This is done by editing the config.js file found in the js directory so that the front end knows where the images and videos are (relatively to the webroot of the UI):
vitrivr offers multiple query modes which are outlined below.
The left side of the vitrivr UI contains the input canvases. They are used to specify the visual part of the query by either sketching directly on them or adding existing images using drag and drop. Longer sequences can be specified by adding multiple canvases.
To search for particaular types of movement, the motion canvas can be used. To select the motion input, the mode has to be switched using the menu on the very top of the input area. Motion paths can be drawn the same way as lines in the sketch mode. Arrowheads are automatically added to indicate the direction of the motion.
To look for sequences which are similar to a previously retrieved one, the looking glass icon on the result can be used to start a new search using this result as a query.
To use multiple results as a query, the plus and minus signs can be used to select positive and negative examples. Once these examples are selected, the relevance feedback button has to be clicked to start the search.
The top bar contains sliders labeled ‘global color’, ‘local color’, ‘edge’ and so forth. These sliders control how the results of the various features used for retrieval are to be combined. Changing the weights with these sliders will change the order in which results are displayed.
To add additional videos to the collection, the
extract command is used in Cineast. It expects a folder containing one video file as well as an arbitrary number of subtitle files.
The name of the folder will be the name under which the video will be added to the collection. To start the extraction simply start Cineast as described above and type