home
vitrivr Logo
  1. Setting up the vitrivr stack
    1. Step 1: install ADAMpro
    2. Step 2: install Cineast
    3. Step 3: install the front end
  2. Use vitrivr
    1. Search by sketch or image
    2. Add motions to sketch
    3. Query by Example
    4. Relevance feedback
    5. Additional functionality
  3. Extending vitrivr
    1. Add new videos

Setting up the vitrivr stack

The vitrivr stack consists of three components: ADAM/ADAMpro (the database), Cineast (the retrieval engine) and the front-end which is browser-based. The following describes the steps needed to set up all components of the stack and populate it with some exemplary data.

Step 1: install ADAMpro

ADAMpro is the persistent polystore (based on Apache Spark) for all data required for retrieval. For experimenting with ADAMpro, a version of ADAMpro without any data, can be easily run as a Docker image. For this, run

docker run -p 5005:5005 -p 5890:5890 -p 9099:9099 -p 5432:5432 -p 9000:9000 -p 4040:4040 vitrivr/adampro

which will pull the Docker image from Docker Hub and run it with the corresponding ports open. You can access the ADAMpro UI by navigating to http://localhost:9099.

Furthermore, we provide a pre-built Docker image which not only contains ADAMpro but is already populated with the data extracted from the Open Short Video Collection. Simply download the archive and run:

docker load < adampro-built-withdata.tar

You can find a more extended documentation on ADAMpro, the code structure and deployment possibilities, in the repository on github.

Step 2: install Cineast

Cineast is the retrieval engine in charge of feature extraction and query processing. To build and run Cineast you need Java JDK 8. Clone the Cineast github repository and run:

./gradlew deploy

This will create the cineast.jar file which can be found in build/libs.

Edit the cineast.json file and point it to the location where you set up ADAMpro in Step 1:

...

"database": {
		"host": "< your ADAMpro host here >",
		"port": 5890,
		"plaintext": true
	},
	
...
	

If you used the prebuilt osvc images, you need to change the database name from cineast to osvc and the user and the password for adam need to be changed to docker as both user and password.

The freshly created cineast jar file can then be started with:

java -jar cineast.jar

Step 3: install the front end

The front end is web-based and therefore requires a web-server. Any web-server is fine as long as it runs php. Once you have one up and running clone the github repo into a folder accessible by the web-server. If you chose to use our pre-populated ADAM image, also download the videos and thumbnails which go together with the meta-data in the database. Place them somewhere the web-server can find them.

Next you need to configure the front end. This is done by editing the config.js file found in the js directory so that the front end knows where the images and videos are (relatively to the webroot of the UI):

const cineastHost = "api.php";
const thumbnailHost = "thumbnails/";
const videoHost = "collection/";
const thumbnailFileType = "jpg";
const maxFrameGap = 250;

Use vitrivr

vitrivr offers multiple query modes which are outlined below.

Search by sketch or image

The left side of the vitrivr UI contains the input canvases. They are used to specify the visual part of the query by either sketching directly on them or adding existing images using drag and drop. Longer sequences can be specified by adding multiple canvases.

Add motions to sketch

To search for particaular types of movement, the motion canvas can be used. To select the motion input, the mode has to be switched using the menu on the very top of the input area. Motion paths can be drawn the same way as lines in the sketch mode. Arrowheads are automatically added to indicate the direction of the motion.

Query by Example

To look for sequences which are similar to a previously retrieved one, the looking glass icon on the result can be used to start a new search using this result as a query.

Relevance feedback

To use multiple results as a query, the plus and minus signs can be used to select positive and negative examples. Once these examples are selected, the relevance feedback button has to be clicked to start the search.

Additional functionality

The top bar contains sliders labeled ‘global color’, ‘local color’, ‘edge’ and so forth. These sliders control how the results of the various features used for retrieval are to be combined. Changing the weights with these sliders will change the order in which results are displayed.

Extending vitrivr

Add new videos

To add additional videos to the collection, the extract command is used in Cineast. It expects a folder containing one video file as well as an arbitrary number of subtitle files. The name of the folder will be the name under which the video will be added to the collection. To start the extraction simply start Cineast as described above and type extract path/to/folder.