HOME.md

Description

This project is inspired by @johanmodin's project clifs which allows you to search through a video using natural language. For example, if I'd like to find the frames of a video that contain a pancake in the shape of an otter, I can search using the following prompt "pancake shaped otter" and pass the video that I want to search through and voila! I'll be able to find the frames of the video that contain the pancake shaped otter.

This demo project makes this process easier by

1. Hosting it on the awesome Huggingface's Spaces servers
2. Allowing you to simply link to a Youtube video for specifying the video you want to search through
3. Allowing you to use multiple languages. Currently supported languages can be found here: https://arxiv.org/pdf/2004.09813.pdf

Disclaimer!

This project should not be used beyond educational/intellectual purposes. It is very much a work in progress and has not been tested on a large scale. Additionally, this type of technology can be misused for malicious purposes such as infringing on privacy through surveillance. Don't use this project for malicious purposes, please, be cool and ethical :).

With all these potential downsides, why did I make this project even easier to use? My main reason is that I see the potential good uses of this project. Namely, I believe this type of work can be used for helping students or learners in general to search through videos to find answers to their questions. Therefore, I created this project as a good way of showing the power of this technology and to learn more about how it works.