video_features allows you to extract features from video clips.
It supports a variety of extractors and modalities,
i. e. visual appearance, optical flow, and audio.
# clone the repo and change the working directory git clone https://github.com/v-iashin/video_features.git cd video_features # install environment conda env create -f conda_env_torch_zoo.yml # load the environment conda activate torch_zoo # extract r(2+1)d features for the sample videos python main.py \ feature_type=r21d \ device="cuda:0" \ video_paths="[./sample/v_ZNVhz7ctTq0.mp4, ./sample/v_GGSY1Qvo990.mp4]" # if you have many GPUs, just run this command from another terminal with another device # device can also be "cpu"
If you are more comfortable with Docker, there is a Docker image with a pre-installed environment that supports all models. Check out the Docker support. documentation page.
Multi-GPU and Multi-Node Setups
video_features, it is easy to parallelize feature extraction among many GPUs.
It is enough to start the script in another terminal with another GPU (or even the same one)
pointing to the same output folder and input video paths.
The script will check if the features already exist and skip them.
It will also try to load the feature file to check if it is corrupted (i.e. not openable).
This approach allows you to continue feature extraction if the previous script failed for some reason.
If you have an access to a GPU cluster with shared disk space you may scale extraction with as many GPUs as you can by creating several single-GPU jobs with the same command.
Since each time the script is run the list of input files is shuffled, you don't need to worry that workers will be processing the same video. On a rare occasion when the collision happens, the script will rewrite previously extracted features.
Please, let me know if you found this repo useful for your projects or papers.