FabBits

Videos contain plethora of contextual information. For example, in a movie there are fighting scenes, sentimental scenes, romantic scenes, and many others. In a cricket match, there are wickets, sixes, fours et cetera. With the advent of the data-driven age, amateurs, researchers, and organisations alike require some specific part of this contextual information for their needs; maybe for creating a highlights reel of a sports match or mining data from movies for their machine learning models. This makes parts of certain types of videos very useful. FabBits tries to automate finding them. Following are the things it will be able to detect -

  • Action sequences in movies/shows - ✅
  • Summary of movies/shows - ✅
  • Actor-specific scenes in movies/shows - ✅
  • Jokes in sitcoms - ✅
  • Slo-mos in Sports - ❌
  • Goals in Soccer - ✅
  • Goal misses in Soccer - ⭕
  • Three pointers in Basketball - ✅

Project repo - github.com/achie27/FabBits Blog posts - medium.com/@achie27 Samples - Drive folder

You need the following things to run FabBits-

  1. Python3
  2. OpenCV - Used for image and video processing
  3. Moviepy - Used for video editing and audio processing
  4. PyQt5 - Used to make the GUI
  5. Scipy - Used for audio processing
  6. Tesserocr - Used for, well, OCR
  7. Pillow - Used to preprocess images for OCR The python dependencies can be installed by running -

          1
          pip3 install scipy
        
          2
          pip3 install opencv-python
        
          3
          pip3 install moviepy
        
          4
          pip3 install pyqt5
        
          5
          pip3 install Pillow
        
          6
          pip3 install tesserocr
not set

or if you are the Anaconda kind -


          1
          conda install -c conda-forge scipy
        
          2
          conda install -c conda-forge opencv
        
          3
          conda install -c conda-forge moviepy
        
          4
          conda install -c anaconda pyqt
        
          5
          conda install -c conda-forge pillow
        
          6
          conda install -c simonflueckiger tesserocr
not set

Run the main GUI by -

python3 main.py

To find your FabBit of choice -

  • Click MOVIES or SPORTS button for their respective use-cases
  • Select the use-case from the sidebar
  • A pop-up dialog will ask for the actor if actor-specific scene was chosen
  • Click on Choose File to select the input video
  • Click on Find FabBits
  • Move the slider in the blue areas, which are the extracted FabBits, and play the video
  • Click on Save FabBits to save the extracted stuff into a video file

You can also run the respective files of use-cases to get their FabBit, like - python3 goal_detector.py soccer_match.mp4

All the references can be found listed in the repository's readme.