Because there are often application-specific host-side filtering to be done on the stereo neural inference results, and because these calculations are lightweight (i.e. could be done on an ESP32), we leave the triangulation itself to the host.
This 3D visualizer is for the facial landmarks demo, and uses OpenGL and OpenCV. Consider it a draft/reference at this point.
sudo apt-get install python3-pygame
python3 -m pip install -r requirements.txt
Run the application
python3 main.py
You should see 5 windows appear:
mono_left
which will show camera output from left mono camera + face bounding box & facial landmarksmono_right
which will show camera output from right mono camera + face bounding box & facial landmarkscrop_left
which will show 48x48 left cropped image that goes into the second NN + facial landmarsk that get outputed from the second NNcrop_right
which will show 48x48 right cropped image that goes into the second NN + facial landmarsk that get outputed from the second NNpygame window
which will show the triangulation results
If you happen to see the error /bin/sh: 1: sdl-config: not found
during command installation, in a log similar to this one
Collecting pygame==1.9.6
Using cached pygame-1.9.6.tar.gz (3.2 MB)
ERROR: Command errored out with exit status 1:
command: /home/vandavv/dev/luxonis/venvs/develop/bin/python3 -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-clr4n0vn/pygame_2163392aa09745ef942ecb93ecf80adc/setup.py'"'"'; __file__='"'"'/tmp/pip-install-clr4n0vn/pygame_2163392aa09745ef942ecb93ecf80adc/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-a1i9ij3c
cwd: /tmp/pip-install-clr4n0vn/pygame_2163392aa09745ef942ecb93ecf80adc/
Complete output (12 lines):
WARNING, No "Setup" File Exists, Running "buildconfig/config.py"
Using UNIX configuration...
/bin/sh: 1: sdl-config: not found
/bin/sh: 1: sdl-config: not found
/bin/sh: 1: sdl-config: not found
Hunting dependencies...
WARNING: "sdl-config" failed!
Unable to run "sdl-config". Please make sure a development version of SDL is installed.
----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/0f/9c/78626be04e193c0624842090fe5555b3805c050dfaa81c8094d6441db2be/pygame-1.9.6.tar.gz#sha256=301c6428c0880ecd4a9e3951b80e539c33863b6ff356a443db1758de4f297957 (from https://pypi.org/simple/pygame/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Could not find a version that satisfies the requirement pygame==1.9.6 (from versions: 1.9.2rc1, 1.9.2, 1.9.3, 1.9.4.dev0, 1.9.4rc1, 1.9.4, 1.9.5rc1, 1.9.5rc2, 1.9.5, 1.9.6rc1, 1.9.6rc2, 1.9.6, 2.0.0.dev1, 2.0.0.dev2, 2.0.0.dev3, 2.0.0.dev4, 2.0.0.dev6, 2.0.0.dev8, 2.0.0.dev10, 2.0.0.dev12, 2.0.0.dev14, 2.0.0.dev16, 2.0.0.dev18, 2.0.0.dev20, 2.0.0.dev22, 2.0.0.dev24, 2.0.0, 2.0.1.dev1, 2.0.1)
ERROR: No matching distribution found for pygame==1.9.6
Please run the following command in order to fix the issue (per this thread)
$ sudo apt-get install python-dev libsdl-image1.2-dev libsdl-mixer1.2-dev libsdl-ttf2.0-dev libsdl1.2-dev libsmpeg-dev python-numpy subversion libportmidi-dev ffmpeg libswscale-dev libavformat-dev libavcodec-dev libfreetype6-dev