We've seen how to learn an object and then detect it, so now we're going to use this to automate an action.
- You learn to recognize an object by creating a its cascade file (xml).
- You configure your raspberry pi (Jetson Nano) to read in a camera feed.
- This object can then be observed with the raspberry pi (Jetson Nano), for this you use openCV
- You add an object counter that displays in the upper left corner how many of these objects it observes.
- When this object is observed you make sure that the raspberry pi (Jetson Nano) controls an actuator (led, relay, lcd, ...)
- Jetson Nano 2GB or better, i strongly recommend to buy a 4GB version (Jetson Nano B01) as you will need all memory you can get for AI projects. I used the 2GB version (Jetson Nano 2 GByte), more info can be found here >> https://developer.nvidia.com/embedded/jetson-developer-kits
- ICE TOWER for cooling (recommended), see following link for more details >> https://www.jetsonhacks.com/2019/11/30/jetson-nano-extreme-cooling/
- Fast Micro SD Card, 64GB UHS-1 or higher recommended.
- Power supply and a jumper (J48) for power selection, see following links for details >> https://desertbot.io/blog/jetson-nano-power-supply-barrel-vs-micro-usb
- Keyboard, mouse and monitor, it is also possible to controll the Jetson Nano headless using SSH and/or VNC Viewer.
- Raspberry Pi cam, see following link for cam specifications >> https://www.raspberrypi.com/documentation/accessories/camera.html
- Optional 3D printed stand for the Pi Cam, STL files are included in this repository.
- 1 LED, 2 resistors, a transistor, a breadbord and some dupont-wires, more details later.
The first thing we need to do is flash a 64GB or larger micro SD card with a special image that can be found folowing this link >> https://github.com/Qengineering/Jetson-Nano-image/tree/611693cfe4aba56e33987afa9690f75259c545c1 The advantage is that this image has several pre-installed frameworks which we will need as well as CUDA support:
- JetPack 4.6.0
- OpenCV 4.5.3 (includes openCV-python and openCV-contrib-python libraries)
- TensorFLow 2.4.1
- TensorFlow Addons 0.13.0-dev
- Pytorch 1.8.1
- TorchVision 0.9.1
- LibTorch 1.8.1
- ncnn 20210720
- MNN 1.2.1
- JTOP 3.1.1
- TeamViewer aarch64 15.24.5
sudo apt-get install gparted
Patition >> Resize/Move >> select the partition
Reboot the device
sudo reboot
Verify your partition size
df -h
sudo jtop
Follow the instructions in this link, i did not erase the swap afterwards >>> https://qengineering.eu/install-opencv-4.5-on-jetson-nano.html
Verify the memory and swap file after reboot.
free -m
This is a very good and reliable IDE with virtual environments (VENV) and auto-completion. Follow the steps below for installation:
wget https://download.jetbrains.com/python/pycharm-community-2019.1.2.tar.gz
tar xvzf pycharm-community-2019.1.2.tar.gz
pycharm-community-2019.1.2/bin/pycharm.sh
Follow the steps mentioned here >> https://www.jetsonhacks.com/2019/04/02/jetson-nano-raspberry-pi-camera/
You will need a Windows machine for this with lots of computing power !!!!
This is an xml file that will generated by training your machine using thousands of positive pictures (cats in this case) and 3 times as much negative pictures (things related to cats, like dogs ....), the more pictures you provide, the more accuracy you will get. Install the training application and Follow the steps using this link >> https://amin-ahmadi.com/cascade-trainer-gui/
A tip for getting a large amount of pictures is installing the "Download All Images" Plu-in for Google Chrome, this plug-in will download all pictures from the site you are visiting (Google >> Pictures). Generating the Cascade file can take many hours, depending on the amount of pictures and processing power of your Windows machine.
Pro Tips:
- Eliminate all pictures that are not cats in the positives (p) folder
- Eliminate all positive objects, even only parts (cats) in the negative (n) folder
- Eliminate all pictures that are smaller than 400x400 in both the positives (p) and negatives (n) folder
- Make sure to have 2 - 5 times more negative pictures that possitive
- The Positive Image Usage (percentage) cannot be 100%, Use something like 95%
All needed Python scripts can be found in this repository, adapt where needed. All script should work with a PyCam. The only thing you need to change is the Cascade.xml file. I also uploaded my Cascade file for detecting cats in this repository.
Create a new project, give it a name. Make sure to select "New environment using Virtualenv" and "Inherit global site-packages" (containing all needed python libraries).
Put this folder in you "Home" Directory under "Jetson" - "PycharmProjects" - "Your Project Name" - "venv", it will then show up in your project.
The only thing you will need to change is the "path" to the cascade file and the "Objectname", this is the text showing if an object was detected (optional).
Point the camera at an object and run the script. Note*: press "q" on your keyboard to exit the program.
If an object (cat) is detected, it will look something like this >>
Note*: You can play with the Trackbars to optimize the tracking for your environment !!!
If no object (cat) is detected, it will look something like this >>
Note*: You can play with the Trackbars to optimize the tracking for your environment !!!
The goal is to enable an LED if an object (cat) is detected and disable the LED if no object is detected.
For this to work, we need ofcourse connect an LED to the GPIO pins of the Jetson Nano, but there is a CAVEAT. The GPIO control pins do not provide enough current to power the LED, so we need a workaround. We need to use a transistor to regulate the current flowing to the LED and powering it. Follow the tutorial on this page to make this work >> https://www.jetsonhacks.com/2019/06/07/jetson-nano-gpio/
The Jetson Nano GPIO pinout
The only thing you will need to change is the "path" to the cascade file and the "Objectname", this is the text showing if an object was detected (optional). Make sure to connect the LED to GPIO port 11.
Point the camera at an object and run the script. Note*: press "q" on your keyboard to exit the program.
If an object (cat) is detected, the LED should be on.
Note*: You can play with the Trackbars to optimize the tracking for your environment !!!
If no object (cat) is detected, the LED should be off.
Note*: You can play with the Trackbars to optimize the tracking for your environment !!!
The goal is to display in the upper left corner of the screen how many objects are detected.
The only thing you will need to change is the "path" to the cascade file and the "Objectname", this is the text showing if an object was detected (optional).
Point the camera at an object and run the script. Note*: press "q" on your keyboard to exit the program.
If an object (cat) is detected, it should display the number of objects detected.
Note*: You can play with the Trackbars to optimize the tracking for your environment !!!
If no object (cat) is detected, the counter should be "0"
Note*: You can play with the Trackbars to optimize the tracking for your environment !!!
The goal is to display in the upper left corner of the screen how many objects are detected and enable an LED if objects (cat) are detected and disable the LED if no objects are detected.
The only thing you will need to change is the "path" to the cascade file and the "Objectname", this is the text showing if an object was detected (optional).
Point the camera at an object and run the script. Note*: press "q" on your keyboard to exit the program.
If an object (cat) is detected, it should display the number of objects detected and enable the LED.
Note*: You can play with the Trackbars to optimize the tracking for your environment !!!
If no object (cat) is detected, the counter should be "0" and the LED should be disabled.
Note*: You can play with the Trackbars to optimize the tracking for your environment !!!
