The following program detects weeds and plants, implementing OpenCV to threshold and find the edges of green objects. As of right now, the largest object is considered a plant while any other object above a certain size is considered a weed. One of the advantages of this program is that it performs under a range (albeit small) of lighting conditions. However, one of the disadvantages of using this program is that it does not implement any kind of machine learning. I aim to add in the future a neural network to identify plants regardless of size.
This was a project inspired by Farmbot and a group called Sustainable Earth at Carnegie Mellon University. Overall, it has been a rewarding experience to understand to program such a large project in pieces, put everything together, and ultimately see some promising results.
- Python 3.x
- OpenCV 3.0 or higher (currently using 3.4.1)
- numpy (latest available version preferred)
- pillow (image resizing)
- PyQt5 (for the Raspberry Pi interface) For more specific directions: https://raspberrypi.stackexchange.com/questions/62939/pyqt5-on-a-raspberry-pi. PyQt4 is also possible, though the entire interface must be edited for that
- serial (for Arduino communication, currently working on extension)
- (FUTURE) tensorflow (normally runs on a 64-bit operating system, have not implemented)
You can get the latest Arduino from here: https://www.arduino.cc/en/Main/Software. However, the Raspberry Pi we currently have runs on Arduino 1.6+.
To get started with this project:
- Make a Github account
- Download and install Github Desktop (https://desktop.github.com/) on your computer.
- Sign into Github desktop with your Github account information.
- Select File -> Clone Repository to clone (make a copy of) this project onto your desktop.
You can now make changes to the files on your local machine, then use Github Desktop to send those edits to the Github repository here.
To save changes to the shared repository on Github:
- Open Github Dekstop on your computer and select the correct repository. You should see that change have been made if you made local changes.
- A summary (name) for the edits is required, so name your edits, preferable with your name or username at the start
- Commit edits with the "commit to master" button
- Click the "Push to Origin" button in the top center of Github Desktop
Be very careful about pushing edits, as they will affect everyone who pulls in the repository. Pulling edits is done by pressing the "Fetch Origin" button.
OpenCV, Pyserial, and PyQt5 can be installed by the 'pip install' or 'pip3 install' command. The commands should be input in terminal. If you have multiple versions of Python installed, make sure to use the correct version of pip.
pip3 install opencv-python
pip3 install pyserial
pip3 install pyqt5
pip3 install pillow
In the case that you don't have pip or the above commands produce the error "pip is not recognized", you can follow these steps:
-
Go to the Python packages library (https://pypi.org/)
-
Download the desired library (usually a wheel file)
-
If you don't have 7-zip or some kind of file extraction program that can deal with wheel files, you can download 7-zip at https://www.7-zip.org/download.html. Choose the distribution that fits your machine and install it.
-
Once you have 7-zip and have downloaded the library, extract the file with 7-zip. Right click the downloaded file, hover over "7-zip," and select "Extract Files." the following link provides a visual and might be helpful: https://www.newsgroupreviews.com/7-zip-extract-unrar.html.
-
Attempt to import the library on the command line in the Python IDLE shell, in Pyzo, and whatever Python editor you prefer. The commands are:
a. opencv-python -import cv2
b. pyserial -import serial
c. pyqt5 -import PyQt5
d. pillow -import PIL -
If step 5 fails for any particular library, place the library contents of that library in the site-packages directory of your Python distribution and attempt to import again. If you are installing OpenCV on a Mac and you succeeded in pip installing but fail to import cv2, try the directions here: https://www.learnopencv.com/install-opencv3-on-macos/. If all fails, contact us.
Once the libraries are installed, attempt to run some of the test files in the 'Test' folder of this repository. If there are no errors, you are set to go!
To make sure that you've followed instructions correctly, try to run some of the tests. The two tests are located within the "Tests" folder. The first test is called "Test-ImageGrab," which tests the reading capabilities of OpenCV. Play around and see if you can take a live picture with your laptop's built-in camera.
The second test is "Test-Threshold," which is an example of one way to threshold the image. There are many ways to threshold, so don't be afraid to play around with the different thresholds. A quick example is located here: https://medium.com/@ckyrkou/color-thresholding-in-opencv-91049607b06d. You can also look into OpenCV masks and contours. The OpenCV docs are here: https://docs.opencv.org/3.4.1/.
from Image import ImageControl
imgControl = ImageControl.Controller()
imgControl.image_grab()
Must be called in order to initialize the image. Can use a given image in the Camera folder or can take an image with a given camera.
imgControl.find_plants()
Finds a plant and weeds based on default thresholds, including a scale for brightness. Future edits will allow for changes. Currently, the center-most object is identified as a plant, and all other objects identified as weeds.
imgControl.draw_all()
Outlines the plant in green and the weeds in blue, and displays the result.
from Arduino import ArduinoControl
arduinoControl = ArduinoControl.Controller()
Creating an instance of Controller opens up serial communication with an Arduino. The various commands to be given to the Arduino can be found in the Arduino folder.
arduinoControl.water_cycle()
Commands such as watering are still being determined.