Our Team!
- Yang Xu (Sensor_Motor)
- Mayank Shrivastava (detection)
- Aadam Ali (detection)
- Gyeongwon Lee (Interact With Google Assistant)
With amazing innovations being made in various personal voice assistant products in the market (e.g. Google Home, Amazon Echo), these technologies are mostly left inaccessible to people with hearing and sppech impairments. We envisioned to create a way for people to interact with these technologies by using sign languages in front of a rotating webcam -- that knows where to look -- to "speak" to these personal assistants, and "listen" to their responses through text displayed on our web application (web application not fully completed yet).
We used a ton of cool technologies and APIs available in order to complete this project, and we learned a lot in the process. We are very proud to have made a functional product that was actually usable by the end of our 36-hour-long hacking. Feel free to check out our page at:
https://devpost.com/software/commi
Thanks! :)
- We used OpenCV and Microsoft Azure API to build a Machine Learning model that was trained to interpret sign languages into text, which was then delivered to Google Assistant using Google Docs API (for communication between our laptops) and Google Assistant API.
- We also used Arduino, Ultrasonic sensors, a servo motor, and our custom-made cardboard platform to make the webcam rotate towards wherever the user's hand is at.
- All in all, we essentially built a "full product" that allowed users with hearing and speech impairments to access and use personal voice assistant products.