Prototyping physical products with machine learning. A workshop for product design students at University of Art and Design Offenbach.
In a 5-day workshop a group of product design students learned about the basic mechanics of machine learning and applied it to solve their daily struggles.
The students used our framework to explore the idea of products that can be trained by the user to solve specific problems. By wiring up teachable machines with small IoT-Devices the students were able to come up with inventive products to help them find their stuff, keep their kitchen clean, moderate meetings, explore nature and more.
Luigi Galbusera
Lara Bohe
Keep overlooking things in plain sight? Luigi and Lara built a flashlight which has a camera where a normal flashlight has its bulb. With the buttons you can add images of the objects you keep misplacing and toggle through the different objects you trained the device to recognize. When the lost object appears in sight of the camera, the device blinks and beeps :-)
Patrick Witkowski
Lorenz Modler
How get your flatmates to keep the kitchen clean? Lorenz and Patrick built a lock which locks them in the kitchen until the counter is tidy.
Dominik Steinhaus
Lennard Wilde
Meetings can be difficult. Especially with hotheaded colleagues. Dominik and Lennard built a moderator which can be trained to intervene in different situations.
The sound recognition can for example detect if participants talk simultaneously, get angry or loud, bad language is used etc.
Lukas Porstner
Florian Hahn
Florian and Lukas came up with a real-life alternative to Pokémon GO. By using different colors and scents they attracted insects to take a seat in front of their USB-microscope. If the insect matches the bug from the classifier, which is trained with pictures from the web, it takes a picture. The device could be part of a game where players compete in scratching off as many bugs on the list.
The teachable machine interfaces we provided could connect to small wireless devices, such as nodeMCUs and Raspberry Pis. This way the students could quickly generate ideas within a simplified framework which enabled them to then build working prototypes.
The concept of the teachable machine is simple and easy to understand. You feed the machine by adding examples to different classes. These examples can contain images, sound bites or other types of data. After the machine is trained it can distinguish if data samples are similar to one of the created classes. This process is called classification and is a basic application of machine learning. By experimenting with the data, the students can explore which types of data produce more reliant results than others and tinker with the algorithms to optimize them for their specific use cases.
The teachable machines where programmed to connect to Node-Red via WebSockets. The starting point for the sound recognition was the tensorflow simple audio recognition, with added tolerance controls. The image and pose classifiers are based on the ml5 framework. For the standalone devices, we used Coral dev boards with cameras.
To connect the teachable machine interfaces and devices with the nodeMCU to control the LEDs, servos, steppers and speakers, we used Node-Red. The interface is similar to other tools like grasshopper and vvvv, which are more commonly used among industrial design students in comparison to plain coding. For students who do know python or JavaScript, Node-RED offers simple access to include standard code.
Contextualizing Technology
office@same.vision
+49 30 2885 2595
Cotheniusstr. 6
D-10407 Berlin