SMLT-3706A
TEAM
Benedict Hubener
Keyur Jain
MY ROLE
Physical prototyping
Programing Arduino
Storytelling and direction
PROJECT DETAILS
Two week Design Fiction course
on living with Machine Intelligence
in the future
Smart city maintenance worker dignosing malfunctioning smart camera
01
What is the SMLT-3607A?
In the near future, cities are filled with smart infrastructure such as decentralized security cameras, self-sorting trashcans and intelligent street lights. But who do you call when smart things breaks? The SMLT 3607A or Supervised Machine Learning Trainer is a speculative tool for the future city maintenance worker. He/she can use the SMLT to interface with abnormally behaving smart infrastructure and retrain the smart camera by recording new examples in real time. The future maintenance worker will teach the camera what it’s seeing and curate the training dataset.
A speculative training video for SMLT-3607A
02
Why did you make the SMLT-3607A?
Artificial intelligence and automation are having a profound impact on future jobs. However, the conversations in the media around job automation tend to lead to doomsday scenarios. Search online for artificial intelligence and jobs and you will see Terminator-esque photos and predictions. We were inspired by the concept of “Future Mundane” popularized by Nick Foster. We believe our future is not made from transparent glass and Minority Report style interfaces. Our future will much more mundane where the old exists with the new.
We wanted to make a “Knotty Objects” that represent and hint at this mundane future. We wanted people to touch and feel the future living with machine city in our cities. We created the SMLT-3607A so that we can start a conversation around automation and jobs. We want people to talk about how we can co-exist with automation and machine learning and how we can embrace a future with artificial intelligences.
Using SMLT-3607A to train correct classification of pedestrians
03
Tell me about
your process?
We observed and spoke to people who are working in the city of Copenhagen. We talked to a city maintenance man, a scaffolding worker, and a meter maid. We spoke to them about their jobs, their tools, and the effects of automation. The most interesting person that we spoke to is Kal, the city maintenance man. Kal has a whole cart of tools that he wheels around. This inspired us to think about how Kal’s job will be affected by automation and the tools of the future city maintenance person.
Kal the Copenhagen city hall maintenance person
In order to create a believable future, we looked at “weak signals” in Copenhagen. We spoke to city planners and visited a street where the City of Copenhagen is testing smart city technologies. We conducted secondary research and looked at papers and articles on smart cities. The signals suggest that the future smart city will be filled with cameras and machine learning algorithms.
A pollution sensor in the city of Copenhagen
As result, we imagine that the future smart cities will be filled with surveillance cameras that are context aware and have machine vision capabilities. Instead of relying on humans to monitor the security footage and identify issues, the surveillance camera can flag behaviours or people. For example, a surveillance camera might be trained to identify cyclists, pedestrians, etc.. Or it might be trained to identity a fist fight or someone who is littering. When the surveillance camera sees something prohibited, it would automatically alert the appropriate authorities. But because machine vision is based on robust datasets, having a well crafted dataset is important. We imagine the future city maintenance person will be asked to maintenance both the infrastructure and the integrity of the dataset.
Observing pedestrians to train the smart camera
List of classes that the smart camera can identify
04
How do you use
the SMLT-3607A?
City infrastructures break all the time. The smart surveillance camera is no exception. The SMLT-3607A helps the city maintenance person to fix and retrain misbehaving surveillance cameras. To use the SMLT-3607A, a city maintenance person plugs the kit into the camera. He/she monitors the footage and see what the camera is mislabelling. He/she then pressed the record example button every time the camera sees the correct person or object or action. The recorded example is then added to the dataset of the algorithm. Eventually, after many correct examples, the surveillance camera will be able to identify correctly.
The SMLT kit with USB loaded with training data, remote control, and security tape.
SMLT-3607A interface
USB containing additional datasets to aid maintenance person
05
How did you
design the
physical interface?
The interface and operation of the SMLT-3607A is deliberately made simple and repetitive. We believe that in the future, machine learning technology is going to be normalized. The repetitive task of data set labeling is going to be the new blue-collar job. The SMLT-3607A will be a tool that would allow anyone to interface and work with machine learning technology.
At the moment the SMLT-3607A exists as a prop with an embedded Arduino. The screen is aN iPhone controlled by a laptop. To continue with the project, we would hook up all of the buttons and knobs to a WIFI enabled Arduino. The Arduino would send all of the input signals to the machine learning software Wekinator. The Wekinator allows us to access machine learning capabilities and output to some kind of Java or Processing sketch. We went through multiple iterations of interface design to create a believable maintenance tool.
The many iterations of the interface
Early prototype of the interface
Pulling apart old technology for buttons and knobs