SMLT-3706A

TEAM

Benedict Hubener
Keyur Jain

MY ROLE

Physical prototyping
Programing Arduino 
Storytelling and direction

PROJECT DETAILS

Two week Design Fiction course
on living with Machine Intelligence
in the future

01_IMG_0433-squashed

Smart city maintenance worker dignosing malfunctioning smart camera

01

What is 
the SMLT-3607A?

In the near future, cities are filled with smart infrastructure such as decentralized security cameras, self-sorting trashcans and intelligent street lights. But who do you call when smart things breaks? The SMLT 3607A or Supervised Machine Learning Trainer is a speculative tool for the future city maintenance worker. He/she can use the SMLT to interface with abnormally behaving smart infrastructure and retrain the smart camera by recording new examples in real time. The future maintenance worker will teach the camera what it’s seeing and curate the training dataset. 

A speculative training video for SMLT-3607A

02

Why did you make 
the SMLT-3607A?

Artificial intelligence and automation are having a profound impact on future jobs. However, the conversations in the media around job automation tend to lead to doomsday scenarios. Search online for artificial intelligence and jobs and you will see Terminator-esque photos and predictions. We were inspired by the concept of “Future Mundane” popularized by Nick Foster. We believe our future is not made from transparent glass and Minority Report style interfaces. Our future will much more mundane where the old exists with the new.

We wanted to make a “Knotty Objects” that represent and hint at this mundane future. We wanted people to touch and feel the future living with machine city in our cities. We created the SMLT-3607A so that we can start a conversation around automation and jobs. We want people to talk about how we can co-exist with automation and machine learning and how we can embrace a future with artificial intelligences.

03_DSC_0910_PS-squashed

Using SMLT-3607A to train correct classification of pedestrians 

03

Tell me about
your process?

We observed and spoke to people who are working in the city of Copenhagen. We talked to a city maintenance man, a scaffolding worker, and a meter maid. We spoke to them about their jobs, their tools, and the effects of automation. The most interesting person that we spoke to is Kal, the city maintenance man. Kal has a whole cart of tools that he wheels around. This inspired us to think about how Kal’s job will be affected by automation and the tools of the future city maintenance person.

IMG_0510-squashed

Kal the Copenhagen city hall maintenance person

In order to create a believable future, we looked at “weak signals” in Copenhagen. We spoke to city planners and visited a street where the City of Copenhagen is testing smart city technologies. We conducted secondary research and looked at papers and articles on smart cities. The signals suggest that the future smart city will be filled with cameras and machine learning algorithms.

 

IMG_1918-2-squashed

A pollution sensor in the city of Copenhagen

As result, we imagine that the future smart cities will be filled with surveillance cameras that are context aware and have machine vision capabilities. Instead of relying on humans to monitor the security footage and identify issues, the surveillance camera can flag behaviours or people. For example, a surveillance camera might be trained to identify cyclists, pedestrians, etc.. Or it might be trained to identity a fist fight or someone who is littering. When the surveillance camera sees something prohibited, it would automatically alert the appropriate authorities. But because machine vision is based on robust datasets, having a well crafted dataset is important. We imagine the future city maintenance person will be asked to maintenance both the infrastructure and the integrity of the dataset. 
 

757824_90431_74673_ocFcxRrzO

Observing pedestrians to train the smart camera

04_IMG_0297-2-squashed

List of classes that the smart camera can identify

04

How do you use
the SMLT-3607A?

City infrastructures break all the time. The smart surveillance camera is no exception. The SMLT-3607A helps the city maintenance person to fix and retrain misbehaving surveillance cameras. To use the SMLT-3607A, a city maintenance person plugs the kit into the camera. He/she monitors the footage and see what the camera is mislabelling. He/she then pressed the record example button every time the camera sees the correct person or object or action. The recorded example is then added to the dataset of the algorithm. Eventually, after many correct examples, the surveillance camera will be able to identify correctly.

IMG_0658-squashed

The SMLT kit with USB loaded with training data, remote control, and security tape.

IMG_0574-squashed

SMLT-3607A interface

IMG_0117-squashed

USB containing additional datasets to aid maintenance person

05

How did you
design the
physical interface?

The interface and operation of the SMLT-3607A is deliberately made simple and repetitive. We believe that in the future, machine learning technology is going to be normalized. The repetitive task of data set labeling is going to be the new blue-collar job. The SMLT-3607A will be a tool that would allow anyone to interface and work with machine learning technology.


At the moment the SMLT-3607A exists as a prop with an embedded Arduino. The screen is aN iPhone controlled by a laptop. To continue with the project, we would hook up all of the buttons and knobs to a WIFI enabled Arduino. The Arduino would send all of the input signals to the machine learning software Wekinator. The Wekinator allows us to access machine learning capabilities and output to some kind of Java or Processing sketch. We went through multiple iterations of interface design to create a believable maintenance tool. 

WI.010-squashed

The many iterations of the interface

IMG_0069-squashed

Early prototype of the interface

WI.011-squashed

Pulling apart old technology for buttons and knobs