Over the past five years, I have taught graduate and undergraduate students at the California College of the Arts how to design interactive products and services. In my class, “Objects and Spaces,” students learn to design experiences that go beyond the screen, creating a future in which our digital interfaces are more closely integrated with the physical world.
The class begins with a reading of Bret Victor’s “A Brief Rant on the Future of Interaction Design”. Victor makes a compelling argument that our touchscreen devices fail to make proper use of human capabilities. For example, our hands provide us with a sophisticated level of control and feedback - they help us interpret the world and give us the agency to shape it. Yet our touchscreens are only capable of the most rudimentary forms of interaction - they can only interpret a “tap,” “swipe,” or a “pinch”. Victor’s thinking frames the objectives of our class: Don’t extrapolate yesterday’s technology. Instead consider, design for and capitalize on our natural human capabilities.
With this in mind, students explore what it means to design tools that enable users to “think through their hands” making fuller use of our innate ability to organize and manipulate objects in the real world. By augmenting these objects with digital superpowers, students are encouraged to innovate their own interactive vocabularies that transcend existing models.
Continuing with the theme of collaborative tools, the next project in the class involved building a working interactive table, controlled by physical objects placed on the table surface.
Working with co-teacher Aaron Soloman, students work in small teams to design and engineer their own interactive tables using the Reactable software system. Originally conceived and developed by a research team at the Pompeu Fabra University in Barcelona, the Reactable system tracks the physical location of objects using the computer’s camera. Fiducial codes adhered to the underside of the objects visualize the identity, position, and rotational direction of each object. Students then wrote custom code for their systems using the Processing scripting language.
Each team learned how to set up, calibrate and then use the Reactable system to prototype a working interactive table. Projects included an interactive kitchen table that advises cooks on the best recipes for leftover food, a collaborative video editing tool and a food pairing system designed to help chefs create a balanced menu (shown above).
One of the best ways to start imagining how a tangible interface might work is to prototype it using minimal technology and a flexible mindset. Sometimes, the less technology there is to worry about, the faster ideas can take form, and the easier it is to adjust course and try out a new direction.
In this project, students use simple crafting materials and ready-made objects to prototype a tangible interface, imagining how the shapes of various objects, their physical placement, and their interactions with people’s hands might provide useful interactive functionality.
The assignment challenges each team to develop a tangible interface that supports 3 to 4 people working towards a common goal - such as planning a vacation, editing a magazine or organizing ideas during a brainstorming session. Team members role-play the tasks of each user, physically acting out each interaction with their own hands.
Once they have a rough idea of how their system could work, students use video projectors and animation software to augment their prototypes with projected light. By mapping colored light onto their cardboard and plastic pieces, it becomes easier to imagine how their systems would incorporate symbols, states and feedback animations. This ‘smoke and mirrors’ prototyping technique enables students to rapidly test their concepts with their peers, and gather just enough feedback to improve their ideas the next time round.
Students are also encouraged to critique their prototypes through the lens of Don Norman’s excellent book “The Design of Everyday Things”. His ideas about what makes an interface useable provide a robust rubric for helping students evaluate and evolve their concepts.
