The art and craft of making the Tortellino: playing with a digital gesture recognizer for preparing pasta culinary recipes

The innovative aspects introduced by the new hands-free gaming systems, like the Nintendo Wii, Sony Move and Microsoft Kinect, indicate that technology is progressively, and at a limited cost, reaching more natural ways of interacting with human beings, and vice versa. This process can clearly pave the way to beneficial opportunities in a wealth of fields, including those of technical and artistic interactive exhibitions. In fact, while for long all types of tech-inspired exhibitions have been mainly based on artifacts and tools that seldom required any interactions with the public, today this can radically change. The new frontiers of technology for performing arts provide the means for creating active interactions between the public and any object, allowing visitors to enjoy an experience that exceeds what can be solely passively seen or heard. In such domain, we describe the design and implementation of a multimedia system that interactively illustrates the process of preparing one of the masterpieces of the culinary heritage of the Italian city of Bologna, the Tortellino. Step-bystep, with our system, anyone can enjoy a virtual experience learning how to prepare a Tortellino while mimicking the movements a real cook would perform to prepare its recipe when starting from its raw ingredients: eggs and flour. From a scientific viewpoint, the challenging part of this project resides in recognizing cooks’ movements while verifying their correctness. In this paper, we describe how we achieved such result devising the most appropriate techniques able to allow our system to recognize a predefined set of actions (as those performed by a real cook) with the help of no other external hardware device, but a simple camera. Witnessing the importance of the results we got, our multimedia system has been chosen to be part of the expositions that will represent the City of Bologna at the Shanghai Universal Expo, in October 2010.