Claytronics: An Instance of Programmable Matter

Programmable matter refers to a technology that will allow one to control and manipulate three-dimensional physical artifacts (similar to how we already control and manipulate two-dimensional images with computer graphics). In other words, programmable matter will allow us to take a (big) step beyond virtual reality, to synthetic reality, an environment in which all the objects in a user’s environment (including the ones inserted by the computer) are physically realized. Note that the idea is not to transport objects nor is it to recreate an objects chemical composition, but rather to create a physical artifact that will mimic the shape, movement, visual appearance, sound, and tactile qualities of the original object. The enabling hardware technology behind synthetic reality is Claytronics, a form of programmable matter that can organize itself into the shape of an object and render its outer surface to match the visual appearance of that object. Claytronics is made up of individual components, called catoms—for Claytronic atoms—that can move in three dimensions (in relation to other catoms), adhere to other catoms to maintain a 3D shape, and compute state information (with possible assistance from other catoms in the ensemble). In our preliminary designs, each catom is a self-contained unit with a CPU, an energy store, a network device, a video output device, one or more sensors, a means of locomotion, and a mechanism for adhering to other catoms. Creating a physical replica of an arbitrary moving 3D object that can be updated in real time involves many challenges. The research involved in addressing these scientific challenges is likely to have broad impact beyond synthetic reality. Particularly relevant to the ASPLOS community, for example, is how to build and program a robust distributed system containing millions of computers that must cooperate extensively in a harsh environment where their configuration and goals are constantly changing? It is already possible to reproduce static 3D objects [5, 2]. To create a dynamic 3D object, however, we will build upon ideas from such diverse areas as modular and reconfigurable robotics [6, 3] and amorphous computing [1]. A Claytronics system forms a shape through the interaction of the individual catoms. For example, suppose we wish to synthesize a physical “copy” of a person. The catoms would first determine their relative location and orientation. Using that information they would then form a network in a distributed fashion and organize themselves into a hierarchical structure, both to improve locality and to facilitate the planning and coordination tasks. The goal (mimicking a human form) would then be specified abstractly, perhaps as a series of “snapshots” or as a collection of virtual deforming “forces”, and then broadcast to the catoms. Compilation of the specification would then provide each catom with a local plan for achieving the desired global shape. At this point, the catoms would start to move around each other using forces generated on-board, either magnetically or electrostatically, and adhere to each other using, for example, a nanofiber-adhesive mechanism [4]. Finally, the catoms on the surface would display an image; rendering the color and texture characteristics of the source object. If the source object begins to move, a concise description of the movements would be broadcast allowing the catoms to update their positions by moving around each other. The end result is that the system appears to be a single coordinated system. One key motivation for our work is that technology has reached a point where we can realistically build a programmable matter system which is guided by design principles which will allow it to ultimately scale to millions of sub-millimeter catoms. In fact, we expect our prototype for 2D Claytronics to be operational before ASPLOS’04. Our goal is that the system be usable now and scalable for the future. Thus, the guiding design principle, behind both the hardware and the software, is scalability. Hardware mechanisms need to scale towards micronsized catoms and million-catom ensembles. For example, the catom hardware minimizes static power consumption (e.g., no static power is used for adhesion) and avoids moving parts (e.g., the locomotion mechanism currently uses magnetic forces). Software mechanisms need to be scale invariant. For example, our localization and orientation algorithms are completely distributed, parallel, and, are indifferent to catom size. Claytronics will be a test-bed for solving some of the most challenging problems we face today: how to build complex, massively distributed dynamic systems. It is also a step towards truly integrating computers into our lives—by having them integrated into the very artifacts around us and allowing them to interact with the world.