Smart spaces are inherently complex and dynamic systems, where diverse devices, sensors, actuators and computational elements need to interact with one another. A middleware infrastructure can provide suitable abstractions that simplify the task, and allow designers to ignore the details of the underlying elements. Unfortunately, however, existing middleware solutions do not generalize well to different kinds of spaces, since they often fail to address the scalability and dynamism of such spaces. In this paper we propose SeSaMe, a semantic and self-adaptive middleware infrastructure for highly dynamic and massive smart spaces. SeSaMe establishes a “backbone” to let components connect to the system without any prior knowledge of its topology. It is capable of maintaining the system’s overall reliability, even when multiple components leave or fail unexpectedly, and of coping with message congestion, by dynamically altering the system’s topology. SeSaMe also provides a simple declarative language for defining how one wants the system to evolve over time, and semantic technologies for harmonizing the interaction of different kinds of components. The main new features of SeSaMe are exemplified on two example smart spaces with significantly different characteristics.