Bit by Bit

According to the guide to the International System of Units (SI) issued by the National Institute of Standards and Technology (NIST)1, there are a “few highly specialized units” that are nonetheless still “acceptable for use with the SI”. One of them is the bit — the unit of information. I would like to challenge this rather unfair treatment of one of the most important units today, the very one that defines our information society. The bit (a contraction of ‘binary digit’), also known as the shannon, quantifies the information capacity of a two-state system: 0/1, true/false, yes/no, on/off. The term was coined by John Tukey at Bell Labs in 1947, and first used formally by Claude Shannon a year later in his seminal paper laying the foundations of information theory2. In general, the amount of information that can be stored in a classical system with N states is logbN, where the logarithmic base b represents the choice of a unit for measuring information. For b = 2 one has the bit, for b = 3 the trit, for b = 10 the hartley or ban or decimal digit, and for b = e, the base of the natural logarithm, one has the nat. The bit is the best known among these units and also the ‘smallest’ in terms of the logarithmic base defining it. For practical purposes the bit is a bit too small, so people usually think in terms of chunks of eight bits, or bytes. An array of eight bits can represent 28 values, enough to encode alphanumeric characters, punctuation and more. The byte (B), which got its name from Werner Buchholz at IBM in 1956, has become the standard across computer architectures and is used to characterize the size of the active memory or storage capacity of computing devices. The beauty of the bit is that it is not just a mathematical concept. It reflects a real physical quantity, and I am not necessarily referring to the memory size of your hard drive. Information itself is physical. Rolf Landauer was the first to observe that information is not a purely abstract concept, but that it is always tied to a physical representation and is therefore inevitably subject to the laws of physics — in particular, the laws of