An Examination of Information Theory

One of the tasks with which communication engineers are presented is that of devising a mechanism by which a significant sequence of words, a message, produced by somebody, the sender of the message, is reproduced at some other place, with the shortest practical time lag. The reproduction must be such that the receiver of the message will be able to understand what the sender meant by his message, at least, if he knows the sender's language (or, perhaps, his specific use of the language). The following illustration is typical: A writes on a sheet of paper "I love you" and wishes that B, 3000 miles away, should become aware of the full content of this message, with little delay and at a low cost. There will be institutions, in a capitalistic society, which will compete with each other in providing A, for a price, with the required service. Those companies which perform these services most satisfactorily, i.e., with an overall better combination of faithfulness, time lag, and cost, will get the job. The executives of these companies will hire engineers and put them to work on improving this overall combination. Let us scrutinize the situation somewhat more. A will be dissatisfied if he learns either that his message has been scrambled up, whether into something incomprehensible like "K bogl pou" or into something comprehensible like "A long bow" but with an entirely different meaning, or that an undistorted replica has been delivered a day late (and, of course, even more so if a distorted message is delivered too late). This dissatisfaction may lead to a loss of business for the company. In addition, A would appreciate a reduction in the cost of transmitting future messages. So would the company, at least in enlightened capitalism, if it were able to save on the cost of transmission. Cheaper transmission would mean increased business. How will the company transmit the message? Assume that a telegraph line stands at the disposal of the company. Assume, further, that at one side of the line a mechanism is available which is able to create electrical pulses in the line for any length of time, and that at the other side a mechanism is available which is able to detect the existence of a pulse in the line as well as measure its exact duration. Under these conditions, one could imagine that the following instructions would be given to the sending operator: "To transmit the letter "a", press the key for 1 second, to transmit "b", press for 2 seconds, . . ., to transmit "z", press for 26 seconds. Wait 1 second between letters, 2 seconds between words, 3 seconds between sentences." (For the sake of simplicity of illustration, let us disregard the transmission of numerical symbols, punctuation marks, capital letters, etc.) Corresponding instructions would be given the receiving operator. Assuming sufficiently trained operators, little or no loss of signal