Information
Information is the fundamental concept of information theory, the mathematical study of symbolic communication. Information theory was founded in 1948 by one seminal paper, Claude Shannon's "The Mathematical Theory of Communication," and has since proved essential to the development of computers, telecommunications, digital music recordings, and many other technologies of the "Information Age." It has also been applied fruitfully in cryptography, genetics, linguistics, and other disciplines.
Shannon, a quirky mathematician and electrical engineer famous for juggling while riding his unicycle down the hallways of Bell Laboratories in the 1940s and 50s, was not the first theorist to ponder the subject of "information." He was, however, the first to define the term rigorously and to specify its unit of measure--the now-famous "bit" (short for "binary digit").
Shannon defined "information" in what might at first seem an odd way: not as a substance that can exist in fixed quantities, although this is how information is usually spoken of, but as a change in another, more fundamental quantity: as a reduction in uncertainty. "Uncertainty" must thus be defined before "information." Consider the following scenario:
Someone is waiting to receive a message over some communications channel, perhaps a telephone. The message will specify a number chosen somehow at the other end of the channel. Before the message arrives there is, obviously, a certain degree of uncertainty, the precise amount of which depends on how many numbers the sender is choosing from (the more choices, the more uncertainty) and on how likely each choice is (if one choice is a million times more likely than all the others put together, there is little uncertainty). Unlike psychological uncertainty or doubt, which is a state of mind, this "uncertainty" is a quantity that can be calculated precisely and expressed in units of bits.
When the message is received, this original uncertainty is reduced. By how much? There are two basic possibilities: 1) If the communications channel is noiseless (perfectly reliable), then the uncertainty has been reduced to zero, for the person receiving the message now knows what number was sent. 2) If the channel is noisy, the person receiving the message cannot be perfectly sure that the number they have received is that was sent. In this case uncertainty has been reduced by receipt of the message, but not to zero; some doubt may remain. Either way, something has been learned: uncertainty has decreased. Shannon defined this decrease as the information gained by the receiver.
Because the word "information" has so many everyday meanings, a few cautions are in order. First, numbers as such (or other symbols) are not information. A string of numbers may be perfectly random, containing no information at all. Only symbols that reduce uncertainty about a well-defined question convey information in the mathematical sense. Second, as Shannon himself warned in 1948, information is irrelevant to meaning. Meaning is a mood or quality perceived subjectively by persons, and it cannot be discussed mathematically. A single bit of information may answer a life-and-death question or an utterly trivial one. Regardless of what meaning it conveys, however, it remains 1 bit of information, and that is all that can be said about it mathematically.
Until Shannon defined information and its unit of measure, it was not possible to give an exact definition of the capacity of a communications channel to transmit information. With these tools in hand, he was able to write down at once an exact expression for the maximum amount of information conveyable by any channel (the "channel capacity"); furthermore, he was able to prove that it is possible, while operating under this limit, to transmit information with as few errors as desired by paying a toll of redundancy.
With these upper and lower limits clearly in view, the design of practical systems could at last move forward on a systematic basis. Similarly, with an intuitive knowledge of fluids one might design hoses and cylinders, but until the technical concept of "pressure" is invented hydraulic design can only proceed by trial and error. In a sense, nothing was done after Shannon's invention of information theory that was not done before (codes, computers, and electronic communications systems had all been built prior to 1948), but afterward, it was done systematically.
Shannon's very general (and thus portable) approach transformed large areas of technology and science radically in just a few years. Information theory quickly became a key tool in telecommunications, encryption, error detection and correction, radar and sonar, seismology, radio astronomy, and other fields. Efforts were also made to apply its concepts in traditionally less mathematical disciplines such as economics, biology, linguistics, and psychology. Not all such attempts were successful. The study of the psychology of speech perception and attention, for example, was dominated by information theory throughout the 1950s, but a shortage of consistent experimental results led for the most part to abandonment of information-theoretic approaches in these fields.
Perhaps the discipline most profoundly connected to information theory is thermodynamics. Shannon's uncertainty measure is formally equivalent to the thermodynamic expression for entropy, and the two turn out to be intimately related. Just as a change in symbolic uncertainty corresponds to symbolic information, so a change in a physical system's entropy (roughly speaking, its state of disorder) corresponds to "thermodynamic information." In 1951, Leon Brillouin proved the extraordinary theorem that the observation of one bit of information by any possible physical system requires a minimum of kBT ln(2) ergs of energy, where kB is Boltzmann's constant and T is the absolute temperature. As Dennis Gabor, the Nobel prize-winning inventor of holography, put it: "You cannot get something for nothing, not even an observation."
Today, applications of information theory are ubiquitous. Error-correction coding schemes ultimately derived from Shannon's work enable deep-space probes to transmit data across the Solar System despite high noise, immense distances, and signal strengths of only a few watts. They also allow near-perfect reproduction of music from compact discs in the presence of dust, scratches, and fingerprints.
This is the complete article, containing 983 words
(approx. 3 pages at 300 words per page).
