Achievement :

I. Information Theory:

One of the basic postulates of information theory is that information can be treated like a measurable physical quantity, such as density or mass. The theory has widely applied by communication engineers and some of its concepts have found application in psychology and linguistics.

The basic elements of any general communications system include
1. a source of information which is a transmitting device that transforms the information or "message" into a form suitable for transmission by a particular means.
2. the means or channel over which the message is transmitted.
3. a receiving device which decodes the message back into some approximation of its original form.
4. the destination or intended recipient of the message.
5. a source of noise (i.e., interference or distortion) which changes the message in unpredictable ways during transmission.
It is important to note that "information" as understood in information theory has nothing to do with any inherent meaning in a message.
It is rather a degree of order, or nonrandomness, that can be measured and treated mathematically much as mass or energy or other physical quantities are. A mathematical characterization of the generalized communication system yields a number of important quantities, including
1. the rate at which information is produced at the source.
2. the capacity of the channel for handling information.
3. the average amount of information in a message of any particular type.
To a large extent the techniques used with information theory are drawn from the mathematical science of probability. Estimates of the accuracy of a given transmission of information under known conditions of noise interference, for example, are probabilistic, as are the numerous approaches to encoding and decoding that have been developed to reduce uncertainty or error to minimal levels.

Information and Uncertainty are technical terms that describe any process that selects one or more objects from a set of objects. We won't be dealing with the meaning or implications of the information since nobody knows how to do that mathematically.

Suppose we have a device that can produce 3 symbols, A, B, or C. As we wait for the next symbol, we are uncertain as to which symbol it will produce. Once a symbol appears and we see it, our uncertainty decreases, and we remark that we have received some information. That is, information is a decrease in uncertainty.

How should uncertainty be measured? The simplest way should be to say that we have an "uncertainty of 3 symbols". This would work well until we begin to watch a second device at the same time, which, let us imagine, produces symbols 1 and 2. The second device gives us an "uncertainty of 2 symbols". If we combine the devices into one device, there are six possibilities, A1, A2, B1, B2, C1, C2. This device has an "uncertainty of 6 symbols". This is not the way we usually think about information, for if we receive two books, we would prefer to say that we received twice as much information than from one book. That is, we would like our measure to be additive.

II. Symbolic Logic and Switching Theory:

Shannon is as the founding father of electronic communications age since he noticed and discovered the similarity between Boolean algebra and the telephone switching circuits. For example, the fundamental unit of information is a yes-no situation. Either something is or is not. This can be easily expressed in Boolean two-value binary algebra by 1 and 0, so that 1 means "on" when the switch is closed and the power is on, and 0 means "off" when the switch is open and power is off.

Under these circumstances, 1 and 0 are binary digits, a phrase that can be shortened to "bits." Thus the unit of information is the bit. A more complicated information can be viewed as built up out of combinations of bits.

By 1948, He turned his efforts toward a fundamental understanding of the problem and had evolved a method of expressing information in quantitative form.