What is the unit of information when the logarithmic base is taken as 2?

Bit

Riddle: I measure the uncertainty of a source, from zero when certain to log2n when symbols are equal. What am I?

Entropy

True or False: If all emitted symbols from a source are equiprobable, then the entropy is H(X) = 0.

False. If all emitted symbols are equiprobable, then H(X) = log2n.

The average information per symbol from a source is known as ______.

Source entropy

Which of the following coding methods is known for its efficiency in generating binary codes? A) Huffman coding B) Shannon-Fano coding C) Both A and B D) Neither A nor B

C) Both A and B

True or False: The channel capacity is defined as the minimum of mutual information.

False. The channel capacity is defined as the maximum of mutual information.

What is the formula for the information rate of a source with entropy H and message generation rate r?

R = rH bits/sec

Fill in the blank: The process of representing data generated by a discrete source is called ______.

Source encoding

In Huffman coding, what is the primary criterion for combining two symbols?

Lowest probability

If a discrete memoryless channel has n inputs and m outputs, the joint entropy is calculated using which of the following?

H(X,Y) = -ΣP(x,y)logP(x,y)