Page 1 1 7.6 MLE for Transformed Parameters Given PDF p(x;? ) but want an estimate of a = g (? ) What is the MLE for a ?? ? g(? ) Two cases: 1. a = g(? ) is a one-to-one function )) ( ; ( maximizes ˆ 1 a a - g p ML x 2. a = g(? ) is not a one-to-one function ? g(? ) Need to define modified likelihood function: {} ! ! !" ! ! !# $ ) ; ( max ) ; ( ) ( : ? a ? a ? x x p p g T = = • For each a, find all ?’s that map to it • Extract largest value of p(x; ? ) over this set of ?’s ) ; ( maximizes ˆ a a x T ML p Page 2 1 7.6 MLE for Transformed Parameters Given PDF p(x;? ) but want an estimate of a = g (? ) What is the MLE for a ?? ? g(? ) Two cases: 1. a = g(? ) is a one-to-one function )) ( ; ( maximizes ˆ 1 a a - g p ML x 2. a = g(? ) is not a one-to-one function ? g(? ) Need to define modified likelihood function: {} ! ! !" ! ! !# $ ) ; ( max ) ; ( ) ( : ? a ? a ? x x p p g T = = • For each a, find all ?’s that map to it • Extract largest value of p(x; ? ) over this set of ?’s ) ; ( maximizes ˆ a a x T ML p 2 Invariance Property of MLE Another Big Advantage of MLE! Theorem 7.2: Invariance Property of MLE If parameter ? is mapped according to a = g(? ) then the MLE of a is given by where is the MLE for ? found by maximizing p(x;? ) ) ˆ ( ˆ ? a g = ? ˆ Note: when g(? ) is not one-to-one the MLE for a maximizes the modified likelihood function “Proof”: Easy to see when g(? ) is one-to-one Otherwise… can “argue” that maximization over ? inside definition for modified LF ensures the result. Page 3 1 7.6 MLE for Transformed Parameters Given PDF p(x;? ) but want an estimate of a = g (? ) What is the MLE for a ?? ? g(? ) Two cases: 1. a = g(? ) is a one-to-one function )) ( ; ( maximizes ˆ 1 a a - g p ML x 2. a = g(? ) is not a one-to-one function ? g(? ) Need to define modified likelihood function: {} ! ! !" ! ! !# $ ) ; ( max ) ; ( ) ( : ? a ? a ? x x p p g T = = • For each a, find all ?’s that map to it • Extract largest value of p(x; ? ) over this set of ?’s ) ; ( maximizes ˆ a a x T ML p 2 Invariance Property of MLE Another Big Advantage of MLE! Theorem 7.2: Invariance Property of MLE If parameter ? is mapped according to a = g(? ) then the MLE of a is given by where is the MLE for ? found by maximizing p(x;? ) ) ˆ ( ˆ ? a g = ? ˆ Note: when g(? ) is not one-to-one the MLE for a maximizes the modified likelihood function “Proof”: Easy to see when g(? ) is one-to-one Otherwise… can “argue” that maximization over ? inside definition for modified LF ensures the result. 3 Ex. 7.9: Estimate Power of DC Level in AWGN x[n] = A + w[n] noise is N(0,s 2 ) & White Want to Est. Power: a = A 2 ? A a = A 2 ? For each a value there are 2 PDF’s to consider ? ? ? ? ? ? ? ? + - = ? ? ? ? ? ? ? ? - - = ? ? n N T n N T n x p n x p 2 2 2 / 2 2 2 2 / 2 ) ] [ ( 2 1 exp ) 2 ( 1 ) ; ( ) ] [ ( 2 1 exp ) 2 ( 1 ) ; ( 2 1 a s ps a a s ps a x x { } [] 2 2 2 0 ˆ ) ; x ( max arg ) ; ( ), ; x ( max arg ˆ ML A ML A A p x p p = ? ? ? ? ? ? = ? ? ? ? ? ? - = 8 < < 8 - = a a a a Then: Demonstration that Invariance Result Holds for this Example Page 4 1 7.6 MLE for Transformed Parameters Given PDF p(x;? ) but want an estimate of a = g (? ) What is the MLE for a ?? ? g(? ) Two cases: 1. a = g(? ) is a one-to-one function )) ( ; ( maximizes ˆ 1 a a - g p ML x 2. a = g(? ) is not a one-to-one function ? g(? ) Need to define modified likelihood function: {} ! ! !" ! ! !# $ ) ; ( max ) ; ( ) ( : ? a ? a ? x x p p g T = = • For each a, find all ?’s that map to it • Extract largest value of p(x; ? ) over this set of ?’s ) ; ( maximizes ˆ a a x T ML p 2 Invariance Property of MLE Another Big Advantage of MLE! Theorem 7.2: Invariance Property of MLE If parameter ? is mapped according to a = g(? ) then the MLE of a is given by where is the MLE for ? found by maximizing p(x;? ) ) ˆ ( ˆ ? a g = ? ˆ Note: when g(? ) is not one-to-one the MLE for a maximizes the modified likelihood function “Proof”: Easy to see when g(? ) is one-to-one Otherwise… can “argue” that maximization over ? inside definition for modified LF ensures the result. 3 Ex. 7.9: Estimate Power of DC Level in AWGN x[n] = A + w[n] noise is N(0,s 2 ) & White Want to Est. Power: a = A 2 ? A a = A 2 ? For each a value there are 2 PDF’s to consider ? ? ? ? ? ? ? ? + - = ? ? ? ? ? ? ? ? - - = ? ? n N T n N T n x p n x p 2 2 2 / 2 2 2 2 / 2 ) ] [ ( 2 1 exp ) 2 ( 1 ) ; ( ) ] [ ( 2 1 exp ) 2 ( 1 ) ; ( 2 1 a s ps a a s ps a x x { } [] 2 2 2 0 ˆ ) ; x ( max arg ) ; ( ), ; x ( max arg ˆ ML A ML A A p x p p = ? ? ? ? ? ? = ? ? ? ? ? ? - = 8 < < 8 - = a a a a Then: Demonstration that Invariance Result Holds for this Example 4 Ex. 7.10: Estimate Power of WGN in dB x[n] = w[n] WGN w/ var = s 2 unknown Recall: P noise = s 2 ? - = = 1 0 2 ] [ 1 ˆ N n noise n x N P Can show that the MLE for variance is: To get the dB version of the power estimate: Note: You may recall a result for estimating variance that divides by N–1 rather than by N … that estimator is unbiased, this estimate is biased (but asymptotically unbiased) ! ! !" ! ! !# $ ? ? ? ? ? ? = ? - = 1 0 2 10 ] [ 1 log 10 ˆ N n dB n x N P Using Invariance Property ! Page 5 1 7.6 MLE for Transformed Parameters Given PDF p(x;? ) but want an estimate of a = g (? ) What is the MLE for a ?? ? g(? ) Two cases: 1. a = g(? ) is a one-to-one function )) ( ; ( maximizes ˆ 1 a a - g p ML x 2. a = g(? ) is not a one-to-one function ? g(? ) Need to define modified likelihood function: {} ! ! !" ! ! !# $ ) ; ( max ) ; ( ) ( : ? a ? a ? x x p p g T = = • For each a, find all ?’s that map to it • Extract largest value of p(x; ? ) over this set of ?’s ) ; ( maximizes ˆ a a x T ML p 2 Invariance Property of MLE Another Big Advantage of MLE! Theorem 7.2: Invariance Property of MLE If parameter ? is mapped according to a = g(? ) then the MLE of a is given by where is the MLE for ? found by maximizing p(x;? ) ) ˆ ( ˆ ? a g = ? ˆ Note: when g(? ) is not one-to-one the MLE for a maximizes the modified likelihood function “Proof”: Easy to see when g(? ) is one-to-one Otherwise… can “argue” that maximization over ? inside definition for modified LF ensures the result. 3 Ex. 7.9: Estimate Power of DC Level in AWGN x[n] = A + w[n] noise is N(0,s 2 ) & White Want to Est. Power: a = A 2 ? A a = A 2 ? For each a value there are 2 PDF’s to consider ? ? ? ? ? ? ? ? + - = ? ? ? ? ? ? ? ? - - = ? ? n N T n N T n x p n x p 2 2 2 / 2 2 2 2 / 2 ) ] [ ( 2 1 exp ) 2 ( 1 ) ; ( ) ] [ ( 2 1 exp ) 2 ( 1 ) ; ( 2 1 a s ps a a s ps a x x { } [] 2 2 2 0 ˆ ) ; x ( max arg ) ; ( ), ; x ( max arg ˆ ML A ML A A p x p p = ? ? ? ? ? ? = ? ? ? ? ? ? - = 8 < < 8 - = a a a a Then: Demonstration that Invariance Result Holds for this Example 4 Ex. 7.10: Estimate Power of WGN in dB x[n] = w[n] WGN w/ var = s 2 unknown Recall: P noise = s 2 ? - = = 1 0 2 ] [ 1 ˆ N n noise n x N P Can show that the MLE for variance is: To get the dB version of the power estimate: Note: You may recall a result for estimating variance that divides by N–1 rather than by N … that estimator is unbiased, this estimate is biased (but asymptotically unbiased) ! ! !" ! ! !# $ ? ? ? ? ? ? = ? - = 1 0 2 10 ] [ 1 log 10 ˆ N n dB n x N P Using Invariance Property ! 5 7.7: Numerical Determination of MLE Note: In all previous examples we ended up with a closed-form expression for the MLE: ) ( ˆ x f ML = ? Ex. 7.11: x[n] = r n + w[n] noise is N(0,s 2 ) & white Estimate r If –1 < r < 0 then this signal is a decaying oscillation that might be used to model: • A Ship’s “Hull Ping” • A Vibrating String, Etc. ? - = - = - ? = ? ? 1 0 1 0 ) ] [ ( 0 ) ; x ( ln N n n n nr r n x p ? ? To find MLE: No closed-form solution for the MLERead More

Offer running on EduRev: __Apply code STAYHOME200__ to get INR 200 off on our premium plan EduRev Infinity!