Chapter Notes - MLE for Transformed Parameters Notes | EduRev

: Chapter Notes - MLE for Transformed Parameters Notes | EduRev

 Page 1


1
7.6 MLE for Transformed Parameters
Given PDF p(x;? ) but want an estimate of a = g (? )
What is the MLE for a ??
?
g(? )
Two cases:
1. a = g(? ) is a one-to-one function
)) ( ; (  maximizes  
ˆ
1
a a
-
g p
ML
x
2.   a = g(? ) is not a one-to-one function
?
g(? )
Need to define modified likelihood function:
{}
! ! !" ! ! !# $
) ; ( max ) ; (
) ( :
? a
? a ?
x x p p
g
T
=
=
• For each a, find all ?’s that map to it
• Extract largest value of p(x; ? ) over  
this set of ?’s
) ; (  maximizes  
ˆ
a a x
T ML
p
Page 2


1
7.6 MLE for Transformed Parameters
Given PDF p(x;? ) but want an estimate of a = g (? )
What is the MLE for a ??
?
g(? )
Two cases:
1. a = g(? ) is a one-to-one function
)) ( ; (  maximizes  
ˆ
1
a a
-
g p
ML
x
2.   a = g(? ) is not a one-to-one function
?
g(? )
Need to define modified likelihood function:
{}
! ! !" ! ! !# $
) ; ( max ) ; (
) ( :
? a
? a ?
x x p p
g
T
=
=
• For each a, find all ?’s that map to it
• Extract largest value of p(x; ? ) over  
this set of ?’s
) ; (  maximizes  
ˆ
a a x
T ML
p
2
Invariance Property of MLE Another Big 
Advantage of MLE!
Theorem 7.2: Invariance Property of MLE
If parameter ? is mapped according to a = g(? ) then the 
MLE of a is given by
where     is the MLE for ? found by maximizing p(x;? )
)
ˆ
(
ˆ
? a g =
?
ˆ
Note: when g(? ) is not one-to-one the MLE for a maximizes 
the modified likelihood function 
“Proof”:  
Easy to see when g(? ) is one-to-one 
Otherwise… can “argue” that maximization over ? inside 
definition for modified LF ensures the result.
Page 3


1
7.6 MLE for Transformed Parameters
Given PDF p(x;? ) but want an estimate of a = g (? )
What is the MLE for a ??
?
g(? )
Two cases:
1. a = g(? ) is a one-to-one function
)) ( ; (  maximizes  
ˆ
1
a a
-
g p
ML
x
2.   a = g(? ) is not a one-to-one function
?
g(? )
Need to define modified likelihood function:
{}
! ! !" ! ! !# $
) ; ( max ) ; (
) ( :
? a
? a ?
x x p p
g
T
=
=
• For each a, find all ?’s that map to it
• Extract largest value of p(x; ? ) over  
this set of ?’s
) ; (  maximizes  
ˆ
a a x
T ML
p
2
Invariance Property of MLE Another Big 
Advantage of MLE!
Theorem 7.2: Invariance Property of MLE
If parameter ? is mapped according to a = g(? ) then the 
MLE of a is given by
where     is the MLE for ? found by maximizing p(x;? )
)
ˆ
(
ˆ
? a g =
?
ˆ
Note: when g(? ) is not one-to-one the MLE for a maximizes 
the modified likelihood function 
“Proof”:  
Easy to see when g(? ) is one-to-one 
Otherwise… can “argue” that maximization over ? inside 
definition for modified LF ensures the result.
3
Ex. 7.9: Estimate Power of DC Level in AWGN
x[n] = A + w[n] noise is N(0,s
2
) & White
Want to Est. Power: a = A
2
?
A
a = A
2
? For each a value there are 2 PDF’s to consider
?
?
?
?
?
?
?
?
+ - =
?
?
?
?
?
?
?
?
- - =
?
?
n
N
T
n
N
T
n x p
n x p
2
2 2 / 2
2
2 2 / 2
) ] [ (
2
1
exp
) 2 (
1
) ; (
) ] [ (
2
1
exp
) 2 (
1
) ; (
2
1
a
s ps
a
a
s ps
a
x
x
{ }
[]
2
2
2
0
ˆ
) ; x ( max arg
) ; ( ), ; x ( max arg
ˆ
ML
A
ML
A
A p
x p p
=
?
?
?
?
?
?
=
?
?
?
?
?
?
- =
8 < < 8 -
=
a a a
a
Then:
Demonstration that 
Invariance Result 
Holds for this 
Example
Page 4


1
7.6 MLE for Transformed Parameters
Given PDF p(x;? ) but want an estimate of a = g (? )
What is the MLE for a ??
?
g(? )
Two cases:
1. a = g(? ) is a one-to-one function
)) ( ; (  maximizes  
ˆ
1
a a
-
g p
ML
x
2.   a = g(? ) is not a one-to-one function
?
g(? )
Need to define modified likelihood function:
{}
! ! !" ! ! !# $
) ; ( max ) ; (
) ( :
? a
? a ?
x x p p
g
T
=
=
• For each a, find all ?’s that map to it
• Extract largest value of p(x; ? ) over  
this set of ?’s
) ; (  maximizes  
ˆ
a a x
T ML
p
2
Invariance Property of MLE Another Big 
Advantage of MLE!
Theorem 7.2: Invariance Property of MLE
If parameter ? is mapped according to a = g(? ) then the 
MLE of a is given by
where     is the MLE for ? found by maximizing p(x;? )
)
ˆ
(
ˆ
? a g =
?
ˆ
Note: when g(? ) is not one-to-one the MLE for a maximizes 
the modified likelihood function 
“Proof”:  
Easy to see when g(? ) is one-to-one 
Otherwise… can “argue” that maximization over ? inside 
definition for modified LF ensures the result.
3
Ex. 7.9: Estimate Power of DC Level in AWGN
x[n] = A + w[n] noise is N(0,s
2
) & White
Want to Est. Power: a = A
2
?
A
a = A
2
? For each a value there are 2 PDF’s to consider
?
?
?
?
?
?
?
?
+ - =
?
?
?
?
?
?
?
?
- - =
?
?
n
N
T
n
N
T
n x p
n x p
2
2 2 / 2
2
2 2 / 2
) ] [ (
2
1
exp
) 2 (
1
) ; (
) ] [ (
2
1
exp
) 2 (
1
) ; (
2
1
a
s ps
a
a
s ps
a
x
x
{ }
[]
2
2
2
0
ˆ
) ; x ( max arg
) ; ( ), ; x ( max arg
ˆ
ML
A
ML
A
A p
x p p
=
?
?
?
?
?
?
=
?
?
?
?
?
?
- =
8 < < 8 -
=
a a a
a
Then:
Demonstration that 
Invariance Result 
Holds for this 
Example
4
Ex. 7.10: Estimate Power of WGN in dB 
x[n] = w[n] WGN   w/ var = s
2
unknown
Recall: P
noise
= s
2
?
-
=
=
1
0
2
] [
1
ˆ
N
n
noise
n x
N
P
Can show that the MLE for variance is:
To get the dB version of 
the power estimate:
Note: You may recall a 
result for estimating 
variance that divides by N–1 
rather than by N … that 
estimator is unbiased, this 
estimate is biased (but 
asymptotically unbiased)
! ! !" ! ! !# $
?
?
?
?
?
?
=
?
-
=
1
0
2
10
] [
1
log 10
ˆ
N
n
dB
n x
N
P
Using 
Invariance Property !
Page 5


1
7.6 MLE for Transformed Parameters
Given PDF p(x;? ) but want an estimate of a = g (? )
What is the MLE for a ??
?
g(? )
Two cases:
1. a = g(? ) is a one-to-one function
)) ( ; (  maximizes  
ˆ
1
a a
-
g p
ML
x
2.   a = g(? ) is not a one-to-one function
?
g(? )
Need to define modified likelihood function:
{}
! ! !" ! ! !# $
) ; ( max ) ; (
) ( :
? a
? a ?
x x p p
g
T
=
=
• For each a, find all ?’s that map to it
• Extract largest value of p(x; ? ) over  
this set of ?’s
) ; (  maximizes  
ˆ
a a x
T ML
p
2
Invariance Property of MLE Another Big 
Advantage of MLE!
Theorem 7.2: Invariance Property of MLE
If parameter ? is mapped according to a = g(? ) then the 
MLE of a is given by
where     is the MLE for ? found by maximizing p(x;? )
)
ˆ
(
ˆ
? a g =
?
ˆ
Note: when g(? ) is not one-to-one the MLE for a maximizes 
the modified likelihood function 
“Proof”:  
Easy to see when g(? ) is one-to-one 
Otherwise… can “argue” that maximization over ? inside 
definition for modified LF ensures the result.
3
Ex. 7.9: Estimate Power of DC Level in AWGN
x[n] = A + w[n] noise is N(0,s
2
) & White
Want to Est. Power: a = A
2
?
A
a = A
2
? For each a value there are 2 PDF’s to consider
?
?
?
?
?
?
?
?
+ - =
?
?
?
?
?
?
?
?
- - =
?
?
n
N
T
n
N
T
n x p
n x p
2
2 2 / 2
2
2 2 / 2
) ] [ (
2
1
exp
) 2 (
1
) ; (
) ] [ (
2
1
exp
) 2 (
1
) ; (
2
1
a
s ps
a
a
s ps
a
x
x
{ }
[]
2
2
2
0
ˆ
) ; x ( max arg
) ; ( ), ; x ( max arg
ˆ
ML
A
ML
A
A p
x p p
=
?
?
?
?
?
?
=
?
?
?
?
?
?
- =
8 < < 8 -
=
a a a
a
Then:
Demonstration that 
Invariance Result 
Holds for this 
Example
4
Ex. 7.10: Estimate Power of WGN in dB 
x[n] = w[n] WGN   w/ var = s
2
unknown
Recall: P
noise
= s
2
?
-
=
=
1
0
2
] [
1
ˆ
N
n
noise
n x
N
P
Can show that the MLE for variance is:
To get the dB version of 
the power estimate:
Note: You may recall a 
result for estimating 
variance that divides by N–1 
rather than by N … that 
estimator is unbiased, this 
estimate is biased (but 
asymptotically unbiased)
! ! !" ! ! !# $
?
?
?
?
?
?
=
?
-
=
1
0
2
10
] [
1
log 10
ˆ
N
n
dB
n x
N
P
Using 
Invariance Property !
5
7.7: Numerical Determination of MLE
Note: In all previous examples we ended up with a closed-form
expression for the MLE:
) (
ˆ
x f
ML
= ?
Ex. 7.11: x[n] = r
n
+ w[n] noise is  N(0,s
2
) & white
Estimate r  
If –1 < r < 0 then this signal 
is a decaying oscillation that 
might be used to model:
• A Ship’s “Hull Ping”
• A Vibrating String, Etc. 
?
-
=
-
= - ?
=
?
?
1
0
1
0 ) ] [ (
0
) ; x ( ln
N
n
n n
nr r n x
p
?
?
To find MLE:
No closed-form 
solution for the MLE
Read More
Offer running on EduRev: Apply code STAYHOME200 to get INR 200 off on our premium plan EduRev Infinity!