Courses

# Chapter Notes - MLE for Transformed Parameters Notes | EduRev

## : Chapter Notes - MLE for Transformed Parameters Notes | EduRev

``` Page 1

1
7.6 MLE for Transformed Parameters
Given PDF p(x;? ) but want an estimate of a = g (? )
What is the MLE for a ??
?
g(? )
Two cases:
1. a = g(? ) is a one-to-one function
)) ( ; (  maximizes
ˆ
1
a a
-
g p
ML
x
2.   a = g(? ) is not a one-to-one function
?
g(? )
Need to define modified likelihood function:
{}
! ! !" ! ! !# \$
) ; ( max ) ; (
) ( :
? a
? a ?
x x p p
g
T
=
=
• For each a, find all ?’s that map to it
• Extract largest value of p(x; ? ) over
this set of ?’s
) ; (  maximizes
ˆ
a a x
T ML
p
Page 2

1
7.6 MLE for Transformed Parameters
Given PDF p(x;? ) but want an estimate of a = g (? )
What is the MLE for a ??
?
g(? )
Two cases:
1. a = g(? ) is a one-to-one function
)) ( ; (  maximizes
ˆ
1
a a
-
g p
ML
x
2.   a = g(? ) is not a one-to-one function
?
g(? )
Need to define modified likelihood function:
{}
! ! !" ! ! !# \$
) ; ( max ) ; (
) ( :
? a
? a ?
x x p p
g
T
=
=
• For each a, find all ?’s that map to it
• Extract largest value of p(x; ? ) over
this set of ?’s
) ; (  maximizes
ˆ
a a x
T ML
p
2
Invariance Property of MLE Another Big
Theorem 7.2: Invariance Property of MLE
If parameter ? is mapped according to a = g(? ) then the
MLE of a is given by
where     is the MLE for ? found by maximizing p(x;? )
)
ˆ
(
ˆ
? a g =
?
ˆ
Note: when g(? ) is not one-to-one the MLE for a maximizes
the modified likelihood function
“Proof”:
Easy to see when g(? ) is one-to-one
Otherwise… can “argue” that maximization over ? inside
definition for modified LF ensures the result.
Page 3

1
7.6 MLE for Transformed Parameters
Given PDF p(x;? ) but want an estimate of a = g (? )
What is the MLE for a ??
?
g(? )
Two cases:
1. a = g(? ) is a one-to-one function
)) ( ; (  maximizes
ˆ
1
a a
-
g p
ML
x
2.   a = g(? ) is not a one-to-one function
?
g(? )
Need to define modified likelihood function:
{}
! ! !" ! ! !# \$
) ; ( max ) ; (
) ( :
? a
? a ?
x x p p
g
T
=
=
• For each a, find all ?’s that map to it
• Extract largest value of p(x; ? ) over
this set of ?’s
) ; (  maximizes
ˆ
a a x
T ML
p
2
Invariance Property of MLE Another Big
Theorem 7.2: Invariance Property of MLE
If parameter ? is mapped according to a = g(? ) then the
MLE of a is given by
where     is the MLE for ? found by maximizing p(x;? )
)
ˆ
(
ˆ
? a g =
?
ˆ
Note: when g(? ) is not one-to-one the MLE for a maximizes
the modified likelihood function
“Proof”:
Easy to see when g(? ) is one-to-one
Otherwise… can “argue” that maximization over ? inside
definition for modified LF ensures the result.
3
Ex. 7.9: Estimate Power of DC Level in AWGN
x[n] = A + w[n] noise is N(0,s
2
) & White
Want to Est. Power: a = A
2
?
A
a = A
2
? For each a value there are 2 PDF’s to consider
?
?
?
?
?
?
?
?
+ - =
?
?
?
?
?
?
?
?
- - =
?
?
n
N
T
n
N
T
n x p
n x p
2
2 2 / 2
2
2 2 / 2
) ] [ (
2
1
exp
) 2 (
1
) ; (
) ] [ (
2
1
exp
) 2 (
1
) ; (
2
1
a
s ps
a
a
s ps
a
x
x
{ }
[]
2
2
2
0
ˆ
) ; x ( max arg
) ; ( ), ; x ( max arg
ˆ
ML
A
ML
A
A p
x p p
=
?
?
?
?
?
?
=
?
?
?
?
?
?
- =
8 < < 8 -
=
a a a
a
Then:
Demonstration that
Invariance Result
Holds for this
Example
Page 4

1
7.6 MLE for Transformed Parameters
Given PDF p(x;? ) but want an estimate of a = g (? )
What is the MLE for a ??
?
g(? )
Two cases:
1. a = g(? ) is a one-to-one function
)) ( ; (  maximizes
ˆ
1
a a
-
g p
ML
x
2.   a = g(? ) is not a one-to-one function
?
g(? )
Need to define modified likelihood function:
{}
! ! !" ! ! !# \$
) ; ( max ) ; (
) ( :
? a
? a ?
x x p p
g
T
=
=
• For each a, find all ?’s that map to it
• Extract largest value of p(x; ? ) over
this set of ?’s
) ; (  maximizes
ˆ
a a x
T ML
p
2
Invariance Property of MLE Another Big
Theorem 7.2: Invariance Property of MLE
If parameter ? is mapped according to a = g(? ) then the
MLE of a is given by
where     is the MLE for ? found by maximizing p(x;? )
)
ˆ
(
ˆ
? a g =
?
ˆ
Note: when g(? ) is not one-to-one the MLE for a maximizes
the modified likelihood function
“Proof”:
Easy to see when g(? ) is one-to-one
Otherwise… can “argue” that maximization over ? inside
definition for modified LF ensures the result.
3
Ex. 7.9: Estimate Power of DC Level in AWGN
x[n] = A + w[n] noise is N(0,s
2
) & White
Want to Est. Power: a = A
2
?
A
a = A
2
? For each a value there are 2 PDF’s to consider
?
?
?
?
?
?
?
?
+ - =
?
?
?
?
?
?
?
?
- - =
?
?
n
N
T
n
N
T
n x p
n x p
2
2 2 / 2
2
2 2 / 2
) ] [ (
2
1
exp
) 2 (
1
) ; (
) ] [ (
2
1
exp
) 2 (
1
) ; (
2
1
a
s ps
a
a
s ps
a
x
x
{ }
[]
2
2
2
0
ˆ
) ; x ( max arg
) ; ( ), ; x ( max arg
ˆ
ML
A
ML
A
A p
x p p
=
?
?
?
?
?
?
=
?
?
?
?
?
?
- =
8 < < 8 -
=
a a a
a
Then:
Demonstration that
Invariance Result
Holds for this
Example
4
Ex. 7.10: Estimate Power of WGN in dB
x[n] = w[n] WGN   w/ var = s
2
unknown
Recall: P
noise
= s
2
?
-
=
=
1
0
2
] [
1
ˆ
N
n
noise
n x
N
P
Can show that the MLE for variance is:
To get the dB version of
the power estimate:
Note: You may recall a
result for estimating
variance that divides by N–1
rather than by N … that
estimator is unbiased, this
estimate is biased (but
asymptotically unbiased)
! ! !" ! ! !# \$
?
?
?
?
?
?
=
?
-
=
1
0
2
10
] [
1
log 10
ˆ
N
n
dB
n x
N
P
Using
Invariance Property !
Page 5

1
7.6 MLE for Transformed Parameters
Given PDF p(x;? ) but want an estimate of a = g (? )
What is the MLE for a ??
?
g(? )
Two cases:
1. a = g(? ) is a one-to-one function
)) ( ; (  maximizes
ˆ
1
a a
-
g p
ML
x
2.   a = g(? ) is not a one-to-one function
?
g(? )
Need to define modified likelihood function:
{}
! ! !" ! ! !# \$
) ; ( max ) ; (
) ( :
? a
? a ?
x x p p
g
T
=
=
• For each a, find all ?’s that map to it
• Extract largest value of p(x; ? ) over
this set of ?’s
) ; (  maximizes
ˆ
a a x
T ML
p
2
Invariance Property of MLE Another Big
Theorem 7.2: Invariance Property of MLE
If parameter ? is mapped according to a = g(? ) then the
MLE of a is given by
where     is the MLE for ? found by maximizing p(x;? )
)
ˆ
(
ˆ
? a g =
?
ˆ
Note: when g(? ) is not one-to-one the MLE for a maximizes
the modified likelihood function
“Proof”:
Easy to see when g(? ) is one-to-one
Otherwise… can “argue” that maximization over ? inside
definition for modified LF ensures the result.
3
Ex. 7.9: Estimate Power of DC Level in AWGN
x[n] = A + w[n] noise is N(0,s
2
) & White
Want to Est. Power: a = A
2
?
A
a = A
2
? For each a value there are 2 PDF’s to consider
?
?
?
?
?
?
?
?
+ - =
?
?
?
?
?
?
?
?
- - =
?
?
n
N
T
n
N
T
n x p
n x p
2
2 2 / 2
2
2 2 / 2
) ] [ (
2
1
exp
) 2 (
1
) ; (
) ] [ (
2
1
exp
) 2 (
1
) ; (
2
1
a
s ps
a
a
s ps
a
x
x
{ }
[]
2
2
2
0
ˆ
) ; x ( max arg
) ; ( ), ; x ( max arg
ˆ
ML
A
ML
A
A p
x p p
=
?
?
?
?
?
?
=
?
?
?
?
?
?
- =
8 < < 8 -
=
a a a
a
Then:
Demonstration that
Invariance Result
Holds for this
Example
4
Ex. 7.10: Estimate Power of WGN in dB
x[n] = w[n] WGN   w/ var = s
2
unknown
Recall: P
noise
= s
2
?
-
=
=
1
0
2
] [
1
ˆ
N
n
noise
n x
N
P
Can show that the MLE for variance is:
To get the dB version of
the power estimate:
Note: You may recall a
result for estimating
variance that divides by N–1
rather than by N … that
estimator is unbiased, this
estimate is biased (but
asymptotically unbiased)
! ! !" ! ! !# \$
?
?
?
?
?
?
=
?
-
=
1
0
2
10
] [
1
log 10
ˆ
N
n
dB
n x
N
P
Using
Invariance Property !
5
7.7: Numerical Determination of MLE
Note: In all previous examples we ended up with a closed-form
expression for the MLE:
) (
ˆ
x f
ML
= ?
Ex. 7.11: x[n] = r
n
+ w[n] noise is  N(0,s
2
) & white
Estimate r
If –1 < r < 0 then this signal
is a decaying oscillation that
might be used to model:
• A Ship’s “Hull Ping”
• A Vibrating String, Etc.
?
-
=
-
= - ?
=
?
?
1
0
1
0 ) ] [ (
0
) ; x ( ln
N
n
n n
nr r n x
p
?
?
To find MLE:
No closed-form
solution for the MLE
```
Offer running on EduRev: Apply code STAYHOME200 to get INR 200 off on our premium plan EduRev Infinity!