Page 1 1 Maximum Likelihood Estimation Econ 424/Amath 540 Eric Zivot © Eric Zivot 2006 Eric Zivot Summer 2012 Updated: July 26, 2012 Bernoulli Likelihood likelihood.Bernoulli = function(theta, x) { # theta success probability parameter # x vector of data n = length(x) ans = theta^sum(x) * (1-theta)^(n-sum(x)) return(ans) } # plot Bernoulli likelihood > x = rep(0,5) > theta.vals = seq(0,1, length.out=10) > like.vals = likelihood.Bernoulli(theta.vals, x) > plot(theta.vals, like.vals, type="b", col="blue", lwd=2, i "B lliLiklih df (00000)") © Eric Zivot 2006 + main="Bernoulli Likelihood for x=(0,0,0,0,0)") Page 2 1 Maximum Likelihood Estimation Econ 424/Amath 540 Eric Zivot © Eric Zivot 2006 Eric Zivot Summer 2012 Updated: July 26, 2012 Bernoulli Likelihood likelihood.Bernoulli = function(theta, x) { # theta success probability parameter # x vector of data n = length(x) ans = theta^sum(x) * (1-theta)^(n-sum(x)) return(ans) } # plot Bernoulli likelihood > x = rep(0,5) > theta.vals = seq(0,1, length.out=10) > like.vals = likelihood.Bernoulli(theta.vals, x) > plot(theta.vals, like.vals, type="b", col="blue", lwd=2, i "B lliLiklih df (00000)") © Eric Zivot 2006 + main="Bernoulli Likelihood for x=(0,0,0,0,0)") 2 Bernoulli Likelihood Maximum of the likelihood function occurs at ? = 0 Note: for x=(0,0,0,0,0) the sample mean is 0. © Eric Zivot 2006 Bernoulli Likelihood Maximum of the likelihood function occurs at ? = 1 Note: for x=(1,1,1,1,1) the sample mean is 1. © Eric Zivot 2006 Page 3 1 Maximum Likelihood Estimation Econ 424/Amath 540 Eric Zivot © Eric Zivot 2006 Eric Zivot Summer 2012 Updated: July 26, 2012 Bernoulli Likelihood likelihood.Bernoulli = function(theta, x) { # theta success probability parameter # x vector of data n = length(x) ans = theta^sum(x) * (1-theta)^(n-sum(x)) return(ans) } # plot Bernoulli likelihood > x = rep(0,5) > theta.vals = seq(0,1, length.out=10) > like.vals = likelihood.Bernoulli(theta.vals, x) > plot(theta.vals, like.vals, type="b", col="blue", lwd=2, i "B lliLiklih df (00000)") © Eric Zivot 2006 + main="Bernoulli Likelihood for x=(0,0,0,0,0)") 2 Bernoulli Likelihood Maximum of the likelihood function occurs at ? = 0 Note: for x=(0,0,0,0,0) the sample mean is 0. © Eric Zivot 2006 Bernoulli Likelihood Maximum of the likelihood function occurs at ? = 1 Note: for x=(1,1,1,1,1) the sample mean is 1. © Eric Zivot 2006 3 Normal Likelihood likelihood.normal.mu = function(mu, sig2=1, x) { # mu mean of normal distribution for given sig2 # x vector of data n = length(x) a1 = (2*pi*sig2)^-(n/2) a2 = -1/(2*sig2) y = (x-mu)^2 y (x mu) 2 ans = a1*exp(a2*sum(y)) return(ans) } # generate N(0,1) data > n = 50 > x = rnorm(n, mean=0, sd=1) # compute normal likelihood as function of mu © Eric Zivot 2006 > mu.vals = seq(-1,1, length.out=100) > like.vals = rep(0,length(mu.vals)) > for (i in 1:length(like.vals)) { + like.vals[i] = likelihood.normal.mu(mu.vals[i], sig2=1, x=x) + } Normal Likelihood for µ Maximum of the likelihood does not likelihood does not occur at µ = 0. The maximum is at 1 1 ˆ 0.0344 n i i x n ? ? ?? ? © Eric Zivot 2006 Page 4 1 Maximum Likelihood Estimation Econ 424/Amath 540 Eric Zivot © Eric Zivot 2006 Eric Zivot Summer 2012 Updated: July 26, 2012 Bernoulli Likelihood likelihood.Bernoulli = function(theta, x) { # theta success probability parameter # x vector of data n = length(x) ans = theta^sum(x) * (1-theta)^(n-sum(x)) return(ans) } # plot Bernoulli likelihood > x = rep(0,5) > theta.vals = seq(0,1, length.out=10) > like.vals = likelihood.Bernoulli(theta.vals, x) > plot(theta.vals, like.vals, type="b", col="blue", lwd=2, i "B lliLiklih df (00000)") © Eric Zivot 2006 + main="Bernoulli Likelihood for x=(0,0,0,0,0)") 2 Bernoulli Likelihood Maximum of the likelihood function occurs at ? = 0 Note: for x=(0,0,0,0,0) the sample mean is 0. © Eric Zivot 2006 Bernoulli Likelihood Maximum of the likelihood function occurs at ? = 1 Note: for x=(1,1,1,1,1) the sample mean is 1. © Eric Zivot 2006 3 Normal Likelihood likelihood.normal.mu = function(mu, sig2=1, x) { # mu mean of normal distribution for given sig2 # x vector of data n = length(x) a1 = (2*pi*sig2)^-(n/2) a2 = -1/(2*sig2) y = (x-mu)^2 y (x mu) 2 ans = a1*exp(a2*sum(y)) return(ans) } # generate N(0,1) data > n = 50 > x = rnorm(n, mean=0, sd=1) # compute normal likelihood as function of mu © Eric Zivot 2006 > mu.vals = seq(-1,1, length.out=100) > like.vals = rep(0,length(mu.vals)) > for (i in 1:length(like.vals)) { + like.vals[i] = likelihood.normal.mu(mu.vals[i], sig2=1, x=x) + } Normal Likelihood for µ Maximum of the likelihood does not likelihood does not occur at µ = 0. The maximum is at 1 1 ˆ 0.0344 n i i x n ? ? ?? ? © Eric Zivot 2006 4 Bernoulli Log-Likelihood © Eric Zivot 2006 Normal Log-Likelihood Maximum of the log- likelihood does not occur at µ = 0. The maximum is at 1 1 ˆ 0.0344 n i i x n ? ? ?? ? © Eric Zivot 2006 Page 5 1 Maximum Likelihood Estimation Econ 424/Amath 540 Eric Zivot © Eric Zivot 2006 Eric Zivot Summer 2012 Updated: July 26, 2012 Bernoulli Likelihood likelihood.Bernoulli = function(theta, x) { # theta success probability parameter # x vector of data n = length(x) ans = theta^sum(x) * (1-theta)^(n-sum(x)) return(ans) } # plot Bernoulli likelihood > x = rep(0,5) > theta.vals = seq(0,1, length.out=10) > like.vals = likelihood.Bernoulli(theta.vals, x) > plot(theta.vals, like.vals, type="b", col="blue", lwd=2, i "B lliLiklih df (00000)") © Eric Zivot 2006 + main="Bernoulli Likelihood for x=(0,0,0,0,0)") 2 Bernoulli Likelihood Maximum of the likelihood function occurs at ? = 0 Note: for x=(0,0,0,0,0) the sample mean is 0. © Eric Zivot 2006 Bernoulli Likelihood Maximum of the likelihood function occurs at ? = 1 Note: for x=(1,1,1,1,1) the sample mean is 1. © Eric Zivot 2006 3 Normal Likelihood likelihood.normal.mu = function(mu, sig2=1, x) { # mu mean of normal distribution for given sig2 # x vector of data n = length(x) a1 = (2*pi*sig2)^-(n/2) a2 = -1/(2*sig2) y = (x-mu)^2 y (x mu) 2 ans = a1*exp(a2*sum(y)) return(ans) } # generate N(0,1) data > n = 50 > x = rnorm(n, mean=0, sd=1) # compute normal likelihood as function of mu © Eric Zivot 2006 > mu.vals = seq(-1,1, length.out=100) > like.vals = rep(0,length(mu.vals)) > for (i in 1:length(like.vals)) { + like.vals[i] = likelihood.normal.mu(mu.vals[i], sig2=1, x=x) + } Normal Likelihood for µ Maximum of the likelihood does not likelihood does not occur at µ = 0. The maximum is at 1 1 ˆ 0.0344 n i i x n ? ? ?? ? © Eric Zivot 2006 4 Bernoulli Log-Likelihood © Eric Zivot 2006 Normal Log-Likelihood Maximum of the log- likelihood does not occur at µ = 0. The maximum is at 1 1 ˆ 0.0344 n i i x n ? ? ?? ? © Eric Zivot 2006 5 R optimize() function # use optimize() to maximize or minimize function of one variable test.fun = function(x) { return(x^2) } > ans = optimize(test.fun, lower=-1, upper=1, maximum=FALSE) > class(ans) [1] "list" > names(ans) [1] "minimum" "objective" > .6 0.8 1.0 f(x)=x^2 © Eric Zivot 2006 > ans $minimum [1] -2.776e-17 $objective [1] 7.704e-34 -1.0 -0.5 0.0 0.5 1.0 0.0 0.2 0.4 0 x f(x) R optim() function # use optim() to minimize functions of multiple variables test.fun = function(theta) { ans = theta[1]^2 + theta[2]^2 return(ans) } } # set starting values for optimizer > theta.start = c(1,1) # optimize function > ans = optim(par=theta.start, fn=test.fun, + method="BFGS") > class(ans) [1] "list" © Eric Zivot 2006 [] > names(ans) [1] "par" "value" "counts" "convergence" [5] "message"Read More

Offer running on EduRev: __Apply code STAYHOME200__ to get INR 200 off on our premium plan EduRev Infinity!