Lecture 7 - Time - Complexity Notes | EduRev

: Lecture 7 - Time - Complexity Notes | EduRev

 Page 1


Time Complexity
CS60001: Foundations of Computing Science
Pallab Dasgupta
Professor, Dept. of Computer Sc. & Engg.,
Indian Institute of Technology Kharagpur
Page 2


Time Complexity
CS60001: Foundations of Computing Science
Pallab Dasgupta
Professor, Dept. of Computer Sc. & Engg.,
Indian Institute of Technology Kharagpur
Indian Institute of Technology Kharagpur
Pallab Dasgupta
Measuring Complexity
? Definition
¦ Let M be a deterministic Turing machine that halts on all inputs. The 
running time or time complexity of M is the function f: N ?N, where f(n) is 
the running time of M, we say that M runs in time f(n) and that M is an f(n)
time Turing machine. Customarily we use n to represent the length of the 
input
? Complexity Analysis
¦ Worst-case Analysis
? Longest running time of all inputs of a particular length
¦ Average-case Analysis
? Average of all the running times of inputs of a particular length
2
Page 3


Time Complexity
CS60001: Foundations of Computing Science
Pallab Dasgupta
Professor, Dept. of Computer Sc. & Engg.,
Indian Institute of Technology Kharagpur
Indian Institute of Technology Kharagpur
Pallab Dasgupta
Measuring Complexity
? Definition
¦ Let M be a deterministic Turing machine that halts on all inputs. The 
running time or time complexity of M is the function f: N ?N, where f(n) is 
the running time of M, we say that M runs in time f(n) and that M is an f(n)
time Turing machine. Customarily we use n to represent the length of the 
input
? Complexity Analysis
¦ Worst-case Analysis
? Longest running time of all inputs of a particular length
¦ Average-case Analysis
? Average of all the running times of inputs of a particular length
2
Indian Institute of Technology Kharagpur
Pallab Dasgupta
Big-O and Small-o Notations
? Asymptotic Upper Bound (O)
¦ Let f and g be functions f, g: N ?R
+
. Say that f(n) = O(g(n)) if positive 
integers c and n
0
exist such that for every integer n = n
0
f(n) = c.g(n)
¦ When f(n) = O(g(n)) we say that g(n) is an upper bound for f(n), or more 
precisely, that g(n) is an asymptotic upper bound for f(n), to emphasize 
that we are suppressing constant factors
? Asymptotic Strict-Upper Bound (o)
¦ Let f and g be functions f, g: N ?R
+
. Say that f(n) = o(g(n)) if
lim               = 0
¦ In other words, f(n) = o(g(n)) means that, for any real number c > 0, a 
number n
0
exist, where f(n) < c.g(n) for all n = n
0
3
f(n)
g(n)
n ?
8
Page 4


Time Complexity
CS60001: Foundations of Computing Science
Pallab Dasgupta
Professor, Dept. of Computer Sc. & Engg.,
Indian Institute of Technology Kharagpur
Indian Institute of Technology Kharagpur
Pallab Dasgupta
Measuring Complexity
? Definition
¦ Let M be a deterministic Turing machine that halts on all inputs. The 
running time or time complexity of M is the function f: N ?N, where f(n) is 
the running time of M, we say that M runs in time f(n) and that M is an f(n)
time Turing machine. Customarily we use n to represent the length of the 
input
? Complexity Analysis
¦ Worst-case Analysis
? Longest running time of all inputs of a particular length
¦ Average-case Analysis
? Average of all the running times of inputs of a particular length
2
Indian Institute of Technology Kharagpur
Pallab Dasgupta
Big-O and Small-o Notations
? Asymptotic Upper Bound (O)
¦ Let f and g be functions f, g: N ?R
+
. Say that f(n) = O(g(n)) if positive 
integers c and n
0
exist such that for every integer n = n
0
f(n) = c.g(n)
¦ When f(n) = O(g(n)) we say that g(n) is an upper bound for f(n), or more 
precisely, that g(n) is an asymptotic upper bound for f(n), to emphasize 
that we are suppressing constant factors
? Asymptotic Strict-Upper Bound (o)
¦ Let f and g be functions f, g: N ?R
+
. Say that f(n) = o(g(n)) if
lim               = 0
¦ In other words, f(n) = o(g(n)) means that, for any real number c > 0, a 
number n
0
exist, where f(n) < c.g(n) for all n = n
0
3
f(n)
g(n)
n ?
8
Indian Institute of Technology Kharagpur
Pallab Dasgupta
Analyzing Algorithms
? Let t: N ?R
+
be a function. Define the time complexity class, TIME(t(n)), to be 
the collection of all languages that are decidable by an O(t(n)) time Turing 
machine
? Example
¦ Analyze the TM algorithm for the language A = {0
k
1
k
| k = 0}
¦ There can be different TM constructions (M
1
, M
2
, M
3
) deciding the language 
[see Sipser’s Book, pp. 207-209]
¦ The total-time taken by them is different
? M
1
decides A in time O(n
2
), therefore A ? TIME(n
2
)
? M
2
decides A in time O(nlogn), therefore A ? TIME(n.logn)
? M
3
decides A in time O(n), therefore A ? TIME(n)
4
Page 5


Time Complexity
CS60001: Foundations of Computing Science
Pallab Dasgupta
Professor, Dept. of Computer Sc. & Engg.,
Indian Institute of Technology Kharagpur
Indian Institute of Technology Kharagpur
Pallab Dasgupta
Measuring Complexity
? Definition
¦ Let M be a deterministic Turing machine that halts on all inputs. The 
running time or time complexity of M is the function f: N ?N, where f(n) is 
the running time of M, we say that M runs in time f(n) and that M is an f(n)
time Turing machine. Customarily we use n to represent the length of the 
input
? Complexity Analysis
¦ Worst-case Analysis
? Longest running time of all inputs of a particular length
¦ Average-case Analysis
? Average of all the running times of inputs of a particular length
2
Indian Institute of Technology Kharagpur
Pallab Dasgupta
Big-O and Small-o Notations
? Asymptotic Upper Bound (O)
¦ Let f and g be functions f, g: N ?R
+
. Say that f(n) = O(g(n)) if positive 
integers c and n
0
exist such that for every integer n = n
0
f(n) = c.g(n)
¦ When f(n) = O(g(n)) we say that g(n) is an upper bound for f(n), or more 
precisely, that g(n) is an asymptotic upper bound for f(n), to emphasize 
that we are suppressing constant factors
? Asymptotic Strict-Upper Bound (o)
¦ Let f and g be functions f, g: N ?R
+
. Say that f(n) = o(g(n)) if
lim               = 0
¦ In other words, f(n) = o(g(n)) means that, for any real number c > 0, a 
number n
0
exist, where f(n) < c.g(n) for all n = n
0
3
f(n)
g(n)
n ?
8
Indian Institute of Technology Kharagpur
Pallab Dasgupta
Analyzing Algorithms
? Let t: N ?R
+
be a function. Define the time complexity class, TIME(t(n)), to be 
the collection of all languages that are decidable by an O(t(n)) time Turing 
machine
? Example
¦ Analyze the TM algorithm for the language A = {0
k
1
k
| k = 0}
¦ There can be different TM constructions (M
1
, M
2
, M
3
) deciding the language 
[see Sipser’s Book, pp. 207-209]
¦ The total-time taken by them is different
? M
1
decides A in time O(n
2
), therefore A ? TIME(n
2
)
? M
2
decides A in time O(nlogn), therefore A ? TIME(n.logn)
? M
3
decides A in time O(n), therefore A ? TIME(n)
4
Indian Institute of Technology Kharagpur
Pallab Dasgupta
Complexity Relationships among Models
? Definition
¦ Let N
TM
be a non-deterministic Turing machine that is a decider. The 
running time of N
TM
is the function f: N ? N, where f(n) is the maximum 
number of steps that N
TM
uses on any branch of its computation on any 
input n
? Theorems
¦ Let t(n) be a function, where t(n) = n. Then every t(n) time multi-tape Turing 
machine has an equivalent O(t
2
(n)) time single-tape Turing machine
¦ Let t(n) be a function, where t(n) = n. Then every t(n) time non-deterministic 
single-tape Turing machine has an equivalent 2
O(t(n))
time deterministic 
single-tape Turing machine
5
Read More
Offer running on EduRev: Apply code STAYHOME200 to get INR 200 off on our premium plan EduRev Infinity!