0-1 Knapsack Problem | Algorithms - Computer Science Engineering (CSE) PDF Download

Introduction

Given weights and values of n items, put these items in a knapsack of capacity W to get the maximum total value in the knapsack. In other words, given two integer arrays val[0..n-1] and wt[0..n-1] which represent values and weights associated with n items respectively. Also given an integer W which represents knapsack capacity, find out the maximum value subset of val[] such that sum of the weights of this subset is smaller than or equal to W. You cannot break an item, either pick the complete item or don’t pick it (0-1 property).

Methods

1. Method 1: Recursion by Brute-Force algorithm OR Exhaustive Search.
Approach: A simple solution is to consider all subsets of items and calculate the total weight and value of all subsets. Consider the only subsets whose total weight is smaller than W. From all such subsets, pick the maximum value subset.
Optimal Sub-structure: To consider all subsets of items, there can be two cases for every item.

• Case 1: The item is included in the optimal subset.
• Case 2: The item is not included in the optimal set.

Therefore, the maximum value that can be obtained from ‘n’ items is the max of the following two values. 

1. Maximum value obtained by n-1 items and W weight (excluding nth item).
2. Value of nth item plus maximum value obtained by n-1 items and W minus the weight of the nth item (including nth item).

If the weight of ‘nth’ item is greater than ‘W’, then the nth item cannot be included and Case 1 is the only possibility.
Below is the implementation of the above approach: 

• C++
  
  
• C
  
• Java
  
  
• Python
  
• C#
  
  
• PHP
  
• Javascript
  

Output: 220
It should be noted that the above function computes the same sub-problems again and again. See the following recursion tree, K(1, 1) is being evaluated twice. The time complexity of this naive recursive solution is exponential (2^n).  

Complexity Analysis

• Time Complexity: O(2ⁿ).  
  As there are redundant subproblems.
• Auxiliary Space: O(1).
  As no extra data structure has been used for storing values.

Since subproblems are evaluated again, this problem has Overlapping Sub-problems property. So the 0-1 Knapsack problem has both properties (see this and this) of a dynamic programming problem. 

2. Method 2: Like other typical Dynamic Programming(DP) problems, re-computation of same subproblems can be avoided by constructing a temporary array K[][] in bottom-up manner. Following is Dynamic Programming based implementation.
Approach: In the Dynamic programming we will work considering the same cases as mentioned in the recursive approach. In a DP[][] table let’s consider all the possible weights from ‘1’ to ‘W’ as the columns and weights that can be kept as the rows.
The state DP[i][j] will denote maximum value of ‘j-weight’ considering all values from ‘1 to ith’. So if we consider ‘wi’ (weight in ‘ith’ row) we can fill it in all columns which have ‘weight values > wi’. Now two possibilities can take place: 

• Fill ‘wi’ in the given column.
• Do not fill ‘wi’ in the given column.

Now we have to take a maximum of these two possibilities, formally if we do not fill ‘ith’ weight in ‘jth’ column then DP[i][j] state will be same as DP[i-1][j] but if we fill the weight, DP[i][j] will be equal to the value of ‘wi’+ value of the column weighing ‘j-wi’ in the previous row. So we take the maximum of these two possibilities to fill the current state. This visualization will make the concept clear:  

• C++
  
  
• C
  
  
• Java
  
  
• Python
  
• C#
  
  
• PHP
  
• Javascript
  

Output: 220
Complexity Analysis

• Time Complexity: O(N * W).  
  where ‘N’ is the number of weight element and ‘W’ is capacity. As for every weight element we traverse through all weight capacities 1 <= w <= W.
• Auxiliary Space: O(N * W).
  The use of 2-D array of size ‘N * W’.

3. Method 3: This method uses Memoization Technique (an extension of recursive approach).
This method is basically an extension to the recursive approach so that we can overcome the problem of calculating redundant cases and thus increased complexity. We can solve this problem by simply creating a 2-D array that can store a particular state (n, w) if we get it the first time. Now if we come across the same state (n, w) again instead of calculating it in exponential complexity we can directly return its result stored in the table in constant time. This method gives an edge over the recursive approach in this aspect.

• C++
  
  
• Java
  
  
• Python3
  
• C#
  
  

Output: 220
Complexity Analysis

• Time Complexity: O(N * W).
  As redundant calculations of states are avoided.
• Auxiliary Space: O(N * W).
  The use of 2D array data structure for storing intermediate states-:

4. Method 4:  We use the dynamic programming approach but with optimized space complexity.

• Python3
  
  

Output: 220
Complexity Analysis

• Time Complexity: O(N * W). As redundant calculations of states are avoided.
• Auxiliary Space: O(W) As we are using 1-D array instead of 2-D array.