[Coding Made Simple] Matrix Chain Multiplication

Posted 2020-10-01 Push your limit!

Given some matrices, in what order you would multiply them to minimize cost of multiplication.

 The following problem formulation is extracted from this link.

Problem Formulation

Note that although we can use any legal parenthesization, which will lead to a valid result. But, not all parenthesizations involve the same number of operations. To understand this point, consider the problem of a chain A₁, A₂, A₃ of three matrices and suppose

A₁ be of dimension 10 × 100
A₂ be of dimension 100 × 5
A₃ be of dimension 5 × 50

Then,

MultCost[((A₁ A₂) A₃)] = (10 . 100 . 5) +  (10 . 5 . 50) = 7,500 scalar multiplications.

MultCost[(A₁ (A₂ A₃))] = (100 . 5 . 50) + (10 . 100 . 50) = 75,000 scalar multiplications.

It is easy to see that even for this small example, computing the product according to first parenthesization is 10 times faster.

 

The Chain Matrix Multiplication Problem

Given a sequence of n matrices A₁, A₂, ... A_n, and their dimensions p₀, p₁, p₂, ..., p_n, where where i = 1, 2, ..., n, matrix A_i has dimension p_{i ? 1} × p_i, determine the order of multiplication that minimizes the the number of scalar multiplications.

 

Equivalent formulation (perhaps more easy to work with!)

Given n matrices, A₁, A₂, ... A_n, where for 1 ≤ i ≤ n, A_i is a p_{i ? 1} × p_i, matrix, parenthesize the product A₁, A₂, ... A_n so as to minimize the total cost, assuming that the cost of multiplying an p_{i ? 1}× p_i matrix by a p_i × p_{i + 1} matrix using the naive algorithm is p_{i ? 1}× p_i × p_{i + 1}.

 

Note that this algorithm does not perform the multiplications, it just figures out the best order in which to perform the multiplication operations.

 

Solution 1. Simple recursive approach 

The original problem can be solved by recursively solving its subproblems as follows.

cost(i, j) is the min of cost(i, k) + cost(k + 1, j) + matrices[i].row * matrices[k].col * matrices[j].col for all k >= i && k < j.

The base case is when i == j, the cost is 0 as it takes nothing to multipy 1 matrix.

 

Once again the same with a lot of recursive algorithms, it has the overlapping subproblems issue.

For example, to compute cost(0, 3), the following computations are done separately.

cost(0, 2), cost(3, 3);

cost(0,1), cost(2, 3);

cost(0, 0), cost(1, 3);

But in the process of computing cost(0,2), cost(0, 1) is computed, which is computed later in a different parenthess split. This 

redundancy has way more presence as the input grows bigger. 

 

Solution 2. Dynamic Programming, O(n^3) runtime, O(n^2) space 

To avoid the redundany issue in solution 1, we should use dynamic programming.

State: cost[i][j]: the min cost of multiplying matrices i to j.

Function: same with the recursive formula.

 

 1 class Matrix {
 2     int row;
 3     int col;
 4     Matrix(int row, int col){
 5         this.row = row;
 6         this.col = col;
 7     }
 8 }
 9 public class Solution {
10     public int matrixChainMultiRecursion(Matrix[] matrices) {
11         if(matrices == null || matrices.length == 0) {
12             return 0;
13         }        
14         return recursiveHelper(matrices, 0, matrices.length - 1);
15     }
16     private int recursiveHelper(Matrix[] matrices, int startIdx, int endIdx) {
17         if(startIdx == endIdx){
18             return 0;
19         }
20         int miniCost = Integer.MAX_VALUE;
21         for(int i = startIdx; i < endIdx; i++){
22             int leftCost = recursiveHelper(matrices, startIdx, i);
23             int rightCost = recursiveHelper(matrices, i + 1, endIdx);
24             int cost = leftCost + rightCost + matrices[startIdx].row * matrices[i].col * matrices[endIdx].col;
25             miniCost = Math.min(cost, miniCost);
26         }
27         return miniCost;
28     }
29     public int matrixChainMultiDp(Matrix[] matrices) {
30         if(matrices == null || matrices.length == 0) {
31             return 0;
32         }
33         int[][] cost = new int[matrices.length][matrices.length];
34         for(int i = 0; i < matrices.length; i++){
35             int j = i;
36             for(; j < matrices.length; j++){
37                 cost[i][j] = Integer.MAX_VALUE;
38             }
39         }
40         for(int i = 0; i < matrices.length; i++){
41             cost[i][i] = 0;
42         }
43         for(int len = 2; len <= matrices.length; len++){
44             for(int i = 0; i <= matrices.length - len; i++){
45                 for(int j = i; j < i + len - 1; j++){
46                     int c = cost[i][j] + cost[j + 1][i + len - 1] + matrices[i].row * matrices[j + 1].row * matrices[i + len - 1].col;
47                     cost[i][i + len - 1] = Math.min(cost[i][i + len - 1], c);                    
48                 }
49             }
50         }
51         return cost[0][matrices.length - 1];
52     }
53     public static void main(String[] args) {
54         Matrix m1 = new Matrix(2, 3);
55         Matrix m2 = new Matrix(3, 6);
56         Matrix m3 = new Matrix(6, 4);
57         Matrix m4 = new Matrix(4, 5);
58         Matrix[] matrices = {m1, m2, m3, m4};
59         Solution sol = new Solution();
60         System.out.println(sol.matrixChainMultiRecursion(matrices));
61         System.out.println(sol.matrixChainMultiDp(matrices));        
62     }
63 }