Decomposition

Decomposition Problem

Introduction

Decomposition is a approach to solving a problem by breaking it up into smaller ones and solving each of smaller ones separately, or sequentially.

If the problem is separable, i.e., the objective is comprised by the sum of functions of sub-vector \(x_i\) and each constraint only involve only one variable from one of the sub-vector \(x_i\), then each smaller problem involving \(x_i\) separately can be easily solved(called trivially parallelizable or block separable). The more interesting problem is that sub-vectors are tangled.

The core idea of decomposition is: using effective methods to solve sub-problems and combining the results in such a way as to solve the larger problem. Two techniques will be introduced in the following.

Primal decomposition

unconstrainted case

\[\min f(x)=f_1(x_1, y)+f_2(x_2, y) \]

where \(x_1\) and \(x_2\) are called private or local variables, \(y\) is the complicating or coupling variable which complicates the problem and \(x=\{x_1, x_2, y\}\). If \(y\) is fixed, the problem is separable and the subproblem is equivalent to find \(\phi_1(y)=\min_{x_1}f_1(x_1, y)\) and \(\phi_2(y)=\min_{x_2}f_2(x_2, y)\). The original problem becomes

\[\min_y \phi_1(y)+\phi_2(y) \]

which is called master problem.

A decomposition method solve the original problem by a iterative method, such as the subgradient method.

Dual decomposition

consistency constraint

\[\begin{align} \label{eq:DCC} &\min \quad f_1(x_1, y_1)+f_2(x_2, y_2)\\ &\text{s.t.} \quad\quad\quad\quad\quad y_1=y_2 \end{align} \]

The Lagrangian is

\[L(x_1, x_2, y_1, y_2, v)=f_1(x_1, y_1)+f_2(x_2, y_2)+v^Ty_1-v^Ty_2 \]

The dual function is

\[g(v) = g_1(v)+g_2(v) \]

where \(g_1(v)=\inf_{x_1,y_1} f_1(x_1,y_1)+v^Ty_1=-f_1^*(0,-v)\), \(g_2(v)=\inf_{x_2,y_2} f_2(x_2,y_2)-v^Ty_2=-f_2^*(0,-v)\)

The dual problem is

\[\max g_1(v)+g_2(v)=\min f_1^*(0,-v)+f_2^*(0,-v) \]

where \(g_1(v)\) and \(g_2\) can be solved separately, according to the Primal decomposition context.

Decomposition with constraints

\[\begin{alignat}{2} \min \quad &f_1(x_1)+f_2(x_2)\\ \text{s.t.}\quad & x_1 \in \mathcal{C}_1, x_2\in\mathcal{C}_2\\ &h_1(x_1) + h_2(x_2)\preceq 0 \end{alignat} \]

where the last constraint complicates the problem, called complicating constraints.

Primal decomposition

Assigning a fixed amount of resource \(t\) to \(h_1(x_1)\) and the amount of \(h_2(x_2)\) is \(-t\), then the original problem can be separable.

\[\begin{alignat}{2} \label{eq:DCP1} \min \quad &f_1(x_1)\\ \mbox{s.t.} \quad & x_1\in \mathcal{C}_1\\ & h_1(x_1)\preceq t \end{alignat} \]

\[\begin{alignat}{2} \label{eq:DCP2} \min \quad &f_2(x_2)\\ \mbox{s.t.} \quad & x_2\in \mathcal{C}_2\\ & h_2(x_2)\preceq -t \end{alignat} \]

Let \(\phi_1(t)\) and \(\phi_2(t)\) be the optimal value of the problem \(\ref{eq:DCP1}\) and \(\ref{eq:DCP2}\), \(\lambda_1\) and \(\lambda_2\) denote the optimal dual variable of corresponding subproblem. Then the optimization process is:

1. Solve the subproblem \(\ref{eq:DCP1}\), to find an optimal \(x_1\) and \(\lambda_1\)
2. Solve the subproblem \(\ref{eq:DCP2}\), to find an optimal \(x_2\) and \(\lambda_2\)
3. \(t=t-\alpha(\lambda_2-\lambda_1)\)

Dual decomposition

The partial Lagrangian is

\[\begin{align} L(x_1,x_2,\lambda)&=f_1(x_1)+f_2(x_2)+\lambda^T(h_1(x)+h_2(x))\\ &= (f_1(x_1)+\lambda^T h_1(x)) + (f_2(x_2)+\lambda^T h_2(x)) \end{align} \]

The problem is the same with \(\ref{eq:DCC}\)

The more complicated decomposition can be represented by a hypergraph or net.

There are three subproblems and each link represent the coupling variable or constraint between the adjoined subproblems.

How primal and dual decomposition work:

• In primal decomposition, each hyperedge or net has a single variable associated with it. Each subproblem is optimized separately, using the public variable values (asserted) on the nets. Each subproblem produces a subgradient associated with each net it is adjacent to. These are combined to update the variable value on the net, hopefully in such a way that convergence to (global) optimality occurs.
• In dual decomposition, each subproblem has its own private copy of the public variables on the nets it is adjacent to, as well as an associated price vector(dual variable \(\lambda_i\)). The subsystems use these prices to optimize their local variables, including the local copies of public variables. The public variables on each net are then compared, and the prices are updated, hopefully in a way that brings the local copies of public variables into consistency (and therefore also optimality).

General framework for decomposition structures

There are \(K\) subproblems with private variables \(x_i\), public variables \(y_i\), public variable \(f_i\) and the local feasible set \(\mathcal{C}_i\). Let \(y\) denote the set of all public variables, i.e., \(y=(y_1, y_2,\dots,y_K)\) and \((y)_i\) be the \(i\)-th component. Suppose there are \(N\) nets, let \(z\in R^N\) be the common value on the nets. Then \(y=Ez\), where E is the matrix with

\[E=\left\{\begin{matrix} 1 & (y)_i \text{ is in net j}\\ 0 & \text{otherwise} \end{matrix} \right. \]

The whole problem is

\[\begin{alignat}{2} \min \quad &\sum_{i=1}^{K} f_i(x_i,y_i)\\ \mbox{s.t.} \quad &(x_i, y_i)\in\mathcal{C}_i\\ & y_i=Ez_i\\ \end{alignat} \]

Primal Decomposition

In primal decomposition, at each iteration we fix the vector z of net variables, and we fix the
public variables as $y_i = E_iz $. Each subsystem can (separately) find optimal values for its local variables \(x_i\)

\[\begin{alignat}{2} \min \quad &f_i(x_i,y_i)\\ \mbox{s.t.} \quad & (x_i,y_i) \in \mathcal{C}_1\\ \end{alignat} \]

To find a subgradient of \(\phi\), we find \(g_i \in \partial \phi_i(y_i)\) . We then have

\[g=\sum_{i=1}^K E^T_ig_i \]

The entire process is

1. $y_i = E_iz, i = 1, . . . , K $
2. Solve subproblems to find optimal \(x_i\), and \(g_i\in\partial \phi(y_i),\, i=1,2\dots,K\)
3. \(g=\sum_{i=1}^K E^T_ig_i\)
4. $z= z - \alpha_kg. $

Dual Decomposition

The partial Lagrangian is

\[\begin{align} L(x, y, z, v) &= \sum_{i=1}^K f_i(x_i,y_i)+v^T(y-Ez)\\ &=\sum_{i=1}^K f_i(x_i,y_i)+v_i^Ty_i-v^TEz \end{align} \]

the subproblem is

\[\begin{alignat}{2} \min \quad &f_i(x_i,y_i)+v_i^Ty_i\\ \mbox{s.t.} \quad &(x_i, y_i)\in \mathcal{C}_i \end{alignat} \]

Let \(g_i(v_i)\) be the optimal value of the subproblem and the subgradient of \(-g_i\) at \(v_i\) is \(-y_i\). The dual problem is

\[\begin{alignat}{2} \max \quad &\sum_{i=1}^K g_i(v_i)\\ \mbox{s.t.} \quad & E^Tv=0 \end{alignat} \]

where the last constraint comes from \(\min_z L(x,y,z,v)\).

We can solve this dual decomposition master problem using a projected subgradient method. Projection onto the feasible set \(\{v | E^T v = 0\}\), which consists of vectors whose sum over each net is zero. The projection is given by multiplication by \(\underbrace{I}_{\text{identity matrix}} - \underbrace{E(E^T E)^{-1}E^T}_{\text{average value on the graph}}\), where \(E^T E=\text{diag}\{d_1, \dots,d_N\}\), \(d_i\) is the degree of the net \(i\). \(i.e.\), the number of subsystems adjacent to net \(i\).