Chapter 9: Deterministic Overlapping Generations Models --- Python Code

Section 9.1: The Steady State

In the following, we study the computation of the steady state of the 60-Period OLG model described in Section 9.1 of our book on 'Dynamic General Equilibrium Modeling: Computational Methods and Applications'.

We apply value function iteration. In order to maximize the right-hand side of the Bellman equation, we use Golden Section Search as described in Section 11.6.1 of our book. In order to interpolate between grid points of the value function, we use linear or cubic interpolation as described in Section 11.2.3 of our book.

The PYTHON program 'RCh91v.py' can be downloaded from our web page

https://www.uni-augsburg.de/de/fakultaet/wiwi/prof/vwl/maussner/dgebook/

Description of the model

Let me briefly describe the economic model. We consider a 60-period overlapping generations model. All members of a cohort are identical and each cohort has measure 1/60 (you also find the Python code for more complex OLG models with heterogeneous agent and income uncertainty on the web page above).

Agents maximize lifetime utility at the beginning of age 1 in period $t$:

$\sum_{s=1}^{60} \beta^{s-1} u(c^s_{t+s-1},1-l^s_{t+s-1}),$

where $\beta$ denotes the discount factor and $c^s_t$ denotes consumption at age $s$ in period $t$. During the first $T^W=40$ years, the agents are workers. During the remaining $T^R=20$ years, they are retired.

Instantaneous utility is a function of both consumption $c$ and leisure $1-l$:

$u(c,1-l)=\frac{\left( (c+\psi) (1-l)^\gamma \right)^{1-\eta}-1}{1-\eta}.$

The small constant $\psi=0.001$ is added to ensure that utility is finite even for zero consumption in the case of no income. This choice will turn out to be very convenient in the subsequent computations because we will be able to use a grid over the individual capital stock with a lower bound equal to zero.

Agents are born without wealth, $a^1_t=0$, and do not leave bequests, $a_t^{61}=0$. Since capital $a$ is the only asset held by individuals, the terms capital and wealth will henceforth be used interchangeably. Agents receive income from capital $a^s_t$ and labor $l^s_t$. The real budget constraint of the working agent is given by

$a^{s+1}_{t+1}=(1+r_t) a^s_t +(1-\tau_t) w_t l^s_t-c^s_t, \;\;\; s=1,\ldots,T^W,$

where $r_t$ and $w_t$ denote the real interest rate and the real wage rate in period $t$, respectively. Wage income in period $t$ is taxed at rate $\tau_t$. We can also interpret $\tau_t w_t l^s_t$ as the worker's social security contributions.

During retirement, agents receive public pensions $pen$ irrespective of their employment history, and the budget constraint of the retired worker is given by

$a^{s+1}_{t+1}=(1+r_t) a^s_t + pen-c^s_t, \;\;\; s=T^W+1,\ldots,T^W+T^R.$

Goods and factor markets are subject to perfect competition. Production $Y_t$ is characterized by constant returns to scale and assumed to be Cobb-Douglas:

$Y_t= F(K_t,L_t) = K_t^\alpha L_t^{1-\alpha}.$

Profits are represented by $ \Pi_t = Y_t - w_t L_t -r_t K_t -\delta K_t.$

In a factor market equilibrium, factors are rewarded with their marginal product:

$w_t=(1-\alpha) K_t^\alpha L_t^{-\alpha},$

$r_t=\alpha K_t^{\alpha-1} L_t^{1-\alpha}-\delta.$

The government uses revenues from taxing labor to finance its expenditures on social security:

$ \tau_t w_t L_t= \frac{T^R}{T} pen.$

In order to compute the individual policy functions for next-period wealth $a'$, consumption $c$ and labor $l$, we define the value function $V^s(k_t;K_t,L_t,pen)$ as the solution to the following dynamic program:

$V^s(a^s_t;K_t,L_t,pen) = \left\{ \begin{array}{l}\!\!\!\! \underset{a^{s+1}_{t+1},c^s_t,l^s_t}{\max}\; \!\!\!\! \left[ u\left(c^s_t,1-l^s_t\right) +\beta V^{s+1}(a^{s+1}_{t+1};K_{t+1},L_{t+1},pen) \right],\\ \;\;\;\;\; s=1,\ldots,T^W \\ \\ \underset{a^{s+1}_{t+1},c^s_t} \max \left[ u\left(c^s_t,1\right) +\beta V^{s+1}(a^{s+1}_{t+1};K_{t+1},L_{t+1},pen) \right],\\ \;\;\;\;\; s=T^W+1,\ldots,T,\end{array} \right. $

subject to the budget constraints of the worker and retiree, respectively, and the value function in the last period of the life, $T=T^W+T^R=60$:

$V^{T}(a^{T}_t;K_t^{T},L_t^{T},pen) = u(c^{T}_t,1).$

Some authors drop the aggregate state and policy variables as arguments of the value function for notational convenience.

In equilibrium, individual and aggregate behavior are consistent

$L_t=\sum_{s=1}^{T^W} \frac{l^s_t}{T}$

$K_t=\sum_{s=1}^{T} \frac{a^s_t}{T}.$

and the goods market clears:

$Y_t = K_t^\alpha L_t^{1-\alpha} = \sum_{s=1}^{T} \frac{c^s_t}{T} + K_{t+1}-(1-\delta) K_{t}.$

In the following, we compute the steady state.

Setting up the PYTHON computer program

Please download the PYTHON code 'AK60_value.py' and run it. In line 23, you specify the interpolation method. We set 'VI_method = 2' for cubic spline interpolation. If you would like to try linear interpolation instead, just set 'VI_method = 1'.

The computational time amounts to approximately 10 minutes on my computer (the GAUSS code is about 5 times faster). The solution for the aggregate variables are $K=0.876$ and $L=0.223$. The individual age-wealth and age-labor supply profiles are displayed in Figs. 9.1 and 9.2 in our book, respectively.

The program consists of an outer loop over the aggregate variables $(K,L)$ and an inner loop that computes the policy functions as a solution to the value function iteration problem. After 23 iterations of the outer loop, the program execution stops when the aggregate capital stocks in two consecutive iterations diverge by less than 0.0001. In the inner loop, we just need one iteration since we can solve the finite lifetime value function problem backwards starting in the last period of life, $T=60$ and initialize the value function at this age as described above in the economic model.

In the following, we set the number of iterations in the outer loop to $nq=1$ and display the value function and policy functions of the 10-years old worker and 50-years-old retiree. In addition, I will present functions as they appear in the main part of the program. In the download version, we specified the functions in the beginning of the program instead so that it allows for easy editing.

In the first part of the program, we load the necessary libraries numpy, scipy, time, matplotlib and math. If you have not installed them or have difficulties to load them, please consult the above referenced manuscript of THOMAS J. SARGENT and JOHN STACHURSKI on 'Python Programming for Economics and Finance':

https://python-programming.quantecon.org/intro.html

Next, we set some numerical parameters. In particular, we set up an equi-spaced grid for individual wealth a with $na=200$ grid points on the interval $[0,10]$. We will also define some constants for the tolerance of the final solution for the aggregate capital stock, $tol=0.0001$, and the tolerance of our optimization routine, $tol1=1e-10$. The other parameters will be explained as we describe the individual parts of the program in more detail and can be ignored by the reader for now.

Next, we introduce the optimization function which we apply to the right-hand side of the Bellman equation in the value function iteration. We will use Golden Section Search (as described in Section 11.6.1 in our book). The function is called using the command 'GoldenSectionMax(f,ay,by,cy,tol)' where f is a one-dimensional function, the maximum is bracketed by the points ay and cy, by is an intermediate point and tol describes the maximum difference between the upper and lower boundary, cy-ay.

We use a simple function 'testfunc(x)' with the maximum at point x=2.0 to test our function GoldenSectionMax(.).

Next, we parameterize the model. Our benchmark case is characterized by the following calibration: $\eta=2$, $\beta=0.96$, $\alpha=0.36$, $\delta=0.10$, replacement ratio of pensions with respect to average net wage income, $\theta=\frac{pen}{(1-\tau) w \bar l}=0.3$ (where $\bar l$ denotes the average labor supply in the economy), $T^W=40$, and $T^R=20$. $\gamma$ is chosen to imply a steady-state labor supply of the working agents approximately equal to $\bar l=35\%$ of available time and amounts to $\gamma=2.0$. The small constant $\psi$ is set equal to 0.001. In addition, the household is born without any wealth so that $a^1_t=0$.

We have to pick initial values for the steady-state values of aggregate capital stock $K$ and labor $L$. Assuming that the workers work approximately 30% of their available time and that the measure of households is equal to 2/3, we set $L=0.2$ initially. The aggregate capital stock is found with the help of the firms' first-order condition with respect to capital above:

$r = \alpha K^{\alpha-1} L^{1-\alpha}-\delta $

For our initial guess of the interest rate $r=0.045$ and $\delta=0.10$, this implies $K=0.828$.

We save the aggregate variables $(K,L)$ in each outer iteration $q$ in the variable $kq$.

In the outer loop, we iterate over $K$ and $L$. Given $K$ and $L$, we can compute the equilibrium values of the wage and interest rate with the help of the first-order condition of the firms. The two factor prices are computed as functions 'wage_rate' and 'interest_rate' for given values of $K$ and $L$.

Similarly, we can compute the pension $pen$ with the help of the replacement rate of pensions $\theta$ with respect to net wage income. Therefore, we apply the average working time

$\bar l = \frac{3}{2} L$

and compute the pension with the help of

$pen = \theta (1-\tau) w \bar l$

We find the following values:

Value Function Iteration: Retirees

In the inner loop, we compute the policy functions of the worker and retiree using value function iteration. First, we initialize the value function of the retirees using the command 'vr = np.zeros((na,tr))'.

To compute the value functions at retirement age $i=0,\ldots,19$, we start in the last period and iterate backwards in time (or age). The value function in the last period of the life, $T=T^W+T^R=60$, with given exogenous pensions $pen$ and interest rate $r$ is presented by

$V^{T}(a^{T}_t;K_t^{T},L_t^{T},pen) = u(c^{T}_t,1)$

with

$c^T_t = pen + (1+r) a^T.$

Retired agents at age $T=60$ (corresponding to the iteration index $i=19$) consume their total income consisting of pensions and interest income. We compute this value function at all asset grid points $a_l \in \{a_0=0,a_1,\ldots,a_{199}=10\}$ and store them in the value function $vr[l,19]$ for $l=0,\ldots,199$. In order to compute the utility $u(c,1-l)$ for given consumption $c$ and leisure $1-l$, we use the function 'utility(x,y)'. Remember that in old age, labor supply is equal to zero so that leisure amounts to $1-l =1$.

In the next step, we consider the retiree in his second-to-last period at retirement age $i=18$. In order to compute the optimal solution to the right-hand side of the Bellman equation

$v^{18}(a^{18}) = \max_{a^{19}} \left\{ u\left(pen +(1+r) a^{18}- a^{19},1 \right) + \beta v^{19} (a^{19}) \right\} $

we use Golden section search. This basic step is performed below in the line

aropt[l,i-1] = GoldenSectionMax(value1,ax,bx,cx,tol1)

Therefore, we have to bracket the maximum in the interval $[ax,cx]$ with an intermediate value $bx$. The right-hand side of the Bellman equation is computed in the function 'value1(x)'. Before I describe this procedure in detail, let us first run the value function iteration of the retiree and study the solution.

In the top figure, the value function of the retiree at retirement age $i=10$ is displayed. The value function is a monotone increasing and concave function of individual beginning-of-period wealth $a^{10}$.

Next-period wealth $a^{11}$ is also montonously increasing in wealth $a^{10}$ (not presented), while savings, $a'-a$, are a negative function of wealth $a$. The rich retirees also consume more. Accordingly, consumption (in the lower figure) increases with wealth.

Notice that next-period wealth $a^{i+1}(a^{i})$ and the value function $v^i(a^i)$ are monotone functions of $a$ at age $i$.

Let us go through the inner loop $i$ of the retired value function iteration line by line. The optimal policy functions $a^{i+2}(a^{i+1})$ and the value function $v^{i+1}(a^{i+1})$ have been assigned to the variables $vr[:,i+1]$ and $aropt[:,i+1]$ in the previous iteration $i+1$. Remember that we have initialized the value function in the last period of life with the utility that results from the consumption of total income and wealth; similarly, the optimal next-period wealth is equal to zero for $aropt[:,19]$.

In iteration $i$ (the retirement age), we need to solve the Bellman equation

$v^{i}(a^{i}) = \max_{a^{i+1}} \left\{ u\left(pen +(1+r) a^{i}- a^{i+1},1 \right) + \beta v^{i+1} (a^{i+1}) \right\} $

by maximizing with respect to $a^{i+1}$. Since $a^{i+1}$ might not be a grid point, we have to interpolate the next-period value function $v^{i+1} (a^{i+1})$ between grid points $\{a_0,\ldots,a_{na}\}$. Since the value function next period will not change in iteration $i$, we have to determine the interpolation coefficient only once at the beginning of the loop using the command "vr_polate = interpolate.interp1d(a,vr[:,i], fill_value='extrapolate')". The interpolation at any wealth level $a^{19}$ is carried out in the function 'rvalue(k)' whenever we compute the right-hand side of the Bellman equation above.

Next, we iterate over all 200 grid points $a_l$, $l=0,\ldots,199$ to compute the solution of the Bellman equation. Therefore, we have to bracket the maximum so that the solution is constrained by $ax < a^{i+1} < cx$. In order to determine the limits $ax$ and $cx$ of this interval, we start at $a_0=0$ and consider $a_1$, $a_2$, $\ldots$ subsequently. We compute the right-hand side of the Bellman equation (using the function 'value1(k)') as long as the value keeps increasing. We stop as soon as it decreases. Let the last three grid points be $a_{j-1}$, $a_j$ and $a_{j+1}$, where $a_j$ is associated with the maximum value among the last evaluations. We then allocate $ax=a_{j-1}$, $bx = a_j$ and $cx = a_{j+1}$ for the golden section search. Next, we use golden section search to find the maximum value of 'value1(k)' in the interval $[ax,cx]$.

When we search for the bracketing interval $[ax,cx]$, we have to take care of two special cases. First, $a^{19}\ge 0$ might become binding. Second, $a^{19}\le a_{199}$ might become binding. These cases might occur when the maximum over the grid point is located at either $a_0$ or $a_{199}$. In this case, we use a small $\epsilon$ and check whether $a_0+\epsilon$ or $a_{199}-\epsilon$ results in an increase of the value of $v^i(a^i)$. If not, the solution is equal to $a_0$ and $a_{199}$, respectively. Otherwise, we use golden section again. For example, if $a_0$ is the maximum of all grid points, we use the interval $[a_0,a_1]$ and set $ax=a_0$, $bx=a_0+\epsilon$, and $cx=a_1$.

In order to speed up the computation of the value function, we also use the fact that next-period assets are a monotone increasing function of wealth; in other words, $a^{i+1}(a^i)$ has a strictly positive first derivative. Therefore we know that, if, in iteration $l$ for the grid $a_l$, the maximizing grid point for next-period assets was located at $a_m$, we start the search for the maximum at the wealth level $a_{l+1}$ at the next-period grid point $a_m$.

At the end of iteration $l$, we store the value function and the policy functions for next-period wealth and consumption in the variables $vr[l,i]$, $aropt[l,i]$ and $cropt[l,i]$, respectively.

Value Function Iteration of the Worker

Next, we turn to the value function iteration of the worker. The procedure is almost the same as in the case of the retiree. The worker also maximizes the right-hand side of the Bellman equation. However, different from the retiree, he also chooses optimal labor supply.

There are different ways how we can implement this two-dimensional optimization problem. We have choosen to break it up in two nested optimization problems. In the outer loop, we optimize over the next-period wealth $a'$ just like in the case of the retiree. In the inner loop, we compute the optimal labor supply given this and next-period wealth, $a$ and $a'$ using the first-order condition of the worker with respect to his labor supply:

$ l = \frac{1}{1+\gamma} \left( 1-\frac{\gamma}{( 1-\tau) w} \left( \psi + (1+r) a - a'\right) \right)$

where consumption is given by

$c = (1-\tau) l w + (1+r) a - a'.$

We also have to impose the constraint $l\ge 0$.

The implementation of this nested optimization is straightforward. The function "value2(x)" computes the right-hand side of the Bellman equation for next-period wealth $a'=x$. The present period wealth is given by the global variable "k0". With the help of these variables, $a'$ and $a$, and the factor prices $w$ and $r$, we are able to compute the optimal labor supply $l$. We also have to check if $l\ge 0$. If not, we set it equal to zero, $l=0$. In order to compute the Bellman equation, we also have to determine the value of the next-period value function at age $i+1$ and wealth $a'=x$. We again interpolate either linearly or with a cubic spline. Notice that at age $s=40$ (corresponding to $i=39$ in the iteration over the age of the worker), the next-period value function is the value function of the retiree during his first period of retirement. In this case, we have to interpolate $vw0=vr[:,0]$; otherwise, the next-period value function is given by $vw0=vw[:,i+1]$.

In the outer loop, we just use golden section search with respect to next-period wealth $a'$. This is implemented in the line

"awopt[l,i-1] = GoldenSectionMax(value2,ax,bx,cx,tol1)"

The value function of the worker at ages $i=39,\ldots,0$ is computed in the next part of the program.

Again, we have plotted the value function and the policy functions for the worker at age $i=10$. The behavior of the value function, optimal savings and consumption is qualitatively the same as in the case of the retired. Consumption becomes a more increasing function of wealth, however, as the non-negativity constraint on labor becomes binding.

In addition, the bottom figure above displays the optimal labor supply of the worker. Notice that labor supply is a decreasing function of wealth due to the income effect and that labor supply becomes zero at high wealth levels approximately equal to 9.1.

Computation of the Age-Profiles of Wealth and Labor and Aggregation

With the help of the policy functions for optimal next period wealth $a'(a)$ and labor supply $l(a)$ we are able to compute the life-cycle profiles of wealth and labor by forward iteration. We start at age 1 (corresponding to $i=0$ in the program) where the newborn worker holds wealth level zero, $a^1 = 0$. The optimal next-period wealth $a^2$ and labor supply $l^1$ are stored in the policy functions $awopt[0,0]$ und $nopt[0,0]$.

We continue like this iterating over age $i=2,\ldots$. Since $a^2$ (and, similarly, $a^3$, $a^$$, $\ldots$) might not be a grid point of the policy function $awopt[:,1]$, we need to interpolate between grid points using

"aw_polate = interpolate.interp1d(a,awopt[:,j], fill_value='extrapolate')"

The optimal present labor supply and next-period wealth are stored in the variables $kgen[:]$ and $ngen[:]$. We also compute consumption at age $i$ and assign it to the variable $cgen[:]$. The life-cycle profiles of wealth $a$, labor $l$ and consumption $c$ are computed with the help of the following code:

A description of the life-cycle profiles is provided in our textbook. With the help of the wealth-age profile we are able to compute aggregate capital. With our demographic structure where each generation is of equal size and total population is normalized to one, aggregate capital is equal to average wealth in the economy. Therefore, we can simply compute it with the help of the command:

knew = np.mean(kgen)

Similarly, we know that we have 40 cohorts of workers with a measure of 1/60 each so that aggregate labor is presented by

nnew = np.mean(ngen)*2/3

Upate of Aggregate Variables $K$ and $L$

FInally, we update the aggregate variables $K$ and $L$ by using a weighted average of these variables in the last and present outer iteration. The updating parameter $phi$ is set equal to 0.8 (see above in the setting of the numerical parameters). We compute a new capital stock equal to $K=0.8173$ which we use in the next iteration in the outer loop.

The aggregate variables converge smoothly to their steady state values and the program "Rch91v.py" converges quickly after a couple of minutes.

In the remaining part of Chapter 9 and in Chapter 10, we will study much more complex OLG models where the computation becomes much slower and where the value function is a function of a higher number of (continuous) state variables.

If you enjoyed this tutorial on Python code and would like to find out more about the economics you might find it helpful to watch my youtube graduate course on 'Social Security in Dynamic General Equilibrium':

https://www.youtube.com/watch?v=0Lu2KAWLQHE&list=PL-anmMgSYtuFJu4A1biSqpKk-Oh5woKQn

Comments? Suggestions? Send an email to: Burkhard.Heer@wiwi.uni-augsburg.de

Last update: February 3, 2021