Chapter 10 · Stochastic Control and the Hamilton–Jacobi–Bellman Equation
Part III · Optimization, Control, and Learning
Chapter at a glance
Modern Finance Laboratory · Module 10 · Week 10
Module 10 has four panels. The Bellman panel runs the discrete recursion (10.3) on a visible grid—the user watches value propagate backward and the argmax policy paint itself state by state—with Chapter 5’s collar tree loadable as the “no-control” special case. The verification panel reproduces Figure 10.1 live: enter any candidate policy for the consumption problem and watch its performance process’s mean bleed at its HJB deficit, flatlining exactly at the optimum. The spending panel exposes the (𝛾, 𝜌) ↦→ 𝜈 ∗ surface with Meridian’s point marked, plus sustainability fans for any rule. The execution panel is a transition-desk simulator: slide
Learning Outcome Statements
LOS 10.1 Formulate financial decision problems as stochastic control: state, control, admissibility, and objective.
LOS 10.2 State and prove the dynamic programming principle in discrete time by backward induction, and connect it to Chapter 5’s pricing recursions.
LOS 10.3 Derive the Hamilton–Jacobi–Bellman equation from the dynamic programming principle and Itô’s formula, and read its terms.
LOS 10.4 State and prove the verification theorem, and use its supermartingale logic to certify or reject candidate policies.
LOS 10.5 Solve the Merton consumption–portfolio problem by HJB and translate the solution into an endowment spending rule.
LOS 10.6 Solve the optimal execution problem and design a transition schedule on the cost–risk frontier.
Laboratory · Module 10 (book §10.11)
Module 10: The Control Room
Course Website · Week 10
Guided experiment (supports LOS 10.2–10.6). (i) In the Bellman panel, solve a three-date, two-action consumption toy by hand, then check the panel’s recursion state by state. (ii) In the verification panel, enter 𝜋 = 120% and 𝑐 = 8% 𝑤 and record each policy’s bleed rate; then find the flat line. (iii) Locate Meridian on the spending surface; report 𝜈 ∗ at (𝛾, 𝜌) = (2, 5%) and the implied 𝜌 of the 4.5% rule; simulate both the 4.5% and 6% rules to thirty years. (iv) On the execution panel, price the $600M transition at 𝜅 ∈ {0.05, 0.2, 0.4} and choose a point on the frontier, writing one sentence justifying the exchange rate you accepted.
Downloads:
Exercises
Exercises are grouped A–E throughout the book: A concept checks, B computations, C proofs and extensions, D modeling and application, E laboratory. Starred exercises (∗) are on the advanced track.
Part A — Concept checks
Exercise 10.1. Cast each as a control problem—state, control, dynamics, objective, and whether the horizon is finite or infinite: (a) a pension fund de-risking as its funded ratio improves; (b) a market maker managing inventory; (c) a treasury smoothing FX hedges. One line per ingredient.
Exercise 10.2. Explain in at most two sentences each: why the supremum in (10.4) sits inside the PDE rather than outside the expectation; why pricing (Chapter 8) is the no-control special case; and why the verification proof needs the supermartingale direction for all controls but the martingale direction only for one.
Exercise 10.3. The trustee asks: “If the optimal spending rule is proportional, why do universities smooth payouts over trailing averages?” Give the honest two-part answer: which assumption of Proposition 10.3 smoothing responds to, and what cost the smoothing rule pays in the theorem’s terms.
Part B — Computations
Exercise 10.4. Solve by hand the three-date consumption toy: wealth 𝑤 ∈ {1, 2, 3, 4}, actions consume 1 or 2 (utility ln 𝑐), remaining wealth grows by +1 or −1 with equal probability (floored at 0, game over), no discounting. Produce the two value tables and the policy; verify one entry against (10.3).
Exercise 10.5. With the chapter’s market parameters, tabulate 𝜈 ∗ (𝛾, 𝜌) over 𝛾 taking values 1.5, 2, 3 and 𝜌 taking values 3%, 4.23%, 5%; identify the entries reproducing 4.88% and 4.5%; and compute post-spending median growth for each rule at its own optimal 𝜋 ∗ .
Exercise 10.6. For the transition trade: verify 𝜂 = 5 × 10−6 from the desk’s oneday estimate; compute expected impact cost and cost-risk for 𝜅 ∈ {0.05, 0.1, 0.2, 0.4} (reproducing Figure 10.3(b)’s points); and find the 𝜅 whose expected cost is exactly double VWAP’s, reporting the risk reduction purchased.
Exercise 10.7. A wealth of 100 is managed under Proposition 10.3 with 𝛾 = 2, 𝜌 = 4.23%. Compute the year-one expected payout in dollars; the probability the payout falls in year two (i.e. that wealth declines net of spending); and the spending’s own volatility—making concrete the trustee complaint of
Exercise 10.3.
Part C — Proofs and extensions
Exercise 10.8. Prove that the finite-horizon value functions of Theorem 10.1 satisfy the monotonicity and concavity a one-line induction can extract: if 𝑔 and 𝑓 (·, 𝑢) are increasing and concave in the state and the controlled transition is stochastically mono- 10.12 Exercises 235 tone and concavity-preserving, then every 𝑉𝑛 is increasing and concave. (This is the structure that makes threshold policies optimal—Chapter 11’s theme.)
Exercise 10.9. Complete the Merton derivation: perform the two inner maximizations of Section 10.5 in detail, substitute the CRRA ansatz 𝑉 = 𝐾𝑈𝛾 (𝑤), reduce the HJB to the scalar equation for 𝐾, solve it to obtain 𝜈 ∗ = 𝐾 −1/𝛾 as in Proposition 10.3, and state precisely where 𝜈 ∗ > 0 is used to make the verification theorem’s integrability hypotheses hold.
Exercise 10.10. Prove the 𝜅 → 0 and 𝜅 → ∞ limits of (10.6) (straight line; instantaneous liquidation), derive the expected-cost functional in closed form, ∫ 𝑇 𝜂 𝑥¤ ∗ (𝑡) 2 𝑑𝑡 = 0 𝜂 𝑥02 𝜅 h 2 coth(𝜅𝑇) + 𝜅𝑇 sinh2 (𝜅𝑇) i , and verify the two frontier endpoints of Figure 10.3(b).
Exercise 10.11. ∗ Set up the execution problem as a genuine stochastic control problem with state (𝑥, 𝑆) (position and price with permanent impact 𝜃𝑣), write its HJB, and show that with quadratic costs the value function is quadratic in the state and the optimal rate is linear—reducing HJB to Riccati ODEs; confirm that with 𝜃 = 0 the schedule collapses to (10.6).
Part D — Modeling and application
Exercise 10.12. Write the trustee memo deriving the spending rule: the objective as the endowment’s charter formalized; the rule (10.5) with Meridian’s numbers and the implied 𝜌 = 4.23%; the sustainability evidence of Figure 10.2(b) for 4.5% versus 6%; and the three honest limits of the derivation with one sentence each on how practice patches them.
Exercise 10.13. Write the desk instruction for the $600M transition: the objective and the two calibrations (𝜂, 𝜎d ) with their provenance; the chosen 𝜅 with the frontier exchange rate it accepts, stated in dollars; the monitoring plan (realized shortfall versus plan, and the trigger for re-planning); and the two model caveats (permanent impact, impact-law shape) with their expected direction of bias.
Part E — Laboratory
Exercise 10.14. (Laboratory Module 10; supports LOS 10.2–10.6.) Perform the fourpart guided experiment of Section 10.11. Submit: (a) the hand-solved toy against the Bellman panel; (b) the two bleed rates and the flat line; (c) Meridian’s point on the spending surface with both simulations; (d) your chosen frontier point with its onesentence justification. 236 10 Stochastic Control and the Hamilton–Jacobi–Bellman Equation
Exercise 10.15. (Course Website, Week 10, Notebook 10.) Extend the notebook: implement the discrete Bellman recursion for the consumption problem on a wealth grid fine enough to confirm Proposition 10.3’s 𝜈 ∗ to two decimals; then break one hypothesis—impose a no-leverage constraint 𝜋 ≤ 100% at 𝛾 = 0.8—and report where the numerical policy departs from the closed form and what the value cost of the constraint is.
Full solutions are distributed to instructors in the Instructor’s Manual, Chapter 10; they are not posted here. Problem-set files are on the Course Website, Week 10.