Chapter 12 · Filtering, Learning, and Hidden States
Part III · Optimization, Control, and Learning
Chapter at a glance
Modern Finance Laboratory · Module 12 · Week 12
Module 12 has four panels. The update panel animates Proposition 12.1: prior and signal as movable Gaussians, the posterior forming as their precision-weighted product, with the stake’s $153M/$148M fusion preloaded. The Kalman panel runs Theorem 12.2 on simulated streams with sliders for (𝑎, 𝑞, 𝜎𝑣 ), displaying state, estimate, band, gain path, and the innovation autocorrelogram—whiteness as a live diagnostic. The desmoothing panel loads mark series (simulated or the reader’s own), estimates 𝜃 by AR regression, applies (12.7), and reruns the two-signal stake filter of Figure 12.2. The drift panel reproduces Figure 12.3 with prior sliders, rep
Learning Outcome Statements
LOS 12.1 Formulate hidden-state problems: state dynamics, observation equations, and the observation filtration on which estimates must be adapted.
LOS 12.2 Derive the Gaussian conjugate update and interpret precision weighting.
LOS 12.3 Derive and prove the scalar Kalman filter, interpret gains and innovations, and compute steady states via the Riccati recursion.
LOS 12.4 De-smooth appraisal-based return series: recover true volatility and betas from smoothed marks, and quantify the correction.
LOS 12.5 Construct an optimal nowcast for an illiquid asset by fusing stale marks with noisy market proxies, with credible bands.
LOS 12.6 Model parameter uncertainty as filtering: quantify the pace of learning about expected returns and derive shrinkage as the optimal response.
Laboratory · Module 12 (book §12.10)
Module 12: The Learning Machine
Course Website · Week 12
Guided experiment (supports LOS 12.2–12.6). (i) On the update panel, verify the $151.5M ± $3.3M nowcast and find the proxy noise level at which the weights equalize. (ii) On the Kalman panel, mis-set the assumed 𝜎𝑣 by a factor of four and document the innovation autocorrelation that betrays it. (iii) On the de-smoothing panel, recover 𝜃 = 0.8 and the 18% from a simulated 6% mark series; report the RMSE trio (marks
Downloads:
Exercises
Exercises are grouped A–E throughout the book: A concept checks, B computations, C proofs and extensions, D modeling and application, E laboratory. Starred exercises (∗) are on the advanced track.
Part A — Concept checks
Exercise 12.1. For each, name the hidden state, the observations, and whether the linear-Gaussian class plausibly applies: (a) nowcasting GDP from monthly indicators; (b) inferring a competitor fund’s positions from its returns; (c) tracking credit quality from bond prices; (d) detecting a regime change in volatility.
Exercise 12.2. “An optimal forecaster’s errors are unforecastable.” Identify the clause of Theorem 12.2 that formalizes this; explain its relationship to the martingale property of Chapter 5; and state what an autocorrelated innovation series proves about the filter producing it.
Exercise 12.3. The private-markets head objects: “Our appraisers are careful professionals; the low volatility is real.” Draft the three-sentence reply that concedes the care, locates the issue in the process (12.6) rather than the people, and names the observable fingerprint that adjudicates.
Part B — Computations
Exercise 12.4. Verify the nowcast fusion: prior N(153, 42 ), signal N(148, 62 ) gives posterior mean 151.5, sd 3.33, weight 0.69; then add a second independent proxy at N(150, 52 ) and recompute all three quantities.
Exercise 12.5. Reproduce the de-smoothing arithmetic: from 𝜃 = 0.8 and 𝜎𝑀 = 6%, the factor 3 and 𝜎𝑉 = 18%; then tabulate 𝜎𝑉 over 𝜃 ∈ {0.5, 0.65, 0.8, 0.9} and identify 274 12 Filtering, Learning, and Hidden States the 𝜃 at which the committee’s “12% real-assets volatility” assumption would have been the truth.
Exercise 12.6. Run the scalar Riccati recursion by hand for three steps: 𝑎 = 1, 𝑞 = 0.0081, 𝑐 = 1, 𝜎𝑣2 = 0.0025, 𝑃0 = 𝑞; report (𝑃𝑡 , 𝐾𝑡 ) each step and the steady state, and confirm against Figure 12.2(b)’s limit.
Exercise 12.7. Using (12.8) with 𝜎 = 17%: compute the posterior sd of 𝜇 at 𝑇 = 5, 20, 40; the years of data needed for a ±1% band; and, with prior N(6%, 2%2 ) and a twenty-year sample mean of 9%, the posterior mean—explaining to a trustee why it sits so far from the sample.
Part C — Proofs and extensions
Exercise 12.8. Complete Proposition 12.1: derive the joint distribution of (𝑋, 𝑌 ), apply the projection formula of Proposition 3.7, verify both algebraic forms of the posterior mean, and prove the claim that posterior variance is strictly decreasing in each additional independent signal.
Exercise 12.9. Prove the innovations claims of Theorem 12.2 in full: E[𝜀 𝑡 |H𝑡 −1 ] = 0; Var(𝜀 𝑡 ) = 𝑐2 𝑃𝑡− + 𝜎𝑣2 ; pairwise independence across 𝑡; and the innovations representation—that H𝑡 = 𝜎(𝜀1 , . . . , 𝜀 𝑡 ), so the observation stream and its surprise stream carry identical information.
Exercise 12.10. Derive Proposition 12.3(iii) two ways: from the AR(1) variance formula as in the text, and directly from the recovery formula (ii) by computing Var (𝑟 𝑡𝑀 −𝜃𝑟 𝑡𝑀−1 )/(1−𝜃) using the AR autocovariances—and reconcile the appearance of the cross term.
Exercise 12.11. ∗ (Certainty equivalence.) In the LQ setting of Exercise 10.11 with the state observed only through Gaussian noise, prove that the optimal control is the b𝑡 : write the HJB for the full-information LQ rule applied to the Kalman estimate 𝑋 conditional-mean state (whose innovations, by Theorem 12.2, are the new Brownian driver), and show the additional estimation-variance term enters the value additively without touching the argmax—the separation principle.
Part D — Modeling and application
Exercise 12.12. Write the two-page memo resolving agenda item two: the AR fingerprint and estimated 𝜃; the de-smoothed volatility (12.7) and corrected beta with their consequences for the Chapter 9 policy inputs; the fused nowcast with its band and the standing reporting format you propose (mark, proxy, posterior, weights); the two caveats (noise amplification; 𝜃 sensitivity) with a sensitivity row each; and the single sentence 12.11 Exercises 275 for the minutes on how the committee should henceforth receive appraisal-based numbers.
Exercise 12.13. The CIO asks whether the drift-learning framework changes the Chapter 9 ratification. Write the one-page answer: what (12.8) implies for the confidence attached to the 7% equity assumption; how the policy’s implied 𝛾 = 1.74 interacts with parameter uncertainty (the posterior-variance term in the Bayesian Merton weight); and whether the committee’s slow-update governance is, in filtering terms, a feature or a bug.
Part E — Laboratory
Exercise 12.14. (Laboratory Module 12; supports LOS 12.2–12.6.) Perform the fourpart guided experiment of Section 12.10. Submit: (a) the fusion screen with the equalweight noise level; (b) the innovation autocorrelogram under the mis-set filter; (c) the recovered (𝜃, 𝜎𝑉 ) and RMSE trio; (d) the 𝜏0 readout and one paragraph on the OU-drift tracking.
Exercise 12.15. (Course Website, Week 12, Notebook 12.) Extend the notebook: implement the two-signal stake filter with a time-varying proxy quality (noise doubling in simulated crisis quarters); report how the steady-state gains reallocate; and write three sentences on what this implies for relying on listed proxies exactly when markets are stressed—a preview of Chapter 13’s concerns.
Full solutions are distributed to instructors in the Instructor’s Manual, Chapter 12; they are not posted here. Problem-set files are on the Course Website, Week 12.