This post explores a core challenge in the investment advisory landscape: forecasting the price of financial instruments — at the intersection where statistics, machine learning, and regulatory frameworks converge.
Even when developing decision-support tools for advisers, regulatory considerations are inseparable from the modeling process.
Why? Because whether supporting client decision-making, building ML-powered algorithms, or surfacing insights, it's essential not only to ensure methodological soundness — but also to comply with regulatory expectations.
Laws like the EU AI Act, frameworks such as NIST RMF for AI, and core fiduciary principles like Fiduciary Duty demand transparency, explainability, and risk governance.
⸻
🎯 Complexity, Forecasting, and the Bias–Variance Tradeoff
One influential paper — "The Virtue of Complexity in Return Prediction" — offers a valuable perspective on the relationship between model complexity and predictive accuracy, as framed by the classic bias–variance tradeoff.
🔴 Bias: Over-simplified models tend to underfit reality — producing consistent yet inaccurate forecasts. (For instance, pricing housing based only on distance from the city center. In finance, consider the Fama–French 5-factor model estimated via linear regression.)
🔵 Variance: Overly complex models tend to overfit — capturing noise and generating unstable predictions. (Example: an outlier transaction influenced by personal motivations, rather than market forces.)
💡 The optimal balance is typically found where the combined error from bias and variance is minimized.
However, the central argument of the paper challenges this view: what if complexity, properly managed, enhances prediction?
With modern computing capabilities and abundant data, it's increasingly feasible to develop complex models that are not only accurate, but also stable and explainable. In this context, complexity becomes an asset — not a liability.
While the paper has its critics, it serves as a useful segue into the broader discussion around regulatory modeling frameworks.
⸻
🧭 Key Takeaways
Machine learning and AI are reshaping the investment advisory value chain — across:
❗ Client interaction
❗ Data ingestion and processing
✅ And most critically — generating economic insights.
And this is precisely the layer becoming more complex — as models move beyond linearity and simple fundamental assumptions.
This paper (and others like it) opens the hood on predictive modeling — especially in asset pricing and advisory contexts.
Take explainability for instance — a rising regulatory requirement. It's easier in simple models, but what if simplicity comes at the cost of predictive power? These are real-world questions we're only beginning to answer.
The challenge isn't just building accurate models — it's building accurate models that meet regulatory standards for transparency and explainability while maintaining predictive power.