Feature Engineering Finance: Building Inputs That Predict Returns

Key Takeaways

• Good features are stationary, comparable across assets, and encode known economic effects like momentum reversal or earnings surprise direction.
• A carefully chosen set of 10 to 20 features typically produces out-of-sample IC of 0.03 to 0.05; adding hundreds of weak features almost always hurts.
• Using fiscal-period end dates instead of announcement dates for fundamental data injects look-ahead bias that inflates backtest returns.
• Purged cross-validation and a label-horizon embargo are mandatory when feature lookback windows overlap with prediction-label windows.

What It Is

A feature is a numeric quantity derived from source data and supplied as an input to a model. Good features encode domain knowledge the model would otherwise have to rediscover from raw prices. Examples include 20-day realized volatility, the z-score of analyst revisions, a normalized earnings surprise, or a rolling correlation to the market.

In conventional machine-learning applications such as image classification, modern deep networks can often learn useful features from raw pixels. In finance, the signal-to-noise ratio is so low that engineered features outperform end-to-end learning in the large majority of practical settings. This is a central argument of Lopez de Prado's Advances in Financial Machine Learning (2018).

The Intuition

A price chart is not a prediction. It is a record. The job of feature engineering is to compress that record into quantities that are stationary, comparable across assets, and economically meaningful.

Three ideas drive good feature design:

1. Stationarity. Raw prices drift and explode; returns and log-differences are closer to stationary. Models assume (often implicitly) that the statistical properties of inputs do not change over time.
2. Comparability across assets. A $500 stock and a $5 stock must be described in the same units. Returns, ratios, z-scores, and percentiles handle this.
3. Encoding of known effects. If you know that earnings announcements move stocks, build a feature that measures proximity to earnings and another that measures the sign of the last surprise. The model does not have to infer it.

How It Works

Typical feature categories in a quant workflow:

• Return-based: lagged returns over 1, 5, 20, 60, and 252 days; momentum (12-month minus 1-month); reversal (1-month).
• Volatility-based: realized volatility at multiple windows; GARCH-implied volatility; realized range (high-low over close).
• Fundamental ratios: EV/EBITDA, P/E, P/B, ROIC, gross margin, each cross-sectionally z-scored within sector.
• Analyst and positioning: revisions, recommendation changes, earnings surprises, short interest changes, insider transactions.
• Market microstructure: bid-ask spread, Amihud illiquidity, order-flow imbalance, put-call ratio.
• Regime and macro: VIX level, credit spread, yield-curve slope, realized market volatility.

Standard transformations include:

z_score_t   = (x_t - rolling_mean) / rolling_std
log_diff_t  = log(x_t) - log(x_{t-1})
rank_t      = cross_sectional_rank(x_t)  over the universe at time t
winsorize   = clip values at 1st and 99th percentile

For model pipelines, Lopez de Prado emphasizes two operations that separate credible research from curve-fitting:

1. Purged cross-validation. Remove training observations whose labels overlap with the validation set's feature window. Necessary because financial labels often span days or weeks.
2. Embargo. After a validation window ends, skip a gap (typically the label horizon) before allowing training data again, to prevent leakage from serial correlation.

Worked Example

Build a one-month-ahead return forecast for US large caps. A minimal feature set might include:

• mom_12_1: return from t-252 to t-21 (classic momentum with a one-month gap).
• rev_1m: return over the last 21 days (short-term reversal).
• vol_60: realized volatility over last 60 days, winsorized.
• ey: trailing earnings yield, cross-sectionally z-scored within GICS sector.
• rev_surp_z: analyst-revision z-score over last 30 days.
• short_int_chg: change in short interest as a fraction of float.
• vix_level: a regime control.

At each month-end rebalance, the pipeline computes these features for every stock in the universe, applies winsorization and sector-neutral z-scoring where appropriate, and feeds the resulting matrix into a gradient-boosted tree that outputs a predicted one-month return. Walk-forward backtesting with purged folds and a 21-day embargo evaluates performance.

The economic test is whether the feature set survives in-sample to out-of-sample. On typical US equity data, a thoughtful set of 10-20 features produces a stable out-of-sample information coefficient around 0.03-0.05. Adding hundreds of weakly motivated features almost always hurts after the embargo and cost assumptions.

Common Mistakes

• Look-ahead bias in fundamental data. Reported earnings become available a few weeks after quarter-end. A feature built from fiscal-period data dated by period-end rather than announcement-date leaks the future. Always use point-in-time databases.

• Survivorship bias in the universe. Backtesting on today's S&P 500 components quietly excludes delisted firms, producing inflated returns and stable features that look stationary only in hindsight.

• Ignoring feature-label overlap. A 20-day ahead label and a 20-day lookback feature share observations if not purged. Lopez de Prado's purging and embargo mechanics exist precisely for this.

• Over-counting correlated features. Adding ten variants of momentum does not increase information; it inflates dimensionality and noise. Cluster correlated features and keep the strongest representatives.

• Confusing statistical significance with economic value. Harvey, Liu, and Zhu (2016) catalog over 300 published equity-return factors and show that after multiple-testing corrections, most are indistinguishable from noise. Demand economic rationale alongside any statistical edge.

Frequently Asked Questions

Q: What is feature engineering in finance in simple terms? It is the process of transforming raw price, fundamental, and alternative data into numerical inputs that a model can learn from, making the inputs stationary, comparable across assets, and free of the look-ahead biases that would make backtests misleading.

Q: How does feature engineering in finance affect investment decisions? The quality of features determines model performance. A thoughtfully engineered 10-feature set typically outperforms an ad-hoc 100-feature set, because good features encode known economic effects and reduce the noise that leads to overfitting.

Q: What is a real-world example of feature engineering in finance? Classic momentum is engineered as the 12-month return excluding the most recent month, not the raw 12-month return, because the last month's return shows short-term reversal that would cancel the momentum signal if included.

Q: How can investors avoid mistakes in feature engineering in finance? Use point-in-time databases for fundamental data, apply purged cross-validation with a label-horizon embargo, winsorize extreme values at the 1st and 99th percentiles, and demand an economic rationale for every feature before including it in a model.

Q: How is feature engineering in finance different from using raw price data? Raw prices are non-stationary and not comparable across assets. Engineered features like z-scored earnings yield, normalized momentum, and cross-sectional percentile ranks are stationary and comparable, which is what machine-learning models require to generalize beyond the training sample.

Sources

1. Lopez de Prado, M. (2018). Advances in Financial Machine Learning. Wiley. https://www.wiley.com/en-us/Advances+in+Financial+Machine+Learning-p-9781119482086
2. Hastie, T., Tibshirani, R., and Friedman, J. (2009). The Elements of Statistical Learning, 2nd edition. Springer. https://hastie.su.domains/ElemStatLearn/
3. Harvey, C.R., Liu, Y., and Zhu, H. (2016). "...and the Cross-Section of Expected Returns." Review of Financial Studies, 29(1), 5-68. https://doi.org/10.1093/rfs/hhv059
4. Arnott, R., Harvey, C.R., Markov, V., and Markowitz, H. (2019). "Backtesting." Journal of Portfolio Management, 45(7), 93-111. https://doi.org/10.3905/jpm.2019.45.7.093

Disclaimer

This article is educational content only and is not financial advice. Nothing here is a recommendation to buy, sell, or hold any security. Consult a licensed advisor before making investment decisions.