System Design · hard · Asked at Uber

How would you design a feature store for ML?

Asked in ML Engineer interviews, in the System Design round.

Short answer

Offline (batch) + online (low-latency) stores, feature registry, point-in-time correctness.

How to answer it

A feature store solves two problems: training and serving computing the same feature differently, and features that leak the future into training. The design follows from those.

# training set with point-in-time correctness
training = store.get_historical_features(
    entity_df=labels[["user_id", "event_ts", "label"]],
    features=["user:orders_30d", "user:avg_basket_90d", "item:ctr_7d"],
)

Operational concerns worth naming: backfills when a feature definition changes (recompute history, version the feature); monitoring for drift between offline and online values; and TTLs in the online store so a stale feature does not serve forever.

Say when not to build one: a single model with batch predictions does not need it. The store pays off at several models sharing features, or any real-time serving.

Related questions

Practice this for real