The problem
ES futures don't trade as one continuous instrument. They trade as quarterly contracts that expire in March, June, September, and December, and each is the actively traded contract for only about three months. Any analysis longer than a quarter needs those contracts stitched into one series.
Commercial data feeds do this for you, but they make the key decisions silently: when to switch contracts, how to handle the price jump between them, and what to do with missing data. Those choices flow directly into every level and statistic built on top. I wanted to make them explicitly, understand their consequences, and be able to defend each one.
What I built
A Python pipeline that runs as a daily batch, in five stages:
- Pulls one-minute bars for each individual contract from the Massive (formerly Polygon.io) API. It caches each contract locally and requests only the data it's missing, within a free-tier limit of five calls per minute shared across my other projects.
- Decides when to roll from one contract to the next, using actual trading volume rather than a calendar rule.
- Stitches the contracts into one continuous series, switching only at the start of a trading session, so no session's highs, lows, or volume ever mixes two contracts' prices.
- Repairs the timeline onto a complete one-minute grid, removing scheduled market closures and marking filled gaps so they can never distort a calculation.
- Publishes the series and daily reports to a shared PostgreSQL database and a synced folder. The reports include each day's value area (the price range where most of the day's volume traded) and session highs and lows.
The finding: the textbook roll date was wrong for my data
The standard answer to "when does ES roll?" is the CME roll date: the Thursday eight days before expiration. My first versions rolled on a fixed calendar date close to it.
When I measured volume across all seven rolls from March 2025 to September 2026, the data said otherwise. On roll Thursday, the incoming contract held only 3–10% of the day's volume, and no more than 25% the next day. Rolling on the calendar would have built several days of every roll week from a thinly traded contract, distorting exactly the volume-based levels the dataset exists to produce.
The series switches at the start of Monday's session, so no session ever mixes two contracts.
So I changed the rule. The series now rolls at the first session, on or after the CME date, in which the new contract outtrades the old one. In every quarter, that crossover came on Monday of expiration week, four days after the textbook date.
A finding that contradicts conventional wisdom deserves suspicion, so I checked it against an independent source. TradingView's continuous ES contract rolled on the same session in both quarters I checked, June and September 2026. That check also exposed a subtler problem: my first volume-based version switched one session too late, which would have pushed one day's levels per quarter off by the full price gap between contracts, roughly 50 to 68 points. Aligning with the chart I actually trade on fixed it.
Decisions I can defend
Real prices, not adjusted ones
Most continuous series remove the price jump at each roll by adjusting history, but adjusted prices never actually traded. My levels have to line up with live charts, so the series keeps real prices and documents the jump at each seam instead.
No silent fixes
Filling gaps can hide real data loss, so suspicious holes get investigated against the source before anything is patched. Two were genuine API gaps, about 1,100 and 650 missing minutes, and I backfilled them. A third looked identical but was real: the market closed early on January 9, 2025 for the national day of mourning for President Carter. That one stayed as it was.
Decisions that don't change history
Once a roll is decided, new data can never move it. If the cached data is incomplete, the run stops with an error rather than write a truncated series.
Only finished sessions get reported
A day's value area is published only after its session closes, so a partially formed profile never passes as final.
What it powers
This dataset is the shared foundation for the rest of my market analysis work.
ES Market Profile indicator
A TradingView indicator I use every trading day, plotting the prior session's value area and key session levels.Private
Level interaction study
Measures how price reacts at key levels: rejection, breakout, or chop. Each level is only used from the moment it was knowable in real time, so the results contain no hindsight.level_interactions.py
Triple-barrier trade scorer
Scores any trade idea by whether its profit target or its stop would have been hit first, giving every strategy an apples-to-apples comparison.triple_barrier.py
Strategy research
Backtests of initial-balance breakouts, premarket setups, and weekly value-area behavior, all built on the same data.ib_breakout_test.pyweekly_value_area.py
Shared market database
The PostgreSQL database this project writes to, also used by my equity scanner.Private
The indicator, the rest of the strategy research, and the shared database and equity scanner code are private. I'm glad to walk through any of them in an interview.
Technical documentation
The full methodology, covering roll logic, session definitions, gap handling, and the value-area algorithm, is in the project's pipeline documentation.