In 2019, a systematic futures trader we know ran a mean-reversion strategy on five years of daily OHLCV data. The backtest showed a Sharpe ratio of 2.1. After six months live, the strategy bled 18% and was shut down. The cause was not a flawed alpha signal. It was math: the backtest assumed every order filled at the bar close price. In live trading, a $5 million position in thinly traded soybean futures does not fill at "close." It fills at a price that moves against you with every contract you buy.

This is the slippage problem. It is not a footnote. It is the difference between a strategy that survives contact with markets and one that does not.

This article builds a production-grade market impact model from scratch, walks through order book replay mechanics, and demonstrates how to wire real-time depth data into a tick-level backtesting loop. All code is directly runnable with a TickDB API key.

1. Why Your Backtest Is Lying to You

A standard backtest using OHLCV bars applies a flat slippage assumption — typically 5–10 basis points — to every fill. This is equivalent to assuming every market is equally liquid and every order is equally disruptive. Neither is true.

The problem compounds across three dimensions:

Bar aggregation destroys intrabar price discovery. When you buy at the bar high, you are assuming zero market impact. In reality, a large order consumes the bid side of the order book sequentially. Each fill is at a progressively worse price.

Fixed slippage ignores order size effects. A $50,000 order in Apple stock and a $50,000 order in a small-cap OTC stock do not produce the same slippage. The order book depth at each price level is radically different.

Bid-ask spread is not the only cost. The spread is the cost of an infinitesimally small order. As order size grows, the price impact — the movement caused by your own order — dominates the spread cost.

A rigorous backtest must answer one question: given my order size and the current order book depth, what is the expected fill price? That question requires a market impact model and order book data at tick-level granularity.

2. Theoretical Foundation: The Almgren-Chriss Framework

The canonical model for market impact comes from Almgren and Chriss (2000). The core insight: total transaction cost is a sum of two terms — a temporary impact term that decays over time, and a permanent impact term that reflects the information content of the trade.

The simplified form:

$$C(v) = \eta \cdot v + \gamma \cdot v^2$$

Where:

  • $v$ is the fraction of average daily volume (ADV) being traded
  • $\eta$ is the temporary impact coefficient (price movement per unit of volume)
  • $\gamma$ is the permanent impact coefficient

The linear term captures temporary liquidity consumption — the order eats through available bids or asks, and the market reprices. The quadratic term captures information leakage — the market interprets a large order as a signal.

For our purposes, we use an empirical variant that maps directly to order book depth:

$$SI = \sum_{k=1}^{N} \frac{\text{order_size} - \text{remaining_depth at level } k}{\text{depth at level } k} \times \text{spread}_k$$

Where $SI$ is the slippage in basis points and $N$ is the number of order book levels consumed by the order.

3. Order Book Data: The Foundation of Impact Simulation

To simulate impact, you need order book snapshots. TickDB provides the depth channel for real-time order book data:

Market Depth levels Update frequency
US equities L1 (best bid/ask) Sub-second
HK equities L1–L10 Sub-second
Crypto L1–L10 Sub-second

For backtesting, you reconstruct the order book by replaying depth snapshots at each timestamp, then compute the cumulative impact as the order is filled level by level.

The following section provides a data acquisition layer that fetches depth snapshots and prepares them for simulation.

import os
import time
import json
import random
import logging
from datetime import datetime, timedelta
from typing import Optional

import requests

logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s [%(levelname)s] %(name)s — %(message)s"
)
logger = logging.getLogger("order_book_replay")


# ─────────────────────────────────────────────────────────────────────────────
# Configuration
# ─────────────────────────────────────────────────────────────────────────────
API_KEY = os.environ.get("TICKDB_API_KEY")
if not API_KEY:
    raise EnvironmentError("Set TICKDB_API_KEY in your environment")

BASE_URL = "https://api.tickdb.ai/v1"

# Markets where L2–L10 depth is available (crypto and HK equities)
DEPTH_ENABLED_MARKETS = {
    "BTC.USDT", "ETH.USDT", "NVDA.US", "TSLA.US"
}


# ─────────────────────────────────────────────────────────────────────────────
# Standard error handler — mirrors production-grade API conventions
# ─────────────────────────────────────────────────────────────────────────────
def handle_api_error(response_data, status_code: int = 200):
    """Standard TickDB error handler with code-specific routing."""
    if status_code == 200 and response_data.get("code", 0) == 0:
        return response_data.get("data")

    code = response_data.get("code", 0)
    msg = response_data.get("message", "Unknown error")

    if code in (1001, 1002):
        raise ValueError(f"API auth error ({code}): {msg} — verify TICKDB_API_KEY")
    if code == 2002:
        raise KeyError(f"Symbol not found ({code}): {msg}")
    if code == 3001:
        retry_after = int(response_data.headers.get("Retry-After", 5))
        logger.warning(f"Rate limited — backing off {retry_after}s")
        time.sleep(retry_after)
        return None

    raise RuntimeError(f"API error {code}: {msg}")


# ─────────────────────────────────────────────────────────────────────────────
# WebSocket depth subscription with exponential backoff + jitter + heartbeat
# ─────────────────────────────────────────────────────────────────────────────
# ⚠️ This is a REST-based simulation. For production HFT workloads,
#    use aiohttp with the WebSocket endpoint. This module targets
#    backtest preparation and strategy prototyping.
def subscribe_depth_stream(symbol: str, duration_seconds: int = 60) -> list[dict]:
    """
    Polls the TickDB depth endpoint to simulate a live order book feed.
    In production, replace this with the WebSocket /v1/market/depth endpoint
    and handle real-time delta updates.

    Returns a list of depth snapshots: {"timestamp": ..., "bids": [...], "asks": [...]}
    """
    if symbol not in DEPTH_ENABLED_MARKETS:
        logger.warning(
            f"{symbol} — L2+ depth not available. Falling back to L1. "
            "Impact simulation accuracy will be reduced."
        )
        # L1 fallback handled by the endpoint automatically

    snapshots = []
    end_time = time.time() + duration_seconds
    retry_count = 0
    max_retries = 5
    base_delay = 1.0
    max_delay = 30.0

    headers = {"X-API-Key": API_KEY}

    while time.time() < end_time:
        try:
            response = requests.get(
                f"{BASE_URL}/market/depth",
                headers=headers,
                params={"symbol": symbol, "limit": 10},
                timeout=(3.05, 10)  # Connect timeout 3.05s, read timeout 10s
            )
            data = response.json()

            result = handle_api_error(data, response.status_code)
            if result is not None:
                snapshots.append({
                    "timestamp": time.time(),
                    "symbol": symbol,
                    "bids": result.get("bids", []),
                    "asks": result.get("asks", []),
                })
                retry_count = 0  # Reset on success
            else:
                # Rate-limited but handled internally; continue
                pass

            time.sleep(0.5)  # Poll every 500ms — simulate real-time cadence

        except requests.exceptions.Timeout:
            retry_count += 1
            delay = min(base_delay * (2 ** retry_count), max_delay)
            jitter = random.uniform(0, delay * 0.1)
            logger.warning(f"Timeout — retry {retry_count}/{max_retries} in {delay:.1f}s")
            if retry_count >= max_retries:
                raise RuntimeError(f"Max retries exceeded for {symbol}")
            time.sleep(delay + jitter)

        except requests.exceptions.RequestException as e:
            logger.error(f"Request error: {e}")
            raise

    logger.info(f"Collected {len(snapshots)} depth snapshots for {symbol}")
    return snapshots


# ─────────────────────────────────────────────────────────────────────────────
# Historical depth replay for backtesting
# ─────────────────────────────────────────────────────────────────────────────
def fetch_historical_depth(
    symbol: str,
    start_time: datetime,
    end_time: datetime,
    interval_seconds: int = 60
) -> list[dict]:
    """
    Reconstructs an order book timeline from TickDB's kline and trade data.
    For markets with full depth history (HK equities, crypto), extend this
    to call /v1/market/depth with historical timestamps.

    Note: TickDB does not provide historical depth snapshots for US equities.
    US equity backtesting uses L1 best bid/ask from kline data as the baseline.
    """
    headers = {"X-API-Key": API_KEY}

    # Fetch 1-minute klines as the baseline price reference
    response = requests.get(
        f"{BASE_URL}/market/kline",
        headers=headers,
        params={
            "symbol": symbol,
            "interval": "1m",
            "start_time": int(start_time.timestamp()),
            "end_time": int(end_time.timestamp()),
            "limit": 1000,
        },
        timeout=(3.05, 10)
    )

    data = response.json()
    klines = handle_api_error(data, response.status_code)

    if not klines:
        logger.warning(f"No kline data returned for {symbol} in the specified window")
        return []

    snapshots = []
    for k in klines:
        # Build a synthetic L1 depth snapshot from the kline close price.
        # The spread is estimated as 0.01% for liquid stocks; adjust per symbol.
        close_price = float(k["close"])
        spread_bps = 1.0  # 1 bps — adjust based on symbol liquidity
        half_spread = (spread_bps / 10000) * close_price / 2

        snapshots.append({
            "timestamp": k["close_time"] / 1000,
            "symbol": symbol,
            "bids": [[close_price - half_spread, 1000.0]],
            "asks": [[close_price + half_spread, 1000.0]],
            "close": close_price,
            "volume": float(k["volume"]),
        })

    logger.info(
        f"Reconstructed {len(snapshots)} synthetic depth snapshots for {symbol}"
    )
    return snapshots

The code above handles two scenarios: live depth streaming (for forward-testing) and historical reconstruction (for backtesting). Note the critical limitation in the docstring — US equity backtesting must rely on L1 kline data as a baseline because TickDB does not provide historical depth snapshots for US equities. This is not a product gap; it is a market data property. Building your impact model around this constraint is the right engineering decision.

4. Building the Impact Cost Model

With depth snapshots in hand, we can now implement the fill price calculator. The algorithm walks through each order book level and accumulates the filled quantity until the order is complete. The final weighted average price is the simulated fill price.

from dataclasses import dataclass


@dataclass
class FillResult:
    """Encapsulates the output of a market impact simulation."""
    order_size: float           # Original order quantity
    filled_size: float          # Actual quantity filled (may be < order_size)
    avg_fill_price: float       # Volume-weighted average fill price
    slippage_bps: float         # Slippage in basis points relative to mid
    impact_consumed_levels: int # Number of order book levels consumed
    unfilled_ratio: float       # Portion of order that could not be filled
    mid_price: float            # Reference mid price at order submission


def simulate_fill(
    order_book: dict,
    order_size: float,
    side: str = "buy"           # "buy" hits asks; "sell" hits bids
) -> FillResult:
    """
    Simulates order execution against a depth snapshot.

    The algorithm consumes the order book level by level, computing a
    volume-weighted average fill price. If the order exceeds available
    depth, the unfilled portion is returned in the result.

    Parameters
    ----------
    order_book : dict
        Depth snapshot from TickDB: {"bids": [[price, size], ...],
                                      "asks": [[price, size], ...]}
    order_size : float
        Number of contracts/shares to execute.
    side : str
        "buy" consumes the ask side; "sell" consumes the bid side.

    Returns
    -------
    FillResult
        Detailed fill breakdown for post-trade analysis.
    """
    levels = order_book.get("asks" if side == "buy" else "bids", [])

    if not levels:
        raise ValueError(f"Empty {'ask' if side == 'buy' else 'bid'} side — order book may be stale")

    # Reference mid price from the best bid/ask
    bids = order_book.get("bids", [])
    asks = order_book.get("asks", [])
    mid_price = (bids[0][0] + asks[0][0]) / 2 if bids and asks else levels[0][0]

    remaining = order_size
    total_cost = 0.0
    filled_size = 0.0
    levels_consumed = 0

    for price, depth in levels:
        if remaining <= 0:
            break

        fill_at_level = min(remaining, depth)
        total_cost += fill_at_level * price
        filled_size += fill_at_level
        remaining -= fill_at_level
        levels_consumed += 1

    if filled_size == 0:
        raise RuntimeError(f"Order could not be filled — insufficient depth at {side} side")

    avg_fill_price = total_cost / filled_size

    # Slippage: buy orders fill above mid; sell orders fill below mid
    if side == "buy":
        slippage_bps = (avg_fill_price - mid_price) / mid_price * 10000
    else:
        slippage_bps = (mid_price - avg_fill_price) / mid_price * 10000

    return FillResult(
        order_size=order_size,
        filled_size=filled_size,
        avg_fill_price=avg_fill_price,
        slippage_bps=slippage_bps,
        impact_consumed_levels=levels_consumed,
        unfilled_ratio=remaining / order_size if order_size > 0 else 0,
        mid_price=mid_price,
    )


# ─────────────────────────────────────────────────────────────────────────────
# Batch impact estimator: run simulation across a full order book history
# ─────────────────────────────────────────────────────────────────────────────
def run_impact_backtest(
    order_books: list[dict],
    order_size_func,            # Callable: (timestamp, price) -> order_size
    side: str = "buy"
) -> list[FillResult]:
    """
    Backtests a strategy against a sequence of order book snapshots.

    order_size_func: a function that generates order size at each timestamp.
                     Example: lambda ts, px: 5000 if px > sma(ts) else 0

    Returns a list of FillResult for each timestamp with an active order.
    """
    results = []
    for snapshot in order_books:
        if not snapshot.get("bids") or not snapshot.get("asks"):
            continue  # Skip stale snapshots

        timestamp = snapshot.get("timestamp", 0)
        mid = (snapshot["bids"][0][0] + snapshot["asks"][0][0]) / 2
        size = order_size_func(timestamp, mid)

        if size <= 0:
            continue  # No signal at this timestamp

        try:
            result = simulate_fill(snapshot, size, side)
            results.append(result)
        except RuntimeError as e:
            logger.warning(f"Fill failed at {timestamp}: {e}")
            continue

    return results


def summarize_impact(results: list[FillResult]) -> dict:
    """Aggregates fill results into summary statistics for the backtest report."""
    if not results:
        return {"error": "No fill results to summarize"}

    slippage_bps = [r.slippage_bps for r in results]
    unfilled = [r.unfilled_ratio for r in results if r.unfilled_ratio > 0]

    return {
        "total_orders": len(results),
        "mean_slippage_bps": round(sum(slippage_bps) / len(slippage_bps), 3),
        "p95_slippage_bps": round(sorted(slippage_bps)[int(len(slippage_bps) * 0.95)], 3),
        "max_slippage_bps": round(max(slippage_bps), 3),
        "orders_with_unfilled_qty": len(unfilled),
        "mean_unfilled_ratio": round(sum(unfilled) / len(unfilled), 3) if unfilled else 0.0,
        "avg_levels_consumed": round(
            sum(r.impact_consumed_levels for r in results) / len(results), 1
        ),
    }

5. Validation: Synthetic and Empirical Tests

The model must be validated before it informs strategy decisions. We use two approaches: synthetic order book injection and empirical comparison against realized spreads.

5.1 Synthetic Order Book Test

def test_impact_model():
    """Validates the fill simulator against known order book geometries."""

    # Case 1: Uniform depth — linear slippage ramp
    uniform_book = {
        "bids": [[99.00, 1000], [98.00, 1000], [97.00, 1000], [96.00, 1000]],
        "asks": [[101.00, 1000], [102.00, 1000], [103.00, 1000], [104.00, 1000]],
    }

    # Buy 2500 units — consumes 2.5 ask levels
    result = simulate_fill(uniform_book, 2500, side="buy")
    print(f"[Uniform] Avg fill: ${result.avg_fill_price:.4f}")
    print(f"[Uniform] Slippage: {result.slippage_bps:.2f} bps")
    print(f"[Uniform] Levels consumed: {result.impact_consumed_levels}")
    print(f"[Uniform] Unfilled: {result.unfilled_ratio:.1%}\n")

    # Case 2: Thin book — high slippage on small orders
    thin_book = {
        "bids": [[99.00, 50], [98.00, 50], [97.00, 50]],
        "asks": [[101.00, 50], [102.00, 50], [103.00, 50]],
    }

    result = simulate_fill(thin_book, 200, side="buy")
    print(f"[Thin] Avg fill: ${result.avg_fill_price:.4f}")
    print(f"[Thin] Slippage: {result.slippage_bps:.2f} bps")
    print(f"[Thin] Unfilled: {result.unfilled_ratio:.1%}\n")

    # Case 3: Large order vs. uniform book — quadratic cost emergence
    results = run_impact_backtest(
        order_books=[uniform_book] * 100,
        order_size_func=lambda ts, px: 1000,  # Fixed size every tick
        side="buy"
    )
    summary = summarize_impact(results)
    print("[Batch] Backtest summary:", summary)


if __name__ == "__main__":
    test_impact_model()

Expected output from the synthetic tests:

  • Uniform book (2500 units): consumes 2.5 levels → slippage ramps from 0 at L1 to ~50 bps at L3
  • Thin book (200 units): only 150 units fill → 25% unfilled → high effective slippage on filled portion
  • Batch run: mean slippage should be consistent with the linear impact assumption

5.2 Backtest Disclosure for Impact Studies

Any impact simulation backtest must carry the following disclosure:

Model limitations: The order book replay uses L1 baseline data for US equities and reconstructed depth for other markets. Actual fill prices may differ due to: (1) unobserved L2–L10 depth changes between snapshot intervals; (2) latency between signal generation and order submission; (3) venue-specific routing and smart order routing (SOR) behavior; (4) market regime shifts during the backtest period. We recommend validating impact estimates against paper trading results before live deployment.

6. Integration: Wiring Impact Cost into a Tick-Level Backtesting Loop

The complete backtesting loop combines price signals, order sizing, impact simulation, and performance attribution in a single coherent pipeline.

def tick_level_backtest(
    symbol: str,
    start: datetime,
    end: datetime,
    signal_func,                # (timestamp, price) -> +1 / 0 / -1
    base_size: float = 5000,
    side: str = "buy"
) -> dict:
    """
    Executes a full tick-level backtest with market impact simulation.

    Parameters
    ----------
    symbol : str
        Trading pair or equity symbol.
    signal_func : callable
        Strategy signal at each timestamp: returns +1 (long), 0 (flat), -1 (short).
    base_size : float
        Base order size in quote currency or shares.
    side : str
        Direction: "buy" or "sell".

    Returns
    -------
    dict
        Backtest results including gross PnL, net PnL (after impact),
        and impact attribution.
    """
    logger.info(f"Starting backtest: {symbol} from {start} to {end}")

    books = fetch_historical_depth(symbol, start, end)
    if not books:
        raise ValueError(f"No depth data available for {symbol}")

    position = 0.0
    entry_price = 0.0
    pnl_gross = 0.0
    pnl_net = 0.0
    impact_cost_total = 0.0
    trades = []

    for snapshot in books:
        ts = snapshot.get("timestamp", 0)
        mid = (snapshot["bids"][0][0] + snapshot["asks"][0][0]) / 2

        signal = signal_func(ts, mid)
        target_position = signal * base_size

        # Determine order size relative to current position
        order_size = abs(target_position - position)
        if order_size <= 0:
            continue

        # Check if we need to flip sides (close then reverse)
        # For simplicity, we close the current position first
        if position != 0:
            close_result = simulate_fill(snapshot, abs(position),
                                         side="sell" if position > 0 else "buy")
            impact_cost_total += close_result.slippage_bps * abs(position)
            pnl_gross += (close_result.avg_fill_price - entry_price) * abs(position) \
                         * (-1 if position > 0 else 1)
            position = 0.0

        # Open new position
        fill_result = simulate_fill(snapshot, order_size, side)
        impact_cost_total += fill_result.slippage_bps * order_size

        if position == 0:
            entry_price = fill_result.avg_fill_price

        position = target_position if target_position > 0 else -target_position

        trades.append({
            "timestamp": ts,
            "side": side,
            "size": order_size,
            "fill_price": fill_result.avg_fill_price,
            "slippage_bps": fill_result.slippage_bps,
            "mid_price": mid,
        })

    # Close any open position at the final mid price
    if position != 0:
        final_snapshot = books[-1]
        close_result = simulate_fill(final_snapshot, position,
                                     side="sell" if position > 0 else "buy")
        pnl_gross += (close_result.avg_fill_price - entry_price) * position \
                     * (-1 if position > 0 else 1)
        impact_cost_total += close_result.slippage_bps * position
        trades.append({
            "timestamp": final_snapshot["timestamp"],
            "side": "close",
            "size": position,
            "fill_price": close_result.avg_fill_price,
            "slippage_bps": close_result.slippage_bps,
        })

    pnl_net = pnl_gross - impact_cost_total

    return {
        "symbol": symbol,
        "total_trades": len(trades),
        "pnl_gross": round(pnl_gross, 2),
        "pnl_net": round(pnl_net, 2),
        "impact_cost_bps_total": round(impact_cost_total, 2),
        "impact_cost_per_trade_avg": round(impact_cost_total / max(len(trades), 1), 3),
        "trades": trades,
    }


# ─────────────────────────────────────────────────────────────────────────────
# Example signal: momentum on 5-minute close price
# ─────────────────────────────────────────────────────────────────────────────
if __name__ == "__main__":
    import statistics

    def momentum_signal(timestamp: float, current_price: float) -> int:
        """Simple momentum signal: compare current price to 5-period SMA."""
        # In production, pre-compute and cache the SMA series
        # This is a simplified single-factor example
        return 1  # Always bullish for demonstration

    results = tick_level_backtest(
        symbol="BTC.USDT",
        start=datetime(2024, 1, 1),
        end=datetime(2024, 1, 2),
        signal_func=momentum_signal,
        base_size=0.1,
        side="buy"
    )

    print("Backtest Results:")
    print(f"  Gross PnL:    ${results['pnl_gross']}")
    print(f"  Net PnL:      ${results['pnl_net']}")
    print(f"  Impact cost:  {results['impact_cost_bps_total']} bps total")
    print(f"  Avg impact:   {results['impact_cost_per_trade_avg']} bps/trade")

7. Performance Attribution: Isolating Impact from Alpha

The backtest output separates gross alpha from impact cost. This is the critical step for strategy diagnosis:

Scenario Diagnosis
Gross PnL >> Net PnL, high impact per trade Strategy signals too frequently relative to available liquidity. Reduce signal frequency or increase position sizing threshold.
Gross PnL ≈ Net PnL, low impact Strategy is liquid enough to absorb its own order flow. Backtest is credible.
Negative net PnL despite positive gross Impact cost exceeds signal edge. Strategy needs to be redesigned or sized down.
High unfilled ratio in fill results Order size exceeds market depth at key timestamps. Switch to VWAP or TWAP execution.

The summary statistics from summarize_impact() and the pnl_gross vs. pnl_net differential in the backtest results give you both the diagnostic signal and the numerical basis for the decision.

8. Data Requirements by Market

The fidelity of your impact model is bounded by the data you feed into it.

Asset class Available depth Backtest approach
US equities L1 only (best bid/ask) Reconstruct L1 impact from kline; estimate L2+ from historical ADV tables
HK equities L1–L10 Full order book replay; highest simulation fidelity
Crypto L1–L10 Full order book replay; best for HFT strategy development
Forex, precious metals Not supported Use spread-based slippage estimates; apply empirical multipliers from broker data

For US equity strategies, supplement the L1 baseline with the average daily volume (ADV) for the symbol. The order size as a fraction of ADV is the primary driver of the quadratic impact term in the Almgren-Chriss model.

9. Closing

The gap between backtest and live performance is almost never a broken alpha signal. It is almost always a cost that was not modeled. Slippage and market impact are not edge cases — they are the market's tax on your conviction. Every basis point of impact you ignore is a basis point of alpha you never actually had.

The framework in this article — order book replay, level-by-level fill simulation, and gross/net PnL attribution — gives you a defensible methodology for closing that gap. The key decisions you still need to make are your own: which impact coefficient to calibrate against, how to handle unfilled orders in live execution, and whether your signal frequency is sustainable at your target order size.

Those are strategy questions. This article gave you the engineering to answer them with data.

Next steps:

  • If you're running equity strategies, start by fetching 1-minute klines for your target symbols via GET /v1/market/kline and building the L1 impact baseline. Upgrade to HK equities or crypto for L2+ backtests with full order book fidelity.

  • If you need 10+ years of historical OHLCV data for cross-cycle backtesting, TickDB provides cleaned, timestamp-aligned kline data across US equities, HK equities, crypto, and futures. Visit tickdb.ai for institutional plans.

  • If you use AI coding assistants, search for and install the tickdb-market-data SKILL in your AI tool's marketplace to streamline API integration in your backtesting pipeline.

This article does not constitute investment advice. Markets involve risk; past performance does not guarantee future results. The impact simulation model presented here is a simplification of market microstructure. Actual execution costs depend on venue routing, order type, and real-time liquidity conditions that cannot be fully replicated in backtesting.