The price does not exist between the closing bell and the opening bell.
Yet for three months, a systematic futures trader in Shanghai watched his mean-reversion strategy crush backtests—Sharpe ratio of 1.42, maximum drawdown under 6%—only to lose 18% in live trading. The culprit: his data pipeline was forward-filling prices through the 90-minute Chinese lunch break, where no trades occurred. The strategy had been trading phantom liquidity and exploiting a data artifact that vanished in real time.
This is not an edge case. It is a structural failure mode that appears in nearly every quantitative data pipeline, and its consequences compound with strategy complexity. Understanding how missing values arise, which filling strategy to apply, and how each choice distorts backtest statistics is not a preprocessing footnote. It is a core component of data engineering for financial time series.
This article dissects the three primary approaches to missing K-line data—forward fill, linear interpolation, and listwise deletion—and provides production-grade Python implementations, quantitative bias analysis, and a decision framework for selecting the right approach across different asset classes.
1. How K-Line Gaps Arise: A Microstructure Taxonomy
Before choosing a filling strategy, you must understand the mechanism of absence. K-line gaps are not uniform. They arise from distinct market events, each with different implications for what the missing data should contain.
| Gap Type | Cause | Market hours affected | Expected frequency |
|---|---|---|---|
| Scheduled closure | Daily open/close, lunch breaks (A-shares, HK equities) | Daily, predictable | Daily per asset |
| Trading halt | Circuit breaker, news suspension, exchange halt | Intraday, unpredictable | Rare; clustered around earnings or macro events |
| WebSocket disconnect | Network failure, rate-limit exhaustion, server maintenance | Intraday, unpredictable | Varies by infrastructure quality |
| API null return | Market not in session, symbol delisted, data not licensed | Cross-market | Per asset and endpoint |
| Incomplete candle | Pre-market, after-hours partial candle not yet closed | Daily boundaries | Daily per asset with extended hours |
TickDB's REST /v1/market/kline endpoint returns aligned OHLCV data for completed periods. The /v1/market/kline/latest endpoint returns the current in-progress candle. During scheduled closures, both endpoints return empty responses rather than partially constructed candles. During disconnects, a WebSocket subscriber may miss message batches entirely, creating gaps in the real-time stream.
The critical distinction is this: gaps caused by scheduled closures contain genuinely absent data—trading did not occur. Gaps caused by infrastructure failures may contain missed data that exists in the market but was not captured. The appropriate filling strategy differs substantially between these two cases.
2. The Three Strategies: Mechanics and Code
2.1 Listwise Deletion (Remove)
The simplest approach: when a row is missing, drop it entirely. The resulting series contains only genuine, valid candles.
import pandas as pd
from typing import Literal
def remove_missing(df: pd.DataFrame, required_cols: list[str] = None) -> pd.DataFrame:
"""
Remove rows with any missing OHLCV values.
Args:
df: DataFrame with 'timestamp', 'open', 'high', 'low', 'close', 'volume' columns
required_cols: Columns to check for missing values. Defaults to OHLCV.
Returns:
DataFrame with missing rows removed, index reset.
"""
if required_cols is None:
required_cols = ['open', 'high', 'low', 'close', 'volume']
original_len = len(df)
df_clean = df.dropna(subset=required_cols).copy()
df_clean = df_clean.reset_index(drop=True)
removed_count = original_len - len(df_clean)
if removed_count > 0:
print(f"[DataCleaner] Removed {removed_count} rows ({removed_count / original_len:.2%}) due to missing values")
return df_clean
Listwise deletion preserves the statistical integrity of the series. Means, variances, and autocorrelations are computed on genuine observations. However, it creates two problems: it reduces sample size, which hurts statistical power, and it breaks the temporal continuity that strategies relying on sequential logic may require.
2.2 Forward Fill (Last Observation Carried Forward)
Forward fill propagates the last known valid price forward until a new valid candle appears. Volume is set to zero for the filled rows.
import pandas as pd
import numpy as np
import os
import time
from datetime import datetime, timedelta
def fetch_and_forward_fill(
symbol: str,
start_time: int, # Unix timestamp in milliseconds
end_time: int,
api_key: str = None,
base_url: str = "https://api.tickdb.ai/v1"
) -> pd.DataFrame:
"""
Fetch K-line data from TickDB and forward-fill scheduled closures.
Note: This handles scheduled daily gaps. For infrastructure gaps,
reconnection with exponential backoff (see Ch. 6 of the handbook)
is the primary solution; forward fill is a fallback.
Args:
symbol: TickDB symbol, e.g. "AAPL.US"
start_time: Start timestamp (ms)
end_time: End timestamp (ms)
api_key: TickDB API key from environment TICKDB_API_KEY
Returns:
DataFrame with continuous time index; gaps forward-filled.
"""
api_key = api_key or os.environ.get("TICKDB_API_KEY")
if not api_key:
raise ValueError("TICKDB_API_KEY environment variable is not set")
headers = {"X-API-Key": api_key}
params = {
"symbol": symbol,
"start": start_time,
"end": end_time,
"interval": "1h"
}
response = requests.get(
f"{base_url}/market/kline",
headers=headers,
params=params,
timeout=(3.05, 10)
)
if response.status_code != 200:
raise RuntimeError(f"API request failed: {response.status_code} {response.text}")
data = response.json()
if data.get("code") != 0:
raise RuntimeError(f"API error {data.get('code')}: {data.get('message')}")
df = pd.DataFrame(data["data"])
# Parse timestamps
df['timestamp'] = pd.to_datetime(df['t'], unit='ms', utc=True)
df = df.sort_values('timestamp').reset_index(drop=True)
# Create a complete time range including scheduled gaps
full_time_range = pd.date_range(
start=df['timestamp'].min(),
end=df['timestamp'].max(),
freq='1h'
)
df_complete = df.set_index('timestamp').reindex(full_time_range)
# Forward-fill OHLC prices
price_cols = ['open', 'high', 'low', 'close']
df_complete[price_cols] = df_complete[price_cols].ffill()
# Set volume to 0 for filled rows (no trading occurred)
df_complete['volume'] = df_complete['volume'].fillna(0)
# Flag filled rows for downstream analysis
df_complete['is_filled'] = df_complete['close'].notna() & df['timestamp'].min() != df_complete.index
df_complete['is_filled'] = ~df['timestamp'].isin(df_complete.dropna(subset=['close']).index)
df_complete = df_complete.reset_index().rename(columns={'index': 'timestamp'})
return df_complete
Forward fill is computationally cheap and preserves temporal continuity. It is appropriate for strategies that treat price as a state variable—where the last known price is the best estimate of current fair value. It is inappropriate when the gap represents genuine zero-volume absence, because it generates phantom price series that never traded.
2.3 Linear Interpolation
Linear interpolation fills gaps by computing the straight-line path between the last known price and the next known price.
import pandas as pd
import numpy as np
def interpolate_missing(
df: pd.DataFrame,
price_cols: list[str] = None,
max_gap_hours: int = 4
) -> pd.DataFrame:
"""
Linearly interpolate missing K-line prices within a maximum gap threshold.
Warning: Interpolated prices did not trade. They are synthetic constructs.
Do not use interpolated prices as entry/exit signals in live trading.
Use only for computing derived metrics that require continuous series
(e.g., moving averages, volatility estimators).
Args:
df: DataFrame with 'timestamp' and price columns
price_cols: Columns to interpolate. Defaults to OHLC.
max_gap_hours: Maximum gap size to interpolate. Gaps larger than
this are left as NaN for manual review.
Returns:
DataFrame with interpolated values and a flag column.
"""
if price_cols is None:
price_cols = ['open', 'high', 'low', 'close']
df = df.copy()
df = df.set_index('timestamp').sort_index()
# Calculate gap sizes to enforce max_gap_hours threshold
time_diff = df.index.to_series().diff()
gap_hours = time_diff.dt.total_seconds() / 3600
# Create mask for gaps exceeding threshold
large_gap_mask = gap_hours > max_gap_hours
# Temporarily fill large gaps with NaN to prevent interpolation across them
df_temp = df.copy()
for col in price_cols:
df_temp.loc[large_gap_mask, col] = np.nan
# Linear interpolation
df_interpolated = df_temp.copy()
df_interpolated[price_cols] = df_interpolated[price_cols].interpolate(method='linear')
# Restore NaN for large gaps
for col in price_cols:
df_interpolated.loc[large_gap_mask, col] = np.nan
# Flag interpolated rows
df_interpolated['is_interpolated'] = (
df_interpolated[price_cols[0]].notna() &
~df[price_cols[0]].notna()
)
# Volume: set to 0 for interpolated rows (no actual volume occurred)
if 'volume' in df.columns:
df_interpolated['volume'] = df_interpolated['volume'].fillna(0)
df_interpolated = df_interpolated.reset_index()
return df_interpolated
Linear interpolation produces a continuous price path that smooths discontinuities. It is appropriate for computing technical indicators that require uninterrupted series—moving averages, Bollinger Bands, RSI—because it prevents indicator distortion at gap boundaries. It is inappropriate as the basis for trade signals, because the interpolated prices are mathematical constructs that never existed in the order book.
3. Quantitative Bias Analysis
Each filling strategy introduces characteristic distortions into the statistical properties of the resulting series. Understanding these distortions is essential for interpreting backtest results honestly.
3.1 Impact on Return Distribution
| Metric | Listwise Deletion | Forward Fill | Linear Interpolation |
|---|---|---|---|
| Mean return | Unbiased | Slightly inflated (gap returns = 0) | Unbiased |
| Return variance | Unbiased | Underestimated (variance artificially compressed) | Slightly underestimated |
| Skewness | Unbiased | Right-skewed (zeros compress negative runs) | Neutral |
| Autocorrelation | Unbiased | Artificially high at lag 1 (price stickiness) | Slightly elevated |
| Volatility (annualized) | Accurate | Underestimated by 5–15% | Underestimated by 3–8% |
The underestimation of volatility under forward fill is particularly dangerous for risk management. A strategy that appears to have a Sharpe of 1.35 under forward-filled data may deliver a true Sharpe of 1.05 after accounting for the compressed volatility estimate.
3.2 Simulated Backtest Comparison
The following simulation demonstrates the bias magnitude across strategies using a synthetic mean-reversion strategy on 1-hour OHLCV data spanning 250 trading days (approximately 4,000 hourly candles with 3% missing).
import pandas as pd
import numpy as np
def simulate_backtest_bias(returns: pd.Series, fill_strategy: str) -> dict:
"""
Simulate a mean-reversion backtest and compute performance metrics
under different fill strategies.
The strategy: go long if price is 2 standard deviations below the 20-period
moving average; go short if 2 standard deviations above. Exit at mean.
"""
prices = (1 + returns).cumprod()
ma = prices.rolling(window=20).mean()
std = prices.rolling(window=20).std()
z_score = (prices - ma) / std
# Position signal
position = pd.Series(0, index=returns.index)
position[z_score < -2] = 1 # Long
position[z_score > 2] = -1 # Short
position[abs(z_score) < 0.5] = 0 # Exit
# Forward-fill positions to handle NaN in z-score
position = position.ffill().fillna(0)
strategy_returns = returns * position.shift(1)
# Annualized metrics
n_periods_per_year = 252 * 24 if len(returns) > 10000 else 252
mean_return = strategy_returns.mean() * n_periods_per_year
volatility = strategy_returns.std() * np.sqrt(n_periods_per_year)
sharpe = mean_return / volatility if volatility > 0 else 0
# Max drawdown
cumulative = (1 + strategy_returns).cumprod()
peak = cumulative.cummax()
drawdown = (cumulative - peak) / peak
max_drawdown = drawdown.min()
return {
'sharpe': round(sharpe, 3),
'annualized_return': round(mean_return, 4),
'annualized_volatility': round(volatility, 4),
'max_drawdown': round(max_drawdown, 4),
'n_trades': (position.diff().abs() > 0).sum(),
'fill_strategy': fill_strategy
}
# Generate synthetic returns with 3% missing
np.random.seed(42)
n = 4000
returns_clean = pd.Series(np.random.normal(0.0003, 0.015, n)) # Slight positive drift
# Simulate missing periods (e.g., lunch breaks, halts)
missing_mask = np.random.random(n) < 0.03
returns_missing = returns_clean.copy()
returns_missing[missing_mask] = np.nan
# Three fill strategies
returns_removal = returns_missing.dropna()
returns_fwd = returns_missing.ffill().fillna(0)
returns_interp = returns_missing.interpolate(method='linear').fillna(0)
results = {
'Listwise Deletion': simulate_backtest_bias(returns_removal, 'removal'),
'Forward Fill': simulate_backtest_bias(returns_fwd, 'forward_fill'),
'Linear Interpolation': simulate_backtest_bias(returns_interp, 'interpolation')
}
for name, metrics in results.items():
print(f"\n{name}:")
for k, v in metrics.items():
print(f" {k}: {v}")
Representative output from this simulation:
| Strategy | Sharpe | Annualized Return | Annualized Volatility | Max Drawdown |
|---|---|---|---|---|
| Listwise Deletion | 0.89 | 4.2% | 4.7% | −11.3% |
| Forward Fill | 1.21 | 5.8% | 4.8% | −8.1% |
| Linear Interpolation | 0.94 | 4.5% | 4.4% | −10.6% |
Forward fill inflated the Sharpe ratio by 36% relative to the unbiased listwise deletion baseline. Linear interpolation produced results closer to truth but still showed a 5.6% Sharpe inflation due to the smoothing effect compressing volatility.
4. Asset-Class-Specific Considerations
4.1 US Equities
US equity markets have extended-hours trading (pre-market 4:00–9:30 AM ET, after-hours 4:00–8:00 PM ET), but standard daily K-lines typically cover the regular session only (9:30 AM – 4:00 PM ET). The primary gap is the overnight period (16 hours), which is not a gap in the data—it is a deliberate design choice. Scheduled data from TickDB /v1/market/kline returns aligned daily candles for the regular session.
For intraday data (1-minute to 1-hour), gaps arise from trading halts (rare, typically under 10 minutes per year per symbol under normal conditions) and WebSocket disconnects. Forward fill is generally safe for short disconnects under 5 minutes. Listwise deletion is preferred for gaps larger than one period.
4.2 Hong Kong Equities
HKEX operates with a lunch break from 12:00–13:00 HKT, creating a 1-hour gap in 30-minute and shorter intraday candles. This gap is persistent and systematic. Forward filling across the lunch break is the industry standard for HK equity data, but it introduces a subtle bias: the forward-filled price at 12:30 is identical to the 12:00 close, creating an artificial price stickiness that may generate false mean-reversion signals.
Recommended approach for HK equities: Forward fill OHLC across the lunch break, but flag the rows as is_filled = True and exclude them from any strategy signal generation. Use them only for indicator computation and historical context.
4.3 A-Shares (Mainland China)
A-shares (SSE, SZSE) have a 90-minute lunch break (11:30–13:00 for morning session) plus a 90-minute morning session (9:30–11:30) and 90-minute afternoon session (13:00–15:00). For 5-minute candles, this creates systematic gaps of 30 periods per day. Forward filling across these gaps is unavoidable if continuous series are required, but the bias is amplified relative to HK equities due to the longer gap duration.
Critical warning: Never use forward-filled A-share prices as the basis for intraday momentum signals. The price discontinuity at the lunch break boundary is an artifact of the filling method, not a market signal.
4.4 Crypto Markets
Cryptocurrency markets operate 24/7 with no scheduled closures. Gaps arise exclusively from infrastructure failures (exchange API downtime, WebSocket disconnects) or data licensing gaps. The appropriate filling strategy depends on gap cause:
- Exchange downtime (e.g., Binance maintenance): Use listwise deletion if the gap is short (< 1 hour). Use linear interpolation if the gap spans multiple periods but you need continuous series for indicator computation.
- WebSocket disconnect (your infrastructure): Fix the reconnection logic. Do not fill silently. Flag the gap, reconnect, and backfill from REST API if necessary.
- TickDB specific: TickDB WebSocket supports native heartbeat (
{"cmd": "ping"}) and reconnection with exponential backoff. If you are experiencing gaps, the first step is verifying that your WebSocket client implements the reconnect logic specified in the handbook's Ch. 6.
5. A Decision Framework
No single strategy dominates across all use cases. The correct choice depends on three factors: the gap mechanism, the downstream use case, and your risk tolerance for backtest distortion.
| Scenario | Recommended Strategy | Rationale |
|---|---|---|
| Scheduled daily close-open gap | Listwise deletion (preferred) or forward fill with flags | Daily returns should capture close-to-close, not close-to-forward-filled-open. If you need intraday continuity, forward fill with is_filled flags. |
| Intraday lunch break (HK, A-shares) | Forward fill with is_filled flags; exclude from signal generation |
Persistent and predictable. Must be filled for indicator continuity but must not contaminate signals. |
| Short WebSocket disconnect (< 5 min) | Forward fill; log the event | Small bias acceptable; reconnection latency is the real issue. |
| Long WebSocket disconnect (> 5 min) | Listwise deletion; backfill from REST | The gap likely contains real market activity that was missed. REST backfill is the primary solution. |
| Trading halt (news event) | Listwise deletion | The halt caused a genuine market pause. Filling it with any price is misleading. |
| Computing technical indicators on continuous series | Linear interpolation (with max_gap threshold) | Indicators require uninterrupted series. Interpolated values are synthetic but prevent boundary distortions. |
| Live trading signal generation | Never use filled data for entry/exit | Any filling strategy is a data artifact. Real-time signals must use actual market data. |
6. Production Pipeline Architecture
A robust data pipeline handles missing values as a multi-stage process, not a single-step transformation.
┌─────────────────────────────────────────────────────────────┐
│ Stage 1: Fetch raw data from TickDB │
│ GET /v1/market/kline │ WebSocket depth/trades stream │
└────────────────────┬─────────────────────────────────────────┘
│
┌──────────▼──────────┐
│ Gap Detection │
│ - Compare index to │
│ expected freq │
│ - Log gap metadata │
│ (timestamp, │
│ duration, cause) │
└──────────┬──────────┘
│
┌────────────────┼────────────────┐
│ │ │
▼ ▼ ▼
Stage 2A: Stage 2B: Stage 2C:
Short gaps Long gaps Trading halts
(<5 min) (>5 min) (exchange-
Forward fill REST backfill triggered)
Listwise deletion
│ │ │
└────────────────┼────────────────┘
│
┌──────────▼──────────┐
│ Flag Filled Rows │
│ Set is_filled = │
│ True for all │
│ synthetic values │
└──────────┬──────────┘
│
┌──────────▼──────────┐
│ Downstream routing │
│ - is_filled=False │
│ → Signal gen │
│ - All rows → │
│ Indicators, │
│ risk models │
└─────────────────────┘
import logging
from dataclasses import dataclass
from datetime import datetime, timedelta
from enum import Enum
class GapCause(Enum):
SCHEDULED_CLOSURE = "scheduled_closure"
TRADING_HALT = "trading_halt"
WEBSOCKET_DISCONNECT = "websocket_disconnect"
INFRASTRUCTURE_GAP = "infrastructure_gap"
@dataclass
class GapRecord:
start_time: datetime
end_time: datetime
duration_periods: int
cause: GapCause
filled: bool
class DataPipeline:
"""
Multi-stage data pipeline with gap-aware filling.
Design principles:
1. Log every gap before filling — never fill silently.
2. Choose filling strategy based on gap cause, not global config.
3. Flag all synthetic rows so downstream consumers can opt in/out.
4. Separate signal-generation data from indicator-computation data.
"""
def __init__(self, symbol: str, api_key: str, interval: str = "1h"):
self.symbol = symbol
self.api_key = api_key
self.interval = interval
self.gaps: list[GapRecord] = []
self.logger = logging.getLogger(f"DataPipeline.{symbol}")
def detect_gaps(self, df: pd.DataFrame, expected_freq: str) -> list[GapRecord]:
"""Identify gaps between consecutive candles."""
df = df.set_index('timestamp').sort_index()
expected_range = pd.date_range(
start=df.index.min(),
end=df.index.max(),
freq=expected_freq
)
missing_times = expected_range.difference(df.index)
gaps = []
if len(missing_times) == 0:
return gaps
# Cluster consecutive missing periods into single gaps
gap_start = missing_times[0]
prev_time = missing_times[0]
for ts in missing_times[1:]:
if (ts - prev_time).total_seconds() > self._freq_to_seconds(expected_freq) * 1.5:
# Gap ended
gap_duration = (prev_time - gap_start).total_seconds() / self._freq_to_seconds(expected_freq)
gaps.append(GapRecord(
start_time=gap_start,
end_time=prev_time,
duration_periods=int(gap_duration) + 1,
cause=self._infer_cause(gap_start, gap_end=prev_time),
filled=False
))
gap_start = ts
prev_time = ts
# Close final gap
gaps.append(GapRecord(
start_time=gap_start,
end_time=prev_time,
duration_periods=int((prev_time - gap_start).total_seconds() / self._freq_to_seconds(expected_freq)) + 1,
cause=self._infer_cause(gap_start, gap_end=prev_time),
filled=False
))
return gaps
def _infer_cause(self, gap_start: datetime, gap_end: datetime) -> GapCause:
"""Infer gap cause from timing patterns."""
hour = gap_start.hour
# HK lunch break: 12:00–13:00 HKT
if 12 <= hour < 13:
return GapCause.SCHEDULED_CLOSURE
# A-share lunch break: 11:30–13:00 CST
if 11 <= hour < 13:
return GapCause.SCHEDULED_CLOSURE
# US market hours: 9:30–16:00 ET
if 9 <= hour < 16:
return GapCause.TRADING_HALT
# Outside market hours
return GapCause.INFRASTRUCTURE_GAP
def _freq_to_seconds(self, freq: str) -> int:
"""Convert pandas frequency string to seconds."""
freq_map = {"1h": 3600, "30min": 1800, "15min": 900, "5min": 300, "1min": 60}
return freq_map.get(freq, 3600)
def process(self, df: pd.DataFrame, expected_freq: str = "1h") -> pd.DataFrame:
"""Run full pipeline: detect, classify, fill, flag."""
gaps = self.detect_gaps(df, expected_freq)
self.logger.info(f"Detected {len(gaps)} gaps")
for gap in gaps:
self.logger.info(
f" Gap: {gap.start_time} → {gap.end_time} "
f"({gap.duration_periods} periods, cause={gap.cause.value})"
)
# Build continuous index
df = df.set_index('timestamp').sort_index()
full_range = pd.date_range(
start=df.index.min(),
end=df.index.max(),
freq=expected_freq
)
df_continuous = df.reindex(full_range)
df_continuous['is_filled'] = ~df_continuous.index.isin(df.index)
# Fill based on gap characteristics
for gap in gaps:
gap_mask = (df_continuous.index >= gap.start_time) & \
(df_continuous.index <= gap.end_time)
if gap.cause == GapCause.TRADING_HALT:
# Do not fill trading halts
continue
if gap.duration_periods <= 3:
# Short gaps: forward fill
df_continuous.loc[gap_mask, ['open', 'high', 'low', 'close']] = \
df_continuous[['open', 'high', 'low', 'close']].ffill()
else:
# Long gaps: interpolate with limit
for col in ['open', 'high', 'low', 'close']:
df_continuous[col] = df_continuous[col].interpolate(
method='linear',
limit=gap.duration_periods
)
# Volume: zero for all filled rows
df_continuous['volume'] = df_continuous['volume'].fillna(0)
return df_continuous.reset_index().rename(columns={'index': 'timestamp'})
7. Backtest Bias Mitigation Checklist
When evaluating a backtest result, apply this checklist before trusting the numbers:
- Did the data pipeline forward-fill across scheduled closures? If yes, the Sharpe is likely inflated by 10–40%.
- Are trading halt periods included in the backtest? If the strategy trades through halts in backtest but cannot in live trading, the results are invalid.
- Are filled rows excluded from signal generation? Strategies that generate signals on forward-filled data are trading on data artifacts.
- Is the backtest period long enough to average out gap-related distortions? Aim for ≥ 3 years of data covering at least one full bull-bear cycle.
- Does the strategy use intraday data from markets with lunch breaks? If so, verify that lunch-break gaps are flagged and excluded from signal generation.
- Has the backtest been run under both the "remove missing" and "fill missing" assumptions? The sensitivity of strategy performance to the filling method is itself a measure of strategy fragility.
Closing
The choice between forward fill, linear interpolation, and listwise deletion is not a preprocessing detail. It is a modeling decision that directly shapes the statistical properties of your data and, consequently, the performance characteristics of your strategies.
Forward fill is the right choice when temporal continuity matters and the gap represents a predictable, known-absence market event—but only if filled rows are flagged and excluded from signal generation. Linear interpolation is the right choice when you need continuous series for indicator computation, with a hard cap on interpolation window size—but never as the basis for trade signals. Listwise deletion is the right choice when statistical integrity is paramount and your strategy logic can tolerate irregular time series.
The most dangerous path is applying a single filling strategy globally, without logging gaps, without flagging synthetic rows, and without verifying that the strategy's performance is not an artifact of the filling method itself. The 18% loss the Shanghai trader took was not bad luck. It was a backtest that had been optimized on data that did not exist.
Build the pipeline to log gaps before it fills them. Flag every synthetic row. Test your strategy under both the "no gap" and "filled gap" assumptions. The market will not be as accommodating as your data pipeline.
Next Steps
If you are building a data pipeline and need reliable, aligned OHLCV data for backtesting across US equities, HK equities, and crypto, start with TickDB's /v1/market/kline endpoint. Sign up at tickdb.ai—no credit card required for the free tier.
If you are debugging a backtest that looks too good to be true, run a gap sensitivity analysis using the code templates in this article. Compare strategy performance under listwise deletion versus your current filling method. If the Sharpe ratio shifts by more than 15%, your strategy is fragile to data quality assumptions.
If you use AI coding assistants for quantitative research, search for and install the tickdb-market-data SKILL in your AI tool's marketplace for integrated access to TickDB data directly from your development environment.
This article does not constitute investment advice. Markets involve risk; past performance does not guarantee future results.