For eight months, a systematic trader I know ran his entire backtesting pipeline through a WebSocket connection. He justified it — correctly, he thought — because his data provider offered a WebSocket endpoint, and WebSocket was newer, faster, and more "modern." Every night, his Python script would open a long-lived connection, request historical klines going back three years, parse the stream, and write them to a Parquet file. The pipeline worked. Until it didn't.
On a Friday evening, the WebSocket gateway restarted. The reconnect logic had a 30-second backoff with no jitter, and the reconnect storm hit 200 concurrent instances simultaneously, triggering a rate limit. His backtest never ran. He lost a weekend.
The lesson was not about the data provider. It was about protocol semantics. REST and WebSocket are not interchangeable tools. Each is optimized for a fundamentally different communication pattern, and matching the protocol to the use case is not a matter of preference — it is a matter of correctness.
This article examines the architectural trade-offs between REST and WebSocket in the context of financial market data APIs. It explains why the conventional wisdom — REST for historical data, WebSocket for real-time streams — exists, what breaks when you violate it, and how to implement both patterns in production with proper resilience.
The Fundamental Asymmetry: Request-Response vs Continuous Push
The distinction between REST and WebSocket is often presented as a performance debate. It is not. It is an architectural decision rooted in the nature of the data being transmitted.
REST (Representational State Transfer) is a request-response protocol. A client sends a request; the server processes it; the server returns a response; the connection closes. Each interaction is self-contained and stateless. The server does not remember what the client asked for five minutes ago.
WebSocket is a persistent, bidirectional connection. Once established, both client and server can send messages at any time, without re-negotiating the connection. The server can push data to the client the moment an event occurs, without the client having to ask.
This difference has profound implications for the types of data each protocol handles well.
| Characteristic | REST | WebSocket |
|---|---|---|
| Connection model | Ephemeral (open-close) | Persistent (open-maintain) |
| Data direction | Client initiates every request | Server can push without client asking |
| State | Stateless (each request independent) | Stateful (connection carries context) |
| Caching | HTTP caching fully available | No native caching mechanism |
| Replay / idempotency | Natural — same request yields same response | Requires application-level replay logic |
| Overhead per message | ~50–100 bytes (HTTP headers) | ~2–14 bytes (frame overhead only) |
| Latency (first byte) | Higher (connection setup + request + response) | Lower after connection established (no setup cost per message) |
For market data, this table maps directly to use cases:
- Historical OHLCV data is a snapshot. You ask for it, you receive it, it does not change. REST is the natural fit.
- Live order book updates are a continuous stream. Data arrives at unpredictable intervals. The server must push it the moment it changes. WebSocket is the natural fit.
The reason you cannot simply swap them is that the communication patterns are inverted from what each protocol is designed to handle.
Why REST Fails for Real-Time Streams: The Polling Tax
Imagine trying to receive real-time depth updates via REST. Your client would need to poll the server repeatedly — every 100 ms, every 50 ms, perhaps every 10 ms — asking "has the order book changed?" If it has, the server returns the new state. If it has not, the server returns either an empty response or a cached copy with a 304 Not Modified header.
This approach has three fundamental problems.
1. Latency is bounded above by your poll interval. If you poll every 200 ms, you cannot react to events that occur every 50 ms. A liquidity vacuum that lasts 150 ms is invisible to your system.
2. Polling tax scales with client count. If 10,000 clients poll every 100 ms, the server processes 100,000 requests per second — even if the order book changes only 50 times per second. REST was not designed for fan-out scenarios where the same data must be delivered to many recipients simultaneously.
3. HTTP header overhead compounds. A 100 ms poll interval with 80-byte headers means 8,000 bytes/second of overhead per client, just for asking "is there new data?" This is pure waste when the actual data payload might be 200 bytes.
WebSocket solves all three. The server maintains one persistent connection per client and pushes only when the data changes. The latency drops to the network propagation delay — typically 1–10 ms for colocated infrastructure. The server sends exactly as many messages as there are data events. No polling overhead.
For TickDB's real-time endpoints (depth channel for order book, trade channel for transaction data), WebSocket is not a preference — it is the only architecturally sound choice.
Why WebSocket Fails for Historical Data: The Replay Problem
Now consider the inverse: requesting three years of historical kline data over a WebSocket connection.
When you open a WebSocket connection and subscribe to kline for a symbol, the server begins sending you the latest candle updates. It does not "rewind" to January 2023. It sends you what is happening right now.
If you need historical data — for backtesting, for regime analysis, for training a machine learning model — you need the server to replay a time series that is already stored. This is a fundamentally different operation:
- A WebSocket stream is ephemeral. Messages are delivered to connected clients and discarded. Historical data is not re-broadcast to new connections.
- A REST endpoint for historical data is a stored query. You specify a time range; the server retrieves pre-computed, stored data from a time-series database; it returns it as a complete response. The operation is idempotent — the same query always returns the same result.
To receive historical data over WebSocket, you would need the server to implement an application-level replay mechanism: accept a historical time range in the subscription message, fetch the data from storage, and stream it back before switching to live mode. This is technically possible, but it introduces several problems:
1. No caching. HTTP responses from a REST endpoint can be cached at the CDN layer, at the client layer, and at any intermediate proxy. WebSocket messages are point-to-point and cannot be cached without application-level logic.
2. No idempotency guarantee. If your WebSocket connection drops during a historical replay, the server has no standard way to resume from exactly where it left off. You might re-request the same range and get duplicate data, or you might have gaps.
3. Connection management overhead. For a three-year backtest of 1-minute candles on 500 symbols, you might receive millions of messages. Managing this stream state — ordering, deduplication, buffering — becomes the client's problem, whereas a REST endpoint simply returns the complete payload and lets the client process it at its own pace.
4. Resource inefficiency. Maintaining a persistent WebSocket connection consumes server-side file descriptors and memory. Holding a connection open for the duration of a bulk historical download — which might take minutes — is a poor use of a protocol optimized for low-latency event delivery.
TickDB's /v1/market/kline endpoint is a REST endpoint specifically because historical OHLCV data is stored, static, and benefits from REST's caching, idempotency, and statelessness guarantees.
Code: REST for Historical Data
The following Python implementation fetches historical kline data using TickDB's REST API. It demonstrates production-grade patterns: environment-variable-based authentication, explicit timeouts, and proper error handling for the specific error codes that TickDB's API returns.
import os
import time
import requests
from typing import Optional
# Load API key from environment variable. Never hardcode credentials.
TICKDB_API_KEY = os.environ.get("TICKDB_API_KEY")
if not TICKDB_API_KEY:
raise EnvironmentError("TICKDB_API_KEY environment variable is not set")
BASE_URL = "https://api.tickdb.ai/v1"
headers = {
"X-API-Key": TICKDB_API_KEY,
"Content-Type": "application/json",
}
def handle_api_error(response: requests.Response, symbol: Optional[str] = None) -> dict:
"""
Standard TickDB REST error handler.
Maps error codes to actionable exceptions.
"""
# Parse error body (try JSON first, fall back to text)
try:
error_body = response.json()
except ValueError:
error_body = {"message": response.text or "Unknown error"}
code = error_body.get("code", 0)
# 0 = success
if code == 0:
return error_body.get("data")
# Authentication failures
if code in (1001, 1002):
raise ValueError(
"Invalid or missing API key. Verify TICKDB_API_KEY is set correctly."
)
# Symbol not found — raise KeyError so caller knows the symbol is invalid
if code == 2002:
raise KeyError(
f"Symbol '{symbol}' not found. "
"Check available symbols via GET /v1/symbols/available"
)
# Rate limit — raise with retry guidance
if code == 3001:
retry_after = int(response.headers.get("Retry-After", 5))
raise RuntimeError(
f"Rate limit exceeded. Retry after {retry_after} seconds. "
"Consider reducing request frequency or upgrading your plan."
)
# Generic API error
raise RuntimeError(
f"API error {code}: {error_body.get('message', 'Unknown error')}"
)
def fetch_historical_klines(
symbol: str,
interval: str = "1h",
start_time: Optional[int] = None,
end_time: Optional[int] = None,
limit: int = 1000,
) -> list:
"""
Fetch historical OHLCV klines for a given symbol via REST.
REST is appropriate here because:
- The data is stored and static (not changing in real time)
- Idempotent queries enable safe retries
- HTTP caching can reduce redundant server load
- The complete response arrives in one round trip
Args:
symbol: Trading pair (e.g., "AAPL.US", "BTC.BITFINEX")
interval: Candle interval ("1m", "5m", "1h", "1d", etc.)
start_time: Unix timestamp in milliseconds (inclusive)
end_time: Unix timestamp in milliseconds (exclusive)
limit: Maximum number of candles per request (max 1000)
Returns:
List of OHLCV candles, each as a dict with keys:
open_time, open, high, low, close, volume, close_time
"""
params = {
"symbol": symbol,
"interval": interval,
"limit": limit,
}
# Optional time range filters
if start_time is not None:
params["start_time"] = start_time
if end_time is not None:
params["end_time"] = end_time
url = f"{BASE_URL}/market/kline"
# Explicit timeout: (connect_timeout, read_timeout)
# Connect: 3.05s (above the 3s common threshold for load balancers)
# Read: 10s for normal responses, adjust for large limit values
response = requests.get(
url,
headers=headers,
params=params,
timeout=(3.05, 10),
)
if not response.ok:
raise RuntimeError(
f"HTTP {response.status_code} from {url}: {response.text[:200]}"
)
data = handle_api_error(response, symbol=symbol)
if not isinstance(data, list):
raise TypeError(f"Expected list in response data, got {type(data)}")
return data
# Example usage: fetch 3 months of hourly candles for AAPL
if __name__ == "__main__":
# 90 days of hourly data = ~2,160 candles (within default 1000 limit)
# For production: paginate using start_time / end_time if limit < total
try:
candles = fetch_historical_klines(
symbol="AAPL.US",
interval="1h",
limit=1000,
)
print(f"Fetched {len(candles)} candles")
if candles:
print(f"Latest: {candles[-1]}")
except KeyError as e:
print(f"Symbol error: {e}")
except RuntimeError as e:
print(f"API error: {e}")
For a full backtest spanning years, paginate through the data using start_time and end_time in sequential requests. The REST pattern makes this straightforward: each request is independent, so you can parallelize multiple symbol requests without connection management complexity.
Code: WebSocket for Real-Time Depth
The following implementation connects to TickDB's WebSocket endpoint and subscribes to the depth channel for real-time order book updates. It demonstrates the mandatory production-grade patterns: heartbeat handling, exponential backoff with jitter, rate-limit handling, and graceful shutdown.
import json
import os
import random
import threading
import time
from typing import Callable, Optional
try:
import websocket
except ImportError:
raise ImportError(
"websocket-client is required. Install with: pip install websocket-client"
)
TICKDB_API_KEY = os.environ.get("TICKDB_API_KEY")
if not TICKDB_API_KEY:
raise EnvironmentError("TICKDB_API_KEY environment variable is not set")
WS_BASE_URL = "wss://api.tickdb.ai/v1/market/stream"
class TickDBDepthStream:
"""
Production-grade WebSocket client for TickDB depth (order book) data.
Why WebSocket here:
- Order book updates arrive at irregular, high-frequency intervals
- Sub-second latency is required for live execution
- Server-push model eliminates polling overhead and latency floors
- Heartbeat mechanism keeps connection alive through proxies and NAT
⚠️ For HFT workloads (<1ms latency requirements), consider
aiohttp/asyncio or raw sockets instead of this synchronous client.
"""
def __init__(
self,
symbols: list[str],
on_depth_update: Optional[Callable[[dict], None]] = None,
):
self.symbols = symbols
self.on_depth_update = on_depth_update
self.ws: Optional[websocket.WebSocketApp] = None
self._running = False
self._thread: Optional[threading.Thread] = None
self._retry_count = 0
self._max_retries = 10
# Exponential backoff configuration
self._base_delay = 1.0 # seconds
self._max_delay = 60.0 # seconds
self._backoff_multiplier = 2.0
def _compute_backoff(self) -> float:
"""
Compute delay with exponential backoff and full jitter.
Full jitter (random value in [0, base * 2^n]) prevents
thundering herd: all retrying clients reconnecting simultaneously.
"""
delay = min(
self._base_delay * (self._backoff_multiplier ** self._retry_count),
self._max_delay,
)
jitter = random.uniform(0, delay * 0.1)
return delay + jitter
def _send_subscribe(self, ws: websocket.WebSocketApp, symbols: list[str]) -> None:
"""Send subscription message for depth channel."""
subscribe_msg = {
"cmd": "subscribe",
"params": {
"channel": "depth",
"symbols": symbols,
}
}
ws.send(json.dumps(subscribe_msg))
print(f"[TickDB] Subscribed to depth channel for: {symbols}")
def _on_message(self, ws: websocket.WebSocketApp, raw_message: str) -> None:
"""Handle incoming WebSocket messages."""
try:
msg = json.loads(raw_message)
# Heartbeat: respond with pong to keep connection alive
if msg.get("cmd") == "ping":
ws.send(json.dumps({"cmd": "pong"}))
return
# Skip non-data messages
channel = msg.get("channel", "")
if channel != "depth":
return
# Extract order book snapshot
data = msg.get("data", {})
symbol = data.get("s") or data.get("symbol", "unknown")
# Format: bids (list of [price, size]) and asks (list of [price, size])
bids = data.get("b", data.get("bids", []))
asks = data.get("a", data.get("asks", []))
if self.on_depth_update:
self.on_depth_update({
"symbol": symbol,
"bids": bids,
"asks": asks,
"timestamp": msg.get("ts", time.time()),
})
except json.JSONDecodeError:
print(f"[TickDB] Failed to parse message: {raw_message[:100]}")
except Exception as e:
print(f"[TickDB] Error processing message: {e}")
def _on_error(self, ws: websocket.WebSocketApp, error: Exception) -> None:
print(f"[TickDB] WebSocket error: {error}")
def _on_close(self, ws: websocket.WebSocketApp, close_status_code: int, close_msg: str) -> None:
print(f"[TickDB] Connection closed (code={close_status_code}): {close_msg}")
self._running = False
def _on_open(self, ws: websocket.WebSocketApp) -> None:
print(f"[TickDB] Connection established")
self._retry_count = 0 # Reset backoff on successful connection
self._send_subscribe(ws, self.symbols)
def _run_loop(self) -> None:
"""Main WebSocket event loop with reconnection logic."""
self._running = True
while self._running and self._retry_count < self._max_retries:
delay = self._compute_backoff()
if self._retry_count > 0:
print(f"[TickDB] Reconnecting in {delay:.1f}s (attempt {self._retry_count})")
time.sleep(delay)
try:
self.ws = websocket.WebSocketApp(
f"{WS_BASE_URL}?api_key={TICKDB_API_KEY}",
on_message=self._on_message,
on_error=self._on_error,
on_close=self._on_close,
on_open=self._on_open,
)
# run_forever() blocks until the connection closes
# ping_interval=15 sends a ping every 15 seconds to keep NAT mappings alive
self.ws.run_forever(ping_interval=15, ping_timeout=10)
except Exception as e:
print(f"[TickDB] Unexpected exception in WebSocket loop: {e}")
if self._running:
self._retry_count += 1
if self._retry_count >= self._max_retries:
print(f"[TickDB] Max retries ({self._max_retries}) exceeded. Giving up.")
def start(self) -> None:
"""Start the WebSocket client in a background thread."""
if self._thread and self._thread.is_alive():
print("[TickDB] Client already running")
return
self._thread = threading.Thread(target=self._run_loop, daemon=True)
self._thread.start()
print("[TickDB] WebSocket client started")
def stop(self) -> None:
"""Gracefully stop the WebSocket client."""
print("[TickDB] Stopping client...")
self._running = False
if self.ws:
self.ws.close()
if self._thread:
self._thread.join(timeout=5)
# Example: Simple pressure ratio calculator from live depth updates
def calculate_pressure_ratio(bids: list[list], asks: list[list], levels: int = 5) -> float:
"""
Compute buy/sell pressure ratio from order book depth.
Buy pressure = sum of bid sizes at top N levels
Sell pressure = sum of ask sizes at top N levels
Ratio > 1.0 = buy pressure dominant; ratio < 1.0 = sell pressure dominant
This is a foundational microstructure metric for event-driven strategies.
"""
bid_total = sum(float(bid[1]) for bid in bids[:levels])
ask_total = sum(float(ask[1]) for ask in asks[:levels])
if ask_total == 0:
return float('inf')
return bid_total / ask_total
def on_depth(symbol_data: dict) -> None:
"""Callback invoked on each depth update."""
ratio = calculate_pressure_ratio(
symbol_data["bids"],
symbol_data["asks"],
levels=5,
)
print(f"[{symbol_data['symbol']}] Pressure ratio (top 5): {ratio:.2f}")
if __name__ == "__main__":
client = TickDBDepthStream(
symbols=["AAPL.US", "NVDA.US"],
on_depth_update=on_depth,
)
try:
client.start()
# Keep main thread alive for demonstration
time.sleep(60)
except KeyboardInterrupt:
pass
finally:
client.stop()
The distinction is clear in the code. The REST call fetches a complete, bounded response and returns. The WebSocket client maintains a persistent session, receives pushes asynchronously, and manages its own reconnection lifecycle. These are fundamentally different programming models.
Comparison: Where Each Protocol Belongs in the Data Stack
The following table maps TickDB's API endpoints to the recommended protocol, with rationale.
| Endpoint | Protocol | Why REST | Why WebSocket |
|---|---|---|---|
GET /v1/market/kline |
REST | Stored, static data; idempotent; cacheable; bounded response | — |
GET /v1/symbols/available |
REST | Infrequently changing reference data; safe to cache aggressively | — |
GET /v1/market/kline/latest |
REST | One-shot query for current candle; no continuous updates needed | — |
depth (WebSocket channel) |
WebSocket | — | High-frequency, irregular order book updates; server push required |
trade (WebSocket channel) |
WebSocket | — | Transaction stream; every trade is an event; push is essential |
kline (WebSocket channel) |
WebSocket | — | Live candle updates as new ticks arrive; push is more efficient than polling |
A common mistake is using WebSocket kline for historical backfills. WebSocket kline delivers live candle updates. For historical candles, use the REST /v1/market/kline endpoint. These are separate endpoints serving different data access patterns.
Hybrid Workflows: Connecting REST and WebSocket in a Production System
The most robust market data architectures combine REST and WebSocket, using each for what it does best.
The canonical quant workflow:
- REST
/v1/market/kline→ Fetch 3 years of historical data for backtesting. - REST
/v1/market/kline/latest→ Initialize live trading state when the session starts. - WebSocket
depth→ Maintain real-time order book awareness during the trading session. - WebSocket
trade→ Detect aggression signals (large prints, trade size anomalies). - WebSocket
kline→ Update live candle indicators as new ticks arrive.
[Historical Backtest] → REST /v1/market/kline
↓
[Strategy Engine] ← REST /v1/market/kline/latest (initialization)
↓
[Live Trading Session]
├── WebSocket depth → Order book pressure monitoring
├── WebSocket trade → Trade aggression detection
└── WebSocket kline → Real-time indicator updates
The REST endpoints are the foundation. The WebSocket endpoints are the real-time layer built on top. Using REST for real-time data or WebSocket for historical data adds complexity without benefit — and, as the opening anecdote illustrated, that complexity eventually manifests as a failed backtest on a Friday night.
Common Mistakes and How to Avoid Them
| Mistake | Symptom | Fix |
|---|---|---|
Polling REST /v1/market/kline every second for live data |
Missed candles during the poll interval; API rate limit errors | Switch to WebSocket kline channel for live updates |
Subscribing to WebSocket depth for historical backtesting |
Server sends only live data; no historical replay available | Use REST /v1/market/kline with appropriate time range |
| Hardcoding the API key in source code | Key committed to version control; immediate revocation required | Load from environment variable: os.environ.get("TICKDB_API_KEY") |
| No reconnection logic on WebSocket | Single network blip terminates the entire session | Implement exponential backoff + jitter (see code above) |
| No heartbeat / ping-pong | Proxy or NAT timeout closes idle connections silently | Send periodic pings; ws.run_forever(ping_interval=15) |
Using /kline/latest for backtesting loops |
One-shot endpoint does not paginate; limited candle count | Use /v1/market/kline with start_time/end_time pagination |
Summary
The REST versus WebSocket decision is not stylistic. It is determined by the nature of the data and the communication pattern required.
Use REST when you need a complete, bounded, static response — historical data, reference data, symbol metadata. REST is idempotent, cacheable, stateless, and integrates naturally with HTTP infrastructure. It is the correct tool for data retrieval.
Use WebSocket when you need a continuous, event-driven stream — live order book updates, trade prints, real-time candle ticks. WebSocket eliminates polling overhead, delivers sub-second latency, and matches the push nature of real-time market events. It is the correct tool for live data.
Violating this mapping — polling for real-time data, or streaming historical data over a persistent connection — does not just reduce performance. It breaks the fundamental assumptions of the protocol, creating failure modes that are subtle, intermittent, and difficult to diagnose.
Choose the right protocol for the data. Everything else follows.
Next Steps
If you are building a backtesting pipeline, start with the REST /v1/market/kline endpoint. TickDB provides 10+ years of cleaned, aligned US equity OHLCV data suitable for cross-cycle strategy validation. Sign up at tickdb.ai to get a free API key — no credit card required.
If you are building a live trading system, connect to the WebSocket depth and trade channels. The code in this article provides a production-ready starting point with heartbeat, reconnection, and error handling already implemented.
If you need institutional-grade data coverage — multiple asset classes, extended historical depth, or dedicated infrastructure — contact [email protected] for professional and enterprise plans.
This article does not constitute investment advice. Markets involve risk; past performance does not guarantee future results.