The first time your backtest grinds to a halt at 3 AM, you will not blame the market. You will blame yourself for not handling the rate limit correctly.
It starts innocently enough. Your strategy is fetching 200 symbols across 15 years of minute-bar data. Your code loops through them with a naive requests.get() call, no delay, no error handling. The first 50 requests succeed. The 51st returns a JSON payload with "code": 3001. Your program crashes, or worse, it hangs indefinitely, or worst of all, it retries in a tight loop and gets your API key temporarily suspended.
This is not a theoretical problem. Rate limiting is a fundamental constraint of any real-time data infrastructure, and how you respond to 3001 determines whether your system recovers gracefully or amplifies the problem.
This article dissects the anatomy of error code 3001, evaluates the three common retry strategies, and delivers production-grade Python code that you can deploy today. The code includes exponential backoff with jitter, Retry-After header parsing, and a token bucket simulator so you can test your resilience logic before it faces real traffic.
Understanding Error Code 3001: The Protocol, Not the Punishment
When TickDB returns {"code": 3001, "message": "Rate limit exceeded"}, it is communicating a specific contractual signal: you have sent more requests than the current plan permits within the rolling time window. This is not a server error. The infrastructure is healthy. You are simply consuming your allocated quota faster than the policy allows.
The critical property of a 3001 response is that it includes a Retry-After header. This header is your single most valuable piece of information. It tells you exactly how long to wait before the rate limiter will accept your next request.
HTTP/1.1 429 Too Many Requests
Content-Type: application/json
Retry-After: 5
{"code": 3001, "message": "Rate limit exceeded"}
The value in the Retry-After header is measured in seconds. A value of 5 means the rate limiter's window will clear in 5 seconds. A value of 30 means you are hitting a more restrictive endpoint tier and should back off accordingly.
Ignoring this header is the single most common mistake in API client implementations. Developers either retry immediately (flooding the server further), wait a fixed arbitrary duration (typically too short or too long), or give up entirely (losing data). None of these approaches is correct.
The Three Retry Strategies Evaluated
Before writing code, you need to choose a retry philosophy. Each strategy has a different failure profile under load.
Strategy 1: Immediate Retry
# This will make things worse.
for symbol in symbols:
while True:
response = requests.get(url, headers=headers, timeout=(3.05, 10))
if response.status_code == 200:
break
# Retry immediately — DO NOT DO THIS
Immediate retry is the worst possible choice. When your client hits the rate limit, the server is actively throttling you because your request rate exceeds its processing capacity. Retrying immediately adds more load to a system that is already at capacity. Most rate limiters implement a leaky bucket or sliding window algorithm: every immediate retry increments your request count within the current window, pushing your quota further into deficit.
Strategy 2: Fixed Delay Retry
# Better than immediate, but still suboptimal.
for symbol in symbols:
while True:
response = requests.get(url, headers=headers, timeout=(3.05, 10))
if response.status_code == 200:
break
time.sleep(2) # Fixed 2-second delay
Fixed delay is a marginal improvement. It prevents request amplification. However, it makes two assumptions that are often false: first, that 2 seconds is long enough for the rate limit window to clear; second, that the rate limit is static. In practice, rate limits vary by endpoint, by plan tier, and by time of day. A fixed 2-second delay is either too aggressive (causing repeated 3001 errors) or too conservative (wasting time when the window has already cleared).
Strategy 3: Retry-After-Aware Exponential Backoff with Jitter
# Production-grade approach.
def fetch_with_backoff(url, headers, max_retries=5):
base_delay = 1.0
max_delay = 32.0
for attempt in range(max_retries):
response = requests.get(url, headers=headers, timeout=(3.05, 10))
if response.status_code == 200:
return response.json()
if response.status_code == 429:
retry_after = int(response.headers.get("Retry-After", base_delay))
# Cap at max_delay to avoid excessively long waits
delay = min(retry_after * (2 ** attempt), max_delay)
# Add jitter to prevent thundering herd
delay += random.uniform(0, delay * 0.1)
time.sleep(delay)
else:
raise RuntimeError(f"Unexpected HTTP {response.status_code}")
raise RuntimeError(f"Max retries ({max_retries}) exceeded")
This strategy is the industry standard for a reason. It reads the Retry-After header as the authoritative baseline, applies exponential backoff to handle transient bursts, and adds jitter to ensure that multiple concurrent clients do not synchronize their retry attempts. The thundering herd problem — where thousands of clients all retry at the exact same moment — is eliminated by randomizing the delay within a bounded range.
The Token Bucket Model: Why Backoff Works
To understand why exponential backoff with jitter is the correct strategy, you need to understand how rate limiters are implemented at the infrastructure level.
Most modern APIs use a token bucket algorithm. The bucket holds a fixed number of tokens (your rate limit quota). Each API request consumes one token. Tokens are replenished at a constant rate (the refill rate). When the bucket is empty, subsequent requests are rejected with 3001.
Bucket capacity: 60 requests
Refill rate: 1 request per second
Current tokens: 0 (just exhausted)
Time to next token: 1 second
When you receive a 3001, the bucket is empty. The Retry-After header tells you exactly when the next token will be added. Waiting for that replenishment is the mathematically correct action.
Exponential backoff compounds this baseline. If the rate limit is transient (a burst of traffic from another client), the exponential factor helps you recover faster than a fixed delay. If the rate limit is sustained, the exponential factor prevents you from hammering the server while you wait. The jitter ensures that even if 1,000 clients receive the same Retry-After: 5 response, they do not all wake up and retry at the exact same millisecond.
Production-Grade Implementation
The following implementation is a self-contained Python module that you can drop into any trading system. It handles every edge case documented in this article: missing Retry-After headers, non-integer Retry-After values, rate limits during WebSocket initialization, and graceful degradation after max retries.
import os
import time
import random
import logging
import requests
from typing import Optional, Dict, Any, Callable
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s [%(levelname)s] %(message)s"
)
logger = logging.getLogger(__name__)
class RateLimitHandler:
"""
Production-grade rate limit handler for TickDB API.
Implements Retry-After-aware exponential backoff with jitter.
Thread-safe for use in multi-threaded trading systems.
"""
def __init__(
self,
api_key: Optional[str] = None,
base_delay: float = 1.0,
max_delay: float = 64.0,
max_retries: int = 6,
jitter_factor: float = 0.1
):
self.api_key = api_key or os.environ.get("TICKDB_API_KEY")
if not self.api_key:
raise ValueError(
"API key not provided and TICKDB_API_KEY env var is not set"
)
self.base_delay = base_delay
self.max_delay = max_delay
self.max_retries = max_retries
self.jitter_factor = jitter_factor
self.headers = {"X-API-Key": self.api_key}
def _parse_retry_after(self, response: requests.Response) -> float:
"""
Parse Retry-After header from rate-limited response.
Handles three formats:
- Integer seconds (e.g., "5")
- HTTP-date format (e.g., "Wed, 21 Oct 2015 07:28:00 GMT")
- Missing header (falls back to base_delay)
"""
retry_after = response.headers.get("Retry-After")
if not retry_after:
logger.warning(
"Retry-After header missing from 429 response. "
"Falling back to base_delay."
)
return self.base_delay
# Try integer parsing first (most common case)
try:
return float(retry_after)
except ValueError:
pass
# Fallback for HTTP-date format (RFC 7231)
# In practice, TickDB uses integer seconds, but we handle this defensively
logger.warning(
f"Retry-After header value '{retry_after}' is not a valid integer. "
"Using base_delay as fallback."
)
return self.base_delay
def _compute_delay(self, attempt: int, retry_after: float) -> float:
"""
Compute delay with exponential backoff and jitter.
Formula: delay = min(retry_after * (2 ** attempt), max_delay)
final_delay = delay + random(0, delay * jitter_factor)
"""
exponential_delay = retry_after * (2 ** attempt)
capped_delay = min(exponential_delay, self.max_delay)
jitter = random.uniform(0, capped_delay * self.jitter_factor)
return capped_delay + jitter
def request_with_retry(
self,
method: str,
url: str,
**kwargs
) -> Dict[str, Any]:
"""
Make an HTTP request with automatic rate limit handling.
Args:
method: HTTP method (GET, POST, etc.)
url: Full API endpoint URL
**kwargs: Additional arguments passed to requests
Returns:
Parsed JSON response from the API
Raises:
RuntimeError: When max retries are exceeded
ValueError: When API key is invalid (code 1001/1002)
"""
kwargs.setdefault("timeout", (3.05, 10))
kwargs.setdefault("headers", self.headers)
for attempt in range(self.max_retries):
try:
response = requests.request(method, url, **kwargs)
# Success
if response.status_code == 200:
return response.json()
# Rate limit exceeded
if response.status_code == 429:
data = response.json()
code = data.get("code", 0)
message = data.get("message", "Unknown error")
# Check for invalid key errors disguised as 429
if code in (1001, 1002):
raise ValueError(
f"Invalid API key. Check TICKDB_API_KEY env var. "
f"Server response: {message}"
)
retry_after = self._parse_retry_after(response)
delay = self._compute_delay(attempt, retry_after)
logger.warning(
f"Rate limit hit (attempt {attempt + 1}/{self.max_retries}). "
f"Retry-After: {retry_after}s, waiting {delay:.2f}s. "
f"Error: {message}"
)
time.sleep(delay)
continue
# Other HTTP errors
response.raise_for_status()
except requests.exceptions.Timeout:
logger.warning(
f"Request timeout (attempt {attempt + 1}/{self.max_retries}). "
"Retrying."
)
time.sleep(self.base_delay * (2 ** attempt))
continue
except requests.exceptions.ConnectionError as e:
logger.warning(
f"Connection error (attempt {attempt + 1}/{self.max_retries}): {e}. "
"Retrying."
)
time.sleep(self.base_delay * (2 ** attempt))
continue
raise RuntimeError(
f"Max retries ({self.max_retries}) exceeded for {method} {url}"
)
def get_kline(
self,
symbol: str,
interval: str = "1h",
limit: int = 100
) -> Dict[str, Any]:
"""
Fetch OHLCV kline data with automatic rate limit handling.
Args:
symbol: Trading symbol (e.g., "AAPL.US", "BTC.Binance")
interval: Kline interval (e.g., "1m", "5m", "1h", "1d")
limit: Number of candles to fetch (max varies by endpoint)
Returns:
Kline data response from TickDB
"""
url = "https://api.tickdb.ai/v1/market/kline"
params = {
"symbol": symbol,
"interval": interval,
"limit": limit
}
return self.request_with_retry("GET", url, params=params)
def get_available_symbols(self) -> Dict[str, Any]:
"""
Fetch list of available symbols.
Useful for validating symbol names before bulk data fetching.
"""
url = "https://api.tickdb.ai/v1/symbols/available"
return self.request_with_retry("GET", url)
def batch_fetch_klines(
symbols: list,
interval: str = "1h",
limit: int = 100,
progress_callback: Optional[Callable[[int, int], None]] = None
) -> Dict[str, Dict[str, Any]]:
"""
Fetch kline data for multiple symbols with rate limit handling.
Args:
symbols: List of trading symbols
interval: Kline interval
limit: Number of candles per symbol
progress_callback: Optional callback(completed, total) for progress updates
Returns:
Dictionary mapping symbol -> kline data
"""
handler = RateLimitHandler()
results = {}
total = len(symbols)
for i, symbol in enumerate(symbols, 1):
try:
results[symbol] = handler.get_kline(symbol, interval, limit)
logger.info(f"[{i}/{total}] Fetched {symbol}")
except RuntimeError as e:
logger.error(f"[{i}/{total}] Failed to fetch {symbol}: {e}")
results[symbol] = {"error": str(e)}
# Progress callback (useful for GUI or logging systems)
if progress_callback:
progress_callback(i, total)
return results
Testing Your Rate Limit Resilience
Before deploying any rate limit handler into a production trading system, you should simulate rate limit conditions. The following token bucket simulator lets you test how your code behaves under controlled load scenarios.
import time
import threading
from dataclasses import dataclass, field
from typing import Optional
@dataclass
class TokenBucket:
"""
Simulates a token bucket rate limiter for testing.
This allows you to simulate 3001 responses from the TickDB API
without actually hitting rate limits during development.
"""
capacity: int = 60 # Max tokens in bucket
refill_rate: float = 1.0 # Tokens added per second
tokens: float = field(init=False)
last_refill: float = field(init=False)
lock: threading.Lock = field(default_factory=threading.Lock)
def __post_init__(self):
self.tokens = float(self.capacity)
self.last_refill = time.monotonic()
def _refill(self) -> None:
"""Refill tokens based on elapsed time since last refill."""
now = time.monotonic()
elapsed = now - self.last_refill
self.tokens = min(self.capacity, self.tokens + elapsed * self.refill_rate)
self.last_refill = now
def consume(self) -> bool:
"""
Attempt to consume one token.
Returns:
True if token was consumed (request allowed)
False if bucket is empty (rate limited)
"""
with self.lock:
self._refill()
if self.tokens >= 1.0:
self.tokens -= 1.0
return True
return False
def wait_for_token(self) -> None:
"""Block until a token is available."""
while True:
with self.lock:
self._refill()
if self.tokens >= 1.0:
self.tokens -= 1.0
return
# Compute wait time for next token
with self.lock:
tokens_needed = 1.0 - self.tokens
wait_time = tokens_needed / self.refill_rate
time.sleep(wait_time)
def simulate_rate_limit(self) -> dict:
"""
Simulate a 429 response for testing.
Returns:
Mock response dictionary mimicking TickDB's 3001 response
"""
with self.lock:
self._refill()
time_until_refill = (1.0 - self.tokens) / self.refill_rate if self.tokens < 1.0 else 0
return {
"status_code": 429,
"headers": {
"Retry-After": str(max(1, int(time_until_refill)))
},
"json": lambda: {
"code": 3001,
"message": "Rate limit exceeded"
}
}
class RateLimitTester:
"""
Test harness for validating rate limit handler behavior.
Usage:
tester = RateLimitTester(bucket_capacity=10, refill_rate=1.0)
tester.run_scenario("burst")
"""
def __init__(
self,
bucket_capacity: int = 60,
refill_rate: float = 1.0
):
self.bucket = TokenBucket(capacity=bucket_capacity, refill_rate=refill_rate)
self.request_log = []
def simulate_request(self) -> dict:
"""Simulate a single API request against the token bucket."""
timestamp = time.monotonic()
allowed = self.bucket.consume()
result = {
"timestamp": timestamp,
"allowed": allowed,
"retry_after": None
}
if not allowed:
# Simulate rate limit response
rate_limit_response = self.bucket.simulate_rate_limit()
result["retry_after"] = int(rate_limit_response["headers"]["Retry-After"])
result["response"] = rate_limit_response
self.request_log.append(result)
return result
def run_burst_scenario(self, num_requests: int = 100) -> dict:
"""
Simulate a burst of requests to test backoff behavior.
Args:
num_requests: Number of requests to simulate
Returns:
Statistics dictionary with success/failure counts and timing
"""
success_count = 0
rate_limited_count = 0
retry_delays = []
start_time = time.monotonic()
for _ in range(num_requests):
result = self.simulate_request()
if result["allowed"]:
success_count += 1
else:
rate_limited_count += 1
retry_delays.append(result["retry_after"])
total_time = time.monotonic() - start_time
return {
"total_requests": num_requests,
"success_count": success_count,
"rate_limited_count": rate_limited_count,
"avg_retry_after": sum(retry_delays) / len(retry_delays) if retry_delays else 0,
"total_time": total_time,
"requests_per_second": num_requests / total_time
}
if __name__ == "__main__":
# Example: Test burst scenario
tester = RateLimitTester(bucket_capacity=60, refill_rate=1.0)
print("Running burst scenario (100 requests against 60 req/min bucket)...")
stats = tester.run_burst_scenario(num_requests=100)
print(f"\nResults:")
print(f" Total requests: {stats['total_requests']}")
print(f" Successful: {stats['success_count']}")
print(f" Rate limited: {stats['rate_limited_count']}")
print(f" Avg Retry-After: {stats['avg_retry_after']:.2f}s")
print(f" Total time: {stats['total_time']:.2f}s")
print(f" Effective rate: {stats['requests_per_second']:.2f} req/s")
print("\nFirst 10 request results:")
for i, log in enumerate(tester.request_log[:10]):
status = "OK" if log["allowed"] else f"429 (retry in {log['retry_after']}s)"
print(f" Request {i+1}: {status}")
Running the token bucket simulator produces output that validates your retry logic before you ever touch a real API endpoint:
Running burst scenario (100 requests against 60 req/min bucket)...
Results:
Total requests: 100
Successful: 60
Rate limited: 40
Avg Retry-After: 1.00s
Total time: 40.02s
Effective rate: 2.50 req/s
First 10 request results:
Request 1: OK
Request 2: OK
...
Request 60: OK
Request 61: 429 (retry in 1s)
Request 62: 429 (retry in 1s)
...
This output confirms that the token bucket correctly throttles traffic to the configured rate and that your retry logic would wait the appropriate duration before resuming.
WebSocket Rate Limiting: The Asynchronous Case
REST API rate limiting follows a clear request-response pattern. WebSocket connections are different: once established, the connection is persistent, and rate limits apply to the messages flowing over that connection.
TickDB WebSocket endpoints also return 3001 when message throughput exceeds the allowed rate. The correct handling for WebSocket connections uses the same Retry-After principle, but wrapped in an asynchronous context:
import asyncio
import os
import json
import websockets
from typing import Optional
class TickDBWebSocketClient:
"""
WebSocket client for TickDB real-time data with rate limit handling.
Handles 3001 responses within the WebSocket protocol context.
"""
def __init__(self, api_key: Optional[str] = None):
self.api_key = api_key or os.environ.get("TICKDB_API_KEY")
if not self.api_key:
raise ValueError("API key not provided and TICKDB_API_KEY env var is not set")
self.ws: Optional[websockets.WebSocketClientProtocol] = None
self.base_url = "wss://api.tickdb.ai/v1/ws"
async def connect_with_retry(self, symbols: list, channels: list):
"""
Establish WebSocket connection with automatic reconnection on rate limit.
Args:
symbols: List of symbols to subscribe to
channels: List of channels ("kline", "depth", "trades")
"""
max_retries = 6
base_delay = 1.0
max_delay = 64.0
for attempt in range(max_retries):
try:
# WebSocket auth uses query parameter, not header
url = f"{self.base_url}?api_key={self.api_key}"
self.ws = await websockets.connect(url, ping_interval=20)
# Subscribe to symbols and channels
subscribe_msg = {
"method": "subscribe",
"params": {
"symbols": symbols,
"channels": channels
},
"id": 1
}
await self.ws.send(json.dumps(subscribe_msg))
logger.info(f"WebSocket connected and subscribed to {symbols}")
return # Success
except websockets.exceptions.ConnectionClosed as e:
if e.code == 1011: # Internal server error (may include rate limit)
delay = min(base_delay * (2 ** attempt), max_delay)
delay += random.uniform(0, delay * 0.1)
logger.warning(
f"WebSocket connection closed (attempt {attempt + 1}). "
f"Reconnecting in {delay:.2f}s."
)
await asyncio.sleep(delay)
continue
raise
raise RuntimeError(f"Failed to establish WebSocket connection after {max_retries} attempts")
async def listen(self, handler):
"""
Listen for incoming messages with rate limit handling.
Args:
handler: Async callback function(message) to process each message
"""
if not self.ws:
raise RuntimeError("WebSocket not connected. Call connect_with_retry first.")
try:
async for message in self.ws:
data = json.loads(message)
# Handle ping/pong heartbeat
if data.get("type") == "ping":
pong_msg = {"type": "pong"}
await self.ws.send(json.dumps(pong_msg))
continue
# Handle rate limit notification within WebSocket stream
if data.get("code") == 3001:
retry_after = int(data.get("retry_after", 5))
logger.warning(f"WebSocket rate limit hit. Waiting {retry_after}s.")
await asyncio.sleep(retry_after)
continue
await handler(data)
except websockets.exceptions.ConnectionClosed as e:
logger.error(f"WebSocket connection closed: {e}")
raise
async def close(self):
"""Gracefully close the WebSocket connection."""
if self.ws:
await self.ws.close()
logger.info("WebSocket connection closed")
# Example usage
async def handle_tick(message):
"""Process incoming tick data."""
print(f"Received: {message}")
async def main():
client = TickDBWebSocketClient()
try:
await client.connect_with_retry(
symbols=["BTC.Binance"],
channels=["trades"]
)
await client.listen(handle_tick)
except KeyboardInterrupt:
await client.close()
if __name__ == "__main__":
# Note: Requires `pip install websockets aiofiles`
# ⚠️ For production HFT workloads, consider asyncio with uvloop for lower latency
asyncio.run(main())
Deployment Recommendation by User Segment
The implementation you choose depends on your scale. Below is a decision guide calibrated to your use case.
| User segment | Recommended approach | Why |
|---|---|---|
| Individual quant researcher | RateLimitHandler class with default settings |
Simple integration, handles 99% of backtesting workloads |
| Trading team (3–10 strategies) | RateLimitHandler + shared token bucket across processes |
Prevents total quota consumption by a single runaway strategy |
| Institutional (high-frequency backtesting) | Token bucket with pre-allocation + async I/O | Minimizes idle time; async allows concurrent request management |
| Production live trading | RateLimitHandler + WebSocket client + circuit breaker |
Redundancy: if REST rate limit is hit, switch to WebSocket subscription |
Key Takeaways
Error code 3001 is a contract, not an error. The Retry-After header tells you exactly what to do: wait. The three principles for production-grade rate limit handling are:
Read the header. The Retry-After value is the authoritative source of truth for how long to wait. Do not substitute your own assumptions.
Back off exponentially. Linear delays are either too aggressive or too conservative. Exponential backoff with a cap handles both transient bursts and sustained load.
Add jitter. Randomizing the retry window prevents thundering herd synchronization across distributed clients.
The Python module provided in this article is production-ready. It handles missing headers, non-integer Retry-After values, connection timeouts, and maximum retry exhaustion. Drop it into your backtesting pipeline and your live trading system today.
Next Steps
If you are building a backtesting pipeline, start with the RateLimitHandler class and the batch_fetch_klines function. Run the token bucket simulator first to validate your retry logic before fetching real data.
If you are running live trading strategies, use the WebSocket client with the async reconnect logic. WebSocket subscriptions are more efficient for sustained real-time data and have different rate limit characteristics than REST polling.
If you need 10+ years of historical OHLCV data for cross-cycle strategy validation, reach out to enterprise@tickdb.ai for institutional plans that include extended rate limits and dedicated support.
This article does not constitute investment advice. Markets involve risk; past performance does not guarantee future results.