The Moment Everything Breaks

Your backtest runs clean. Your signal holds up across three years of data. You are ready.

Then you hire a second quant. Or a data engineer. Or you bring in a partner who wants to run their own strategies on the same data pipeline. Suddenly, your carefully curated data/ folder—organized by naming conventions only you understand—becomes a coordination nightmare. Who updated the nyse_ohlcv_2024.csv file last Tuesday? Did the preprocessing script change? Why does your partner's Sharpe ratio differ from yours by 0.3?

The transition from solo quant to small team is where most boutique quant operations quietly collapse. Not because the alpha degrades. Because the infrastructure was never designed for multiple humans.

This article builds the foundation: a production-grade data infrastructure for a 3-person quant team. We cover shared data access patterns, Git-based code collaboration, secure API key management, and role-based permissions. Every architecture decision is justified by the failure modes it prevents.

Why Small Teams Fail at Infrastructure (And How to Avoid It)

The root cause is almost never malicious. Small quant teams fail because their infrastructure evolves by accretion rather than by design.

The Four Canonical Failure Modes

Failure Mode Symptom Root Cause
Data fragmentation Each team member maintains their own copy of historical data No shared data source of record
Version drift Backtests produce different results across team members No version-controlled preprocessing scripts
Credential sprawl API keys scattered across .env files, Slack DMs, spreadsheets No centralized secrets management
Permission ambiguity Junior quant accidentally overwrites shared feature store No role-based access control

These failures compound. A 3-person team where each member maintains their own data copy is already spending roughly 30% of engineering time on data reconciliation. Add a fourth person and that cost becomes untenable.

The solution is not a single tool. It is a small number of well-chosen tools with clear ownership boundaries.

Architecture Overview: The Three-Layer Model

For a 3-person quant team, we recommend a three-layer infrastructure architecture:

┌─────────────────────────────────────────────────────────────┐
│                    PRESENTATION LAYER                        │
│  (Analysts' notebooks, research environments, dashboards)     │
├─────────────────────────────────────────────────────────────┤
│                    SERVICE LAYER                             │
│  (Shared API gateway, backtest runner, feature store)        │
├─────────────────────────────────────────────────────────────┤
│                    DATA LAYER                                │
│  (Market data warehouse, tick database, preprocessing jobs)  │
└─────────────────────────────────────────────────────────────┘

Layer Responsibilities

Layer Components Owner in 3-person team
Presentation Jupyter notebooks, FastAPI dashboards, report generators All team members
Service API gateway, backtest orchestration, webhook listeners Lead engineer
Data TickDB ingestion, PostgreSQL feature store, Airflow scheduler Data engineer

In a 3-person team, you do not have the luxury of dedicated DevOps. Every team member touches every layer. The architecture must be simple enough for a generalist to maintain and robust enough to survive neglect.

Data Sharing: The Shared Database Pattern

The Problem with File-Based Data Sharing

The naive approach is to share a network drive or a cloud storage bucket with raw CSV files. This works until someone runs a preprocessing job that mutates the shared data in place, breaking every other team member's notebook.

The fix is a database of record: a single source of truth that all team members query, never mutate directly.

Implementing a Shared Data Warehouse

For a 3-person quant team, we recommend the following stack:

  • TickDB as the primary market data API (ingestion layer)
  • PostgreSQL as the shared feature and signal store
  • Apache Airflow (or a lightweight alternative like Cron + scripts) for scheduled ingestion jobs

The key principle: raw market data enters through a controlled ingestion path. Preprocessed features are written to the shared database. Analysts read from the database, never from raw files.

# ingestion/shared_ingestion.py
"""
Shared data ingestion job — runs on Airflow or cron.
All team members read from the resulting tables; no one writes to raw data.
"""
import os
import time
import psycopg2
from psycopg2.extras import execute_values
import requests
from datetime import datetime, timedelta

# ─── Configuration ────────────────────────────────────────────
POSTGRES_CONN = os.environ.get("SHARED_DB_URL")
TICKDB_API_KEY = os.environ.get("TICKDB_API_KEY")  # Loaded once, not scattered in .env files
BATCH_SIZE = 1000

# ─── Error handling template ───────────────────────────────────
def handle_api_error(response, attempt, max_retries=5):
    """Standard error handler with exponential backoff."""
    code = response.get("code", 0)
    
    if code == 0:
        return  # Success
    
    if code in (1001, 1002):
        raise ValueError("Invalid TickDB API key — check SHARED_DB_URL or TICKDB_API_KEY")
    
    if code == 2002:
        raise KeyError(f"Symbol not found — verify via /v1/symbols/available")
    
    if code == 3001:
        retry_after = int(response.headers.get("Retry-After", 5))
        print(f"Rate limited. Retrying after {retry_after} seconds.")
        time.sleep(retry_after)
        return  # Retry the same request
    
    raise RuntimeError(f"Unexpected error {code}: {response.get('message')}")

# ─── Ingestion function ─────────────────────────────────────────
def fetch_and_store_klines(symbol: str, interval: str, lookback_days: int = 30):
    """
    Fetch historical OHLCV data from TickDB and store in shared PostgreSQL.
    This is the single write path to the shared data warehouse.
    """
    headers = {"X-API-Key": TICKDB_API_KEY}
    
    end_time = datetime.utcnow()
    start_time = end_time - timedelta(days=lookback_days)
    
    params = {
        "symbol": symbol,
        "interval": interval,
        "start_time": int(start_time.timestamp()),
        "end_time": int(end_time.timestamp()),
        "limit": BATCH_SIZE
    }
    
    response = requests.get(
        "https://api.tickdb.ai/v1/market/kline",
        headers=headers,
        params=params,
        timeout=(3.05, 10)  # (connect_timeout, read_timeout)
    )
    
    response.raise_for_status()
    data = response.json()
    
    # Check for API-level errors
    handle_api_error(data, attempt=1)
    
    candles = data.get("data", {}).get("klines", [])
    
    if not candles:
        print(f"No data returned for {symbol}. Skipping.")
        return
    
    # Prepare batch insert
    records = [
        (
            symbol,
            c["timestamp"],
            c["open"],
            c["high"],
            c["low"],
            c["close"],
            c["volume"],
            datetime.utcnow()
        )
        for c in candles
    ]
    
    insert_sql = """
        INSERT INTO ohlcv_1h (symbol, timestamp, open, high, low, close, volume, ingested_at)
        VALUES %s
        ON CONFLICT (symbol, timestamp) DO UPDATE SET
            open = EXCLUDED.open,
            high = EXCLUDED.high,
            low = EXCLUDED.low,
            close = EXCLUDED.close,
            volume = EXCLUDED.volume,
            ingested_at = EXCLUDED.ingested_at
    """
    
    conn = psycopg2.connect(POSTGRES_CONN)
    with conn:
        with conn.cursor() as cur:
            execute_values(cur, insert_sql, records, page_size=500)
    conn.close()
    
    print(f"Ingested {len(records)} candles for {symbol}")

# ─── Entry point ────────────────────────────────────────────────
if __name__ == "__main__":
    symbols = ["AAPL.US", "MSFT.US", "GOOGL.US"]
    for symbol in symbols:
        fetch_and_store_klines(symbol, "1h", lookback_days=30)

Engineering warning: This ingestion script runs on a shared server. API keys are loaded from environment variables at runtime, never hardcoded. If you need to rotate the API key, update the environment variable in one place; the next scheduled run picks it up automatically.

Git Collaboration: Structuring a Quant Research Repository

The Repository Layout

A 3-person quant team should maintain a single monorepo with the following structure:

quant-research/
├── data/                    # Generated data (gitignored)
│   ├── features/
│   └── backtests/
├── src/
│   ├── ingestion/           # Data pipeline code (version-controlled)
│   ├── features/            # Feature engineering
│   ├── strategies/          # Strategy implementations
│   └── utils/               # Shared utilities
├── notebooks/               # Research notebooks
├── tests/                   # Unit and integration tests
├── configs/                 # Strategy configs (YAML)
├── .env.example             # Template for environment variables
├── README.md
└── requirements.txt

The .env File Problem

Never commit .env files. Instead, commit .env.example:

# .env.example — commit this to the repository
# Each team member copies this file to .env and fills in their values

# Market data API
TICKDB_API_KEY=your_api_key_here

# Shared database
SHARED_DB_URL=postgresql://user:password@host:5432/quantdb

# Feature store credentials
FEATURE_STORE_URL=http://localhost:8080

# Slack webhook for alerts
SLACK_WEBHOOK_URL=https://hooks.slack.com/services/YOUR/WEBHOOK/URL

# .gitignore — never commit these
.env
data/
*.csv
*.parquet
__pycache__/
.ipynb_checkpoints/
.pytest_cache/

Branching Strategy for a 3-Person Team

For a small quant team, we recommend a simplified GitFlow:

Branch Purpose Who pushes
main Stable, backtested strategies only Lead engineer (PR + review required)
develop Integration branch for feature candidates All team members
feature/strategy-name Individual strategy development Assigned researcher
hotfix/urgent-fix Critical production fixes Any team member

The critical rule: A strategy cannot move from develop to main without a backtest run on the shared data warehouse. This prevents the "it works on my data copy" problem.

# Example: Merging a strategy to main
git checkout develop
git merge feature/momentum-factor
# Run shared backtest
python -m pytest tests/backtest/test_momentum_factor.py --shared-db
# If all tests pass:
git checkout main
git merge develop
git tag -a v1.2 -m "Momentum factor live"
git push origin main --tags

API Key Management: Centralized Secrets

The Credential Sprawl Problem

In a 3-person team, each member typically signs up for their own API key. This creates multiple problems:

  1. Billing fragmentation: Individual keys make it impossible to track which team member's usage is generating costs.
  2. Revocation complexity: If a team member leaves, revoking their key requires knowing which services they accessed.
  3. Security surface: Scattered API keys across personal machines are a compliance and security risk.

The Solution: Centralized Secrets Management

All team members share a single team API key for production data access. Individual keys are used only for personal experimentation.

# config/team_config.py
"""
Team configuration — loaded from environment variables.
Centralizes all API credentials in one place.
"""
import os
from dataclasses import dataclass

@dataclass
class TeamConfig:
    tickdb_api_key: str
    shared_db_url: str
    feature_store_url: str
    slack_webhook_url: str
    
    @classmethod
    def from_env(cls):
        required = ["TICKDB_API_KEY", "SHARED_DB_URL"]
        missing = [k for k in required if not os.environ.get(k)]
        
        if missing:
            raise EnvironmentError(
                f"Missing required environment variables: {', '.join(missing)}.\n"
                "Copy .env.example to .env and fill in your values."
            )
        
        return cls(
            tickdb_api_key=os.environ.get("TICKDB_API_KEY"),
            shared_db_url=os.environ.get("SHARED_DB_URL"),
            feature_store_url=os.environ.get("FEATURE_STORE_URL", "http://localhost:8080"),
            slack_webhook_url=os.environ.get("SLACK_WEBHOOK_URL", "")
        )

# Usage in any script
config = TeamConfig.from_env()

Key Rotation Protocol

When a team member leaves or a key is compromised:

# 1. Generate new key in TickDB dashboard
# 2. Update the environment variable on the shared ingestion server
# 3. Notify team members to update their local .env
# 4. Revoke the old key in the dashboard
# 5. Document the rotation in the team wiki

This is a 5-minute process that eliminates the risk of a departed team member retaining API access.

Role-Based Access Control: Who Can Do What

Three Roles for a 3-Person Quant Team

Role Capabilities Typical owner
Admin Full access: manage team members, rotate API keys, delete data Lead engineer
Researcher Read/write shared database, run backtests, commit to develop Quant researchers
Observer Read-only access to shared database and dashboards External advisors, investors

Implementing Permissions in PostgreSQL

-- admin_access.sql
-- Run as superuser on the shared database

-- Create roles
CREATE ROLE quant_admin;
CREATE ROLE quant_researcher;
CREATE ROLE quant_observer;

-- Admin: full access to all schemas
GRANT ALL PRIVILEGES ON DATABASE quantdb TO quant_admin;
GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO quant_admin;
GRANT ALL PRIVILEGES ON ALL SEQUENCES IN SCHEMA public TO quant_admin;

-- Researcher: read/write on data schemas, no DDL on production tables
GRANT CONNECT ON DATABASE quantdb TO quant_researcher;
GRANT USAGE ON SCHEMA public TO quant_researcher;
GRANT SELECT, INSERT, UPDATE, DELETE ON ALL TABLES IN SCHEMA public TO quant_researcher;
GRANT USAGE, SELECT ON ALL SEQUENCES IN SCHEMA public TO quant_researcher;

-- Observer: read-only access to views and materialized tables
GRANT CONNECT ON DATABASE quantdb TO quant_observer;
GRANT USAGE ON SCHEMA public TO quant_observer;
GRANT SELECT ON ALL TABLES IN SCHEMA public TO quant_observer;

-- Assign users to roles
-- Adjust usernames to match your PostgreSQL setup
CREATE ROLE alice WITH LOGIN PASSWORD 'secure_password_alice';
CREATE ROLE bob WITH LOGIN PASSWORD 'secure_password_bob';
CREATE ROLE charlie WITH LOGIN PASSWORD 'secure_password_charlie';

GRANT quant_admin TO alice;       -- Lead engineer
GRANT quant_researcher TO bob;    -- Quant researcher 1
GRANT quant_researcher TO charlie; -- Quant researcher 2

Why This Matters More Than You Think

When a junior quant accidentally runs a destructive query on the shared feature store, the consequence is not just data loss. It is a 4-hour debugging session where two other team members cannot run their backtests. Role-based access controls are not bureaucratic overhead. They are circuit breakers that prevent team-wide disruption from individual mistakes.

Deployment by Team Size

Team size Recommended stack Key difference
1 person Local Python scripts, SQLite No shared infrastructure needed
3 persons Shared PostgreSQL, Git monorepo, TickDB Single team API key, basic RBAC
5–10 persons Add Airflow, feature store, CI/CD pipeline Per-feature API keys, automated testing
10+ persons Full data platform, dedicated DevOps Microservices, audit logging, SLA monitoring

For a 3-person team, the goal is to establish the right patterns, not to over-engineer. A shared PostgreSQL instance and a disciplined Git workflow will take you further than a complex Kubernetes cluster that no one has time to maintain.

Common Pitfalls and How to Prevent Them

Pitfall Why it happens Prevention
Everyone uses their own data copy Shared database not set up on day one Establish the shared warehouse before the first backtest
Preprocessing scripts diverge No version-controlled preprocessing All feature engineering code goes in src/features/ with tests
Backtests produce different results Different data timestamps or preprocessing logic Require shared data warehouse + CI/CD backtest validation
API keys in Slack DMs No centralized secrets management Enforce .env pattern; rotate keys on team changes
Nobody knows who broke the pipeline No audit trail on shared resources Log all ingestion jobs with timestamps and user identity

Next Steps

If you're a solo quant ready to bring on a partner, start with the shared database. Set up PostgreSQL, point all your ingestion scripts to it, and establish the .env pattern before the second person joins. Retrofitting is always harder than building right the first time.

If you already have a team with scattered data, the migration path is: (1) stand up a shared PostgreSQL instance, (2) run a one-time backfill script from your TickDB historical data, (3) update each team member's scripts to read from the shared database instead of local files, (4) deprecate the local copies. Budget one sprint (1–2 weeks) for this migration.

If you need enterprise-grade data access for your team, TickDB offers team plans with centralized API key management, usage reporting by endpoint, and priority support. Visit tickdb.ai to explore plans that support multi-user access.

This article does not constitute investment advice. Markets involve risk; past performance does not guarantee future results. Infrastructure decisions should be evaluated against your team's specific regulatory and operational requirements.