Real-Time Backend
Financial Trading
Every microsecond on NYSE and Binance, dozens of algorithms fight for the same order. The loser loses money.
- **Binance** processes up to 1.4 million orders per second and supports WebSocket streams with <10 ms latency for 2,000+ trading pairs at once
- **NYSE** moves up to 5B messages per day over XDP (eXtreme Data Platform); a direct data feed runs $50,000+ per month for institutional clients
- **Virtu Financial** is the HFT firm that traded profitably 1,237 of 1,238 trading days in a row (2009-2014) using co-location and FIX with 50-100 us latency
- **Jump Trading** built a microwave link Chicago-NJ in 2010 for $14M, cutting latency from 13 to 8.1 ms; it paid off in months
Order Book
An order book is a two-sided queue of buy orders (bids) and sell orders (asks) for a financial instrument, sorted by price. Binance handles up to 100,000 order-book updates per second on the BTC/USDT pair during volatility. Each update carries a price level and the new volume at that level.
The order book lives in two modes: full snapshot (complete state, 5-20 MB JSON for top pairs) and incremental updates (deltas, 50-200 bytes). On connect a client gets a snapshot, then subscribes to the delta stream. Missing even one delta makes the local copy invalid, so a fresh snapshot is required.
- **Best bid** - the highest buy price; **best ask** - the lowest sell price
- **Spread** = best ask - best bid; on Binance BTC/USDT it is usually $0.01-$0.10
- **Depth** - total volume across N levels; used to gauge liquidity
- Quantity=0 in a delta means the level is removed (do not skip it!)
The client received an order-book snapshot with lastUpdateId=500, then a delta with lastUpdateId=498. What to do?
Market Data
A market data feed is the stream of trading activity: trades, prices, volumes, indicators. NYSE pushes up to 5 billion messages per day through XDP (eXtreme Data Platform). The data ships via UDP multicast: one stream to thousands of subscribers at once without the TCP overhead per connection.
Market data has two classes: Level 1 (best bid/ask and last trade only) and Level 2 (full order book to N levels). Bloomberg Terminal delivers Level 1 for 35M instruments worldwide. Level 2 for one exchange is already gigabytes per second at peak, requiring co-located servers next to the exchange.
- **Consolidated tape** - aggregated stream across every venue (SIP in the US); up to 500 us latency
- **Direct feed** - direct connection to the exchange; 50-200 us latency, more expensive
- **Snapshot vs stream**: a periodic snapshot is simpler but misses intra-second moves
- Store timestamps in nanoseconds - microseconds occasionally collide on NYSE
Why do exchanges use UDP multicast instead of TCP to distribute market data?
FIX Protocol
FIX (Financial Information eXchange) is the standard protocol for electronic trading, created in 1992 by Fidelity and Salomon Brothers. Today 90%+ of institutional trading instructions move over FIX. The protocol is text-based: fields are separated by the SOH character (0x01), which simplifies parsing but inflates message size compared to binary formats.
FIX/FAST (Fast Application-layer Encoding) is the binary FIX extension for market data with compression of repeating fields. CME Group uses FIX/FAST for MDP 3.0 (Market Data Platform), delivering updates with 3-8 us latency on co-located servers. Order routing (sending orders) uses classic text FIX or proprietary binary protocols like OUCH (NASDAQ).
- **Session layer**: Logon (A), Heartbeat (0), Logout (5) - connection maintenance
- **Application layer**: NewOrderSingle (D), ExecutionReport (8), OrderCancelRequest (F)
- **Checksum**: the last field (tag 10) is the sum of all message bytes mod 256
- **CompIDs**: SenderCompID (49) and TargetCompID (56) identify the parties
- FIX/FAST compresses a typical 200-byte message down to 20-40 bytes through delta encoding
A broker received a FIX message with tag 35=8. What does it mean?
Trading Latency
In algorithmic trading, latency is a competitive edge. HFT firm Virtu Financial earns on the spread by reacting to market changes faster than anyone else. Co-location in the exchange's data center gives 50-200 us from event to responsive order. Over the public internet it is 5-50 ms, which makes competing with HFT impossible.
- **Co-location**: a server physically inside the exchange data center; lease from $5,000/mo on NYSE
- **Kernel bypass**: DPDK, io_uring - bypass the OS to talk directly to the NIC
- **FPGA**: Field-Programmable Gate Array for market-data parsing in hardware; <1 us
- **Fiber vs microwave**: Chicago-NJ over microwave 8.1 ms vs 13.1 ms over fiber - HFT uses microwave
- **Jitter matters more than the mean**: P99 latency, not the average, decides real losses
Higher CPU clock speed means lower latency in trading systems
For latency-critical systems a CPU with low jitter, NUMA support, and core isolation matters more than peak clock speed
Modern CPUs with Turbo Boost change frequency unpredictably. HFT firms often pin clock speed below max for stability. Intel Xeon with fixed frequency and DDIO (Data Direct I/O) is more popular than fast consumer chips
An HFT strategy is losing money to latency jitter. Which change cuts jitter most effectively?
Takeaways
- **Order book** is the primary source of truth on liquidity; the client maintains a local copy through snapshot + delta updates with sequence numbers
- **FIX protocol** is the industry standard (1992) for trading instructions; text format with SOH separators; FIX/FAST is the binary extension for market data with up to 5x compression
- **Trading latency** is measured in microseconds; co-location, kernel bypass (DPDK), FPGA, and CPU pinning form the standard HFT stack; jitter hurts more than mean latency
Related topics
Financial trading intersects several real-time-system areas:
- WebSocket internals — Binance and Coinbase deliver market data over WebSocket; understanding framing and backpressure is critical
- Real-time rate limiting — Exchanges throttle API requests (Binance: 1,200 weight/min); a violation triggers a 24h IP ban
- WebSocket security — Trading APIs require an HMAC signature on every request; key compromise means loss of funds
Вопросы для размышления
- Binance updates the order book 100 times per second. A client with slow internet drops some deltas. How do you detect the gap and recover consistency?
- FIX is text and takes 200 bytes per order. Binance uses its own binary WebSocket API. In what scenarios does FIX still win despite the inefficiency?
- An HFT firm spends $14M on a microwave link to save 5 ms. What daily trading volume and spread would pay that back in a year?