Fetching orderbooks via CCXT Pro websockets library

CCXT Pro has been free for sometime now, in olden days we need write lower level websocket calls. With CCXT Pro, it’s very simple to fetch order book via websockets. But there are quite a few basic things we need in additional to the most simplistic example from the doc.

Features I added?

1. Initial snapshot is overlayed with incremental updates. For example, when exchange push you incremental updates, you need remove price levels if amount is zero. And also add the new levels.
2. Validations

a. Check local clock vs timestamp in update from exchange. If breach ‘ts_delta_observation_ms_threshold’, orderbook.is_valid is flagged to False.

b. Check gap between consecutive updates. If breach ‘ts_delta_consecutive_ms_threshold’, orderbook.is_valid is flagged to False)

c. If best ask is lower than best bid

3. Publish updates to redis

Next?

1. Pinned memory
2. ping pong (Exchange heart beats). Example, Binance requires “The websocket server will send a ping frame every 3 minutes. If the websocket server does not receive a pong frame back from the connection within a 10 minute period, the connection will be disconnected.”
3. Automatic disconnect, reconnect then resubscribe on issues
4. In an actual trading infrastructure, one provider may be assigned multiple tickers. And some universe providers will be responsible for defining the universe. Typically, redis will sit between these order book providers and universe providers. One order book provider may only be able to keep up with some many tickers and you may need multiple order book provider instances (Here’s where pinned memory and pypy will help).

Usage

python ccxt_ws_ob_provider.py --ticker BTC/USDT:USDT --ts_delta_observation_ms_threshold 150 --ts_delta_consecutive_ms_threshold 150

Keep track of latency issues:

a) ts_delta_observation_ms (Default 150 ms): Keep track of server clock vs timestamp from exchange

b) ts_delta_consecutive_ms (Default 150 ms): Keep track of gap between consecutive updates

To point to a different exchange, modify ‘instantiate_exhange’ and fix ccxt import.

To fetch spot market instead of swap? Change ‘exchange_param’.

Code

It’s very simple, and below is a completely standalone python script.