Connector Architecture

Connector Architecture Diagram

The Architecture Diagram, given above, depicts the high-level design of a Connector.

Directory and File Structure

Each exchange connector consists of the following files:

1
2
3
4
5
6
7
8
9
hummingbot/market/<market_name> # folder for specific exchange
├── *_market.[pyx,pxd]                    
├── *_auth.py                      
├── *_data_source.py               
├── *_order_book.[pyx,pxd]                
├── *_order_book_tracker.py        
├── *_active_order_tracker.[pyx,pxd]
├── *_user_stream_tracker.py 
└── *_in_flight_order.[pyx,pxd]

Component Overview

Each exchange connector is comprised of the following components. Below are the detailed descriptions of tasks for each component and its corresponding files

Component
Description
Market
*_market.pyx
REQUIRED
Connector modules are centered around a Market class, which are children of MarketBase. Each Market class contains a separate OrderBookTracker and UserStreamTracker to pass along changes to the relevant order books and user accounts, respectively.

Market instances also contain a list of InFlightOrders, which are orders placed by Hummingbot that are currently on the order book. Typically, it is also helpful to have a market-specific Auth class, which generates the necessary authentication parameters to access restricted endpoints, such as for trading.
MarketAuth
*_auth.py
OPTIONAL
This class handles the work of creating access requests from information provided by Hummingbot. Arguments tend to include:
  • HTTP Request Type
  • Endpoint URL
  • Mandatory parameters to pass on to the exchange (e.g. API key, secret, passphrase, request body)

Depending on the specific exchange, different information may be needed for authentication. Typically, the Auth class will:
  • Generate a timestamp.
  • Generate a signature based on the time, access method, endpoint, provided parameters, and private key of the user.
  • Compile the public key, timestamp, provided parameters, and signature into a dictionary to be passed via an http or ws request.
Note: This module is typically required for centralized exchange only. Generally, auth for DEXs is handled by the respective wallet.
OrderBookTracker
*_order_book_tracker.py
REQUIRED

UserStreamTracker
*_user_stream_tracker.py
OPTIONAL
Each Market class contains an OrderBookTracker and a UserStreamTracker, to maintain a real-time order book of a particular trading pair and to access and maintain the current state of the user’s account and orders respectively.
Both the OrderBookTracker and UserStreamTracker have subsidiary classes which handle data retrieval and processing.
  • OrderBookTrackerDataSource and UserStreamDataSource classes contain API calls to pull data from the exchange and user accounts and WebSocket feeds to capture state changes.
  • The OrderBook class contains methods which convert raw snapshots from exchanges into usable dictionaries of active bids and asks.
OrderBookTrackerDataSource
*_order_book_data_source.py
REQUIRED
Consists of the OrderBookTrackerDataSource class. This class is responsible for initializing the OrderBook and also deals with order book data retrieval. It simply collects, parses and queues the data stream to be processed by OrderBookTracker. Generally, this would mean pulling data from the exchange's API/WebSocket servers.

To maintain a consistent and up-to-date order book, it is necessary to track the timestamp of each message received from the exchange API servers. Depending on the exchange responses, we can maintain an order book in the following ways:
  1. Presence of Timestamp/Nonce
    In this ideal scenario, we will only 'apply' delta messages onto the order book if and only if the timestamp of the message received is above or +1 of _last_diff_uid in the order book.
  2. Absence of Timestamp/Nonce
    In this scenario, we would have to assign a timestamp to every message received from the exchange and similarly apply the delta messages sequentially only if it is received after the snapshot message.
Note: It is important that the order book being maintained reflects all changes and is consistent with the order book in the exchange. As a safeguard/fallback, in the event when Hummingbot is unable to adequately maintain the order book, executing periodic order book snapshot requests can help to ensure that any deltas missed would be corrected.
UserStreamTrackerDataSource
*_user_stream_data_source.py
OPTIONAL
The UserStreamTrackerDataSource class deals with user data retrieval. It simply collects, parses and queues the data stream to be processed by UserStreamTracker.

Unlike OrderBookTrackerDataSource, UserStreamTrackerDataSource retrieves messages pertaining to user account balances and orders.
ActiveOrderTracker
*_active_order_tracker.pyx
OPTIONAL
Mainly deals with tracking of an open order placed by the user. It also consists of functions like convert_snapshot_message_to_order_book_row and convert_diff_message_to_order_book_row to help parse the incoming data that will be subsequently used by OrderBookTrackerDataSource and OrderBookTracker to maintain a real-time order book.
Note: This class is not necessary for all exchanges
InFlightOrder
*_in_flight_order.pyx
REQUIRED
Stores all details pertaining to the current state of an order.
Note: It is important to keep a consistent and accurate state of all active orders placed by the user. This ensures that the strategies are given the correct information and are able to perform their tasks accordingly.

For more details on how to begin implementing the components, please refer to Tutorial