Building Connectors

Introduction

This guide is intended to get you familiarized with basic structure of a connector in Hummingbot. It will guide you through the scope of creating/modifying the necessary components to implement a connector.

By the end of this guide, you should:

  • Have a general understanding of the base classes that serve as building blocks of a connector
  • Be able to integrate new connectors from scratch

Implementing a new connector can generally be split into 3 major tasks:

  1. Data Source & Order Book Tracker
  2. User Stream Tracker
  3. Market Connector

Task 1. Data Source & Order Book Tracker

Generally the first 2 components you should begin with when implementing your own connector are the OrderBookTrackerDataSource and OrderBookTracker.

The OrderBookTracker contains subsidiary classes that help maintain the real-time order book of a market. Namely, the classes are OrderBookTrackerDataSource and ActiveOrderTracker.

OrderBookTrackerDataSource

The OrderBookTrackerDataSource class is responsible for making API calls and/or WebSocket queries to obtain order book snapshots, order book deltas and miscellaneous information on order book.

Integrating your own data source component would require you to extend from the OrderBookTrackerDataSource base class here.

The table below details the required functions in OrderBookTrackerDataSource:

Function
Input Parameter(s) Expected Output(s) Description
get_active_exchange_markets None pandas.DataFrame Performs the necessary API request(s) to get all currently active trading pairs on the exchange and returns a pandas.DataFrame with each row representing one active trading pair.

Note: If none of the API requests returns a traded USDVolume of a trading pair, you are required to calculate it and include it as a column in the DataFrame.

Also the the base and quote currency should be represented under the baseAsset and quoteAsset columns respectively in the DataFrame.

Refer to Calling a Class method for an example on how to test this particular function.
get_trading_pairs None List[str] Calls get_active_exchange_market to retrieve a list of active trading pairs.

Ensure that all trading pairs are in the right format.
get_snapshot client: aiohttp.ClientSession, trading_pair: str Dict[str, any] Fetches order book snapshot for a particular trading pair from the exchange REST API.
Note: Certain exchanges do not add a timestamp/nonce to the snapshot response. In this case, to maintain a real-time order book would require generating a timestamp for every order book snapshot and delta messages received and applying them accordingly.

In Bittrex, this is performed by invoking the queryExchangeState topic on the SignalR WebSocket client.
get_tracking_pairs None Dict[str, OrderBookTrackerEntry] Initializes order books and order book trackers for the list of trading pairs.
listen_for_trades ev_loop: asyncio.BaseEventLoop, output: asyncio.Queue None Subscribes to the trade channel of the exchange. Adds incoming messages(of filled orders) to the output queue, to be processed by
listen_for_order_book_diffs ev_loop: asyncio.BaseEventLoop, output: asyncio.Queue None Fetches or Subscribes to the order book snapshots for each trading pair. Additionally, parses the incoming message into a OrderBookMessage and appends it into the output Queue.
listen_for_order_book_snapshots ev_loop: asyncio.BaseEventLoop, output: asyncio.Queue None Fetches or Subscribes to the order book deltas(diffs) for each trading pair. Additionally, parses the incoming message into a OrderBookMessage and appends it into the output Queue.

ActiveOrderTracker

The ActiveOrderTracker class is responsible for parsing raw data responses from the exchanges API servers.
This is not required on all exchange connectors depending on API responses from the exchanges. This class is mainly used by DEXes to facilitate the tracking of orders

The table below details the required functions in ActiveOrderTracker:

Function
Input Parameter(s) Expected Output(s) Description
active_asks None Dict[Decimal, Dict[str, Dict[str, any]]] Get all asks on the order book in dictionary format.
active_bids None Dict[Decimal, Dict[str, Dict[str, any]]] Get all bids on the order book in dictionary format.
convert_snapshot_message_to_order_book_row object: message Tuple[List[OrderBookRow],List[OrderBookRow]] Convert an incoming snapshot message to Tuple of np.arrays, and then convert to OrderBookRow.
convert_diff_message_to_order_book_row object: message Tuple[List[OrderBookRow],List[OrderBookRow]] Convert an incoming diff message to Tuple of np.arrays, and then convert to OrderBookRow.
convert_trade_message_to_order_book_row object: message Tuple[List[OrderBookRow],List[OrderBookRow]] Convert an incoming trade message to Tuple of np.arrays, and then convert to OrderBookRow.
c_convert_snapshot_message_to_np_arrays object: message Tuple[numpy.array, numpy.array] Parses an incoming snapshot messages into numpy.array data type to be used by convert_snapshot_message_to_order_book_row().
c_convert_diff_message_to_np_arrays object: message Tuple[numpy.array, numpy.array] Parses an incoming delta("diff") messages into numpy.array data type to be used by convert_diff_message_to_order_book_row().
c_convert_trade_message_to_np_arrays object: message numpy.array Parses an incoming trade messages into numpy.array data type to be used by convert_diff_message_to_order_book_row().

Warning

OrderBookRow should only be used in the ActiveOrderTracker class, while ClientOrderBookRow should only be used in the Market class. This is due to improve performance especially since calculations in float fair better than that of Decimal.

OrderBookTracker

The OrderBookTracker class is responsible for maintaining a real-time order book on the Hummingbot client. By using the subsidiary classes like OrderBookTrackerDataSource and ActiveOrderTracker(as required), it applies the market snapshot/delta messages onto the order book.

Integrating your own tracker would require you to extend from the OrderBookTracker base class here.

The table below details the required functions to be implemented in OrderBookTracker:

Function
Input Parameter(s) Expected Output(s) Description
data_source None OrderBookTrackerDataSource Retrieves the OrderBookTrackerDataSource object for this OrderBookTracker.
exchange_name None str Returns the exchange name.
_refresh_tracking_tasks None None Starts tracking for any new trading pairs, and stop tracking for any inactive trading pairs.
Note: Requires the get_tracking_pairs() function from data source to obtain the available pairs on the exchange.
_order_book_diff_router None None Route the real-time order book diff messages to the correct order book.

Each tracked trading pair has their own _saved_message_queues, this would subsequently be used by _track_single_book to apply the messages onto the respective order book.
_order_book_snapshot_router None None Route the real-time order book snapshot messages to the correct order book.

Each tracked trading pair has their own _saved_message_queues, this would subsequently be used by _track_single_book to apply the messages onto the respective order book.
_track_single_book None None Update an order book with changes from the latest batch of received messages.
Constantly attempts to retrieve the next available message from _save_message_queues and applying the message onto the respective order book.
Note: Might require convert_[snapshot|diff]_message_to_order_book_row from the ActiveOrderTracker to convert the messages into OrderBookRow
start None None Start all custom listeners and tasks in the OrderBookTracker component.
Note: You may be required to call start in the base class by using await super().start(). This is optional as long as there is a task listening for trade messages and emitting the TradeEvent as seen in c_apply_trade in OrderBook

Additional Useful Function(s)

The table below details some functions already implemented in the OrderBookTracker base class:

Function
Input Parameter(s) Expected Output(s) Description
order_books None Dict[str, OrderBook] Retrieves all the order books being tracked by OrderBookTracker.
ready None bool Returns a boolean variable to determine if the OrderBookTracker is in a state such that the Hummingbot client can begin its operations.
snapshot None Dict[str, Tuple[pd.DataFrame, pd.DataFrame]] Returns the bids and asks entries in the order book of the respective trading pairs.
start None None Start listening on trade messages.
Note: This is to be overridden and called by running super().start() in the custom implementation of start.
stop None None Stops all tasks in OrderBookTracker.
_emit_trade_event_loop None None Attempts to retrieve trade_messages from the Queue _order_book_trade_stream and apply the trade onto the respective order book.

Task 2. User Stream Tracker

The UserStreamTracker main responsibility is to fetch user account data and queues it accordingly.

UserStreamTracker contains subsidiary classes that help maintain the real-time wallet/holdings balance and open orders of a user. Namely, the classes required are UserStreamTrackerDataSource, UserStreamTracker and MarketAuth(if applicable).

Note

This is only required in Centralized Exchanges.

UserStreamTrackerDataSource

The UserStreamTrackerDataSource class is responsible for making API calls and/or WebSocket queries to obtain order book snapshots, order book deltas and miscellaneous information on order book.

Integrating your own data source component would require you to extend from the OrderBookTrackerDataSource base class here.

The table below details the required functions in UserStreamTrackerDataSource:

Function
Input Parameter(s) Expected Output(s) Description
order_book_class None OrderBook Get relevant order book class ot access class specific methods.

Note: You are also required to implement your own OrderBook class that converts JSON data into a standard OrderBookMessage format.
listen_for_user_stream ev_loop: asyncio.BaseEventLoop
output: asyncio.Queue
None Subscribe to user stream via web socket, and keep the connection open for incoming messages

UserStreamTracker

The UserStreamTracker class is responsible for maintaining the real-time account balances and orders of the user.

This can be achieved in 2 ways(depending on the available API on the exchange):

  1. REST API

    In this scenario, we would have to periodically make API requests to the exchange to retrieve information on the user's account balances and order statuses. An example of this can be seen in the Huobi connector.

  2. WebSocket API

    When an exchange does have WebSocket API support to retrieve user account details and order statuses, it would be ideal to incorporate it into the Hummingbot client when managing account balances and updating order statuses. This is especially important since Hummingbot needs knows to the available account balances and order statuses at all times.

    Tip

    In most scenarios, as seen in most other Centralized Exchanges(Binance, Coinbase Pro, Bittrex), a simple WebSocket integration is used to listen on selected topics and retrieving messages to be processed in Market class.

The table below details the required functions to be implemented in UserStreamTracker:

Function
Input Parameter(s) Expected Output(s)(s) Description
data_source None UserStreamTrackerDataSource Initializes a user stream data source.
start None None Starts all listeners and tasks.
user_stream None asyncio.Queue Returns the message queue containing all the messages pertaining to user account balances and order statues.

Authentication

The Auth class is responsible for crafting the request parameters and bodies that are necessary for certain API requests.

For a more detailed explanation and implementation details, please refer to the Authentication section in the Task 3.

Task 3. Market Connector

The primary bulk of integrating a new exchange connector is in the section.

The role of the Market class can be broken down into placing and tracking orders. Although this might seem pretty straightforward, it does require a certain level of understanding and knowing the expected side-effect(s) of certain functions.

Authentication

Placing and tracking of orders on the exchange normally requiring a form of authentication tied to every requests to ensure protected access/actions to the assets that users have on the respective exchanges.

As such, it is would only make sense to have a module dedicated to handling authentication.

As briefly mentioned, the Auth class is responsible for creating the request parameters and/or data bodies necessary to authenticate an API request.

Note

Mainly used in the Market class, but may be required in the UserStreamTrackerDataSource to authenticate subscribing to a WebSocket connection in listen_for_user_stream.

Function
Input Parameter(s) Expected Output(s)(s) Description
generate_auth_dict http_method: str,
url: str,
params: Dict[str, any],
body: Dict[str, any]
Dict[str, any] Generates the url and the valid signature to authenticate a particular API request.

Tip

The input parameters and return value(s) can be modified accordingly to suit the exchange connectors. In most cases, the above parameters are required when creating a signature.

Market

The section below will describe in detail what is required for the Market class to place and track orders.

Placing Orders

execute_buy and execute_sell are the crucial functions when placing orders on the exchange,. The table below will describe the task of these functions.

Function
Input Parameter(s) Expected Output(s) Description
execute_buy order_id: str,
symbol: str,
amount: Decimal,
order_type: OrderType,
price: Optional[Decimal] = s_decimal_0
None Function that takes the strategy inputs, auto corrects itself with trading rules, and places a buy order by calling the place_order function.

This function also begins to track the order by calling the c_start_tracking_order and c_trigger_event function.
execute_buy order_id: str,
symbol: str,
amount: Decimal,
order_type: OrderType,
price: Optional[Decimal] = s_decimal_0
None Function that takes the strategy inputs, auto corrects itself with trading rules, and places a buy order by calling the place_order function.

Tip

The execute_buy and execute_sell methods verify that the trades would be allowed given the trading rules obtained from the exchange and calculate applicable trading fees. They then must do the following:

  • Quantize the order amount to ensure that the precision is as required by the exchange
  • Create a params dictionary with the necessary parameters for the desired order
  • Pass the params to an Auth object to generate the signature and place the order
  • Pass the resulting order ID and status along with the details of the order to an InFlightOrder

InFlightOrders are stored within a list in the Market class, and are Hummingbot’s internal records of orders it has placed that remain open on the market. When such orders are either filled or canceled, they are removed from the list and the relevant event completion flag is passed to the strategy module.

Considering that placing of orders normally involves a POST request to a particular buy/sell order REST API endpoint. This would require additional parameters like :

Variable(s)
Type Description
order_id str A generated, client-side order ID that will be used to identify an order by the Hummingbot client.
The order_id is generated in the c_buy function.
symbol str The trading pair string representing the market on which the order should be placed. i.e. (ZRX-ETH)
Note: Some exchanges have the trading pair symbol in Quote-Base format. Hummingbot requires that all trading pairs to be in Base-Quote format.
amount Decimal The total value, in base currency, to buy/sell.
order_type OrderType OrderType.LIMIT or OrderType.MARKET
price Optional[Decimal] If order_type is LIMIT, it represents the rate at which the amount base currency is being bought/sold at.
If order_type is MARKET, this is not used(price = s_decimal_0).
Note: s_decimal_0 = Decimal(0)

Cancelling Orders

The execute_cancel function is the primary function used to cancel any particular tracked order. Below is a quick overview of the execute_cancel function

Function
Input Parameter(s) Expected Output(s) Description
execute_cancel symbol: str,
order_id: str
order_id: str Function that makes API request to cancel an active order and returns the order_id if it has been successfully cancelled or no longer needs to be tracked.
Note: This function also stops tracking the order by calling the c_stop_tracking_order and c_trigger_event functions.

Tracking Orders & Balances

The Market class tracks orders in several ways:

  • Listening on user stream
    This is primarily done in the _user_stream_event_listener function. This is only done when the exchange has a WebSocket API.

  • Periodic status polling
    This serves as a fallback for when user stream messages are not caught by the UserStreamTracker. This is done by the _status_polling_loop

The table below details the required functions to implement

Function
Description
_user_stream_event_listener Update order statuses and/or account balances from incoming messages from the user stream.
_update_balances Pulls the REST API for the latest account balances and updates _account_balances and _account_available_balances.
_update_order_status Pulls the REST API for the latest order statuses and updates the order statuses of locally tracked orders.

Tip

Refer to Order Lifecycle for a more detailed description on how orders are being tracked in Hummingbot.

It is necessary that the above functions adhere to the flow as defined in the order lifecycle for the connector to work as intended.

Additional Required Function(s)

Below are a list of required functions for the Market class to be fully functional.

Function
Input Parameter(s) Expected Output(s) Description
name None str Returns a lower case name / id for the market. Must stay consistent with market name in global settings.
order_books None Dict[str, OrderBook Returns a mapping of all the order books that are being tracked.
*_auth None *Auth Returns the Auth class of the market.
status_dict None Dict[str, bool] Returns a dictionary of relevant status checks. This is necessary to tell the Hummingbot client if the market has been initialized.
ready None bool This function calls status_dict and returns a boolean value that indicates if the market has been initialized and is ready for trading.
limit_orders None List[LimitOrder] Returns a list of active limit orders being tracked.
tracking_states None Dict[str, any] Returns a mapping of tracked client order IDs to the respective InFlightOrder. Used by the MarketsRecorder class to orchestrate market classes at a high level.
restore_tracking_states None None Updates InFlight order statuses from API results. This is used by the MarketRecorder class to orchestrate market classes at a higher level.
get_active_exchange_markets None pandas.DataFrame Used by the discovery strategy to read order books of all actively trading markets, and find opportunities for profit.
start_network None None An asynchronous wrapper function used by NetworkBase class to handle when a single market goes online.
stop_network None None An asynchronous wrapper function for _stop_network. Used by NetworkBase class to handle when a single market goes offline.
check_network None NetworkStatus An asynchronous function used byNetworkBase` class to check if the market is online/offline.
get_order client_order_id:str Dict[str, Any] Gets status update for a particular order via rest API.

Note: You are required to retrieve the exchange order ID for the specified client_order_id. You can do this by calling the get_exchange_order_id function available in the InFlightOrderBase.
place_order order_id:str
symbol:str
amount:Decimal
is_buy:bool
order_type:OrderType
price:Decimal
Dict[str, Any] An asynchronous wrapper for placing orders through the REST API. Returns a JSON response from the API.
cancel_all timeout_seconds:float List[CancellationResult] An asynchronous function that cancels all active orders. Used by Hummingbot's top level "stop" and "exit" commands(cancelling outstanding orders on exit). Returns a List of CancellationResult.

A CancellationResult is an object that indicates if an order has been successfully cancelled with a boolean variable.
_stop_network None None Synchronous function that handles when a single market goes offline
_http_client None aiohttp.ClientSession Returns a shared HTTP client session instance.
Note: This prevents the need to establish a new session on every API request.
_api_request http_method:str
path_url:str
url:str
data:Optional[Dict[str,Any]]
Dict[str, Any] An asynchronous wrapper function for submitting API requests to the respective exchanges. Returns the JSON response form the endpoints. Handles any initial HTTP status error codes.
_update_balances None None Gets account balance updates from the corresponding REST API endpoints and updates _account_available_balances and _account_balances class variables in the MarketBase class.
_update_trading_rules None None Gets the necessary trading rules definitions from the corresponding REST API endpoints. Calls _format_trading_rules, to parse and subsequently updates _trading_rules variable.
_format_trading_rules List[Any] List[TradingRule] Parses the raw JSON response into a list of TradingRule.
Note: This is important since exchanges might only accept certain precisions and impose a minimum trade size on the order.
_status_polling_loop None None A background process that periodically polls for any updates on the REST API. This is responsible for calling _update_balances and _update_order_status.
_trading_rules_polling_loop None None A background process that periodically polls for trading rule changes. Since trading rules tend not to change as often as account balances and order statuses, this is done less often. This function is responsible for calling _update_trading_rules.
c_start Clock clock
double timestamp
None A function used by the top level Clock to orchestrate components of Hummingbot.
c_tick double timestamp None Used by top level Clock to orchestrate components of Hummingbot. This function is called frequently with every clock tick.
c_buy str symbol,
object amount,
object order_type=OrderType.MARKET,
object price=s_decimal_0,
dict kwargs={}
str A synchronous wrapper function that generates a client-side order ID and schedules a buy order. It calls the execute_buy function and returns the client-side order ID.
c_sell str symbol,
object amount,
object order_type=OrderType.MARKET,
object price=s_decimal_0,
dict kwargs={}
str A synchronous wrapper function that generates a client-side order ID and schedules a sell order. It calls the execute_buy function and returns the client-side order ID.
c_cancel str symbol,
str order_id
str A synchronous wrapper function that schedules an order cancellation.
Note: The order_id here refers to the client-side order ID as tracked by Hummingbot.
c_did_timeout_tx str tracking_id None Triggers MarketEvent.TransactionFailure when an Ethereum transaction has timed out.
c_get_fee str base_currency,
str quote_currency,
object order_type,
object order_side,
object amount,
object price
TradeFee A function that calculates the fees for a particular order. Returns a TradeFee object.
c_get_order_book str symbol OrderBook Returns the OrderBook for a specific trading pair(symbol).
c_start_tracking_order str client_order_id,
str symbol,
object order_type,
object trade_type,
object price,
object amount
None Adds a new order to the _in_flight_orders class variable. This essentially begins tracking the order on the Hummingbot client.
c_stop_tracking_order str order_id None Deletes an order from _in_flight_orders class variable. This essentially stops the Hummingbot client from tracking an order.
c_get_order_price_quantum str symbol,
object price
Decimal Gets the minimum increment interval for an order price.
c_get_order_size_quantum str symbol,
object order_size
Decimal Gets the minimum increment interval for order size. (i.e. 0.01 USD)
c_quantize_order_amount str symbol,
object amount,
object price=s_decimal_0
Decimal Checks the current order amount against the trading rules, and correct any rule violations. Returns a valid order amount in Decimal format.

Task 4. Hummingbot Client

This section will define the necessary files that need to be modified to allow users configure Hummingbot to use the new exchange connector.

Below are the files and the respective changes that require to be modified.

  • conf/__init_.py

    1
    2
    new_market_api_key = os.getenv("NEW_MARKET_API_KEY")
    new_market_secret_key = os.getenv("NEW_MARKET_SECRET_KEY")
    

  • hummingbot/client/config/global_config_map.py

    1
    2
    3
    4
    5
    6
    7
    8
    "new_market_api_key": ConfigVar(key="new_market_api_key",
                                 prompt="Enter your NewMarket API key >>> ",
                                 required_if=using_exchange("new_market"),
                                 is_secure=True),
    "new_market_secret_key": ConfigVar(key="new_market_secret_key",
                                    prompt="Enter your NewMarket secret key >>> ",
                                    required_if=using_exchange("new_market"),
                                    is_secure=True),
    

  • hummingbot/client/hummingbot_application.py

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    MARKET_CLASSES = {
        .
        .
        .
        "new_market": NewMarket
    }
    .
    .
    .
      def _initialize_markets(self, market_names: List[Tuple[str, List[str]]]):
        ...
        ...
           ...
           elif market_name == "new_market":
             new_market_api_key = global_config_map.get("new_market_api_key").value
             new_market_secret_key = global_config_map.get("new_market_secret_key").value
             new_market_passphrase = global_config_map.get("new_market_passphrase").value
    
             market = NewMarket(new_market_api_key,
                                new_market_secret_key,
                                new_market_passphrase,
                                symbols=symbols,
                                trading_required=self._trading_required)
    

  • hummingbot/client/settings.py

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    EXCHANGES = {
        "bamboo_relay",
        .
        .
        .
        "new_market",
    }   }
    
    DEXES = {
        "bamboo_relay",
        .
        .
        .
        "new_market", # if it is a DEX
    }
    
    EXAMPLE_PAIRS = {
        "binance": "ZRXETH",
        .
        .
        .
        "new_market": "EXAMPLE_PAIR",
    }
    
    EXAMPLE_ASSETS = {
        "binance": "ZRX",
        .
        .
        .
        "new_market": "EXAMPLE_ASSET",
    }
    

  • hummingbot/core/utils/trading_pair_fetcher.py

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    @staticmethod
    async def fetch_new_market_trading_pairs() -> List[str]:
        # Returns a List of str, representing each active trading pair on the exchange.
        async with aiohttp.ClientSession() as client:
                async with client.get(NEW_MARKET_ENDPOINT, timeout=API_CALL_TIMEOUT) as response:
                    if response.status == 200:
                        try:
                            all_trading_pairs: List[Dict[str, any]] = await response.json()
                            return [item["symbol"]
                                    for item in all_trading_pairs
                                    if item["status"] == "ONLINE"]  # Only returns active trading pairs
                        except Exception:
                            pass
                            # Do nothing if the request fails -- there will be no autocomplete available
                    return []
    .
    .
    .
    
    async def fetch_all(self):
        binance_trading_pairs = await self.fetch_binance_trading_pairs()
        .
        .
        .
        new_market_trading_pairs = await self.fetch_new_market_trading_pairs()
        self.trading_pairs = {}
            "binance": binance_trading_pairs,
            .
            .
            .
            "new_market": new_market_trading_pairs,
    

  • hummingbot/logger/report_aggregator.py

    1
    2
    3
    4
    5
    6
    7
    MARKET = {
        "ddex": DDEXMarket,
        .
        .
        .
        "new_market": NewMarket,
    }
    

Additional: Debugging & Testing

This section will breakdown some of the ways to debug and test the code. You are not entirely required to use the options during your development process.

Warning

As part of the QA process, for each tasks(Task 1 through 3) you are required to include the unit test cases for the code review process to begin. Refer to Option 1: Unit Test Cases to build your unit tests.

Option 1. Unit Test Cases

For each tasks(1->3), you are required to create a unit test case. Namely they are test_*_order_book_tracker.py, test_*_user_stream_tracker.py and test_*_market.py. Examples can be found in the test/integration folder.

Below are a list of items required for the Unit Tests:

  1. Data Source & Order Tracker | test_*_order_book_tracker.py
    The purpose of this test is to ensure that the OrderBookTrackerDataSource and OrderBookTracker and all its functions are working as intended. Another way to test its functionality is using a Debugger to ensure that the contents OrderBook mirrors that on the exchange.

  2. User Stream Tracker | test_*_user_stream_tracker.py
    The purpose of this test is to ensure that the UserStreamTrackerDataSource and UserStreamTracker components are working as intended. This only applies to exchanges that has a WebSocket API. As seen in the examples for this test, it simply outputs all the user stream messages. It is still required that certain actions(buy and cancelling orders) be performed for the tracker to capture. Manual message comparison would be required.

i.e. Placing a single LIMIT-BUY order on Bittrex Exchange should return the following(some details are omitted)

1
2
3
4
Trading Pair: ZRX-ETH
Order Type: LIMIT-BUY
Amount: 100ZRX
Price: 0.00160699ETH
1
2
1. Placed LIMIT BUY order.
2. Cancel order.
  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
# Below is the outcome of the test. Determining if this is accurate would still be necessaru.

<Queue maxsize=0 _queue=[
    BittrexOrderBookMessage(
        type=<OrderBookMessageType.DIFF: 2>, 
        content={
            'event_type': 'uB',
            'content': {
                'N': 4,
                'd': {
                    'U': '****', 
                    'W': 3819907,
                    'c': 'ETH',
                    'b': 1.13183357, 
                    'a': 0.96192245, 
                    'z': 0.0,
                    'p': '0x****',
                    'r': False, 
                    'u': 1572909608900,
                    'h': None
                }
            }, 
            'error': None, 
            'time': '2019-11-04T23:20:08'
        },
        timestamp=1572909608.0
    ), 
    BittrexOrderBookMessage(
        type=<OrderBookMessageType.DIFF: 2>,
        content={
            'event_type': 'uO',
            'content': {
                'w': '****',
                'N': 44975,
                'TY': 0,
                'o': {
                    'U': '****',
                    'I': 3191361360,
                    'OU': '****',
                    'E': 'XRP-ETH',
                    'OT': 'LIMIT_BUY',
                    'Q': 100.0,
                    'q': 100.0,
                    'X': 0.00160699,
                    'n': 0.0,
                    'P': 0.0,
                    'PU': 0.0,
                    'Y': 1572909608900,
                    'C': None,
                    'i': True,
                    'CI': False,
                    'K': False,
                    'k': False,
                    'J': None,
                    'j': None,
                    'u': 1572909608900,
                    'PassthroughUuid': None
                }
            },
            'error': None,
            'time': '2019-11-04T23:20:08'
        }, 
        timestamp=1572909608.0
    ),
    BittrexOrderBookMessage(
        type=<OrderBookMessageType.DIFF: 2>,
        content={
            'event_type': 'uB',
            'content': {
                'N': 5,
                'd': {
                    'U': '****',
                    'W': 3819907,
                    'c': 'ETH', 
                    'b': 1.13183357, 
                    'a': 1.1230232,
                    'z': 0.0,
                    'p': '****',
                    'r': False,
                    'u': 1572909611750,
                    'h': None
                }
            }, 
            'error': None, 
            'time': '2019-11-04T23:20:11'
        }, 
        timestamp=1572909611.0
    ), 
    BittrexOrderBookMessage(
        type=<OrderBookMessageType.DIFF: 2>,
        content={
            'event_type': 'uO',
            'content': {
                'w': '****',
                'N': 44976, 
                'TY': 3, 
                'o': {
                    'U': '****', 
                    'I': 3191361360, 
                    'OU': '****', 
                    'E': 'XRP-ETH', 
                    'OT': 'LIMIT_BUY', 
                    'Q': 100.0, 
                    'q': 100.0, 
                    'X': 0.00160699, 
                    'n': 0.0, 
                    'P': 0.0, 
                    'PU': 0.0, 
                    'Y': 1572909608900, 
                    'C': 1572909611750, 
                    'i': False, 
                    'CI': True,
                    'K': False,
                    'k': False, 
                    'J': None, 
                    'j': None, 
                    'u': 1572909611750, 
                    'PassthroughUuid': None
                }
            }, 
            'error': None, 
            'time': '2019-11-04T23:20:11'
        }, 
        timestamp=1572909611.0
    )
] tasks=4>
  1. Market Connector | test_*_market.py
    The purpose of this test is to ensure that all components and the order life cycle is working as intended. This test determines if the connector is able to place and manage orders.
    Below are a list of tests that are required:
Function
Description
test_get_fee Tests the get_fee function in the Market class. Ensures that calculation of fees are accurate.
test_limit_buy Utilizes the place_order function in the Market class and tests if the market connector is capable of placing a LIMIT buy order on the respective exchange. Asserts that a BuyOrderCompletedEvent and OrderFilledEvent(s) have been captured.
Note: Important to ensure that the amount specified in the order has been completely filled.
test_limit_sell Utilizes the place_order function in the Market class and tests if the market connector is capable of placing a LIMIT sell order on the respective exchange.
test_market_buy Utilizes the place_order function in the Market class and tests if the market connector is capable of placing a MARKET buy order on the respective exchange.
test_market_sell Utilizes the place_order function in the Market class and tests if the market connector is capable of placing a MARKET sell order on the respective exchange.
test_cancel_order Utilizes the cancel_order function in the Market class and tests if the market connector is capable of cancelling an order.
Note: Ensures that the Hummingbot client is capable of resolving the client_order_id to obtain the exchange_order_id before posting the cancel order request.
test_cancel_all Tests the cancel_all function in the Market class. All orders(being tracked by Hummingbot) would be cancelled.
test_list_orders Places an order before checking calling the list_orders function in the Market class. Checks the number of orders and the details of the order.
test_order_saving_and_restoration Tests if tracked orders are being recorded locally and determines if the Hummingbot client is able to restore the orders.
test_order_fill_record Tests if trades are being recorded locally.
test_get_wallet_balances (DEXes only) Tests the get_all_balances function in the Market class.
Note: This is only required in Decentralized Exchanges.
test_wrap_eth (DEXes only) Tests the wrap_eth function in the Wallet class.
Note: This is only required in Decentralized Exchanges that support WETH wrapping and unwrapping.
test_unwrap_eth (DEXes only) Tests the unwrap_eth function in the `Wallet class.
Note: This is only required in Decentralized Exchanges that support WETH wrapping and unwrapping.

Note

Ensure that you have enough asset balance before testing. Also document the minimum and recommended asset balance to run the tests. This is to aid testing during the PR review process.

Option 2. aiopython console

This option is mainly used to test for specific functions. Considering that many of the functions are asynchronous functions, it would be easier to test for these in the aiopython console. Click here for some documentation on how to use aiopython.

Writing short code snippets to examine API responses and/or how certain functions in the code base work would help you understand the expected side-effects of these functions and the overall logic of the Hummingbot client.

Issue a API Request

Below is just a short example on how to write a short asynchronous function to mimic a API request to place an order and displaying the response received.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
# Prints the response of a sample LIMIT-BUY Order
# Replace the URL and params accordingly.

>>> import aiohttp
>>> URL="api.test.com/buyOrder"
>>> params = {
...     "symbol": "ZRXETH",
...     "amount": "1000",
...     "price": "0.001",
...     "order_type": "LIMIT"
... }
>>> async with aiohttp.ClientSession() as client:
...    async with client.request("POST",
...                              url=URL,
...                              params=params) as response:
...        if response == 200:
...            print(await response.json())

Calling a Class Method

i.e. Printing the output from get_active_exchange_markets() function in OrderBookTrackerDataSource.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
# In this example, we will be using BittrexAPIOrderBookDataSource

>>> from hummingbot.market.bittrex.BittrexAPIOrderBookDataSource import BittrexAPIOrderBookDataSource as b
>>> await b.get_active_exchange_markets() 

                 askRate baseAsset        baseVolume  ...             volume     USDVolume old_symbol
symbol                                                ...
BTC-USD    9357.49900000       BTC  2347519.11072768  ...       251.26097386  2.351174e+06    USD-BTC
XRP-BTC       0.00003330       XRP       83.81218622  ...   2563786.10102864  7.976883e+05    BTC-XRP
BTC-USDT   9346.88236735       BTC   538306.04864142  ...        57.59973765  5.379616e+05   USDT-BTC
.
.
.
[339 rows x 18 columns]

Option 3. Custom Scripts

This option, like in Option 2, is mainly used to test specific functions. This is mainly useful when debugging how various functions/classes interact with one another.

i.e. Initializing a simple websocket connection to listen and output all captured messages to examine the user stream message when placing/cancelling an order. This is helpful when determining the exact response fields to use.

i.e. A simple function to craft the Authentication signature of a request. This together with POSTMAN can be used to check if the you are generating the appropriate authentication signature for the respective requests.

API Request: POST Order

Below is a sample code for POST-ing a LIMIT-BUY order on Bittrex. This script not only tests the BittrexAuth class but also outputs the response from the API server.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
#!/usr/bin/env python3

import asyncio
import aiohttp
from typing import Dict
from hummingbot.market.bittrex.bittrex_auth import BittrexAuth

BITTREX_API_ENDPOINT = "https://api.bittrex.com/v3"

async def _api_request(http_method: str,
                       path_url: str = None,
                       params: Dict[str, any] = None,
                       body: Dict[str, any] = None,
                       ):
    url = f"{BITTREX_API_ENDPOINT}{path_url}"

    auth = BittrexAuth(
        "****",
        "****"
    )

    auth_dict = auth.generate_auth_dict(http_method, url, params, body, '')

    headers = auth_dict["headers"]

    if body:
        body = auth_dict["body"]

    client = aiohttp.ClientSession()

    async with client.request(http_method,
                              url=url,
                              headers=headers,
                              params=params,
                              data=body) as response:
        data: Dict[str, any] = await response.json()
        if response.status not in [200,201]:
            print(f"Error occured. HTTP Status {response.status}: {data}")
        print(data)

# POST order
path_url = "/orders"

body = {
    "marketSymbol": "FXC-BTC",
    "direction": "BUY",
    "type": "LIMIT",
    "quantity": "1800",
    "limit": "3.17E-7",  # Note: This will throw an error
    "timeInForce": "GOOD_TIL_CANCELLED"
}

loop = asyncio.get_event_loop()
loop.run_until_complete(_api_request("POST",path_url=path_url,body=body))
loop.close()

Examples / Templates

Please refer to Examples / Templates for some existing reference when implementing a connector.