/
Launch Hummingbot Miner

Task 3 — Exchange Connector & InFlightOrder


Overview

In Task 3, we will be required to implement both InFlightOrder and Exchange Class. The primary bulk of implementing a new exchange connector is in this task.

InFlightOrder Class

As seen in the Exchange Component Overview, the Exchange class depends on the InFlightOrder Class. The InFlightOrder abstracts the details of an order and is primary used by the Exchange class to manage all active orders.

The InFlightOrder Class Diagram, given below, details the critical variables and functions in the InFlightOrder class.

InFlightOrderUMLDiagram

Below are the functions that need to be implemented in the new InFlightOrder class.

Function(s)InputOutputDescription
is_doneNoneboolReturns True if the order state is completely filled or cancelled.
is_failureNoneboolReturns True if placing the order is unsuccesfully.
is_cancelledNoneboolReturns True if the order state is cancelled.
from_jsondata: Dict[str, Any]InFlightOrderConverts the order data from a JSON object to an InFlightOrder object.
update_with_trade_updatetrade_update: Dict[str, Any]boolUpdates the in flight order with trade update from the REST or WebSocket API. Returns True if the order gets updated.

Exchange Class

The Exchange class is the middle-man between the strategies and the exchange API servers. It provides the necessary order book and user data to the strategies and communicates the order proposals to the exchanges.

The functions of the Exchange class can be catergorized into:

(1) Placing Orders
(2) Cancelling Orders
(3) Tracking Orders & Balances
(4) Managing Trading Rules
(5) Additional Functions
(6) Class Properties

Although this might seem pretty straightforward, it does require a certain level of understanding and knowing the expected side-effect(s) of certain functions.

The Exchange Class Diagram, given below, details the critical variables and functions in the Exchange class.

ExchangeUMLDiagram

NOTE:
The 6 categories of functions broadly covers the necessary functions that need to be implemented in the Exchange class, so feel free to include other utility functions as needed.

(1) Placing Orders

The Exchange class places orders by either calling the buy() or sell() method. Both these methods first generates a client order ID for the order which will be used locally by Hummingbot to track the orders before calling the _create_order() method.

buy()

Function that takes the strategy inputs, generates a client order ID(used by Hummingbot for local order tracking), and places a buy order by calling the _create_order() function.

Input Parameter(s):

Parameter(s)TypeDescription
trading_pairstrName of the trading pair symbol(in Hummingbot's format i.e. BASE-QUOTE)
priceDecimalPrice in which the order will be placed in Decimal
amountDecimalAmount in which the order will be placed in Decimal
order_typeOrderTypeSpecifies the order type of the order(i.e. OrderType.LIMIT and OrderType.LIMIT_MAKER)

Expected Output(s): order_id: str

Output(s)TypeDescription
order_idstrClient Order ID

sell()

Function that takes the strategy inputs, generates a client order ID(used by Hummingbot for local order tracking), and places a sell order by calling the _create_order() function.

Input Parameter(s):

Parameter(s)TypeDescription
trading_pairstrName of the trading pair symbol(in Hummingbot's format i.e. BASE-QUOTE)
priceDecimalPrice in which the order will be placed in Decimal
amountDecimalAmount in which the order will be placed in Decimal
order_typeOrderTypeSpecifies the order type of the order(i.e. OrderType.LIMIT and OrderType.LIMIT_MAKER)

Expected Output(s):

Output(s)TypeDescription
order_idstrClient Order ID

async _create_order()

This function is responsible for executing the API request to place the order on the exchange. It does the following:

  • Verifies that the order satisfies exchange trading rules.
  • Quantize the order amount to ensure that the precision is as required by the exchange.
  • Create a params dictionary with the necessary parameters for the desired order.
  • Pass the params to an Auth object to generate the request signature.
  • Begin tracking the order by calling start_tracking_order(...).
  • Places the order by calling the _api_request() method with the relevant order parameters.
  • Upon successfully placing the order, the tracked order will be updated with the resulting exchange order ID from the API Response.
NOTE:
The tracked order is an InFlightOrder that is within a dictionary variable(_in_flight_orders) in the Exchange class. InFlightOrder are Hummingbot’s internal records of orders it has placed that remain open in the exchange. When such orders are either filled or canceled, they are removed from the dictionary by calling  stop_tracking_order() method, and the relevant event completion flag is passed to the strategy module.

Input Parameter(s):

Parameter(s)TypeDescription
order_idstrID used to track an order in Hummingbot.
trading_pairstrName of the trading pair symbol(in Hummingbot's format i.e. BASE-QUOTE)
priceDecimalPrice in which the order will be placed in Decimal
amountDecimalAmount in which the order will be placed in Decimal
order_typeOrderTypeSpecifies the order type of the order(i.e. OrderType.LIMIT and OrderType.LIMIT_MAKER)
trade_typeTradeTypeSpecifies the trade type of the order(i.e. TradeType.BUY and TradeType.SELL)

Expected Output(s):

Output(s)TypeDescription
order_idstrClient Order ID

(2) Cancelling Orders

The strategy and the exit command cancels orders by calling cancel() or cancel_all() methods respectively.

cancel()

Function that takes in the trading pair and client order ID from the strategy as inputs and proceeds to a cancel the order by calling the _execute_cancel() function.

Input Parameter(s):

Parameter(s)TypeDescription
trading_pairstrName of the trading pair symbol(in Hummingbot's format i.e. BASE-QUOTE)
order_idstrClient Order ID

Expected Output(s): order_id: str

async cancel_all()

Function that is primarily triggered by the ExitCommand that cancels all InFlightOrders being tracked by the Exchange class. It confirms the successful cancellation of the orders by calling the

Calls the _api_request() function with the relevant parameters.

Input Parameter(s): None

Expected Output(s):

Output(s)TypeDescription
cancellation_resultList[CancellationResult]List of CancellationResult. CancellationResult.success is assigned True if the particular order(identified by Client Order ID) is successfully cancelled
NOTE:
In some cases, where an exchange does not have a cancel all orders endpoint, the we will have to call _execute_cancel() for each in-flight order

async _execute_cancel()

Cancels the specified in-flight order and simply returns the client order ID.

Input Parameter(s):

Parameter(s)TypeDescription
trading_pairstrName of the trading pair symbol(in Hummingbot's format i.e. BASE-QUOTE)
order_idstrClient Order ID

Expected Output(s):

Output(s)TypeDescription
order_idstrClient Order ID

(3) Tracking Orders & Balances

The functions listed in this section details how the connector should process and track orders and balances.

start_tracking_order()

Starts tracking an order by simply adding it into _in_flight_orders dictionary.

Input Parameter(s):

Parameter(s)TypeDescription
order_idstrID used to track an order in Hummingbot.
exchange_order_idstrID used to uniquely identify the order on the Exchange.
trading_pairstrName of the trading pair symbol(in Hummingbot's format i.e. BASE-QUOTE)
priceDecimalPrice in which the order will be placed in Decimal
amountDecimalAmount in which the order will be placed in Decimal
order_typeOrderTypeSpecifies the order type of the order(i.e. OrderType.LIMIT and OrderType.LIMIT_MAKER)
trade_typeTradeTypeSpecifies the trade type of the order(i.e. TradeType.BUY and TradeType.SELL)

Expected Output(s): None

NOTE:
In most cases, the exchange_order_id is only provided after the place order APi request is succesfully processed by the exchange. As such, the exchange_order_id can be None initially. It can be updated later by using the  InFlightOrderBase.update_exchange_order_id() function.

stop_tracking_order()

Stops tracking an order by simply removing it from _in_flight_orders dictionary.

Input Parameter(s):

Parameter(s)TypeDescription
order_idstrID used to track an order in Hummingbot.

Expected Output(s): None

async _user_stream_event_listener()

Wait for new messages from _user_stream_tracker.user_stream queue and processes them according to their message channels. These messages are queued by the respective UserStreamDataSource.

Below are the function(s) called from within _user_stream_event_listener() when a message is received.

Function(s)Description
_process_order_message()Process user's order update messages.
_process_trade_message()Process user's trade messages.
_process_balance_message()Process user's balance messages.

Input Parameter(s): None

Expected Output(s): None

_status_polling_loop()

Periodically update user balances and order status via REST API. This serves as a fallback measure for websocket API updates. Calling of both _update_balances() and _update_order_status() functions is determined by the _poll_notifier variable.

_poll_notifier is an asyncio.Event object that is set in the tick() function. It is set after every SHORT_POLL_INTERVAL or LONG_POLL_INTERVAL depending on the last_recv_time of the _user_stream_tracker.

Input Parameter(s): None

Expected Output(s): None

_update_balances()

Calls the REST API to update total and available balances.

Input Parameter(s): None

Expected Output(s): None

_update_order_status()

Calls the REST API to get order/trade updates for each in-flight order.

If needed, it will call either _process_order_message() or _process_trade_message() or both.

NOTE:
  • _process_trade_message() must be called before _process_order_message() for any in-flight orders.
  • Each partial fill should be accompanied by a call to _process_trade_message(). This ensures that Hummingbot captures every trade executed on an order.
  • Input Parameter(s): None

    Expected Output(s): None

    _process_order_message()

    Updates the in-flight order's order status and triggers the OrderCancelledEvent if needed.

    Input Parameter(s):

    Parameter(s)TypeDescription
    order_msgDict[str, Any]The order response from either REST or websocket API.

    Expected Output(s): None

    _process_trade_message()

    Updates in-flight order trade information by calling the NewInFlightOrder.update_with_trade_update() function and triggers the OrderFilledEvent .

    It will also trigger either BuyOrderCompletedEvent or SellOrderCompletedEvent if needed.

    Input Parameter(s):

    Parameter(s)TypeDescription
    order_msgDict[str, Any]The trade history response from either REST or websocket API.

    Expected Output(s): None

    _process_balance_message()

    Updates the user's available and total asset balance.

    Input Parameter(s):

    Parameter(s)TypeDescription
    balance_msgDict[str, Any]The user balance response from either REST or websocket API.

    Expected Output(s): None

    (4) Managing Trading Rules

    The Exchange is also responsible for managing the trading rules of the trading pairs since there are rules enforced by the exchange themselves.

    Below are the functions used to ensure that orders being placed meet the requirements and ensure that the trading rules are up to date.:

    async _trading_rules_polling_loop()

    Asynchronous task that periodically update trading rules.

    Calls _update_trading_rules()

    Input Parameter(s): None

    Expected Output(s): None

    async _update_trading_rules()

    Queries the necessary API endpoint and intialize the TradingRule object for each trading pair being traded.

    Calls _api_request() and subsequently _format_trading_rules()

    Input Parameter(s): None

    Expected Output(s): None

    _format_trading_rules()

    Converts JSON API response into a dictionary of TradingRule.

    Input Parameter(s):

    Parameter(s)TypeDescription
    api_responseDict[str, Any]The JSON API response for the exchanges trading rules

    Expected Output(s): None

    NOTE:
    The trading rules can generally be found in the API endpoint that lists all supported trading pairs/markets.

    get_order_price_quantum()

    Returns a price step, a minimum price increment for a given trading pair.

    Input Parameter(s):

    Parameter(s)TypeDescription
    trading_pairstrTrading pair of the order being placed.
    priceDecimalPrice of the order being placed.

    Expected Output(s):

    Output(s)TypeDescription
    min_price_incrementDecimalMinimum Price increment of specified trading pair

    get_order_size_quantum()

    Returns an order amount step, a minimum amount increment for a given trading pair.

    Input Parameter(s):

    Parameter(s)TypeDescription
    trading_pairstrTrading pair of the order being placed.
    order_sizeDecimalOrder size of the order being placed.

    Expected Output(s):

    Output(s)TypeDescription
    min_incrementDecimalMinimum base asset size increment of specified trading pair

    (5) Additional Function(s)

    The list below are the utility functions and its description that the Exchange class use.

    async _api_request()

    start_network()

    This function is required by NetworkIterator base class and is called automatically. It starts tracking order book, polling trading rules, updating statuses and tracking user data.

    Input Parameter(s): None

    Expected Output(s): None

    stop_network()

    This function is required by NetworkIterator base class and is called automatically. It performs the necessary shut down procedure.

    Input Parameter(s): None

    Expected Output(s): None

    check_network()

    This function is required by NetworkIterator base class and is called periodically to check the network connection. Simply ping the network (or call any light weight public API).

    Input Parameter(s): None

    Expected Output(s): None

    get_order_book()

    Used by the OrderBookCommand to display the order book in the terminal.

    Input Parameter(s):

    Parameter(s)TypeDescription
    trading_pairstrTrading pair used to identify the OrderBook from the OrderBookTracker

    Expected Output(s):

    Output(s)TypeDescription
    order_bookOrderBookOrderBook object of the specified trading pair

    get_open_orders()

    Queries the open order endpoint and parses the response as a List of OpenOrder objects.

    Input Parameter(s): None

    Expected Output(s):

    Output(s)TypeDescription
    ordersList[OpenOrder]List of open orders

    restore_tracking_states()

    Restore in-flight orders from saved tracking states, this is such that the connector can pick up on where it left off should it crash unexpectedly.

    The saved tracking states are stored locally as a sqlite file in the /data folder.

    Input Parameter(s):

    Parameter(s)TypeDescription
    saved_statesDict[str, any]JSON value of all the saved order states from the locally saved sqlite file.

    Expected Output(s): None

    supported_order_types()

    Returns list of OrderType supported by this connector.

    Input Parameter(s): None

    Expected Output(s):

    Output(s)TypeDescription
    typesList[OrderType]List of OrderType. Generally connectors would support OrderType.LIMIT and OrderType.LIMIT_MAKER

    async _http_client

    Returns the shared aiohttp.ClientSession used by the connector. This HTTP client is used for all REST API requests.

    Input Parameter(s): None

    Expected Output(s):

    Output(s)TypeDescription
    clientaiohttp.ClientSessionA HTTP client used to interact with the exchanges API servers.

    get_fee()

    To get trading fee, this function is simplified by using fee override configuration. Most parameters to this function are ignore except order_type. Use OrderType.LIMIT_MAKER to specify you want trading fee for maker order.

    Input Parameter(s):

    Parameter(s)TypeDescription
    base_currencystrBase currency of the order.
    quote_currencystrQuote currency of the order.
    priceDecimalPrice in which the order will be placed in Decimal
    amountDecimalAmount in which the order will be placed in Decimal
    order_typeOrderTypeSpecifies the order type of the order(i.e. OrderType.LIMIT and OrderType.LIMIT_MAKER)
    trade_typeTradeTypeSpecifies the trade type of the order(i.e. TradeType.BUY and TradeType.SELL)

    Expected Output(s):

    Output(s)TypeDescription
    feeTradeFeeEstimated trade fee of the order.

    (6) Class Properties

    Below are the property functions of the Exchange class.

    Property Function(s)OutputDescription
    namestrName of the exchange. Used to identify the exchange across Hummingbot.
    order_booksDict[str, OrderBook]Dictionary of the OrderBook of each active trading pair on the bot. Utilizes the order_books property from OrderBookTracker
    trading_rulesDict[str, TradingRule]Returns the _trading_rule class variable; a dictionary of the TradingRule of each active trading pair on the bot.
    in_flight_ordersDict[str, InFlightOrder]Dictionary of the all InFlightOrder by its client order ID.
    status_dictDict[str, bool]A dictionary of statuses of various connector's components. Used by the ready() property function.
    readyboolTrue when all statuses pass, this might take 5-10 seconds for all the connector's components and services to be ready.
    limit_ordersList[LimitOrder]Returns a list of InFlightOrder.
    tracking_statesDict[str, Any]All InFlightOrder in JSON format. Used to save in sqlite db.

    Debugging & Testing

    As part of the QA process, for each tasks(Task 1 through 3) you are required to include the unit test cases for the code review process to begin. Refer to Option 1: Unit Test Cases to build your unit tests.

    Edit on GitHub