/
Launch Hummingbot Miner

Task 3 — Market Connector

Info on market connector


Task 3. Market Connector

The primary bulk of integrating a new exchange connector is in this section.

The role of the Market class can be broken down into placing and tracking orders. Although this might seem pretty straightforward, it does require a certain level of understanding and knowing the expected side-effect(s) of certain functions.

Market classes place orders via execute_buy and execute_sell commands, which require the following arguments:

  • The order ID
  • The market symbol
  • The amount of the order
  • The type (limit or market)
  • The price, if limit order

The execute_buy and execute_sell methods verify that the trades would be legal given the trading rules pulled from the exchange and calculate applicable trading fees. They then must do the following:

  • Quantize the order amount to ensure that the precision is as required by the exchange
  • Create a params dictionary with the necessary parameters for the desired order
  • Pass the params to an Auth object to generate the signature and place the order
  • Pass the resulting order ID and status along with the details of the order to an InFlightOrder

InFlightOrders are stored within a list in the Market class, and are Hummingbot’s internal records of orders it has placed that remain open on the market. When such orders are either filled or canceled, they are removed from the list and the relevant event completion flag is passed to the strategy module.

Authentication

Placing and tracking of orders on the exchange normally requiring a form of authentication tied to every requests to ensure protected access/actions to the assets that users have on the respective exchanges.

As such, it would only make sense to have a module dedicated to handling authentication.

As briefly mentioned, the Auth class is responsible for creating the request parameters and/or data bodies necessary to authenticate an API request.

Note: Mainly used in the Market class, but may be required in the UserStreamTrackerDataSource to authenticate subscribing to a WebSocket connection in listen_for_user_stream.

generate_auth_dict

Generates the url and the valid signature to authenticate a particular API request.

Input Parameter: http_method: str,url: str,params: Dict[str, any],body: Dict[str, any]
Expected Output(s): Dict[str, any]

Tip: The input parameters and return value(s) can be modified accordingly to suit the exchange connectors. In most cases, the above parameters are required when creating a signature.

Market

The following describes in detail what is required for the Market class to place and track orders.

Placing Orders

execute_buy and execute_sell are the crucial functions when placing orders on the exchange,. The following describes the task of these functions.

execute_buy

Function that takes the strategy inputs, auto corrects itself with trading rules, and places a buy order by calling the place_order function.

This function also begins to track the order by calling the c_start_tracking_order and c_trigger_event function.

Input Parameter:
order_id: str,
symbol: str,
amount: Decimal,
order_type: OrderType,
price: Optional[Decimal] = s_decimal_0
Expected Output(s): None

execute_sell

Function that takes the strategy inputs, auto corrects itself with trading rules, and places a sell order by calling the place_order function.

Input Parameter:
order_id: str,
symbol: str,
amount: Decimal,
order_type: OrderType,
price: Optional[Decimal] = s_decimal_0
Expected Output(s): None

Tip: The execute_buy and execute_sell methods verify that the trades would be allowed given the trading rules obtained from the exchange and calculate applicable trading fees. They then must do the following:

  • Quantize the order amount to ensure that the precision is as required by the exchange
  • Create a params dictionary with the necessary parameters for the desired order
  • Pass the params to an Auth object to generate the signature and place the order
  • Pass the resulting order ID and status along with the details of the order to an InFlightOrder
  • InFlightOrders are stored within a list in the Market class, and are Hummingbot’s internal records of orders it has placed that remain open on the market. When such orders are either filled or canceled, they are removed from the list and the relevant event completion flag is passed to the strategy module.

Considering that placing of orders normally involves a POST request to a particular buy/sell order REST API endpoint. This would require additional parameters like :

Variable(s)
TypeDescription
order_idstrA generated, client-side order ID that will be used to identify an order by the Hummingbot client.
The order_id is generated in the c_buy function.
symbolstrThe trading pair string representing the market on which the order should be placed. i.e. (ZRX-ETH)
Note: Some exchanges have the trading pair symbol in Quote-Base format. Hummingbot requires that all trading pairs to be in Base-Quote format.
amountDecimalThe total value, in base currency, to buy/sell.
order_typeOrderTypeOrderType.LIMIT, OrderType.LIMIT_MAKER or OrderType.MARKET
Note: LIMIT_MAKER should be used as market maker and LIMIT as market taker(using price to cross the orderbook) for exchanges that support LIMIT_MAKER. Otherwise, the the usual MARKET OrderType should be used as market taker and LIMIT as market taker.
priceOptional[Decimal]If order_type is LIMIT, it represents the rate at which the amount base currency is being bought/sold at.
If order_type is LIMIT_MAKER, it also represents the rate at which the amount base currency is being bought/sold at. However, this OrderType is expected to be a post only order(i.e should ideally be rejected by the exchange if it'll cross the market)
If order_type is MARKET, this is not used(price = s_decimal_0).
Note: s_decimal_0 = Decimal(0)

Cancelling Orders

The execute_cancel function is the primary function used to cancel any particular tracked order. Below is a quick overview of the execute_cancel function

execute_cancel

Function that makes API request to cancel an active order and returns the order_id if it has been successfully cancelled or no longer needs to be tracked.

Note: This function also stops tracking the order by calling the c_stop_tracking_order and c_trigger_event functions.

Input Parameter:
symbol: str,
order_id: str
Expected Output(s): order_id: str

Tracking Orders & Balances

The Market class tracks orders in several ways:

  • Listening on user stream
    This is primarily done in the _user_stream_event_listener function. This is only done when the exchange has a WebSocket API.

  • Periodic status polling
    This serves as a fallback for when user stream messages are not caught by the UserStreamTracker. This is done by the _status_polling_loop

The table below details the required functions to implement

Function
Description
_user_stream_event_listenerUpdate order statuses and/or account balances from incoming messages from the user stream.
_update_balancesPulls the REST API for the latest account balances and updates _account_balances and _account_available_balances.
_update_order_statusPulls the REST API for the latest order statuses and updates the order statuses of locally tracked orders.

If the exchange doesn't provide user balance updates in real-time (web socket), you will need to set self._real_time_balance_update = False in the market constructor (init).

And, you will need to take in_flight_orders snapshot during _update_balances as below:

    self._in_flight_orders_snapshot = {k: copy.copy(v) for k, v in self._in_flight_orders.items()}
    self._in_flight_orders_snapshot_timestamp = self._current_timestamp

This is so that the connector can use default current balance calculation (in market_base) for available balances.

Tip: Refer to Order Lifecycle for a more detailed description on how orders are being tracked in Hummingbot.

It is necessary that the above functions adhere to the flow as defined in the order lifecycle for the connector to work as intended.

Trading Rules

Trading Rules are defined by the respective exchanges. It is crucial that Hummingbot manage and maintain the set of trading rules to ensure that there will be no issues when placing orders.

A list of some common rules can be seen in trading_rule.pyx.

Tip: Most exchanges have a minimum trading size. Not all rules need to be defined, however, it is essential to meet the rules as specified by the exchange.

All trading rules are stored in self._trading_rules in the Market class.

The following details the functions responsible for maintaining the TradeRule for each trading pair

_trading_rules_polling_loop

A background process that periodically polls for trading rule changes. Since trading rules tend not to change as often as account balances and order statuses, this is done less often. This function is responsible for calling _update_trading_rules

Input Parameter: None
Expected Output(s): None

_update_trading_rules

Gets the necessary trading rules definitions form the corresponding REST API endpoints. Calls _format_trading_rules; that parses and updates the _trading_rules variable in the Market class.

Input Parameter: None
Expected Output(s): None

_format_trading_rules

Parses the raw JSON response into a list of TradingRule.

Note: This is important since exchanges might only accept certain precisions and impose a minimum trade size on the order.

Input Parameter: List[Any]
Expected Output(s): List[TradingRule]

Order Price/Size Quantum & Quantize Order Amount

This, together with the trading rules, ensure that all orders placed by Hummingbot meet the required specifications as defined by the exchange.

Note:

  • These checks are performed in execute_buy and execute_sell before placing an order.
  • In the event where the orders proposed by the strategies fail to meet the requirements as defined in the trading rules, the intended behaviour for the Market class is simply to raise an error and not place the order.
  • You may be required to add additional functions to help determine if an order meets the necessary requirements.

The following details some functions responsible for this.

c_get_order_price_quantum

Gets the minimum increment interval for an order price.

Input Parameter: str trading_pair,object price
Expected Output(s): Decimal

c_get_order_size_quantum

Gets the minimum increment interval for an order size (i.e. 0.01 .USD)

Input Parameter: str trading_pair,object order_size
Expected Output(s): Decimal

c_quantize_order_amount

Checks the current order amount against the trading rules, and corrects(i.e simple rounding) any rule violations. Returns a valid order amount in Decimal format.

Input Parameter:
str trading_pair,object amount,
object price=s_decimal_0
Expected Output(s): Decimal

c_quantize_order_price

Checks the current order price against the trading rules, and corrects(i.e. simple rounding) any rule violations. Returns a valid order price in Decimal format.

Input Parameter:
str trading_pair,
object price,br/>object price=s_decimal_0
Expected Output(s): Decimal

Additional Required Function(s)

The following is a list of required functions for the Market class to be fully functional.

name

Returns a lower case name / id for the market. Must stay consistent with market name in global settings.

Input Parameter: None
Expected Output(s): str

order_books

Returns a mapping of all the order books that are being tracked.

Input Parameter: None
Expected Output(s): Dict[str, OrderBook]

*_auth

Returns the Auth class of the market.

Input Parameter: None
Expected Output(s): *Auth

status_dict

Returns a dictionary of relevant status checks. This is necessary to tell the Hummingbot client if the market has been initialized.

Input Parameter: None
Expected Output(s): Dict[str, bool]

ready

This function calls status_dict and returns a boolean value that indicates if the market has been initialized and is ready for trading.

Input Parameter: None
Expected Output(s): bool

limit_orders

Returns a list of active limit orders being tracked.

Input Parameter: None
Expected Output(s): List[LimitOrder]

in_flight_orders

Returns a dictionary of all in flight orders.

Input Parameter: None
Expected Output(s): Dict[str, InFlightOrderBase]

tracking_states

Returns a mapping of tracked client order IDs to the respective InFlightOrder. Used by the MarketsRecorder class to orchestrate market classes at a high level.

Input Parameter: None
Expected Output(s): Dict[str, any]

restore_tracking_states

Updates InFlight order statuses from API results. This is used by the MarketRecorder class to orchestrate market classes at a higher level.

Input Parameter: None
Expected Output(s): None

restore_tracking_states

Updates InFlight order statuses from API results. This is used by the MarketRecorder class to orchestrate market classes at a higher level.

Input Parameter: None
Expected Output(s): None

start_network

An asynchronous wrapper function used by NetworkBase class to handle when a single market goes online.

Input Parameter: None
Expected Output(s): None

stop_network

An asynchronous wrapper function for _stop_network. Used by NetworkBase class to handle when a single market goes offline.

Input Parameter: None
Expected Output(s): None

check_network

An asynchronous function used by NetworkBase` class to check if the market is online/offline.

Input Parameter: None
Expected Output(s): NetworkStatus

get_order

Gets status update for a particular order via rest API.

Note: You are required to retrieve the exchange order ID for the specified client_order_id. You can do this by calling the get_exchange_order_id function available in the InFlightOrderBase.

Input Parameter: client_order_id:str
Expected Output(s): Dict[str, Any]

supported_order_types

Returns a list of OrderType(s) supported by the exchange. Examples are: OrderType.LIMIT, OrderType.LIMIT_MAKER and OrderType.MARKET.

Input Parameter: None
Expected Output(s): List[OrderType]

place_order

An asynchronous wrapper for placing orders through the REST API. Returns a JSON response from the API.

Input Parameter:
order_id:str
symbol:str
amount:Decimal
is_buy:bool
order_type:OrderType
price:Decimal
Expected Output(s): Dict[str, Any]

cancel_all

An asynchronous function that cancels all active orders. Used by Hummingbot's top level "stop" and "exit" commands(cancelling outstanding orders on exit). Returns a List of CancellationResult.

A CancellationResult is an object that indicates if an order has been successfully cancelled with a boolean variable.

Input Parameter: timeout_seconds:float
Expected Output(s): List[CancellationResult]

_stop_network

Synchronous function that handles when a single market goes offline.

Input Parameter: None
Expected Output(s): None

_http_client

Returns a shared HTTP client session instance.

Note: This prevents the need to establish a new session on every API request.

Input Parameter: None
Expected Output(s): aiohttp.ClientSession

_api_request

An asynchronous wrapper function for submitting API requests to the respective exchanges. Returns the JSON response form the endpoints. Handles any initial HTTP status error codes.

Input Parameter:
http_method:str
path_url:str
url:str
data:Optional[Dict[str,Any]]
Expected Output(s): Dict[str, Any]

_update_balances

Gets account balance updates from the corresponding REST API endpoints and updates _account_available_balances and _account_balances class variables in the MarketBase class.

Input Parameter: None
Expected Output(s): None

_status_polling_loop

A background process that periodically polls for any updates on the REST API. This is responsible for calling _update_balances and _update_order_status.

Input Parameter: None
Expected Output(s): None

c_start

A function used by the top level Clock to orchestrate components of Hummingbot.

Input Parameter: Clock clock,double timestamp
Expected Output(s): None

c_tick

Used by top level Clock to orchestrate components of Hummingbot. This function is called frequently with every clock tick.

Input Parameter: double timestamp
Expected Output(s): None

c_buy

A synchronous wrapper function that generates a client-side order ID and schedules a buy order. It calls the execute_buy function and returns the client-side order ID.

Input Parameter:
str symbol,
object amount,
object order_type=OrderType.MARKET,
object price=s_decimal_0,
dict kwargs={}
Expected Output(s): str

c_sell

A synchronous wrapper function that generates a client-side order ID and schedules a sell order. It calls the execute_sell function and returns the client-side order ID.

Input Parameter:
str symbol,
object amount,
object order_type=OrderType.MARKET,
object price=s_decimal_0,
dict kwargs={}
Expected Output(s): str

c_cancel

A synchronous wrapper function that schedules an order cancellation.

Note: The order_id here refers to the client-side order ID as tracked by Hummingbot.

Input Parameter: str symbol,str order_id
Expected Output(s): str

c_did_timeout_tx

Triggers MarketEvent.TransactionFailure when an Ethereum transaction has timed out.

Input Parameter: str tracking_id
Expected Output(s): None

c_get_fee

A function that calculates the fees for a particular order. Use estimate_fee module to get the fee. Returns a TradeFee object.

Input Parameter:
`str base_currency,
str quote_currency,
object order_type,
object order_side,
object amount,
object price
Expected Output(s): TradeFee

c_get_order_book

Returns the OrderBook for a specific trading pair(symbol).

Input Parameter: str symbol
Expected Output(s): OrderBook

c_start_tracking_order

Adds a new order to the _in_flight_orders class variable. This essentially begins tracking the order on the Hummingbot client.

Input Parameter:
str client_order_id,
str symbol,
object order_type,
object trade_type,
object price,
object amount
Expected Output(s): None

c_stop_tracking_order

Deletes an order from _in_flight_orders class variable. This essentially stops the Hummingbot client from tracking an order.

Input Parameter: str order_id
Expected Output(s): None

Debugging & Testing

As part of the QA process, for each tasks(Task 1 through 3) you are required to include the unit test cases for the code review process to begin. Refer to Option 1: Unit Test Cases to build your unit tests.

Edit on GitHub