Websocket orderbook buffering (#333)

* Initial commit setting up a map orderbook system with a buffer. It will write to the buffer, sort apply to main orderbook and then process.

* Moves namespaces again

* Updates orderbook to use a sweet new WebsocketOrderbookUpdate type to handle all updates whether its using ID or not. So good. Adds many tests

* Starting to implement orderbook update handling per exchange. Updates namespaces again. Hopefuylly will find a way to update via ID not timestamp, too many endpoints dont provide update timestamps

* Changes orderbookbuffer to use BufferUpdate type instead of orderbook.Base to achieve more functionality and no need for type conversion functions. Updates tests

* Updates all instances of ws.orderbook.Update. Simplifies some orderbook logic

* Introduces toggleable buffer. Renames orderbooks. Completes implementation for everywhere but OKGroup due to hash calculation

* Implements orderbook update for okgroup, but forgets about the orderbook hash checking

* Fixes okgroup checksum calculation. Fixes linting issue. Removes redundant Kraken tests.

* Introduces sorting toggle and separates from buffer toggle. Uses benchmarks to highlight performance gains

* Fixes Gemini rate limit and parsing. Removes comments and fixes typos

* Fixes bitfinex orderbook processing

* Inbuilt sorting, minor fixes for websocket implementations. Improves test coverage

* Adds surprise LakeBTC websocket support

* Fixes data race

* Fixes rebasing issues due to namespace movements

* Addresses PR nits: moves folder namespace from ws to websocket. Removes line spaces in imports. Fixes lakebtc websocket returns and defer fucntions. Fixes comments

* Adds poloniex orderook sorting support

* Enables bitstamp and hitbtc orderbook sorting. Fixes poloniex's sorting

* Renames namespaces and combines monitor and connection into wshandler. Removes unused SPOT const. Changes how orderbook stuff is loaded. It is done in startup with a setup. Removes exchange name from loadsnapshot as well

* Removes the connection.go from rebasing issues. Removes error response from functions used in goroutines

* Fixes test with exchange name output change

* Fixes issues where copy and paste and replace all were used poorly
This commit is contained in:
Scott
2019-08-13 09:32:59 +10:00
committed by Adrian Gallagher
parent 2078ba907f
commit 0fbf8b172a
110 changed files with 2197 additions and 1909 deletions

View File

@@ -16,7 +16,7 @@ import (
"github.com/thrasher-corp/gocryptotrader/exchanges/orderbook"
"github.com/thrasher-corp/gocryptotrader/exchanges/request"
"github.com/thrasher-corp/gocryptotrader/exchanges/ticker"
"github.com/thrasher-corp/gocryptotrader/exchanges/wshandler"
"github.com/thrasher-corp/gocryptotrader/exchanges/websocket/wshandler"
log "github.com/thrasher-corp/gocryptotrader/logger"
)
@@ -32,6 +32,8 @@ const (
DefaultWebsocketResponseCheckTimeout = time.Millisecond * 30
// DefaultWebsocketResponseMaxLimit is the default max wait for an expected websocket response before a timeout
DefaultWebsocketResponseMaxLimit = time.Second * 7
// DefaultWebsocketOrderbookBufferLimit is the maximum number of orderbook updates that get stored before being applied
DefaultWebsocketOrderbookBufferLimit = 5
)
// FeeType custom type for calculating fees based on method
@@ -267,6 +269,7 @@ type Base struct {
RESTPollingDelay time.Duration
WebsocketResponseCheckTimeout time.Duration
WebsocketResponseMaxLimit time.Duration
WebsocketOrderbookBufferLimit int64
AuthenticatedAPISupport bool
AuthenticatedWebsocketAPISupport bool
APIWithdrawPermissions uint32