mirror of
https://github.com/d0zingcat/gocryptotrader.git
synced 2026-06-02 23:16:51 +00:00
bybit: enable multiconnection handling across websocket endpoints (#1670)
* glorious: whooops * gk: nits * Leak issue and edge case * Websocket: Add SendMessageReturnResponses * whooooooopsie * gk: nitssssss * Update exchanges/stream/stream_match.go Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com> * Update exchanges/stream/stream_match_test.go Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com> * linter: appease the linter gods * gk: nits * gk: drain brain * started * more changes before merge match pr * gateio: still building out * gateio: finish spot * fix up tests in gateio * Add tests for stream package * rm unused field * glorious: nits * rn files, specifically set function names to asset and offload routing to websocket type. * linter: fix * Add futures websocket request support * gateio: integrate with IBOTExchange (cherry pick my nose) * linter: fix * glorious: nits * add counter and update gateio * fix collision issue * Update exchanges/stream/websocket.go Co-authored-by: Scott <gloriousCode@users.noreply.github.com> * glorious: nits * add tests * linter: fix * After merge * Add error connection info * upgrade to upstream merge * Fix edge case where it does not reconnect made by an already closed connection * stream coverage * glorious: nits * glorious: nits removed asset error handling in stream package * linter: fix * rm block * Add basic readme * fix asset enabled flush cycle for multi connection * spella: fix * linter: fix * Add glorious suggestions, fix some race thing * reinstate name before any routine gets spawned * stop on error in mock tests * glorious: nits * Set correct price * glorious: nits found in CI build * Add test for drain, bumped wait times as there seems to be something happening on macos CI builds, used context.WithTimeout because its instant. * mutex across shutdown and connect for protection * lint: fix * test time withoffset, reinstate stop * fix whoops * const trafficCheckInterval; rm testmain * y * fix lint * bump time check window * stream: fix intermittant test failures while testing routines and remove code that is not needed. * spells * cant do what I did * protect race due to routine. * update testURL * use mock websocket connection instead of test URL's * linter: fix * remove url because its throwing errors on CI builds * connections drop all the time, don't need to worry about not being able to echo back ws data as it can be easily reviewed _test file side. * remove another superfluous url thats not really set up for this * spawn overwatch routine when there is no errors, inline checker instead of waiting for a time period, add sleep inline with echo handler as this is really quick and wanted to ensure that latency is handing correctly * linter: fixerino uperino * fix ID bug, why I do this, I don't know. * glorious: panix * linter: things * whoops * dont need to make consecutive Unix() calls * websocket: fix potential panic on error and no responses and adding waitForResponses * bybit: enable multiconnection handling across websocket endpoints * rm debug lines * rm json parser and handle in json package instead * in favour of json package unmarshalling * fix processing issues with tickers * linter: fix * linter: fix again * * change field name OutboundRequestSignature to WrapperDefinedConnectionSignature for agnostic inbound and outbound connections. * change method name GetOutboundConnection to GetConnection for agnostic inbound and outbound connections. * drop outbound field map for improved performance just using a range and field check (less complex as well) * change field name connections to connectionToWrapper for better clarity * spells and magic and wands * merge: fixup * linter: fix * spelling: fix * glorious: nits * comparable check for signature * mv err var * glorious: nits and stuff * attempt to fix race * linter: fix * fix tests * types/time: strict usage of time type for usage with unix timestamps * fix tests etc * glorious: nits * gk: nits; engine log cleanup * gk: nits; OCD * gk: nits; move function change file names * gk: nits; 🚀 * gk: nits; convert variadic function and message inspection to interface and include a specific function for that handling so as to not need nil on every call * gk: nits; continued * gk: engine nits; rm loaded exchange * gk: nits; drop WebsocketLoginResponse * stream: Add match method EnsureMatchWithData * gk: nits; rn Inspect to IsFinal * gk: nits; rn to MessageFilter * linter: fix * gateio: update rate limit definitions (cherry-pick) * Add test and missing * Shared REST rate limit definitions with Websocket service, set lookup item to nil for systems that do not require rate limiting; add glorious nit * integrate rate limits for websocket trading spot * bybit: split public and private processing to dedicated handler add supporting function and tests * use correct handler for private inbound connection * conform to match upstream changes * standardise names to upstream style * fix wrapper standards test when sending a auth request through a websocket connection * whoops * Update exchanges/gateio/gateio_types.go Co-authored-by: Scott <gloriousCode@users.noreply.github.com> * glorious: nits * linter: fix * linter: overload * whoops * spelling fixes on recent merge * glorious: nits * linter: fix? * glorious: nits * gk: assert errors touched * gk: unexport derive functions * gk: nitssssssss * fix test * gk: nitters v1 * gk: http status * gk/nits: Add getAssetFromFuturesPair * gk: nits single response when submitting * gk: new pair with delimiter in tests * gk: param update slice to slice of pointers * gk: add asset type in params, includes t.Context() for tests * linter: fix * linter: fix * fix merge whoopsie * glorious: nits * gk: nit * linter: fix * glorious: nits * linter/misc: fix and remove meows * okx: update requestID gen func without func wrapping * RM: functions not needed * Update docs/ADD_NEW_EXCHANGE.md Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com> * gk: nitsssssss * linter: fix * Update exchanges/bybit/bybit_test.go Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com> * Update exchanges/bybit/bybit_test.go Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com> * gk: nit words * cranktakular: nits * linter: fix * cranktakular: nits and expand coverage * linter: fix? * misc fix * cranktakular: missing nit which I thumbed up but did not do. Sillllllly billlyyyy nilllyyy * cranktakular: nits * cranktakular: purge DCP ref/handling and add another TODO * Update exchanges/bybit/bybit_websocket.go Co-authored-by: Scott <gloriousCode@users.noreply.github.com> * glorious: nits * fix test * fix alignment issue and rm println * Update exchanges/bybit/bybit_websocket.go Co-authored-by: Scott <gloriousCode@users.noreply.github.com> * Update exchanges/bybit/bybit_websocket.go Co-authored-by: Scott <gloriousCode@users.noreply.github.com> * glorious: fix * Update exchanges/bybit/bybit_websocket.go Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io> * Update common/common.go Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com> * Update common/common_test.go Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com> * Update exchanges/bybit/bybit_test.go Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com> * gk: nits * gk: nit with test --------- Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io> Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com> Co-authored-by: Scott <gloriousCode@users.noreply.github.com> Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
This commit is contained in:
Ryan O'Hara-Reid
GitHub
@@ -27,6 +27,7 @@ import (
|
||||
"github.com/thrasher-corp/gocryptotrader/exchanges/orderbook"
|
||||
"github.com/thrasher-corp/gocryptotrader/exchanges/protocol"
|
||||
"github.com/thrasher-corp/gocryptotrader/exchanges/request"
|
||||
"github.com/thrasher-corp/gocryptotrader/exchanges/subscription"
|
||||
"github.com/thrasher-corp/gocryptotrader/exchanges/ticker"
|
||||
"github.com/thrasher-corp/gocryptotrader/exchanges/trade"
|
||||
"github.com/thrasher-corp/gocryptotrader/log"
|
||||
@@ -67,12 +68,6 @@ func (e *Exchange) SetDefaults() {
|
||||
}
|
||||
}
|
||||
|
||||
for _, a := range []asset.Item{asset.CoinMarginedFutures, asset.USDTMarginedFutures, asset.USDCMarginedFutures, asset.Options} {
|
||||
if err := e.DisableAssetWebsocketSupport(a); err != nil {
|
||||
log.Errorf(log.ExchangeSys, "%s error disabling %q asset type websocket support: %s", e.Name, a, err)
|
||||
}
|
||||
}
|
||||
|
||||
e.Features = exchange.Features{
|
||||
CurrencyTranslations: currency.NewTranslations(
|
||||
map[currency.Code]currency.Code{
|
||||
@@ -188,12 +183,17 @@ func (e *Exchange) SetDefaults() {
|
||||
|
||||
e.API.Endpoints = e.NewEndpoints()
|
||||
err := e.API.Endpoints.SetDefaultEndpoints(map[exchange.URL]string{
|
||||
exchange.RestSpot: bybitAPIURL,
|
||||
exchange.RestCoinMargined: bybitAPIURL,
|
||||
exchange.RestUSDTMargined: bybitAPIURL,
|
||||
exchange.RestFutures: bybitAPIURL,
|
||||
exchange.RestUSDCMargined: bybitAPIURL,
|
||||
exchange.WebsocketSpot: spotPublic,
|
||||
exchange.RestSpot: bybitAPIURL,
|
||||
exchange.RestCoinMargined: bybitAPIURL,
|
||||
exchange.RestUSDTMargined: bybitAPIURL,
|
||||
exchange.RestFutures: bybitAPIURL,
|
||||
exchange.RestUSDCMargined: bybitAPIURL,
|
||||
exchange.WebsocketSpot: spotPublic,
|
||||
exchange.WebsocketCoinMargined: inversePublic,
|
||||
exchange.WebsocketUSDTMargined: linearPublic,
|
||||
exchange.WebsocketUSDCMargined: linearPublic,
|
||||
exchange.WebsocketOptions: optionPublic,
|
||||
exchange.WebsocketPrivate: websocketPrivate,
|
||||
})
|
||||
if err != nil {
|
||||
log.Errorln(log.ExchangeSys, err)
|
||||
@@ -214,65 +214,176 @@ func (e *Exchange) SetDefaults() {
|
||||
|
||||
// Setup takes in the supplied exchange configuration details and sets params
|
||||
func (e *Exchange) Setup(exch *config.Exchange) error {
|
||||
err := exch.Validate()
|
||||
if err != nil {
|
||||
if err := exch.Validate(); err != nil {
|
||||
return err
|
||||
}
|
||||
if !exch.Enabled {
|
||||
e.SetEnabled(false)
|
||||
return nil
|
||||
}
|
||||
if err := e.SetupDefaults(exch); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
err = e.SetupDefaults(exch)
|
||||
if err := e.Websocket.Setup(&websocket.ManagerSetup{
|
||||
ExchangeConfig: exch,
|
||||
Features: &e.Features.Supports.WebsocketCapabilities,
|
||||
OrderbookBufferConfig: buffer.Config{SortBuffer: true, SortBufferByUpdateIDs: true},
|
||||
TradeFeed: e.Features.Enabled.TradeFeed,
|
||||
UseMultiConnectionManagement: true,
|
||||
}); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
wsSpotURL, err := e.API.Endpoints.GetURL(exchange.WebsocketSpot)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
wsRunningEndpoint, err := e.API.Endpoints.GetURL(exchange.WebsocketSpot)
|
||||
// Spot
|
||||
if err := e.Websocket.SetupNewConnection(&websocket.ConnectionSetup{
|
||||
URL: wsSpotURL,
|
||||
ResponseCheckTimeout: exch.WebsocketResponseCheckTimeout,
|
||||
ResponseMaxLimit: exch.WebsocketResponseMaxLimit,
|
||||
RateLimit: request.NewWeightedRateLimitByDuration(time.Microsecond),
|
||||
Connector: e.WsConnect,
|
||||
GenerateSubscriptions: e.generateSubscriptions,
|
||||
Subscriber: e.SpotSubscribe,
|
||||
Unsubscriber: e.SpotUnsubscribe,
|
||||
Handler: func(_ context.Context, conn websocket.Connection, resp []byte) error {
|
||||
return e.wsHandleData(conn, asset.Spot, resp)
|
||||
},
|
||||
RequestIDGenerator: e.messageIDSeq.IncrementAndGet,
|
||||
}); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
wsOptionsURL, err := e.API.Endpoints.GetURL(exchange.WebsocketOptions)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
err = e.Websocket.Setup(
|
||||
&websocket.ManagerSetup{
|
||||
ExchangeConfig: exch,
|
||||
DefaultURL: spotPublic,
|
||||
RunningURL: wsRunningEndpoint,
|
||||
RunningURLAuth: websocketPrivate,
|
||||
Connector: e.WsConnect,
|
||||
Subscriber: e.Subscribe,
|
||||
Unsubscriber: e.Unsubscribe,
|
||||
GenerateSubscriptions: e.generateSubscriptions,
|
||||
Features: &e.Features.Supports.WebsocketCapabilities,
|
||||
OrderbookBufferConfig: buffer.Config{
|
||||
SortBuffer: true,
|
||||
SortBufferByUpdateIDs: true,
|
||||
},
|
||||
TradeFeed: e.Features.Enabled.TradeFeed,
|
||||
})
|
||||
if err != nil {
|
||||
// Options
|
||||
if err := e.Websocket.SetupNewConnection(&websocket.ConnectionSetup{
|
||||
URL: wsOptionsURL,
|
||||
ResponseCheckTimeout: exch.WebsocketResponseCheckTimeout,
|
||||
ResponseMaxLimit: exch.WebsocketResponseMaxLimit,
|
||||
RateLimit: request.NewWeightedRateLimitByDuration(time.Microsecond),
|
||||
Connector: e.WsConnect,
|
||||
GenerateSubscriptions: e.GenerateOptionsDefaultSubscriptions,
|
||||
Subscriber: e.OptionsSubscribe,
|
||||
Unsubscriber: e.OptionsUnsubscribe,
|
||||
Handler: func(_ context.Context, conn websocket.Connection, resp []byte) error {
|
||||
return e.wsHandleData(conn, asset.Options, resp)
|
||||
},
|
||||
RequestIDGenerator: e.messageIDSeq.IncrementAndGet,
|
||||
}); err != nil {
|
||||
return err
|
||||
}
|
||||
err = e.Websocket.SetupNewConnection(&websocket.ConnectionSetup{
|
||||
URL: e.Websocket.GetWebsocketURL(),
|
||||
ResponseCheckTimeout: exch.WebsocketResponseCheckTimeout,
|
||||
ResponseMaxLimit: bybitWebsocketTimer,
|
||||
})
|
||||
|
||||
wsUSDTLinearURL, err := e.API.Endpoints.GetURL(exchange.WebsocketUSDTMargined)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
return e.Websocket.SetupNewConnection(&websocket.ConnectionSetup{
|
||||
URL: websocketPrivate,
|
||||
// Linear - USDT margined futures.
|
||||
if err := e.Websocket.SetupNewConnection(&websocket.ConnectionSetup{
|
||||
URL: wsUSDTLinearURL,
|
||||
ResponseCheckTimeout: exch.WebsocketResponseCheckTimeout,
|
||||
ResponseMaxLimit: exch.WebsocketResponseMaxLimit,
|
||||
Authenticated: true,
|
||||
})
|
||||
}
|
||||
RateLimit: request.NewWeightedRateLimitByDuration(time.Microsecond),
|
||||
Connector: e.WsConnect,
|
||||
GenerateSubscriptions: func() (subscription.List, error) {
|
||||
return e.GenerateLinearDefaultSubscriptions(asset.USDTMarginedFutures)
|
||||
},
|
||||
Subscriber: func(ctx context.Context, conn websocket.Connection, sub subscription.List) error {
|
||||
return e.LinearSubscribe(ctx, conn, asset.USDTMarginedFutures, sub)
|
||||
},
|
||||
Unsubscriber: func(ctx context.Context, conn websocket.Connection, unsub subscription.List) error {
|
||||
return e.LinearUnsubscribe(ctx, conn, asset.USDTMarginedFutures, unsub)
|
||||
},
|
||||
Handler: func(_ context.Context, conn websocket.Connection, resp []byte) error {
|
||||
return e.wsHandleData(conn, asset.USDTMarginedFutures, resp)
|
||||
},
|
||||
RequestIDGenerator: e.messageIDSeq.IncrementAndGet,
|
||||
MessageFilter: asset.USDTMarginedFutures, // Unused but it allows us to differentiate between the two linear futures types.
|
||||
}); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// AuthenticateWebsocket sends an authentication message to the websocket
|
||||
func (e *Exchange) AuthenticateWebsocket(ctx context.Context) error {
|
||||
return e.WsAuth(ctx)
|
||||
wsUSDCLinearURL, err := e.API.Endpoints.GetURL(exchange.WebsocketUSDCMargined)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// Linear - USDC margined futures.
|
||||
if err := e.Websocket.SetupNewConnection(&websocket.ConnectionSetup{
|
||||
URL: wsUSDCLinearURL,
|
||||
ResponseCheckTimeout: exch.WebsocketResponseCheckTimeout,
|
||||
ResponseMaxLimit: exch.WebsocketResponseMaxLimit,
|
||||
RateLimit: request.NewWeightedRateLimitByDuration(time.Microsecond),
|
||||
Connector: e.WsConnect,
|
||||
GenerateSubscriptions: func() (subscription.List, error) {
|
||||
return e.GenerateLinearDefaultSubscriptions(asset.USDCMarginedFutures)
|
||||
},
|
||||
Subscriber: func(ctx context.Context, conn websocket.Connection, sub subscription.List) error {
|
||||
return e.LinearSubscribe(ctx, conn, asset.USDCMarginedFutures, sub)
|
||||
},
|
||||
Unsubscriber: func(ctx context.Context, conn websocket.Connection, unsub subscription.List) error {
|
||||
return e.LinearUnsubscribe(ctx, conn, asset.USDCMarginedFutures, unsub)
|
||||
},
|
||||
Handler: func(_ context.Context, conn websocket.Connection, resp []byte) error {
|
||||
return e.wsHandleData(conn, asset.USDCMarginedFutures, resp)
|
||||
},
|
||||
RequestIDGenerator: e.messageIDSeq.IncrementAndGet,
|
||||
MessageFilter: asset.USDCMarginedFutures, // Unused but it allows us to differentiate between the two linear futures types.
|
||||
}); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
wsInverseURL, err := e.API.Endpoints.GetURL(exchange.WebsocketCoinMargined)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// Inverse - Coin margined futures.
|
||||
if err := e.Websocket.SetupNewConnection(&websocket.ConnectionSetup{
|
||||
URL: wsInverseURL,
|
||||
ResponseCheckTimeout: exch.WebsocketResponseCheckTimeout,
|
||||
ResponseMaxLimit: exch.WebsocketResponseMaxLimit,
|
||||
RateLimit: request.NewWeightedRateLimitByDuration(time.Microsecond),
|
||||
Connector: e.WsConnect,
|
||||
GenerateSubscriptions: e.GenerateInverseDefaultSubscriptions,
|
||||
Subscriber: e.InverseSubscribe,
|
||||
Unsubscriber: e.InverseUnsubscribe,
|
||||
Handler: func(_ context.Context, conn websocket.Connection, resp []byte) error {
|
||||
return e.wsHandleData(conn, asset.CoinMarginedFutures, resp)
|
||||
},
|
||||
RequestIDGenerator: e.messageIDSeq.IncrementAndGet,
|
||||
}); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
wsPrivateURL, err := e.API.Endpoints.GetURL(exchange.WebsocketPrivate)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// Private
|
||||
return e.Websocket.SetupNewConnection(&websocket.ConnectionSetup{
|
||||
URL: wsPrivateURL,
|
||||
ResponseCheckTimeout: exch.WebsocketResponseCheckTimeout,
|
||||
ResponseMaxLimit: exch.WebsocketResponseMaxLimit,
|
||||
RateLimit: request.NewWeightedRateLimitByDuration(time.Microsecond),
|
||||
Authenticated: true,
|
||||
Connector: e.WsConnect,
|
||||
GenerateSubscriptions: e.generateAuthSubscriptions,
|
||||
Subscriber: e.authSubscribe,
|
||||
Unsubscriber: e.authUnsubscribe,
|
||||
Handler: e.wsHandleAuthenticatedData,
|
||||
RequestIDGenerator: e.messageIDSeq.IncrementAndGet,
|
||||
Authenticate: e.WebsocketAuthenticateConnection,
|
||||
})
|
||||
}
|
||||
|
||||
// FetchTradablePairs returns a list of the exchanges tradable pairs
|
||||
|
||||
Reference in New Issue
Block a user