bybit: enable multiconnection handling across websocket endpoints (#1670)

* glorious: whooops * gk: nits * Leak issue and edge case * Websocket: Add SendMessageReturnResponses * whooooooopsie * gk: nitssssss * Update exchanges/stream/stream_match.go Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com> * Update exchanges/stream/stream_match_test.go Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com> * linter: appease the linter gods * gk: nits * gk: drain brain * started * more changes before merge match pr * gateio: still building out * gateio: finish spot * fix up tests in gateio * Add tests for stream package * rm unused field * glorious: nits * rn files, specifically set function names to asset and offload routing to websocket type. * linter: fix * Add futures websocket request support * gateio: integrate with IBOTExchange (cherry pick my nose) * linter: fix * glorious: nits * add counter and update gateio * fix collision issue * Update exchanges/stream/websocket.go Co-authored-by: Scott <gloriousCode@users.noreply.github.com> * glorious: nits * add tests * linter: fix * After merge * Add error connection info * upgrade to upstream merge * Fix edge case where it does not reconnect made by an already closed connection * stream coverage * glorious: nits * glorious: nits removed asset error handling in stream package * linter: fix * rm block * Add basic readme * fix asset enabled flush cycle for multi connection * spella: fix * linter: fix * Add glorious suggestions, fix some race thing * reinstate name before any routine gets spawned * stop on error in mock tests * glorious: nits * Set correct price * glorious: nits found in CI build * Add test for drain, bumped wait times as there seems to be something happening on macos CI builds, used context.WithTimeout because its instant. * mutex across shutdown and connect for protection * lint: fix * test time withoffset, reinstate stop * fix whoops * const trafficCheckInterval; rm testmain * y * fix lint * bump time check window * stream: fix intermittant test failures while testing routines and remove code that is not needed. * spells * cant do what I did * protect race due to routine. * update testURL * use mock websocket connection instead of test URL's * linter: fix * remove url because its throwing errors on CI builds * connections drop all the time, don't need to worry about not being able to echo back ws data as it can be easily reviewed _test file side. * remove another superfluous url thats not really set up for this * spawn overwatch routine when there is no errors, inline checker instead of waiting for a time period, add sleep inline with echo handler as this is really quick and wanted to ensure that latency is handing correctly * linter: fixerino uperino * fix ID bug, why I do this, I don't know. * glorious: panix * linter: things * whoops * dont need to make consecutive Unix() calls * websocket: fix potential panic on error and no responses and adding waitForResponses * bybit: enable multiconnection handling across websocket endpoints * rm debug lines * rm json parser and handle in json package instead * in favour of json package unmarshalling * fix processing issues with tickers * linter: fix * linter: fix again * * change field name OutboundRequestSignature to WrapperDefinedConnectionSignature for agnostic inbound and outbound connections. * change method name GetOutboundConnection to GetConnection for agnostic inbound and outbound connections. * drop outbound field map for improved performance just using a range and field check (less complex as well) * change field name connections to connectionToWrapper for better clarity * spells and magic and wands * merge: fixup * linter: fix * spelling: fix * glorious: nits * comparable check for signature * mv err var * glorious: nits and stuff * attempt to fix race * linter: fix * fix tests * types/time: strict usage of time type for usage with unix timestamps * fix tests etc * glorious: nits * gk: nits; engine log cleanup * gk: nits; OCD * gk: nits; move function change file names * gk: nits; 🚀 * gk: nits; convert variadic function and message inspection to interface and include a specific function for that handling so as to not need nil on every call * gk: nits; continued * gk: engine nits; rm loaded exchange * gk: nits; drop WebsocketLoginResponse * stream: Add match method EnsureMatchWithData * gk: nits; rn Inspect to IsFinal * gk: nits; rn to MessageFilter * linter: fix * gateio: update rate limit definitions (cherry-pick) * Add test and missing * Shared REST rate limit definitions with Websocket service, set lookup item to nil for systems that do not require rate limiting; add glorious nit * integrate rate limits for websocket trading spot * bybit: split public and private processing to dedicated handler add supporting function and tests * use correct handler for private inbound connection * conform to match upstream changes * standardise names to upstream style * fix wrapper standards test when sending a auth request through a websocket connection * whoops * Update exchanges/gateio/gateio_types.go Co-authored-by: Scott <gloriousCode@users.noreply.github.com> * glorious: nits * linter: fix * linter: overload * whoops * spelling fixes on recent merge * glorious: nits * linter: fix? * glorious: nits * gk: assert errors touched * gk: unexport derive functions * gk: nitssssssss * fix test * gk: nitters v1 * gk: http status * gk/nits: Add getAssetFromFuturesPair * gk: nits single response when submitting * gk: new pair with delimiter in tests * gk: param update slice to slice of pointers * gk: add asset type in params, includes t.Context() for tests * linter: fix * linter: fix * fix merge whoopsie * glorious: nits * gk: nit * linter: fix * glorious: nits * linter/misc: fix and remove meows * okx: update requestID gen func without func wrapping * RM: functions not needed * Update docs/ADD_NEW_EXCHANGE.md Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com> * gk: nitsssssss * linter: fix * Update exchanges/bybit/bybit_test.go Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com> * Update exchanges/bybit/bybit_test.go Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com> * gk: nit words * cranktakular: nits * linter: fix * cranktakular: nits and expand coverage * linter: fix? * misc fix * cranktakular: missing nit which I thumbed up but did not do. Sillllllly billlyyyy nilllyyy * cranktakular: nits * cranktakular: purge DCP ref/handling and add another TODO * Update exchanges/bybit/bybit_websocket.go Co-authored-by: Scott <gloriousCode@users.noreply.github.com> * glorious: nits * fix test * fix alignment issue and rm println * Update exchanges/bybit/bybit_websocket.go Co-authored-by: Scott <gloriousCode@users.noreply.github.com> * Update exchanges/bybit/bybit_websocket.go Co-authored-by: Scott <gloriousCode@users.noreply.github.com> * glorious: fix * Update exchanges/bybit/bybit_websocket.go Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io> * Update common/common.go Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com> * Update common/common_test.go Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com> * Update exchanges/bybit/bybit_test.go Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com> * gk: nits * gk: nit with test --------- Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io> Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com> Co-authored-by: Scott <gloriousCode@users.noreply.github.com> Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
2026-05-24 23:16:52 +00:00 · 2025-08-08 14:22:29 +10:00
parent ba92ba3254
commit dcf596c72b
32 changed files with 1475 additions and 939 deletions
--- a/exchanges/gateio/gateio.go
+++ b/exchanges/gateio/gateio.go
@@ -188,8 +188,9 @@ func timeInForceFromString(tif string) (order.TimeInForce, error) {

 // Exchange implements exchange.IBotExchange and contains additional specific api methods for interacting with GateIO
 type Exchange struct {
-	Counter common.Counter // Must be first	due to alignment requirements
 	exchange.Base
+
+	messageIDSeq  common.Counter
 	wsOBUpdateMgr *wsOBUpdateManager
 }

--- a/exchanges/gateio/gateio_test.go
+++ b/exchanges/gateio/gateio_test.go
@@ -2324,7 +2324,7 @@ func TestSubscribe(t *testing.T) {
 	subs, err := e.Features.Subscriptions.ExpandTemplates(e)
 	require.NoError(t, err, "ExpandTemplates must not error")
 	e.Features.Subscriptions = subscription.List{}
-	err = e.Subscribe(t.Context(), &DummyConnection{}, subs)
+	err = e.Subscribe(t.Context(), &FixtureConnection{}, subs)
 	require.NoError(t, err, "Subscribe must not error")
 }

@@ -2866,15 +2866,10 @@ func TestGetSettlementCurrency(t *testing.T) {
 	}
 }

-func TestGenerateWebsocketMessageID(t *testing.T) {
-	t.Parallel()
-	require.NotEmpty(t, e.GenerateWebsocketMessageID(false))
-}
+type FixtureConnection struct{ websocket.Connection }

-type DummyConnection struct{ websocket.Connection }
-
-func (d *DummyConnection) GenerateMessageID(bool) int64 { return 1337 }
-func (d *DummyConnection) SendMessageReturnResponse(context.Context, request.EndpointLimit, any, any) ([]byte, error) {
+func (d *FixtureConnection) GenerateMessageID(bool) int64 { return 1337 }
+func (d *FixtureConnection) SendMessageReturnResponse(context.Context, request.EndpointLimit, any, any) ([]byte, error) {
 	return []byte(`{"time":1726121320,"time_ms":1726121320745,"id":1,"conn_id":"f903779a148987ca","trace_id":"d8ee37cd14347e4ed298d44e69aedaa7","channel":"spot.tickers","event":"subscribe","payload":["BRETT_USDT"],"result":{"status":"success"},"requestId":"d8ee37cd14347e4ed298d44e69aedaa7"}`), nil
 }

@@ -2883,12 +2878,12 @@ func TestHandleSubscriptions(t *testing.T) {

 	subs := subscription.List{{Channel: subscription.OrderbookChannel}}

-	err := e.handleSubscription(t.Context(), &DummyConnection{}, subscribeEvent, subs, func(context.Context, websocket.Connection, string, subscription.List) ([]WsInput, error) {
+	err := e.handleSubscription(t.Context(), &FixtureConnection{}, subscribeEvent, subs, func(context.Context, websocket.Connection, string, subscription.List) ([]WsInput, error) {
 		return []WsInput{{}}, nil
 	})
 	require.NoError(t, err)

-	err = e.handleSubscription(t.Context(), &DummyConnection{}, unsubscribeEvent, subs, func(context.Context, websocket.Connection, string, subscription.List) ([]WsInput, error) {
+	err = e.handleSubscription(t.Context(), &FixtureConnection{}, unsubscribeEvent, subs, func(context.Context, websocket.Connection, string, subscription.List) ([]WsInput, error) {
 		return []WsInput{{}}, nil
 	})
 	require.NoError(t, err)
--- a/exchanges/gateio/gateio_websocket.go
+++ b/exchanges/gateio/gateio_websocket.go
@@ -729,11 +729,6 @@ func (e *Exchange) Unsubscribe(ctx context.Context, conn websocket.Connection, s
 	return e.manageSubs(ctx, unsubscribeEvent, conn, subs)
 }

-// GenerateWebsocketMessageID generates a message ID for the individual connection
-func (e *Exchange) GenerateWebsocketMessageID(bool) int64 {
-	return e.Counter.IncrementAndGet()
-}
-
 // channelName converts global channel names to gateio specific channel names
 func channelName(s *subscription.Subscription) string {
 	if name, ok := subscriptionNames[s.Channel]; ok {
--- a/exchanges/gateio/gateio_websocket_request_spot.go
+++ b/exchanges/gateio/gateio_websocket_request_spot.go
@@ -46,7 +46,7 @@ func (e *Exchange) WebsocketSpotSubmitOrders(ctx context.Context, orders ...*Cre
 	for i := range orders {
 		if orders[i].Text == "" {
 			// API requires Text field, or it will be rejected
-			orders[i].Text = "t-" + strconv.FormatInt(e.Counter.IncrementAndGet(), 10)
+			orders[i].Text = "t-" + strconv.FormatInt(e.messageIDSeq.IncrementAndGet(), 10)
 		}
 		if orders[i].CurrencyPair.IsEmpty() {
 			return nil, currency.ErrCurrencyPairEmpty
--- a/exchanges/gateio/gateio_wrapper.go
+++ b/exchanges/gateio/gateio_wrapper.go
@@ -208,17 +208,17 @@ func (e *Exchange) Setup(exch *config.Exchange) error {
 	}
 	// Spot connection
 	err = e.Websocket.SetupNewConnection(&websocket.ConnectionSetup{
-		URL:                      gateioWebsocketEndpoint,
-		ResponseCheckTimeout:     exch.WebsocketResponseCheckTimeout,
-		ResponseMaxLimit:         exch.WebsocketResponseMaxLimit,
-		Handler:                  e.WsHandleSpotData,
-		Subscriber:               e.Subscribe,
-		Unsubscriber:             e.Unsubscribe,
-		GenerateSubscriptions:    e.generateSubscriptionsSpot,
-		Connector:                e.WsConnectSpot,
-		Authenticate:             e.authenticateSpot,
-		MessageFilter:            asset.Spot,
-		BespokeGenerateMessageID: e.GenerateWebsocketMessageID,
+		URL:                   gateioWebsocketEndpoint,
+		ResponseCheckTimeout:  exch.WebsocketResponseCheckTimeout,
+		ResponseMaxLimit:      exch.WebsocketResponseMaxLimit,
+		Handler:               e.WsHandleSpotData,
+		Subscriber:            e.Subscribe,
+		Unsubscriber:          e.Unsubscribe,
+		GenerateSubscriptions: e.generateSubscriptionsSpot,
+		Connector:             e.WsConnectSpot,
+		Authenticate:          e.authenticateSpot,
+		MessageFilter:         asset.Spot,
+		RequestIDGenerator:    e.messageIDSeq.IncrementAndGet,
 	})
 	if err != nil {
 		return err
@@ -236,10 +236,10 @@ func (e *Exchange) Setup(exch *config.Exchange) error {
 		GenerateSubscriptions: func() (subscription.List, error) {
 			return e.GenerateFuturesDefaultSubscriptions(asset.USDTMarginedFutures)
 		},
-		Connector:                e.WsFuturesConnect,
-		Authenticate:             e.authenticateFutures,
-		MessageFilter:            asset.USDTMarginedFutures,
-		BespokeGenerateMessageID: e.GenerateWebsocketMessageID,
+		Connector:          e.WsFuturesConnect,
+		Authenticate:       e.authenticateFutures,
+		MessageFilter:      asset.USDTMarginedFutures,
+		RequestIDGenerator: e.messageIDSeq.IncrementAndGet,
 	})
 	if err != nil {
 		return err
@@ -258,9 +258,9 @@ func (e *Exchange) Setup(exch *config.Exchange) error {
 		GenerateSubscriptions: func() (subscription.List, error) {
 			return e.GenerateFuturesDefaultSubscriptions(asset.CoinMarginedFutures)
 		},
-		Connector:                e.WsFuturesConnect,
-		MessageFilter:            asset.CoinMarginedFutures,
-		BespokeGenerateMessageID: e.GenerateWebsocketMessageID,
+		Connector:          e.WsFuturesConnect,
+		MessageFilter:      asset.CoinMarginedFutures,
+		RequestIDGenerator: e.messageIDSeq.IncrementAndGet,
 	})
 	if err != nil {
 		return err
@@ -275,12 +275,12 @@ func (e *Exchange) Setup(exch *config.Exchange) error {
 		Handler: func(ctx context.Context, conn websocket.Connection, incoming []byte) error {
 			return e.WsHandleFuturesData(ctx, conn, incoming, asset.DeliveryFutures)
 		},
-		Subscriber:               e.DeliveryFuturesSubscribe,
-		Unsubscriber:             e.DeliveryFuturesUnsubscribe,
-		GenerateSubscriptions:    e.GenerateDeliveryFuturesDefaultSubscriptions,
-		Connector:                e.WsDeliveryFuturesConnect,
-		MessageFilter:            asset.DeliveryFutures,
-		BespokeGenerateMessageID: e.GenerateWebsocketMessageID,
+		Subscriber:            e.DeliveryFuturesSubscribe,
+		Unsubscriber:          e.DeliveryFuturesUnsubscribe,
+		GenerateSubscriptions: e.GenerateDeliveryFuturesDefaultSubscriptions,
+		Connector:             e.WsDeliveryFuturesConnect,
+		MessageFilter:         asset.DeliveryFutures,
+		RequestIDGenerator:    e.messageIDSeq.IncrementAndGet,
 	})
 	if err != nil {
 		return err
@@ -288,16 +288,16 @@ func (e *Exchange) Setup(exch *config.Exchange) error {

 	// Futures connection - Options
 	return e.Websocket.SetupNewConnection(&websocket.ConnectionSetup{
-		URL:                      optionsWebsocketURL,
-		ResponseCheckTimeout:     exch.WebsocketResponseCheckTimeout,
-		ResponseMaxLimit:         exch.WebsocketResponseMaxLimit,
-		Handler:                  e.WsHandleOptionsData,
-		Subscriber:               e.OptionsSubscribe,
-		Unsubscriber:             e.OptionsUnsubscribe,
-		GenerateSubscriptions:    e.GenerateOptionsDefaultSubscriptions,
-		Connector:                e.WsOptionsConnect,
-		MessageFilter:            asset.Options,
-		BespokeGenerateMessageID: e.GenerateWebsocketMessageID,
+		URL:                   optionsWebsocketURL,
+		ResponseCheckTimeout:  exch.WebsocketResponseCheckTimeout,
+		ResponseMaxLimit:      exch.WebsocketResponseMaxLimit,
+		Handler:               e.WsHandleOptionsData,
+		Subscriber:            e.OptionsSubscribe,
+		Unsubscriber:          e.OptionsUnsubscribe,
+		GenerateSubscriptions: e.GenerateOptionsDefaultSubscriptions,
+		Connector:             e.WsOptionsConnect,
+		MessageFilter:         asset.Options,
+		RequestIDGenerator:    e.messageIDSeq.IncrementAndGet,
 	})
 }