mirror of
https://github.com/d0zingcat/gocryptotrader.git
synced 2026-06-03 07:26:45 +00:00
bybit: enable multiconnection handling across websocket endpoints (#1670)
* glorious: whooops * gk: nits * Leak issue and edge case * Websocket: Add SendMessageReturnResponses * whooooooopsie * gk: nitssssss * Update exchanges/stream/stream_match.go Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com> * Update exchanges/stream/stream_match_test.go Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com> * linter: appease the linter gods * gk: nits * gk: drain brain * started * more changes before merge match pr * gateio: still building out * gateio: finish spot * fix up tests in gateio * Add tests for stream package * rm unused field * glorious: nits * rn files, specifically set function names to asset and offload routing to websocket type. * linter: fix * Add futures websocket request support * gateio: integrate with IBOTExchange (cherry pick my nose) * linter: fix * glorious: nits * add counter and update gateio * fix collision issue * Update exchanges/stream/websocket.go Co-authored-by: Scott <gloriousCode@users.noreply.github.com> * glorious: nits * add tests * linter: fix * After merge * Add error connection info * upgrade to upstream merge * Fix edge case where it does not reconnect made by an already closed connection * stream coverage * glorious: nits * glorious: nits removed asset error handling in stream package * linter: fix * rm block * Add basic readme * fix asset enabled flush cycle for multi connection * spella: fix * linter: fix * Add glorious suggestions, fix some race thing * reinstate name before any routine gets spawned * stop on error in mock tests * glorious: nits * Set correct price * glorious: nits found in CI build * Add test for drain, bumped wait times as there seems to be something happening on macos CI builds, used context.WithTimeout because its instant. * mutex across shutdown and connect for protection * lint: fix * test time withoffset, reinstate stop * fix whoops * const trafficCheckInterval; rm testmain * y * fix lint * bump time check window * stream: fix intermittant test failures while testing routines and remove code that is not needed. * spells * cant do what I did * protect race due to routine. * update testURL * use mock websocket connection instead of test URL's * linter: fix * remove url because its throwing errors on CI builds * connections drop all the time, don't need to worry about not being able to echo back ws data as it can be easily reviewed _test file side. * remove another superfluous url thats not really set up for this * spawn overwatch routine when there is no errors, inline checker instead of waiting for a time period, add sleep inline with echo handler as this is really quick and wanted to ensure that latency is handing correctly * linter: fixerino uperino * fix ID bug, why I do this, I don't know. * glorious: panix * linter: things * whoops * dont need to make consecutive Unix() calls * websocket: fix potential panic on error and no responses and adding waitForResponses * bybit: enable multiconnection handling across websocket endpoints * rm debug lines * rm json parser and handle in json package instead * in favour of json package unmarshalling * fix processing issues with tickers * linter: fix * linter: fix again * * change field name OutboundRequestSignature to WrapperDefinedConnectionSignature for agnostic inbound and outbound connections. * change method name GetOutboundConnection to GetConnection for agnostic inbound and outbound connections. * drop outbound field map for improved performance just using a range and field check (less complex as well) * change field name connections to connectionToWrapper for better clarity * spells and magic and wands * merge: fixup * linter: fix * spelling: fix * glorious: nits * comparable check for signature * mv err var * glorious: nits and stuff * attempt to fix race * linter: fix * fix tests * types/time: strict usage of time type for usage with unix timestamps * fix tests etc * glorious: nits * gk: nits; engine log cleanup * gk: nits; OCD * gk: nits; move function change file names * gk: nits; 🚀 * gk: nits; convert variadic function and message inspection to interface and include a specific function for that handling so as to not need nil on every call * gk: nits; continued * gk: engine nits; rm loaded exchange * gk: nits; drop WebsocketLoginResponse * stream: Add match method EnsureMatchWithData * gk: nits; rn Inspect to IsFinal * gk: nits; rn to MessageFilter * linter: fix * gateio: update rate limit definitions (cherry-pick) * Add test and missing * Shared REST rate limit definitions with Websocket service, set lookup item to nil for systems that do not require rate limiting; add glorious nit * integrate rate limits for websocket trading spot * bybit: split public and private processing to dedicated handler add supporting function and tests * use correct handler for private inbound connection * conform to match upstream changes * standardise names to upstream style * fix wrapper standards test when sending a auth request through a websocket connection * whoops * Update exchanges/gateio/gateio_types.go Co-authored-by: Scott <gloriousCode@users.noreply.github.com> * glorious: nits * linter: fix * linter: overload * whoops * spelling fixes on recent merge * glorious: nits * linter: fix? * glorious: nits * gk: assert errors touched * gk: unexport derive functions * gk: nitssssssss * fix test * gk: nitters v1 * gk: http status * gk/nits: Add getAssetFromFuturesPair * gk: nits single response when submitting * gk: new pair with delimiter in tests * gk: param update slice to slice of pointers * gk: add asset type in params, includes t.Context() for tests * linter: fix * linter: fix * fix merge whoopsie * glorious: nits * gk: nit * linter: fix * glorious: nits * linter/misc: fix and remove meows * okx: update requestID gen func without func wrapping * RM: functions not needed * Update docs/ADD_NEW_EXCHANGE.md Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com> * gk: nitsssssss * linter: fix * Update exchanges/bybit/bybit_test.go Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com> * Update exchanges/bybit/bybit_test.go Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com> * gk: nit words * cranktakular: nits * linter: fix * cranktakular: nits and expand coverage * linter: fix? * misc fix * cranktakular: missing nit which I thumbed up but did not do. Sillllllly billlyyyy nilllyyy * cranktakular: nits * cranktakular: purge DCP ref/handling and add another TODO * Update exchanges/bybit/bybit_websocket.go Co-authored-by: Scott <gloriousCode@users.noreply.github.com> * glorious: nits * fix test * fix alignment issue and rm println * Update exchanges/bybit/bybit_websocket.go Co-authored-by: Scott <gloriousCode@users.noreply.github.com> * Update exchanges/bybit/bybit_websocket.go Co-authored-by: Scott <gloriousCode@users.noreply.github.com> * glorious: fix * Update exchanges/bybit/bybit_websocket.go Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io> * Update common/common.go Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com> * Update common/common_test.go Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com> * Update exchanges/bybit/bybit_test.go Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com> * gk: nits * gk: nit with test --------- Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io> Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com> Co-authored-by: Scott <gloriousCode@users.noreply.github.com> Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
This commit is contained in:
Ryan O'Hara-Reid
GitHub
@@ -88,11 +88,11 @@ type ConnectionSetup struct {
|
||||
// received from the exchange's websocket server. This function should
|
||||
// handle the incoming message and pass it to the appropriate data handler.
|
||||
Handler func(ctx context.Context, conn Connection, incoming []byte) error
|
||||
// BespokeGenerateMessageID is a function that returns a unique message ID.
|
||||
// RequestIDGenerator is a function that returns a unique message ID.
|
||||
// This is useful for when an exchange connection requires a unique or
|
||||
// structured message ID for each message sent.
|
||||
BespokeGenerateMessageID func(highPrecision bool) int64
|
||||
Authenticate func(ctx context.Context, conn Connection) error
|
||||
RequestIDGenerator func() int64
|
||||
Authenticate func(ctx context.Context, conn Connection) error
|
||||
// MessageFilter defines the criteria used to match messages to a specific connection.
|
||||
// The filter enables precise routing and handling of messages for distinct connection contexts.
|
||||
MessageFilter any
|
||||
@@ -112,23 +112,23 @@ type Response struct {
|
||||
|
||||
// connection contains all the data needed to send a message to a websocket connection
|
||||
type connection struct {
|
||||
Verbose bool
|
||||
connected int32
|
||||
writeControl sync.Mutex // Gorilla websocket does not allow more than one goroutine to utilise write methods
|
||||
RateLimit *request.RateLimiterWithWeight // RateLimit is a rate limiter for the connection itself
|
||||
RateLimitDefinitions request.RateLimitDefinitions // RateLimitDefinitions contains the rate limiters shared between WebSocket and REST connections
|
||||
Reporter Reporter
|
||||
ExchangeName string
|
||||
URL string
|
||||
ProxyURL string
|
||||
Wg *sync.WaitGroup
|
||||
Connection *gws.Conn
|
||||
shutdown chan struct{}
|
||||
Match *Match
|
||||
ResponseMaxLimit time.Duration
|
||||
Traffic chan struct{}
|
||||
readMessageErrors chan error
|
||||
bespokeGenerateMessageID func(highPrecision bool) int64
|
||||
Verbose bool
|
||||
connected int32
|
||||
writeControl sync.Mutex // Gorilla websocket does not allow more than one goroutine to utilise write methods
|
||||
RateLimit *request.RateLimiterWithWeight // RateLimit is a rate limiter for the connection itself
|
||||
RateLimitDefinitions request.RateLimitDefinitions // RateLimitDefinitions contains the rate limiters shared between WebSocket and REST connections
|
||||
Reporter Reporter
|
||||
ExchangeName string
|
||||
URL string
|
||||
ProxyURL string
|
||||
Wg *sync.WaitGroup
|
||||
Connection *gws.Conn
|
||||
shutdown chan struct{}
|
||||
Match *Match
|
||||
ResponseMaxLimit time.Duration
|
||||
Traffic chan struct{}
|
||||
readMessageErrors chan error
|
||||
requestIDGenerator func() int64
|
||||
}
|
||||
|
||||
// Dial sets proxy urls and then connects to the websocket
|
||||
@@ -339,8 +339,8 @@ func (c *connection) parseBinaryResponse(resp []byte) ([]byte, error) {
|
||||
// If a bespoke function is set (by using SetupNewConnection) it will use that,
|
||||
// otherwise it will use the defaultGenerateMessageID function.
|
||||
func (c *connection) GenerateMessageID(highPrec bool) int64 {
|
||||
if c.bespokeGenerateMessageID != nil {
|
||||
return c.bespokeGenerateMessageID(highPrec)
|
||||
if c.requestIDGenerator != nil {
|
||||
return c.requestIDGenerator()
|
||||
}
|
||||
return c.defaultGenerateMessageID(highPrec)
|
||||
}
|
||||
|
||||
@@ -305,12 +305,7 @@ func (m *Manager) SetupNewConnection(c *ConnectionSetup) error {
|
||||
return err
|
||||
}
|
||||
|
||||
if c == nil || c.ResponseCheckTimeout == 0 &&
|
||||
c.ResponseMaxLimit == 0 &&
|
||||
c.RateLimit == nil &&
|
||||
c.URL == "" &&
|
||||
c.ConnectionLevelReporter == nil &&
|
||||
c.BespokeGenerateMessageID == nil {
|
||||
if c.ResponseCheckTimeout == 0 && c.ResponseMaxLimit == 0 && c.RateLimit == nil && c.URL == "" && c.ConnectionLevelReporter == nil && c.RequestIDGenerator == nil {
|
||||
return fmt.Errorf("%w: %w", errConnSetup, errExchangeConfigEmpty)
|
||||
}
|
||||
|
||||
@@ -394,20 +389,20 @@ func (m *Manager) getConnectionFromSetup(c *ConnectionSetup) *connection {
|
||||
match = NewMatch()
|
||||
}
|
||||
return &connection{
|
||||
ExchangeName: m.exchangeName,
|
||||
URL: connectionURL,
|
||||
ProxyURL: m.GetProxyAddress(),
|
||||
Verbose: m.verbose,
|
||||
ResponseMaxLimit: c.ResponseMaxLimit,
|
||||
Traffic: m.TrafficAlert,
|
||||
readMessageErrors: m.ReadMessageErrors,
|
||||
shutdown: m.ShutdownC,
|
||||
Wg: &m.Wg,
|
||||
Match: match,
|
||||
RateLimit: c.RateLimit,
|
||||
Reporter: c.ConnectionLevelReporter,
|
||||
bespokeGenerateMessageID: c.BespokeGenerateMessageID,
|
||||
RateLimitDefinitions: m.rateLimitDefinitions,
|
||||
ExchangeName: m.exchangeName,
|
||||
URL: connectionURL,
|
||||
ProxyURL: m.GetProxyAddress(),
|
||||
Verbose: m.verbose,
|
||||
ResponseMaxLimit: c.ResponseMaxLimit,
|
||||
Traffic: m.TrafficAlert,
|
||||
readMessageErrors: m.ReadMessageErrors,
|
||||
shutdown: m.ShutdownC,
|
||||
Wg: &m.Wg,
|
||||
Match: match,
|
||||
RateLimit: c.RateLimit,
|
||||
Reporter: c.ConnectionLevelReporter,
|
||||
requestIDGenerator: c.RequestIDGenerator,
|
||||
RateLimitDefinitions: m.rateLimitDefinitions,
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -769,7 +769,7 @@ func TestGenerateMessageID(t *testing.T) {
|
||||
ids[i] = id
|
||||
}
|
||||
|
||||
wc.bespokeGenerateMessageID = func(bool) int64 { return 42 }
|
||||
wc.requestIDGenerator = func() int64 { return 42 }
|
||||
assert.EqualValues(t, 42, wc.GenerateMessageID(true), "GenerateMessageID should use bespokeGenerateMessageID")
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user