Commit Graph

7 Commits

Author SHA1 Message Date
Gareth Kirwan
52c6b3bf0b Websocket: Various refactors and test improvements (#1466)
* Websocket: Remove IsInit and simplify SetProxyAddress

IsInit was basically the same as IsConnected.
Any time Connect was called both would be set to true.
Any time we had a disconnect they'd both be set to false
Shutdown() incorrectly didn't setInit(false)

SetProxyAddress simplified to only reconnect a connected Websocket.
Any other state means it hasn't been Connected, or it's about to
reconnect anyway.
There's no handling for IsConnecting previously, either, so I've wrapped
that behind the main mutex.

* Websocket: Expand and Assertify tests

* Websocket: Simplify state transistions

* Websocket: Simplify Connecting/Connected state

* Websocket: Tests and errors for websocket

* Websocket: Make WebsocketNotEnabled a real error

This allows for testing and avoids the repetition.
If each returned error is a error.New() you can never use errors.Is()

* Websocket: Add more testable errors

* Websocket: Improve GenerateMessageID test

Testing just the last id doesn't feel very robust

* Websocket: Protect Setup() from races

* Websocket: Use atomics instead of mutex

This was spurred by looking at the setState call in trafficMonitor and
the effect on blocking and efficiency.
With the new atomic types in Go 1.19, and the small types in use here,
atomics should be safe for our usage. bools should be truly atomic,
and uint32 is atomic when the accepted value range is less than one byte/uint8 since
that can be written atomicly by concurrent processors.
Maybe that's not even a factor any more, however we don't even have to worry enough to check.

* Websocket: Fix and simplify traffic monitor

trafficMonitor had a check throttle at the end of the for loop to stop it just gobbling the (blocking) trafficAlert channel non-stop.
That makes sense, except that nothing is sent to the trafficAlert channel if there's no listener.
So that means that it's out by one second on the trafficAlert, because any traffic received during the pause is doesn't try to send a traffic alert.

The unstopped timer is deliberately leaked for later GC when shutdown.
It won't delay/block anything, and it's a trivial memory leak during an infrequent event.

Deliberately Choosing to recreate the timer each time instead of using Stop, drain and reset

* Websocket: Split traficMonitor test on behaviours

* Websocket: Remove trafficMonitor connected status

trafficMonitor does not need to set the connection to be connected.
Connect() does that. Anything after that should result in a full
shutdown and restart. It can't and shouldn't become connected
unexpectedly, and this is most likely a race anyway.

Also dropped trafficCheckInterval to 100ms to mitigate races of traffic
alerts being buffered for too long.

* Websocket: Set disconnected earlier in Shutdown

This caused a possible race where state is still connected, but we start
to trigger interested actors via ShutdownC and Wait.
They may check state and then call Shutdown again, such as
trafficMonitor

* Websocket: Wait 5s for slow tests to pass traffic draining

Keep getting failures upstream on test rigs.
Think they can be very contended, so this pushes the boundary right out
to 5s
2024-02-23 18:39:25 +11:00
Adrian Gallagher
08da42ddb7 CI/build: Update Go version, linters and fix minor issues (#1473)
* CI/build: Update Go version, linters and fix minor issues

* Bump golangci-lint to v1.56.1

* BinanceUS: Make uint usage consistent

* Throw blank identifiers into the trash
2024-02-14 11:02:06 +11:00
Gareth Kirwan
e007f69f7c exchanges/websocket: Implement subscription configuration (#1394)
* Websockets: Move Subscription to its own package

This allows the small type to be imported from both `config` and from
`stream` without an import cycle, so we don't have to repeat ourselves

* Subs: Renamed Currency to Pair

This was being mis-used through much of the code, and since we're
already touching everything, we might as well fix it

* Websockets: Add Subscription configuration

* Binance: Add subscription configuration

* Kucoin: Subscription configuration

* Simplify GenerateDefaultSubs
* Improve TestGenSubs coverage
* Test Candle Sub generation
* Support Candle intervals
* Full responsibility for formatting Channel name on GenerateDefaultSubs
  OR consumer of Subscribe
* Simplify generatePayloads as a result
* Fix test coverage of asset types in processMarketSnapshot

* Exchanges: Abstract ParallelChanOp

* Tests: Generic ws mock instances

* Kucoin: Fix intermittent conflict in test currs

Use isolated test instance for `TestGetOpenInterest`.

`TestGetOpenInterest` would occassionally change pairs before
GenerateDefault Subs.
2024-01-24 15:54:07 +11:00
Ryan O'Hara-Reid
81a8b4a575 common: remove SimpleTimeFormat const for time package layout const DateTime (#1246)
* switch over to package defined const for time layout

* bump appveyor playa

* bumperino to latest while setting patherino

* whoooops

* bump VS version set GOROOT

* puge build cache

* Revert "puge build cache"

This reverts commit 315bb578afc19529457f435e52af2172f5143bc5.

* bumperino to test

* purge setting of golang directory for version and allow default

* purge cache state when file change

* whoops

* thrasher: nits

* don't need to flusherino the cacherino

---------

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
2023-07-06 11:17:43 +10:00
Ryan O'Hara-Reid
f843b7d277 exchange: upgrade UpdatePair method (#991)
* exchange: upgrade update pair

* exchanges: Add enabled string matching and format handling if discrepency is found.

* linter: fixes

* bithumb: fix tests

* BTSE: api change fix ordering

* huobi: fix tests

* gloriousnits: stage 1

* gloriousnits: stage 2

* currency: more nits

* bitmex: add spot and process pairs before currency package call.

* bitmex: finished correct orderbook matching and other implementations

* linter: fix issue

* currency: Fix linter

* currency: segregate and protect pair store, update tests

* currency/manager: clean code, rm log output

* currency: Add store method and make sure formatting stays nil if not stored.

* gct: check errors

* engine/websocketroutineman: fix tests

* bybit: fix duplication bug

* huobi: fix test

* btse: fix tests?

* ob/buffer: fix tests

* Update currency/manager.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* glorious: nits strikes again.

* exchange: add bypassConfigFormatUpgrades to stop formatting

* GLORIOUS LINTER

* Update exchanges/bithumb/bithumb_wrapper.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* exchange: fix pair upgrade issue when duplications are in both avail and enabled pairs

* linter: fix shadow dec

* config: fix test

* Update currency/pair_test.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2022-09-16 08:59:27 +10:00
Ryan O'Hara-Reid
66fbd43cf0 websocket: fix deadlock when enabling/disabling via gctrpc (#754)
* websocket: select case error if no receiver, add in functionality to reset to initial sync for books on a new websocket connection

* websocket: fix tests

* websocket: log error instead of losing it

* websocket: fix whoopsie

* exchanges: fix test

* websocket: force requirement of specific functionality

* exchanges: fix tests

* exchanges/websocket: move waitgroup add before scheduling across exchanges

* gateio: add feature subscribe

* bithumb/bittrex: include connection state reset, fix reconnection bug for Bithumb

* huobi: Add listen to shutdown to routine so it actually returns and stops being a naughty boy.

* huobi: add missing waitgroup add.

* exchanges: bleed comms channels

* binance: fix reconnection bug with buffer

* bithumb: fix reconnection bug with ws orderbook when websocket is diabled/enabled

* bithumb/bittrex: add bleeders for ws orderbook jobs

* linter: fix

* kraken: reduce code block from double assertion

* This bug ruined my day.

* glorious: error checking

* zb: add correct path for websocket connection

* exchange: Add verbosity when config conflicts and overwrites default values

* zb: add https to path

* exchanges: glorious nits

* stream: Add checkAndSetMonitoring to reduce potential routine bundling, increase timeout and check state in tests

* stream: remove check that is not needed.

* glorious: nits addr.

* lint: test
2021-09-03 17:21:23 +10:00
Ryan O'Hara-Reid
736c92a99b bithumb: Add websocket support (#745)
* bithumb: Add websocket support (WIP)

* bithumb: cont impl.

* bithumb: finish, issues with orderbook needs review

* linter issues

* bithumb: move to separate file

* bithumb: change to pointer for book

* bithumb: Add tests

* bithumb: Address nits

* websocket: Export subscription error and wrap returns

* bithumb: cleanup/ fix tests

* gctrpc/bithumb: fix misspelling

* bithumb: gofmt

* readme: update support table, regen doc

* tradesReadme: update

* readme: update template
2021-08-16 16:53:23 +10:00