* qa: Enforce JSON lint for GCT configs
* Makefile/GHA: Make output more verbose on success, clean duplicate PHONY and rid excess newlines
* Makefile: Use printf for OS cross-compatibility output
* gateio: fix spot deployment issue
* fix status bug add test
* to actual return type
* fix linter
* ch type
* glorious: nits
* rm space
* glorious: nits
---------
Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
* Currency: Remove Pair Index formatting/parsing
This feature was originally for exchanges with only one pair (e.g. KRW) which made parsing easier.
However there's no examples of this left, and we can reduce complexity
overall by removing it.
* Exchange: Partial assertify tests and fixes
* Currency: Fix panic on a delimiterless small currency
* Kraken: Fix wsCancelOrders not erroring order id
We were using the "cancel many" facility of the Kraken api.
However since that doesn't actually report errors individually, it seems
saner to just multiplex over it.
We were going to get N+ responses anyway. Might as well send N+ requests
* Common: Add ErrorCollector and methods
* improv. timed mutex
* Add all protection back in and jankyness because races. :'(
* Add intial benchmarkeroos
* Add master benchmarks
* goodness me
* what?
* what again?
* glorious: nits
* just a swaperino instead
* clean up package nonce so that we only need to aquire mutex once
* unlock before checking master
* commentary
* wha
* more comment
* ch comment
* nonce: Allow for broad customisation externally with a ~2ns overhead
* glorious: nits maybe works?
---------
Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
* Shift wrapper function GetDefaultConfig to exchange.Base method definition, to ensure set defaults doesn't get called twice and to reduce code
* rm alphapoint bootstrap method as is defined as exchange.Base method
* add tests
* glorious: make it a function and make it IBOTEXCHANGE
---------
Co-authored-by: shazbert <ryan.oharareid@thrasher.io>
* improv. timed mutex
* Add all protection back in and jankyness because races. :'(
* Add intial benchmarkeroos
* Add master benchmarks
* goodness me
* what?
* what again?
* glorious: nits
* just a swaperino instead
---------
Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
WsConnect is calling GetInstruments, and when that fails, erroring out
and not subscribing to anything.
The response to get instruments is an object {}, which skips the
IncomingWithData check in wsReadData for arrays [].
The check in wsHandleData depended on client_ord_id, but I figure so
long as there's a nonce which matches, we can palm it off.
This results simutaneously in having to move the login handler back to
it's waiting nonce-parser, and also simplifying/deduping it.
Failed again due to kraken re-setting limits due to price increases.
Switch from specific PriceStepIncrementSize and MinimumBaseAmount to just ensuring they are positive so we don't have to do this again
* Websocket: Prevent panic on Disconnect/Connect
Previously we'd set the websocket to disconnected when *either* of the
Conn/AuthConn got a disconnect message.
This meant that:
* The other connection was still functioning
* A user would be free to call Connect()
* wsReadData() may not have exited
Any call to Connect would be acceptable, and we'd probably get a panic
from the underlying shared/re-used gorilla websocket:
`panic: runtime error: slice bounds out of range [:13501] with capacity 8192`
when a new wsReadData goro is started and the old tries to ReadMessage
as well, overallocating the buffer.
This wouldn't normally occur because trafficMonitor would see traffic
(on either connection) and then set the websocket to being connected
again.
The solution is to treat a Disconnect on either websocket as a call to
Shutdown the whole websocket, and then trafficMonitor can reconnect it
properly.
Unit Testing for this has been difficult. So far I've had to rely on a
disruption inside websocket's connectionMonitor itself to reproduce the
panic, but from there it's been very consistent.
* Websocket: Fix race on DataHandler during shutdown
Previously we would encounter situations where shutdown would race and
fail TestConnectionMessageErrors, demonstrating that consumers might not
be told why their connection is closing.
* Do not drain DataHandler on dataMonitor shutdown
Consumers should be allowed to process whatever messages were in flight
prior to a socket shutdown
* Ensure all DataHandler messages are sent to ToRoutine before shutdown
This avoids a race where we'd read a message from DataHandler, but then
process a shutdown before trying to relay it. The relay is non-blocking
anyway, so we can trust that we'd pick up the Shutdown during the next
loop.
* Send errors to DataHandler before starting a shutdown
This just reduces the chance of a race on shutdown processing
* Websocket: Log counter of dropped messages
* gateio: fix unmarshal bug and update fields
* gateio: fix wrapper function function, add helper methods
* update order types and add kucoin wrapper fix
* currency pairs
* Add tests
* gateio; inspect error and continue for no funds in account, kucoin: fetch all settlement amounts
* futures: order fixit
* finish off gateio updates for market orders
* cute line
* Update exchanges/kucoin/kucoin_wrapper.go
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
* Update exchanges/kucoin/kucoin_wrapper.go
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
* Update exchanges/gateio/gateio.go
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
* Update exchanges/gateio/gateio_wrapper.go
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
* Update exchanges/gateio/gateio_wrapper.go
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
* glorious: nits
* glorious: nits - filter by pair match and fix bug where the endpoint returns details instead of message
* Add fix for leverage check (non-merge) my ip has been blocked from gateio still... scammmmmmmm
* glorious: nitters
* Update exchanges/gateio/gateio_test.go
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
* Update exchanges/gateio/gateio_test.go
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
* Update exchanges/gateio/gateio_test.go
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
---------
Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
* kucoin: quick batching support for ticker/trades and orderbooks
* fix test
* kucoin: move pieces add commentry
* kucoin: optimise listOfAssetsCurrencyPairEnabledFor and refactor implementations, address specific orderbook channel subscription handling
* glorious: nits
* thx @thrasher-: nits addressed
* rm types and tests that are not needed
* rm subs checking code, and convert to types.Number
* not needed anymore
* fix tests
* set up reader routine to process updates before init a potential slow websocket subscriber
* implement glorious suggestion
* glorious: nitters
---------
Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
* Websocket: Remove IsInit and simplify SetProxyAddress
IsInit was basically the same as IsConnected.
Any time Connect was called both would be set to true.
Any time we had a disconnect they'd both be set to false
Shutdown() incorrectly didn't setInit(false)
SetProxyAddress simplified to only reconnect a connected Websocket.
Any other state means it hasn't been Connected, or it's about to
reconnect anyway.
There's no handling for IsConnecting previously, either, so I've wrapped
that behind the main mutex.
* Websocket: Expand and Assertify tests
* Websocket: Simplify state transistions
* Websocket: Simplify Connecting/Connected state
* Websocket: Tests and errors for websocket
* Websocket: Make WebsocketNotEnabled a real error
This allows for testing and avoids the repetition.
If each returned error is a error.New() you can never use errors.Is()
* Websocket: Add more testable errors
* Websocket: Improve GenerateMessageID test
Testing just the last id doesn't feel very robust
* Websocket: Protect Setup() from races
* Websocket: Use atomics instead of mutex
This was spurred by looking at the setState call in trafficMonitor and
the effect on blocking and efficiency.
With the new atomic types in Go 1.19, and the small types in use here,
atomics should be safe for our usage. bools should be truly atomic,
and uint32 is atomic when the accepted value range is less than one byte/uint8 since
that can be written atomicly by concurrent processors.
Maybe that's not even a factor any more, however we don't even have to worry enough to check.
* Websocket: Fix and simplify traffic monitor
trafficMonitor had a check throttle at the end of the for loop to stop it just gobbling the (blocking) trafficAlert channel non-stop.
That makes sense, except that nothing is sent to the trafficAlert channel if there's no listener.
So that means that it's out by one second on the trafficAlert, because any traffic received during the pause is doesn't try to send a traffic alert.
The unstopped timer is deliberately leaked for later GC when shutdown.
It won't delay/block anything, and it's a trivial memory leak during an infrequent event.
Deliberately Choosing to recreate the timer each time instead of using Stop, drain and reset
* Websocket: Split traficMonitor test on behaviours
* Websocket: Remove trafficMonitor connected status
trafficMonitor does not need to set the connection to be connected.
Connect() does that. Anything after that should result in a full
shutdown and restart. It can't and shouldn't become connected
unexpectedly, and this is most likely a race anyway.
Also dropped trafficCheckInterval to 100ms to mitigate races of traffic
alerts being buffered for too long.
* Websocket: Set disconnected earlier in Shutdown
This caused a possible race where state is still connected, but we start
to trigger interested actors via ShutdownC and Wait.
They may check state and then call Shutdown again, such as
trafficMonitor
* Websocket: Wait 5s for slow tests to pass traffic draining
Keep getting failures upstream on test rigs.
Think they can be very contended, so this pushes the boundary right out
to 5s
Kucoin delisted XMR temporarily for margin trading.
That seems quite likely long term too, due to XMR's privacy features.
Replace XMR with TRX to hopefully avoid in future
Retain the .msg field of a go fmt.Errorf .msg field returned by .Error()
when wrapping multiple errors.
This fixes a situation where a nested stack of errors would lose
formatting information, which is often used to supply identifying
context.
e.g.
```
err = fmt.Errorf("%w `%s`: %w", errParsingField, fieldName,
parsingError)
errs = common.AppendError(errs, err)
```
This isn't really an issue with our implementation; Calling Unwrap() on a
fmt.Errorf() which returns a wrapErrors will lose that formatting.
Our issue was that we were using just Unwrap() to bind together our
chain-of-custody.
* CI/build: Update Go version, linters and fix minor issues
* Bump golangci-lint to v1.56.1
* BinanceUS: Make uint usage consistent
* Throw blank identifiers into the trash
Calls for limits for margin asset were not doing anything if spot is
enabled.
This caused intermittent failures when the sequence of tests meant
margin was tested before spot, since the limits wouldn't be loaded.
But it also highlighted that API users calling Update Limits for margin
outside the context of a loop on all assets would not get an update.
Also intermittently when the loop of UpdateLimits on assets puts margin first
