* Tests: Use currency.NewUSD and NewUSDT
Simple refactor to use the provided shortcut methods
* Github: Add CI check to ensure NewPair not used
Add a step to ensure NewPair(BTC, USD*) isn't used
* golangci-lint: Enable usetesting and unused linters
* tests: Improve assertions in various test cases for clarity and accuracy
* tests: Enhance error assertions in TestExecuteStrategyFromFile for improved clarity
* tests: Update assertions for improved clarity and accuracy
* tests: Replace assert with require for task count checks
* config/versions/v7: Replace context.Background() with t.Context()
* Bithumb: Centralise and consoliate testPair, relax UpdateTickers check
with some glorious Doom Eternal music
* Bithumb: Use UpdatePairsOnce and update remaining pair string
* Bithumb: Add UpdatePairsOnce to TestUpdateTickers
* Exchanges: Remove cancel order walletAddress
* Order: Refactor TestMatchFilter
TestMatchFilter had inconsistent testing of empty values, and was
painful when a field was removed due to index methodology.
This should provide equal test clarity, but improve maintainability and
improve coverage on empty values.
* cmd/documentation: Add pagination support for contributors
* Drop break and return immediately
* documentation: change defaultGithubAPIPerPageLimit to an integer and update usage in GetContributorList
* gateio: Add multi asset websocket support WIP.
* meow
* Add tests and shenanigans
* integrate flushing and for enabling/disabling pairs from rpc shenanigans
* some changes
* linter: fixes strikes again.
* Change name ConnectionAssociation -> ConnectionCandidate for better clarity on purpose. Change connections map to point to candidate to track subscriptions for future dynamic connections holder and drop struct ConnectionDetails.
* Add subscription tests (state functional)
* glorious:nits + proxy handling
* Spelling
* linter: fixerino
* instead of nil, dont do nil.
* clean up nils
* cya nils
* don't need to set URL or check if its running
* stream match update
* update tests
* linter: fix
* glorious: nits + handle context cancellations
* stop ping handler routine leak
* * Fix bug where reader routine on error that is not a disconnection error but websocket frame error or anything really makes the reader routine return and then connection never cycles and the buffer gets filled.
* Handle reconnection via an errors.Is check which is simpler and in that scope allow for quick disconnect reconnect without waiting for connection cycle.
* Dial now uses code from DialContext but just calls context.Background()
* Don't allow reader to return on parse binary response error. Just output error and return a non nil response
* Allow rollback on connect on any error across all connections
* fix shadow jutsu
* glorious/gk: nitters - adds in ws mock server
* linter: fix
* fix deadlock on connection as the previous channel had no reader and would hang connection reader for eternity.
* glorious: whooops
* gk: nits
* Leak issue and edge case
* Websocket: Add SendMessageReturnResponses
* whooooooopsie
* gk: nitssssss
* Update exchanges/stream/stream_match.go
Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>
* Update exchanges/stream/stream_match_test.go
Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>
* linter: appease the linter gods
* gk: nits
* gk: drain brain
* started
* more changes before merge match pr
* gateio: still building out
* gateio: finish spot
* fix up tests in gateio
* Add tests for stream package
* rm unused field
* glorious: nits
* rn files, specifically set function names to asset and offload routing to websocket type.
* linter: fix
* Add futures websocket request support
* gateio: integrate with IBOTExchange (cherry pick my nose)
* linter: fix
* glorious: nits
* add counter and update gateio
* fix collision issue
* Update exchanges/stream/websocket.go
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
* glorious: nits
* add tests
* linter: fix
* After merge
* Add error connection info
* upgrade to upstream merge
* Fix edge case where it does not reconnect made by an already closed connection
* stream coverage
* glorious: nits
* glorious: nits removed asset error handling in stream package
* linter: fix
* rm block
* Add basic readme
* fix asset enabled flush cycle for multi connection
* spella: fix
* linter: fix
* Add glorious suggestions, fix some race thing
* reinstate name before any routine gets spawned
* stop on error in mock tests
* glorious: nits
* Set correct price
* glorious: nits found in CI build
* Add test for drain, bumped wait times as there seems to be something happening on macos CI builds, used context.WithTimeout because its instant.
* mutex across shutdown and connect for protection
* lint: fix
* test time withoffset, reinstate stop
* fix whoops
* const trafficCheckInterval; rm testmain
* y
* fix lint
* bump time check window
* stream: fix intermittant test failures while testing routines and remove code that is not needed.
* spells
* cant do what I did
* protect race due to routine.
* update testURL
* use mock websocket connection instead of test URL's
* linter: fix
* remove url because its throwing errors on CI builds
* connections drop all the time, don't need to worry about not being able to echo back ws data as it can be easily reviewed _test file side.
* remove another superfluous url thats not really set up for this
* spawn overwatch routine when there is no errors, inline checker instead of waiting for a time period, add sleep inline with echo handler as this is really quick and wanted to ensure that latency is handing correctly
* linter: fixerino uperino
* fix ID bug, why I do this, I don't know.
* glorious: panix
* linter: things
* whoops
* dont need to make consecutive Unix() calls
* websocket: fix potential panic on error and no responses and adding waitForResponses
* rm json parser and handle in json package instead
* in favour of json package unmarshalling
* linter: fix
* linter: fix again
* * change field name OutboundRequestSignature to WrapperDefinedConnectionSignature for agnostic inbound and outbound connections.
* change method name GetOutboundConnection to GetConnection for agnostic inbound and outbound connections.
* drop outbound field map for improved performance just using a range and field check (less complex as well)
* change field name connections to connectionToWrapper for better clarity
* spells and magic and wands
* glorious: nits
* comparable check for signature
* mv err var
* glorious: nits and stuff
* attempt to fix race
* glorious: nits
* gk: nits; engine log cleanup
* gk: nits; OCD
* gk: nits; move function change file names
* gk: nits; 🚀
* gk: nits; convert variadic function and message inspection to interface and include a specific function for that handling so as to not need nil on every call
* gk: nits; continued
* gk: engine nits; rm loaded exchange
* gk: nits; drop WebsocketLoginResponse
* stream: Add match method EnsureMatchWithData
* gk: nits; rn Inspect to IsFinal
* gk: nits; rn to MessageFilter
* linter: fix
* gateio: update rate limit definitions (cherry-pick)
* Add test and missing
* Shared REST rate limit definitions with Websocket service, set lookup item to nil for systems that do not require rate limiting; add glorious nit
* integrate rate limits for websocket trading spot
* conform to match upstream changes
* standardise names to upstream style
* fix wrapper standards test when sending a auth request through a websocket connection
* whoops
* Update exchanges/gateio/gateio_types.go
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
* glorious: nits
* linter: fix
* linter: overload
* whoops
* spelling fixes on recent merge
* glorious: nits
* linter: fix?
* glorious: nits
* gk: assert errors touched
* gk: unexport derive functions
* gk: nitssssssss
* fix test
* gk: nitters v1
* gk: http status
* gk/nits: Add getAssetFromFuturesPair
* gk: nits single response when submitting
* gk: new pair with delimiter in tests
* gk: param update slice to slice of pointers
* gk: add asset type in params, includes t.Context() for tests
* linter: fix
* linter: fix
* fix merge whoopsie
* glorious: nits
* gk: nit
* shift over to websocket package error
* internal/exchange/websocket -> exchange/websocket
* PEAK OCD!
* appease the OCD gods
* thrasher: nits
---------
Co-authored-by: shazbert <ryan.oharareid@thrasher.io>
Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
* Websocket: Rename stream package
* Websocket: Rename Websocket to Manager
* Websocket: Replace explicit errs with common.NilGuard
* Websocket: Move websocket_types.go to types.go
* Websocket: Minor field comment and alignment in types
* Webosocket: Rename WebsocketConnection to Connection
* Alphapoint: Make gorilla ws import explicit
Just to avoid confusion with our own packages.
* Websocket: Move stream_match to match
* Websocket: Move websocket_connection to connection
* Websocket: Move websocket.go to manager.go
* Websocket: Break out all subscription methods into subscriptions.go
* Websocket: Move connection type into its file
* Websocket: Remove PositionUpdated
Type is not used anywhere
* Kraken: Use local constant for pong
Was the only use of websocket.Pong and doesn't really feel right to
represent kraken's api resp in one of our packages
* Websocket: Move connection sub-types to connection package
* Websocket: Move manager types into manager
* Websocket: Move ConnectionWrapper into manager
* Websocket: Move websocket_test to manager_test
* Websocket: Privatise connectionWrapper
* Websocket: Remaining types into types.go
These really belong somewhere else mostly, but this will do for now
* Websocket: Tidy up connection method vars
* Gofumpt: Moving package imports around
* Websocket: Rename errDuplicateConnectionSetup
* Websocket: Fix duplicate import of gws
* Websocket: Fix gofumpt -extra
* Websocket: Standardise import of gws across other pkgs
* Kraken: Remove unused sub conf consts
These were replaced by the generic Levels and Depth fields on all subs
* Websocket: Privitise ConnectioWrapper fields
* Websocket: inline single use var WebsocketNotAuthenticatedUsingRest
* Websocket: Move documentation to template
* Bithumb: Assertify TestWsHandleData
* portfolio: Fix CryptoID balance issue and assertify tests
* portfolio: Expand context usage, enhance tests and a few other minor improvements
* portfolio: Further improvements and enhance common.IsValidCryptoAddress
* config, portfolio: Use v6.DefaultConfig, switch to context.WithCancel
* modernise: Run new gopls modernise tool against codebase
* Address shazbert's nits
* apichecker, gctcli: Simplify HTML scraping functions and improve depth limit handling
* refactor: Create minSyncInterval const and update order book limit handling for binance and binanceUS
* refactor: Various slice usage improvements and rename TODO
* tranches: Revert deleteByID changes due to performance decrease
Shazbert was a F1 driver in a past lifetime 🏎️
* tranches: Simply retrieve copy
Thanks to shazbert
* documentation: Sort contributors list by contributions
* tranches: Remove deadcode in deleteByID
* Sonic: Add sonic_on build tag
Thinking about other ways to do this, but they amount to the same thing.
It's messy, but I don't have another idea.
* Okx: Remove redundant useAsIs on slices
Slices are automatically used-as-is, so passing true for these types was
unnecessary.
Removing the field simplifies rationalising inverting the flag
* Okx: Unify timestamp response types
* Okx: Change ResetRFQMMPStatus to return a types.Time
* Okx: Move withdrawData type to types
* Okx: Fix AccountConfiguration slice return
* Okx: Improve SendHTTPRequest documentation
* Okx: Extend tests for UseAsIs non-defaults
* Okx: Fix GetPublicUnderlying with sonic 1.12.9
Using **result for slices with useAsItIs causes sonic to fail.
This might be addressed upstream, but it's also not clear what the
unmarshal behaviour for an untyped reference to a typed reference should
be in the RFC, so we could get a golang.org/encoding/json regression on
this too.
There's no harm in fixing this, for consistency, to match our handling
for non-slice []any wrapping to just use the pointer as is.
Note: As of today this requires sonic:main for this to work, because of
the other bug:
```
M go.mod
- github.com/bytedance/sonic v1.12.9
+ github.com/bytedance/sonic v1.12.10-0.20250224121557-e30ac4f2e4fe
```
* Okx: Remove redundant slice check
This code didn't work, and if I make it look at rv.Elem().Kind() it
errors.
Haven't dug too deeply but right now I think we just remove it.
* Okx: Simplify SendHTTPRequest by removing UseAsIs
Looks to be entirely derivable
* Okx: Remove http check that resps must contain data
GetAnnouncementTyeps failing in US because of empty response.
But also any situation where there really is no data.
e.g. GetCandlesticks might return no candlesticks for a period and instrument, because there aren't any.
That shouldn't be an error.
More saliently if you request orders, or something similar.
So, since that check wasn't working before, and it's causing issues now, I'm going to remove it.
* Okx: Fix TestGetAnnouncementTypes failing in US
announcement-types returns empty in the US, where our github actions
run.
That's kinda okay. Just don't test we get any back
* Sonic: Default to sonic off
We've seen too many fatal panics and races with sonic, both in GCT runs
and being reported in sonic, to default to it being turned on right now.
Whilst we have faith sonic will get through these with time, for now the
sensible thing to do for our users is make sonic opt-in.
This also removes any of the conditions around 386, etc.
If someone wants to run with sonic, they can. Most notably if they're
trying out an experimental sonic branch that supports 386, etc.
* Config: Move assetEnabled upgrade to Version management
* Assets: Do not error on asset not enabled, or disabled
This became more messy with Disabling something that's defaulted to
disabled.
Taking an idealogical stance against erroring that what you want to have
done is already done.
* CurrencyManager: Set AssetEnabled when StorePairs(enabled)
* RPCServer: Fix tests expecting StoreAssetPairFormat to enable the asset
Also assertifies
* Bitfinex: Fix tests for MarginFunding subs
* GCTWrapper: Improve TestMain clarity
* BTSE: Add futures to testconfig
* Exchanges: Rename StoreAssetPairStore
Previously we were calling it "Format", but accepting everything from
the PairStore.
We were also defaulting to turning the Asset on.
Now callers need to get their AssetEnabled set as they want it, so
there's no magic
This change also moves responsibility for error wrapping outside to the
caller.
* Config: AssetEnabled upgrade should respect assetTypes
Previously we ignored the field and just turned on everything.
I think that was because we couldn't get at the old value.
In either case, we have the option to do better, and respect the
assetEnabled value
* Config: Improve exchange config version upgrade error messages
* build/ci: Update Go to v1.24, golangci-lint to v1.64.5 and fix issues
* Address shazbert's nitters
* linter/config: Fix new linter issue and use versionSize const
* Address gk's nitters and fix additional linter issue after rebase
* Address glorious nits
* staticcheck: Fix additional linter issues after upgrading to Go 1.24.1 and golangci-lint v1.64.6
Also addresses nits
* Improve testing, assertify usage and use common.ErrParsingWSField
* TestCreateNewStrategy: Replace must > should wording
* Config: Tighten config version handling as uint16
This constrains the versions to uint16 and improves error handling.
LatestVersion becomes literally that.
Fixes handling for negative or overflowing versions in config
* Config: Rename LatestVersion to UseLatestVersion
* CI: Replace docker amd64 with native runner, switch Docker test arch to amd64
* tests: Remove spaces and new lines from test JSON timestamp fields
* builds: Disable sonic for arm64 due to OKX unmarshal issue
* nits: Improve Dockerfile comment and fix build tag
* tag optional sonic and allow full library conversion
* Add workflow and disallow arm and darwin usage
* Add basic hotswap benchmark
* linter: fix
* use bash
* linter: fix?
* Fix whoopsie, add to make file, also add mention in features list.
* test enforcement
* actually read documentation see if this works
* linter: fix
* linter: fix
* sonic: bump tagged version
* encoding/json: drop build tag arch and os filters
* encoding/json: consolidate tests
* encoding/json: log build tag usage
* rm superfluous builds
* glorious/nits: add template change and regen docs
* glorious/nits: update commentary on nolint directive
* glorious/nits: rm init func and log results in main.go
* Test to actually pull flag in
* linter: fix
* thrasher: nits
* gk: nits 4 goflags goooooooooo!
* gk: nits rn
* make sonic default json implementation
* screen 386
* linter: fix
* Add commentary
* glorious: nits Makefile not working
* gk: nits
* gk: nits whoops
* whoopsirino
* mention 32bit systems won't be sonic
* gk: super-duper nit of extremes
---------
Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
* Config: Rename DecryptConfigFile to DecryptConfigData
Because this isn't really a file, it's a byte slice
* Config: Rename EncryptConfigFile to EncryptConfigData
Because it's not actually a file
* Config: Fix config version downgrade
Fixes#1769
As a GCT user with spot and margin assets enabled, but only margin asset enabled websocket subscriptions,
I should still get subscriptions for all the pairs in margin which are also in spot
Currently it only works when spot subscriptions are enabled. Otherwise the spot pairs are ignored.
Fixes#1755
* Kline: Fix Raw Short, Marshal and Unmarshal
* Deribit: Rename GenerateDefaultSubs
* Deribit: Remove custom GetDefaultConfig
Moved to exchange base by #1472
* Deribit: Straight Rename of eps to endpoints
Since I had to ask what this abbreviation meant, I think we should
abandon it
* Deribit: Add Subscription configuration
* Deribit: Fix race on Setup with optionsRegex
Calling Setup twice would race on the assignment to this package var.
There was an option to just move the assignment to the package var declaration, but this
change improves the performance and allocations:
```
BenchmarkOptionPairToString-8 1000000 1239 ns/op 485 B/op 10 allocs/op
BenchmarkOptionPairToString2-8 3473804 656.2 ns/op 348 B/op 7 allocs/op
```
I've also removed the t.Run because even success the -v output from
tests would be very noisy, and I don't think we were getting any benefit
from it at all:
```
=== RUN TestOptionPairToString
=== PAUSE TestOptionPairToString
=== CONT TestOptionPairToString
=== RUN TestOptionPairToString/BTC-30MAY24-61000-C
=== PAUSE TestOptionPairToString/BTC-30MAY24-61000-C
=== RUN TestOptionPairToString/ETH-1JUN24-3200-P
=== PAUSE TestOptionPairToString/ETH-1JUN24-3200-P
=== RUN TestOptionPairToString/SOL_USDC-31MAY24-162-P
=== PAUSE TestOptionPairToString/SOL_USDC-31MAY24-162-P
=== RUN TestOptionPairToString/MATIC_USDC-6APR24-0d98-P
=== PAUSE TestOptionPairToString/MATIC_USDC-6APR24-0d98-P
=== CONT TestOptionPairToString/BTC-30MAY24-61000-C
=== CONT TestOptionPairToString/SOL_USDC-31MAY24-162-P
=== CONT TestOptionPairToString/ETH-1JUN24-3200-P
=== CONT TestOptionPairToString/MATIC_USDC-6APR24-0d98-P
--- PASS: TestOptionPairToString (0.00s)
--- PASS: TestOptionPairToString/BTC-30MAY24-61000-C (0.00s)
--- PASS: TestOptionPairToString/ETH-1JUN24-3200-P (0.00s)
--- PASS: TestOptionPairToString/SOL_USDC-31MAY24-162-P (0.00s)
--- PASS: TestOptionPairToString/MATIC_USDC-6APR24-0d98-P (0.00s)
```
( And that got worse with me adding more tests )
* Config: Version Management
* Engine: Improve visibility of TestConfigAllJsonResponse failures
* Config: Update cmd/config to allow upgrades
* Config: Add Version2 to rename GDAX
* Config: Restructure versioning to share types
This restructure allows us to share types between versions, avoids
needing to import the versions, and puts the test fixtures in same
package.
It's a win on all fronts
* Config: Fix SetNTPCheck using log
Called from engine before logger is inited, and also just wrong to use
log to communicate with user
* Config: Improve TestMigrateConfig
* Config: Drop requirement for versions to be registered in sequence
Checking the versions at Deploy is much saner.
* Config: Fix file encrypted but flag not set
* Config: Add -edit and encryption upgrade to cmd/config
This simplifies the handling for encryption prompts by moving it to a
field on config, allowing us to simplify all the places were were
passing around config
Also moves password entry to being secure (echo-off)
* Tests: Fix inconsistent should/must assertions
* CI/build: Update Go version, linters and fix minor issues
* linters: Add intrange, copyloopvar, additional go vet linters to match gopls and fix issues
* Currency: Variadic Pairs.Add
This version of Pairs.Add is simpler and [more
performant](https://gist.github.com/gbjk/06a1fc1832d04ee41213ca518938cf74)
Behavioural difference: If there's nothing to add, the same slice is
returned unaltered. This seems like good sauce
* Currency: Variadic Remove
* Common: Add Batch function
* Common: Add common.SortStrings for stringers
* Subscriptions: Add batching to templates
* Subscriptions: Sort list of pairs
* Kucoin: Switch to sub templating
* Kucoin: Simplify channel prefix usage
* Kucoin: Fix race on fetchedFuturesOrderbook
* Subscriptions: Filter AssetPairs
Now only the assetPairs relevant to the subscription are in the context
* Subscriptions: Respect subscription Pairs
* Subscriptions: Trim AssetSeparator early
We want to trim before checking for "AssetSeparator vs All" because a
template should be allowed to reuse a range template and generate just one trailing AssetSeparator
whilst using a specific Asset
* Kucoin: Fix empty margin asset added
* Kucoin: Add Subscription batching
Turns out that contary to the documentation, kucoin supports batching of
all symbols and currencies
* Kucoin: Fix checkSubscriptions and coverage
* Subscriptions: Simplify error checking
This reduces the complexity of error checking to just be "do we get the
correct numbers".
Fixes Asset.All with only one asset erroring on xpandPairs, because we
trimmed the only asset separator, and then errored that we're not
xpanding Assets and the asset on the sub is asset.All
This use-case conflicted with commit 6bbd546d74, which required:
```
Subscriptions: Trim AssetSeparator early
We want to trim before checking for "AssetSeparator vs All" because a
template should be allowed to reuse a range template and generate just one trailing AssetSeparator
whilst using a specific Asset
```
Now we set up the assets earlier, and we remove the check for xpandAssets, since the number of asset lines matching is all that matters.
I've removed the asset tests for this, but they were correctly erroring
on the number of asset lines instead.
Everything hits coverage, as well.
* Kucoin: Remove deprecated fundingBook endpoint
* BTCMarkets: Use common.Batch
* Subscriptions: Add List.AssetPairs
* Subscriptions: Add Template and QualifiedChannel
These fields separate the concept of what the channel is from the
qualified resource name
* Subscriptions: Add List.SetStates()
* Subscriptions: Add List.QualifiedChannels
* Subscriptions: Rename testsubs.EqualLists
* Binance: Switch to ExpandTemplates
* Binance: Update ConfigTest format
* Subscriptions: Test Coverage improvements
* Subscriptions: Reenterant List.ExpandTemplates
* Subscriptions: Move templates from subscriptions to exchanges
* Binance: Inline subscription template and improvements
* Add in initial handling for quote/base currency deployment requirements
* include client order ID checking
* glorious: suggestions
* spell and fix
* linter/context/test
* rework tests and order side specific requirements
* linter
* mend panic at the disco
* mending more panics at the disco
* anudda fix brudda
* glorious: NITTTTTT BOOOOOMB
* leftover things and stuff
* whoops
* tie in gateio
* glorious: nit fixes life and everything.
* thrasher: nits
---------
Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
* initial consolidation of rate limiting code to request package to reduce bespoke code implementation
* continued
* finish abstraction
* lint
* exchanges: fix tests
* linter: fix
* poloniex: fix auth rate limit not being set
* ratelimiter: convert from token to weight
* glorious: nits addressed with fire
* linter: rip
* change func name set -> get
* fix test
* derbit: impl
---------
Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
* deribit implementation
* add ws impll
* cleanup
* Update deribit_wrapper.go
* Add missing endpoints
* Fix config file
* asset type update
* Update code structure
* Update authenticated private endpoints unit tests
* Updating websocket
* Updating websocket connection and subscription handling
* Finishing up adding subscription push data
* Adding websocket public endpoint
* Adding WS endpoints
* Adding websocket unit tests
* Minor clean-up
* Integrating websocket endpoints into the wrapper funcs
* Updating exchange documentations
* Fixing test issues
* Code cleaning-up
* fix test issues
* Updating validations and logic errors
* Updating wrapper issues
* fix test issues
* Slight test update
* Unit test and code structure update
* Update websocket tempos
* Slight update on code structure
* Minor update on unit tests
* Update depending on review comments
* Minor code fix and doc re-generating
* Update on Candlestick wrapper functions
* Minor updates
* minor unit test updates
* Minor updates on weboscket and unit tests
* minor linter fix
* codespell and rate limiter issues
* single linter issue fix
* adding rate limiter
* Add ratelimiter to websocket conn and overall code update
* fix websocket push data issue
* Implementing missing wrapper function
* Websocket fix
* Minor update on missing endpoint and other
* fixing websocket issues and cleaningup
* Minor tempo fix
* Minor linter issues
* unit test update
* Indexing error fix
* Websocket connection fix
* string formatting fix
* Small fix on unit tests
* fix minor json conversion issue
* websocket and documentation update
* websocket, wrapper and unit test updates
* Documentation and unit tests update
* Fix unit tests
* wrapper fix for new change
* Unit test fix
* timestamp conversion and unit tests update
* Minor instrument ID conversion fix
* instrument formats and unit test update
* formatting and unit test fix
* config update
* Updating websocket and adding the Spot support
* Add small unit test fix
* unit test and websocket handlers update
* Linter issues fix
* minor documentation and code update
* Minor fix
* added a wrapper func GetLatestFundingRates
* Types, wrapper update, and unit tests
* Minor config update
* fix wrapper unit tests
* Resolve all panic and wrapper test issues
* minor unit test fix
* fix issues and adding newly added endpoints
* updates and added remaining endpoints with unit tests
* Update unit tests using assert
* Added missing endpoints and unit tests
* Minor updates and clean-ups
* Resolve tradable pair fetching panic
* Mutex fix
* Added Options assets test and minor fixes
* subscription mothod updated
* Remove misadded code lines
* resolve websocket
* Updating tests, types, endpoint methods and others
* Added GetFuturesContractDetails and minor fix
* fix linter issue
* revert change on candlestic time
* Added filters to candles
* minor unit test and wrapper fix
* Minor unit tests update
* cahnge param key for GetOrderMarginByID
* updating unit tests and resolve issues
* Update websocket unit tests
* Minor fix based on review
* Revert unit test change
* fix pair config issue
* Added missing wrapper functions
* Fix missing review changes
* Fix options request pair formatting
* fix AllExchangeWrappers test issue
* Changes with unit test and wrapper based on the review
* Fix to options reg-exp
* wrapper functions fix
* Update MaximumFundingRateHistory filter and minor fixes
* Fix besed on review comment
* Fix issues on review comment
* linter fix
* fix minor unit test issue
* Fix unit test issues
* Update trade order cancellation responses
* fix config files issue
* lint update config files
* Update unit tests
* Update return values and response handling
* added missing endpoint and fixes based on review comment
* toggle useTestNet back
* Update cancel by label and other fix
* fix forgotten cancel all response type
* update CancelResp type
* Fix unmarshaling error
* updated websocket orderbook load issue
* fix websocket lock and groups
* Change Items to Tranche and fix linter issues
* Fix orderbook issue
* Update unit tests offline error handling, and endpoints argument and error handling
* Contributors documentation update and change error return type
* Updated unit tests based on review comment
* Update unit tests and removed password change endpoint
* Fix race condition
* Update on tests, test pairs, and wrapper config
* Update test tradable pairs loading
* Update unit tests, fix linter issues, and update wrapper functions
* remove credentials
* Update test and fix authentication method and few authenticated endpoints
* fix codespell issue
* group the repeated currency code check to a func
* added unit test for repeated pair check func
* Added a base coin and related updates
---------
Co-authored-by: E Sequeira <earncef@earncef.com>
* initial purge and benchmarks proof before rn overhaul
* rn LinkedList -> Tranche(s) and purge references
* roll out acrost exchanges
* linterino
* rn silly billy label
* linter strikes AAAAAGAIN!
* fix some things
* rm comment
* Add actual comparison from master to branch benchmark for sorting algorithms
* lower case via git mv YAAY!
* drop code
* convert type name
* glorious: nits
---------
Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
* Shift wrapper function GetDefaultConfig to exchange.Base method definition, to ensure set defaults doesn't get called twice and to reduce code
* rm alphapoint bootstrap method as is defined as exchange.Base method
* add tests
* glorious: make it a function and make it IBOTEXCHANGE
---------
Co-authored-by: shazbert <ryan.oharareid@thrasher.io>
* gateio: fix unmarshal bug and update fields
* gateio: fix wrapper function function, add helper methods
* update order types and add kucoin wrapper fix
* currency pairs
* Add tests
* gateio; inspect error and continue for no funds in account, kucoin: fetch all settlement amounts
* futures: order fixit
* finish off gateio updates for market orders
* cute line
* Update exchanges/kucoin/kucoin_wrapper.go
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
* Update exchanges/kucoin/kucoin_wrapper.go
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
* Update exchanges/gateio/gateio.go
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
* Update exchanges/gateio/gateio_wrapper.go
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
* Update exchanges/gateio/gateio_wrapper.go
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
* glorious: nits
* glorious: nits - filter by pair match and fix bug where the endpoint returns details instead of message
* Add fix for leverage check (non-merge) my ip has been blocked from gateio still... scammmmmmmm
* glorious: nitters
* Update exchanges/gateio/gateio_test.go
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
* Update exchanges/gateio/gateio_test.go
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
* Update exchanges/gateio/gateio_test.go
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
---------
Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
* Websocket: Remove IsInit and simplify SetProxyAddress
IsInit was basically the same as IsConnected.
Any time Connect was called both would be set to true.
Any time we had a disconnect they'd both be set to false
Shutdown() incorrectly didn't setInit(false)
SetProxyAddress simplified to only reconnect a connected Websocket.
Any other state means it hasn't been Connected, or it's about to
reconnect anyway.
There's no handling for IsConnecting previously, either, so I've wrapped
that behind the main mutex.
* Websocket: Expand and Assertify tests
* Websocket: Simplify state transistions
* Websocket: Simplify Connecting/Connected state
* Websocket: Tests and errors for websocket
* Websocket: Make WebsocketNotEnabled a real error
This allows for testing and avoids the repetition.
If each returned error is a error.New() you can never use errors.Is()
* Websocket: Add more testable errors
* Websocket: Improve GenerateMessageID test
Testing just the last id doesn't feel very robust
* Websocket: Protect Setup() from races
* Websocket: Use atomics instead of mutex
This was spurred by looking at the setState call in trafficMonitor and
the effect on blocking and efficiency.
With the new atomic types in Go 1.19, and the small types in use here,
atomics should be safe for our usage. bools should be truly atomic,
and uint32 is atomic when the accepted value range is less than one byte/uint8 since
that can be written atomicly by concurrent processors.
Maybe that's not even a factor any more, however we don't even have to worry enough to check.
* Websocket: Fix and simplify traffic monitor
trafficMonitor had a check throttle at the end of the for loop to stop it just gobbling the (blocking) trafficAlert channel non-stop.
That makes sense, except that nothing is sent to the trafficAlert channel if there's no listener.
So that means that it's out by one second on the trafficAlert, because any traffic received during the pause is doesn't try to send a traffic alert.
The unstopped timer is deliberately leaked for later GC when shutdown.
It won't delay/block anything, and it's a trivial memory leak during an infrequent event.
Deliberately Choosing to recreate the timer each time instead of using Stop, drain and reset
* Websocket: Split traficMonitor test on behaviours
* Websocket: Remove trafficMonitor connected status
trafficMonitor does not need to set the connection to be connected.
Connect() does that. Anything after that should result in a full
shutdown and restart. It can't and shouldn't become connected
unexpectedly, and this is most likely a race anyway.
Also dropped trafficCheckInterval to 100ms to mitigate races of traffic
alerts being buffered for too long.
* Websocket: Set disconnected earlier in Shutdown
This caused a possible race where state is still connected, but we start
to trigger interested actors via ShutdownC and Wait.
They may check state and then call Shutdown again, such as
trafficMonitor
* Websocket: Wait 5s for slow tests to pass traffic draining
Keep getting failures upstream on test rigs.
Think they can be very contended, so this pushes the boundary right out
to 5s
* CI/build: Update Go version, linters and fix minor issues
* Bump golangci-lint to v1.56.1
* BinanceUS: Make uint usage consistent
* Throw blank identifiers into the trash
* Exchanges: Remove Pair upgrade handling
Now redundant behind #1401. These paths should never be met.
Several legacy coin upgrade paths being deprecated as well: ZUSD and CNY
Expecting any users with bad config from 3+ years ago would have to
reset anyway.
Also: At the time the intention of this was to upgrade the config
format.
However now, instead, it'd mostly serve to reset enabled pairs if
there's a config mistake, which doesn't feel right.
* Kraken: Fix typo in Kraken type struct
* Exchanges: Abstract exchange Start() and Run()
* Exchanges: Add test for abstracted Start
* Exchanges: Move Start to Bootstrap
* Simplify waitgroup usage
* Add call to exchange.Bootstrap to allow overide or supplementation
* Exchanges: Concurrent common bootstap actions
* Gateio: Remove incorrect Run in test
* GateIO: Fix pair dependencies in tests
This ensures that the pairs are initialised no more than needed and
kind-of just-in-time.
Better pattern might be to use a function to get these pairs when we
need them.
* Exchanges: Complete UpdatePairs before ExecLims
If we're going to update pairs, it needs to complete before we check for
limits to avoid errors on old pairs
* Exchanges: Remove Start and Run from tmpl
Since they're replaced by bootstrap now and shouldn't need customisation
normally
* Alphapoint: Move Start to Bootstrap
* GateIO: Fix linter shadow var
* Websockets: Move Subscription to its own package
This allows the small type to be imported from both `config` and from
`stream` without an import cycle, so we don't have to repeat ourselves
* Subs: Renamed Currency to Pair
This was being mis-used through much of the code, and since we're
already touching everything, we might as well fix it
* Websockets: Add Subscription configuration
* Binance: Add subscription configuration
* Kucoin: Subscription configuration
* Simplify GenerateDefaultSubs
* Improve TestGenSubs coverage
* Test Candle Sub generation
* Support Candle intervals
* Full responsibility for formatting Channel name on GenerateDefaultSubs
OR consumer of Subscribe
* Simplify generatePayloads as a result
* Fix test coverage of asset types in processMarketSnapshot
* Exchanges: Abstract ParallelChanOp
* Tests: Generic ws mock instances
* Kucoin: Fix intermittent conflict in test currs
Use isolated test instance for `TestGetOpenInterest`.
`TestGetOpenInterest` would occassionally change pairs before
GenerateDefault Subs.
* adds open interest to exchanges
* ADDS TESTING YEAH
* New endpoints, BTSE, RPCS, cached
* slight design change, begin gateio
You will need to get cached for
each exchange that supports it
* gateio, huobi, rpc
* fix up kraken, cache retrieval
* okx, gateio
* finalising all implementations and tests
* definitely my final ever commit on this
* Well, well, well
* final v2
* quick fix of bug
* test coverage, assert notempty, test helper
Added a new testhelper for currency
management because its very annoying
in a parallel test setting which wastes
so much space otherwise
* minimises REST requests for Open Interest
* types.Number merge misses
* Minimises Kraken REST calls
* len change, value -> pointer receiver
* further fixup
* fixes gateio, batch calculates open interest
* single gateio, lint const fixes
* rejig and more thorough oi for huobi
* formatting expansion
* minor fix for handling expiring contracts
* rm unused Binance strings
* add bybit support, fix bybit issues
* oopsie doopsie, dont look at my whoopsie
* Fix issue, remove feature
* move an irrelevant function for the pr
* mini bybit upgrades
* fixes cli request bug
* CI: Add macOS, Windows and Linux arm64 support
* Modify arm64 job name to be inline with the others
* linter: Prevent cache from causing "Cannot open: File exists" issues
* Use setup-go and setup-node's inbuilt caching
