* Exchanges: Remove Pair upgrade handling
Now redundant behind #1401. These paths should never be met.
Several legacy coin upgrade paths being deprecated as well: ZUSD and CNY
Expecting any users with bad config from 3+ years ago would have to
reset anyway.
Also: At the time the intention of this was to upgrade the config
format.
However now, instead, it'd mostly serve to reset enabled pairs if
there's a config mistake, which doesn't feel right.
* Kraken: Fix typo in Kraken type struct
* Exchanges: Abstract exchange Start() and Run()
* Exchanges: Add test for abstracted Start
* Exchanges: Move Start to Bootstrap
* Simplify waitgroup usage
* Add call to exchange.Bootstrap to allow overide or supplementation
* Exchanges: Concurrent common bootstap actions
* Gateio: Remove incorrect Run in test
* GateIO: Fix pair dependencies in tests
This ensures that the pairs are initialised no more than needed and
kind-of just-in-time.
Better pattern might be to use a function to get these pairs when we
need them.
* Exchanges: Complete UpdatePairs before ExecLims
If we're going to update pairs, it needs to complete before we check for
limits to avoid errors on old pairs
* Exchanges: Remove Start and Run from tmpl
Since they're replaced by bootstrap now and shouldn't need customisation
normally
* Alphapoint: Move Start to Bootstrap
* GateIO: Fix linter shadow var
* Websockets: Fix subscription failure on reconnect
If a websocket is disconnected then the subscription map was left with
old subscriptions, causing:
`Okx websocket: subscription failure, channel already subscribed for ...`
* fixup! Websockets: Fix subscription failure on reconnect
Fixes review comment https://github.com/thrasher-corp/gocryptotrader/pull/1457#discussion_r1468947509
* Bitfinex: Handle Errors in tickers without panic
* Bitfinex: Use TypeAssertError for tickers
* Bitfinex: Refactor and improve ticker handling
* Unify Ticker response handling
* Simplify/Humanify errors in parsing ticker responses
* Remove polymorphic response handling for < 10 fields, seems [antiquated according to docs](https://docs.bitfinex.com/reference/rest-public-ticker)
* Add test coverage for tickers
* Bitfinex: Ignore resp format errs in GetTickers
We're still getting:
`received 'invalid ticker response format for tALT2612:USD field BidSize
from [100 <nil> 100 <nil> <nil> <nil> <nil> <nil> <nil> <nil>]`
too frequently right now.
Considered:
* Special casing tALT2612:*; However if it's happening with this curr
it'll probably happen again
* Warning about the error; However it'd just be persistent,
unactionable and annoying noise in the logs of a running server
So the conclusion is to just silently ignore it
* Bitfinex: Accept locked market in TestUpdateTicker
* Websockets: Move Subscription to its own package
This allows the small type to be imported from both `config` and from
`stream` without an import cycle, so we don't have to repeat ourselves
* Subs: Renamed Currency to Pair
This was being mis-used through much of the code, and since we're
already touching everything, we might as well fix it
* Websockets: Add Subscription configuration
* Binance: Add subscription configuration
* Kucoin: Subscription configuration
* Simplify GenerateDefaultSubs
* Improve TestGenSubs coverage
* Test Candle Sub generation
* Support Candle intervals
* Full responsibility for formatting Channel name on GenerateDefaultSubs
OR consumer of Subscribe
* Simplify generatePayloads as a result
* Fix test coverage of asset types in processMarketSnapshot
* Exchanges: Abstract ParallelChanOp
* Tests: Generic ws mock instances
* Kucoin: Fix intermittent conflict in test currs
Use isolated test instance for `TestGetOpenInterest`.
`TestGetOpenInterest` would occassionally change pairs before
GenerateDefault Subs.
* Dispatch: Assertify tests
* Dispatch: Fix race during stop
If we have blocking writers, then we need to synchronise them exiting
before closing off their channels.
* Dispatch: Rename Routes mutex for clarity
* adds open interest to exchanges
* ADDS TESTING YEAH
* New endpoints, BTSE, RPCS, cached
* slight design change, begin gateio
You will need to get cached for
each exchange that supports it
* gateio, huobi, rpc
* fix up kraken, cache retrieval
* okx, gateio
* finalising all implementations and tests
* definitely my final ever commit on this
* Well, well, well
* final v2
* quick fix of bug
* test coverage, assert notempty, test helper
Added a new testhelper for currency
management because its very annoying
in a parallel test setting which wastes
so much space otherwise
* minimises REST requests for Open Interest
* types.Number merge misses
* Minimises Kraken REST calls
* len change, value -> pointer receiver
* further fixup
* fixes gateio, batch calculates open interest
* single gateio, lint const fixes
* rejig and more thorough oi for huobi
* formatting expansion
* minor fix for handling expiring contracts
* rm unused Binance strings
* add bybit support, fix bybit issues
* oopsie doopsie, dont look at my whoopsie
* Fix issue, remove feature
* move an irrelevant function for the pr
* mini bybit upgrades
* fixes cli request bug
This test was failing with `-tags mock_test_off` because the API imposes
a default limit of 500 now, so we'd just get 500 trades and no batching.
If the user doesn't provide a limit, that means we need to assume
batching
* Adding Bybit public endpoints
* Completed adding market endpoints
* Added trade endpoints
* Adding position endpoints
* completing position endpoints
* Adding Pre-upgrade endpoints
* Completed adding Pre-upgrade and Account endpoints
* Added asset endpoints
* Added user endpoints and unit tests
* Adding spot leverage and margin trade endpoints
* spot margin trade added
* added spot-margin-trade, institutional lending, c2c lending, and broker
* Adding wrapper funnctions
* Working on wrapper public methods
* Added wrapper functions and unit tests
* Added websocket support with unit tests
* Update websocket handlers and added rate-limiter
* wrapper function, websocket handlers, and linter issues fixe
* unit tests fixes and codespell correction
* Update documentation
* Minor websocket handling fix and URL consts merging
* types, unit test other updates
* Updated websocket and methods based on review
* Added GetFeeByType method with unit test and fixes
* add filter for Unified and Normal endpoints
* Mock recording and unit tests update
* minor linter issue fix
* websocket and rest tests and fix
* change asset types and wrapper methods update
* rm: forgotten panic message
* endpoints, websocket and unit tests update
* Added and updated endpoints and unit test
* linter and spell fix
* unit test and orders update
* Update on endpoints, fields, config pairs and formating, and unit tests
* minor update on responses
* Fix unit test and types
* Unit tests, models, and wrapper issues fix and mock test recording
* rm print statement
* Fix issue, add FundingRate wrapper func, mock record
* minor type and unit test update
* Update on order handling and unit test
* Minor test
* Minor fix in wrapper
* unit tests update, recording, and documentation update
* Unit tests and minor wrapper function update
* minor unit test fix
* Added newly added endpoints, unit tests, and mock recording
* Rename GetInstruments -> GetInstrumentInfo
* doc update
* Minor unit tests update
* Minor unit test and wrapper update
* Fix linter issue
* Add unit test and minor updates
* Revert websocket error declaration
* Revert websocket error declaration
* Balace --> Balance
* Fix config issues
* Added next funding time minor fix
* Update GetLatestFundingRates and record mock test data
* Added LatestFundingRate time
* Fix test issue of TestAllExchangeWrappers
* config pairs update
* configtest spot pairs update
* Minor update on options UpdateOrderExecutionLimits wrapper func
* Linter issue fix and added new currency codes
* Added new currency codes
* Update bybit pairs in config_example
* config assets pair format update
* Bitfinex: Fix TestUpdateTickers on ICE
Accept 5% failure rate of available pairs not working as tickers.
When Bitfinex [delisted ICE](https://www.bitfinex.com/posts/990) it's still coming back in all pub:conf and pub:info listings:
```
❯ curl -s https://api.bitfinex.com/v2/conf/pub:info:pair | jq -c '.[0][] | select(.[0] | test("(BTC|ICE|ZIL)USD")) ' ["BTCUSD",[null,null,null,"0.00006","2000.0",null,null,null,0.1,0.05,null,null]]
["ICEUSD",[null,null,null,"4.0","25000.0",null,null,null,null,null,null,null]]
["ZILUSD",[null,null,null,"40.0","1500000.0",null,null,null,null,null,null,null]]
```
_( I included ZIL to show a tradable pair without Margin fields )_
There's absolutely no sign it's not a tradable pair _until_ you ask for a ticker for it:
```
❯ curl -s https://api.bitfinex.com/v2/ticker/tICEUSD ["error",10020,"symbol: invalid"]⏎
❯ curl -s https://api.bitfinex.com/v2/ticker/tBTCUSD [42854,11.8920918,42855,12.71095442,-290,-0.00672292,42846,725.08132142,43288,41850]⏎
```
* fixup! Bitfinex: Fix TestUpdateTickers on ICE
* fixup! Bitfinex: Fix TestUpdateTickers on ICE
* Bitfinex: Fix UpdateTickers stopping on first error
* CI: Add macOS, Windows and Linux arm64 support
* Modify arm64 job name to be inline with the others
* linter: Prevent cache from causing "Cannot open: File exists" issues
* Use setup-go and setup-node's inbuilt caching
* Dispatch: Assertify TestMuxPublish
* Dispatch: Fix errDispatcherJobsAtLimit test fails
This test would fail intermittently when the jobs queue drained quickly
enough.
This sets the overload ceiling based on the default settings, and seems
a safe way of ensuring we get an error every time.
It adds a done channel guard around the goro test because otherwise
we'll get a panic occassionally when the goro outlives TestMaxPublish
* Dispatch: Add test for Publish receiving data
* Dispatch: Publish to all subscribers
* Types: Add Number type
* Types: Switch StringToFloat64 for Number
This change mostly just renames the type.
convert package and StringToFloat64 represent actions, not types,
and make it misleading to use outside of the API context,
especially when using it for a Float64ToString operation.
* Common: Remove StringToFloat64
Replaced by types.Number
* fixup! Types: Switch StringToFloat64 for Number
Second pass at Okx
* Spellcheck: Fix whitespace handling for okx line
* Orderbook: Fix test failures on different arch
Fixes floating point inaccuracies on different architectures
Moves the depth tests over to table driven tests
* Kline: Fix test failures on different arch
* BTSE: Fix duplicate error on Million pairs (M_*)
BTSE has listed Pitbull token with two symbols:
PIT-USD and M_PIT-USD for millons of PIT / USD.
The native token is not tradable, so we ignore them and
get a base of M_PIT because that's what later APIs will accept
* BTSE: Fix test errors on locked market
* Common: Improve AppendError and ExcludeError
This change switches from a stateful multiError to caring more about the
Unwrap() []error interface, the same as [go standard
lib](https://github.com/golang/go/blob/go1.21.4/src/errors/wrap.go#L54-L68)
Notably, if we implement Unwrap() []error and do NOT implement Is() then
we get free compatibility with the core functions.
The only distateful thing here is needing to deeply unwrap fmt.Errorf
errors, since they don't flatten. I can't see any way around that
* Pairs: Fix exchange config Pairs loading
When a pair string contained two punctuation runes, the first one is used,
and the configFormat is ignored.
This fix checks the list and corrects any with the wrong delimiter, or
errors if the format is inconsistent.
* BTSE: Fix all tickers retrieved by GetTicker
PR #764 introduced GetTickers, but it wasn't rolled out to BTSE.
This fix ensures that when one ticker is a locked market, the rest continue to
function. Particularly important if the locked market wasn't even
enabled anyway.
* Kucoin: Fix test config future pairs
* BTSE: Remove PIT tests; Token removed
BTSE have removed the PIT token pairs
All these changes stand, and this just removes the test
* ITBit: Fix fatal error on second run
This fix removes incorrect config pair delimiter, because it would be
re-inserted into config the first run, and then error the second time.
This delimiter doesn't match the config we have.
There's no implementation of fetching pairs, so what's in config files
now is all that matters
* Engine: Fix TestConfigAllJsonResponse
* Clarity of non-matching json improved
* Handling for fixing pair delimiters
* Okx: Remove UpdateTradablePairs from TestMain
When running individual tests we do not want to be forced to update
trading pairs.
It also does not seem to be required, since disabling UpdateTradingPairs
entirely doesn't stop any tests from passing.
If it is required we can enact this pattern:
```
func TestUpdateTradablePairs(t *testing.T) {
t.Parallel()
updatePairsOnce(t)
}
var updatePairsGuard sync.Once
func updatePairsOnce(tb testing.TB) {
tb.Helper()
updatePairsGuard.Do(func() {
err := ok.UpdateTradablePairs(context.Background(), true)
assert.NoError(tb, err, "UpdateTradablePairs should not error")
})
```
* Okx: Fix GetBlockTrades url
Endpoint was wrong
Not convinced we should prefix with market vs public, but not
committing a driveby
* Okx: Expand test coverage for GetBlockTrades
* Okx: Fix GetPublicBlockTrades
Was marked as an authenticated API call, and it's not.
Also renames it because without the context of "Block" (or RFQ) it's
misleading.
Adds tests.
* Okx: Fix GetPublicBlockTrades to be public
Expand coverage of GetBlockTrade and GetPublicBlockTrades
* Kucoin: Fix unmarshalling of FuturesTicker resp
* Fix several string float64 fields not unmarshalling
* Fix time field misnamed; Make a time.Time
* Use order.Side for sides
* Add tests
This change opts to expose float64 instead of StringToFloat64 because
there's no wrapper to convert the type over, and it's not a websocket.
Essentially choosing to not expose internal convert types for methods
called directly.
Possible we'll eventually harmonise this with a types.Number or
something.
* Kucoin: FuturesRealTimeTicker => FuturesTicker
Kucoin only call it RealTime in the first sentence of their docs. None of the API names, endpoints or any thing mention RealTime.
This is just a ticker
* Tests: Improve clarity of failed push data
* Kucoin: Move PushData to a fixture file
This avoids both the linter errors, and the bloat.
Loses the name of the "test", but I think that's okay
* Okx: Fix GetInsuranceFundInformation "" resp
When amount is "" this was erroring
* Test: Standardise configtest.json
Simple jq formatting of configtest
git diff -w or github ignore whitespaces should show no changes.
Any conflicts, just accept --ours and reformat it. Or not :o)
* Currency: Do not use a default forex provider
exchangerate.host now requires an API key.
Instead of finding a new Free (for now) default, this change simply
disables the currency exchange when nothing is enabled.
* SyncManager: Report ?.?? for an unknown forex amount
In a situation where we thought forex was available but we got an error,
this avoids showing 0.00 when there was actually an error.
* Currency: Tests for no default forex
* Currency: Use mock provider for tests
* Currency: Add API key to exchangerate.host
* Currency: Remove Exchangerate.host
Exchangerate.host was bought by apilayer, the old API deprecated, and
replaced with a proxy to the apilayer api.
We already have currencylayer support, so ther's no reason to keep exh.
Worth noting: New ERH keys actually work on currencylayer
* Currencies: Add test coverage for currency layer
* fixup! Currency: Tests for no default forex
Remove duplicate assignment
Fixes [review comment](https://github.com/thrasher-corp/gocryptotrader/pull/1395#discussion_r1395178513)
* fixup! Currency: Add API key to exchangerate.host
Remove unused ErrVar
Fixes [review
comment](https://github.com/thrasher-corp/gocryptotrader/pull/1395#discussion_r1396647418)
* fixup! Currency: Tests for no default forex
Fix spelling of override in test
Fixes [review comment](https://github.com/thrasher-corp/gocryptotrader/pull/1395#discussion_r1396701476)
* fixup! SyncManager: Report ?.?? for an unknown forex amount
Fix display of non-positive currency conversions.
Fixes [review comment](https://github.com/thrasher-corp/gocryptotrader/pull/1395/files#r1398527134)
