Commit Graph

200 Commits

Author SHA1 Message Date
Ryan O'Hara-Reid
e99adca86f encoding/json: Add custom JSON package with build tag support for Sonic (#1623)
* tag optional sonic and allow full library conversion

* Add workflow and disallow arm and darwin usage

* Add basic hotswap benchmark

* linter: fix

* use bash

* linter: fix?

* Fix whoopsie, add to make file, also add mention in features list.

* test enforcement

* actually read documentation see if this works

* linter: fix

* linter: fix

* sonic: bump tagged version

* encoding/json: drop build tag arch and os filters

* encoding/json: consolidate tests

* encoding/json: log build tag usage

* rm superfluous builds

* glorious/nits: add template change and regen docs

* glorious/nits: update commentary on nolint directive

* glorious/nits: rm init func and log results in main.go

* Test to actually pull flag in

* linter: fix

* thrasher: nits

* gk: nits 4 goflags goooooooooo!

* gk: nits rn

* make sonic default json implementation

* screen 386

* linter: fix

* Add commentary

* glorious: nits Makefile not working

* gk: nits

* gk: nits whoops

* whoopsirino

* mention 32bit systems won't be sonic

* gk: super-duper nit of extremes

---------

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
2025-02-20 16:05:55 +11:00
Gareth Kirwan
4c7f48ae0e GateIO: Fix GetFuturesContractDetails for delivery futures and minor other fixes (#1766)
* GateIO: Fix GetFuturesContractDetails for Deliveries

Was returning the product of all the contracts, so 1444 instead of 38
contracts.

* GateIO: Fix GetOpenInterest returning asset.ErrNotEnabled

Using wrong error for pair not enabled

* GateIO: Rename GetSingleContract and GetSingleDeliveryContracts

Especially fixes GetSingleContract, which seems misleading to not say
Futures.
There's a load of `GetSingle*` here that should probably also be fixed,
but these two justified a dyno

* GateIO: Rename GateIOGetPersonalTradingHistory to GetMySpotTradingHistory

* GateIO: Rename GetMyPersonalTradingHistory to GetMyFuturesTradingHistory

* GateIO: Remove duplicate DeliveryTradingHistory

* GateIO: Rename Get*PersonalTradingHistory to GetMy*TradingHistory

* Linter: Disable shadow linting for err

It's been a year, and I'm still getting caught out by govet demanding I
don't shadow a var I was deliberately shadowing.
Made worse by an increase in clashes with stylecheck when they both want
opposite things on the same line.

* GateIO: Add missing Futures and tradinghistory fields

* GateIO: Improve WS Header parsing

This unifies handling for time_ms and time in response headers, since
options and delivery have only time, but spot has time_ms as well.
We use the better of the two results.

Also [improves performance 2x](https://gist.github.com/gbjk/7cacb63b9a256e745534bb05ca853c48)

* GateIO: Use time_ms WS fields where available

Removes the deprecated _time json fields and populates our Time fields
with the time_ms values
2025-01-14 15:19:17 +11:00
Ryan O'Hara-Reid
4bbaf75d34 currency: Fix edge case in NewCode where all string characters are digits (#1593)
* add fixes and test

* glorious: nits, privatise upper case field, add item field for case sensitivity for format checks, rm dead code.

* fix potential panic

* gk/glorious: nits

* gk: nits and other things

* improve commentary lol

* glorious+gk: nits and improvements (with no sillyness this time)

* Update currency/pairs.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* thrasher: nitssssss

* linter: fix

* bye bye silly boy

---------

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
2024-11-25 14:51:35 +11:00
Adrian Gallagher
69bb645fc2 docs: Migrate development tracking from Trello to GitHub Projects (#1696)
* docs: Migrate development tracking from Trello to GitHub Projects

* docs: Replace non-template based references and correct templates
2024-11-08 10:10:49 +11:00
Gareth Kirwan
33e82c170f Kraken: Subscription improvements (#1587)
* Convert: Fix TimeFromUnixTimestampDecimal using local

All parsed times should be in UTC

* Subscriptions: Add IgnoringAssetsKey

* Tests: Pass tb to curried WS handlers

* Websocket: Make ErrNoMessageListener a public error

* Kraken: Fix URLMap ignored for websocket URLs

* Kraken: Move SeedAssets from Setup to Bootstrap

Having SeedAssets in Setup is cruel and unusual because it calls the
API. Most other interactive data seeding happens in Bootstrap.

This made it so that fixing and creating unit tests for Kraken was
painfully slow, particularly on flaky internet.

* Kraken: Remove convert test

Duplicate of convert_test.go TestTimeFromUnixTimestampDecimal

* Kraken: Test config upgrades

* Kraken: Sub Channel improvements

* Use Websocket subscriptionChannels instead of local slice
* Remove ChannelID - Deprecated in docs
* Simplify ping handlers and hardcodes message
* Add Depth as configurable orderbook channel param
* Simplify auth/non-auth channel updates
* Add configurable Book depth
* Add configurable Candle timeframes

Kraken: Simplify all WS handlers with reqId

* Kraken: Subscription templating

* Generate N+ subs for pairs
If we generate one sub for all pairs, but then fan it out in the
responses, we end up with a mis-match between the sub store and
GenerateSubs, and when we do FlushChannels it will try to resub
everything again.

* Kraken: Rename channelName var throughout

Avoid shadowing func of same name

* Kraken: Add TestEnforceStandardChannelNames

* Websocket: Fix Resubscribe erroring Duplicate
2024-10-08 10:34:10 +11:00
Adrian Gallagher
b8e836d74f common: Replace StringDataCompare with slices.Contains and cleanup string funcs (#1631)
* common: Replace StringDataCompare with slices.Contains and cleanup string funcs

* common/docs: Update SliceDifference and remove outdated steps from ADD_NEW_EXCHANGE.md

* common: Improve SliceDifference
2024-09-13 10:43:20 +10:00
Adrian Gallagher
225429bda6 CI/build: Update Go version, linters and fix minor issues (#1612)
* CI/build: Update Go version, linters and fix minor issues

* linters: Add intrange, copyloopvar, additional go vet linters to match gopls and fix issues
2024-08-16 17:41:11 +10:00
Ryan O'Hara-Reid
facf291069 currency/exchanges: Add bespoke exchange translator and pair matching helper (#1556)
* currency: translation and matching pairs

* Update exchanges/exchange_types.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* linter: fix?

* translation

* fix cherry pick

* gateio: translation for mbabydoge with 1e6 divisor

* okx: add translation

* cherry-pick: fix

* glorious: todos

* thrasher: nits

---------

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2024-08-16 16:47:17 +10:00
Gareth Kirwan
b41fe27684 Kucoin: Add subscription templating and various fixes (#1579)
* Currency: Variadic Pairs.Add

This version of Pairs.Add is simpler and [more
performant](https://gist.github.com/gbjk/06a1fc1832d04ee41213ca518938cf74)

Behavioural difference: If there's nothing to add, the same slice is
returned unaltered. This seems like good sauce

* Currency: Variadic Remove

* Common: Add Batch function

* Common: Add common.SortStrings for stringers

* Subscriptions: Add batching to templates

* Subscriptions: Sort list of pairs

* Kucoin: Switch to sub templating

* Kucoin: Simplify channel prefix usage

* Kucoin: Fix race on fetchedFuturesOrderbook

* Subscriptions: Filter AssetPairs

Now only the assetPairs relevant to the subscription are in the context

* Subscriptions: Respect subscription Pairs

* Subscriptions: Trim AssetSeparator early

We want to trim before checking for "AssetSeparator vs All" because a
template should be allowed to reuse a range template and generate just one trailing AssetSeparator
whilst using a specific Asset

* Kucoin: Fix empty margin asset added

* Kucoin: Add Subscription batching

Turns out that contary to the documentation, kucoin supports batching of
all symbols and currencies

* Kucoin: Fix checkSubscriptions and coverage

* Subscriptions: Simplify error checking

This reduces the complexity of error checking to just be "do we get the
correct numbers".

Fixes Asset.All with only one asset erroring on xpandPairs, because we
trimmed the only asset separator, and then errored that we're not
xpanding Assets and the asset on the sub is asset.All

This use-case conflicted with commit 6bbd546d74, which required:
```
Subscriptions: Trim AssetSeparator early

We want to trim before checking for "AssetSeparator vs All" because a
template should be allowed to reuse a range template and generate just one trailing AssetSeparator
whilst using a specific Asset
```

Now we set up the assets earlier, and we remove the check for xpandAssets, since the number of asset lines matching is all that matters.

I've removed the asset tests for this, but they were correctly erroring
on the number of asset lines instead.

Everything hits coverage, as well.

* Kucoin: Remove deprecated fundingBook endpoint

* BTCMarkets: Use common.Batch
2024-08-09 12:33:15 +10:00
cranktakular
90fee94c76 currency: Make pairs.Add variadic (#1566)
* Making Pairs(Add) Variatic

* Slight improvements

* Implementation overhaul

* Improvements

* Changing code which can use the variadic functionality

* Fixing silliness

* thing left over
2024-07-12 14:36:05 +10:00
Gareth Kirwan
1199f38546 subscriptions: Encapsulate, replace Pair with Pairs and refactor; improve exchange support
* Websocket: Use ErrSubscribedAlready

instead of errChannelAlreadySubscribed

* Subscriptions: Replace Pair with Pairs

Given that some subscriptions have multiple pairs, support that as the
standard.

* Docs: Update subscriptions in add new exch

* RPC: Update Subscription Pairs

* Linter: Disable testifylint.Len

We deliberately use Equal over Len to avoid spamming the contents of large Slices

* Websocket: Add suffix to state consts

* Binance: Subscription Pairs support

* Bitfinex: Subscription Pairs support

* Bithumb: Subscription Pairs support

* Bitmex: Subscription Pairs support

* Bitstamp: Subscription Pairs support

* BTCMarkets: Subscription Pairs support

* BTSE: Subscription Pairs support

* Coinbase: Subscription Pairs support

* Coinut: Subscription Pairs support

* GateIO: Subscription Pairs support

* Gemini: Subscription Pairs support and improvement

* Hitbtc: Subscription Pairs support

* Huboi: Subscription Pairs support

* Kucoin: Subscription Pairs support

* Okcoin: Subscription Pairs support

* Poloniex: Subscription Pairs support

* Kraken: Add subscription Pairs support

Note: This is a naieve implementation because we want to rebase the
kraken websocket rewrite on top of this

* Bybit: Subscription Pairs support

* Okx: Subscription Pairs support

* Bitmex: Subsription configuration

* Fixes unauthenticated websocket left as CanUseAuth
* Fixes auth subs happening privately

* CoinbasePro: Subscription Configuration

* Consolidate ProductIDs when all subscriptions are for the same list

* Websocket: Log actual sent message when Verbose

* Subscriptions: Improve clarity of which key is which in Match

* Subscriptions: Lint fix for HugeParam

* Subscriptions: Add AddPairs and move keys from test

* Subscriptions: Simplify subscription keys and add key types

* Subscriptions: Add List.GroupPairs Rename sub.AddPairs

* Subscription: Fix ExactKey not matching 0 pairs

* Subscriptions: Remove unused IdentityKey and HasPairKey

* Subscriptions: Fix GetKey test

* Subscriptions: Test coverage improvements

* Websocket: Change State on Add/Remove

* Subscriptions: Improve error context

* Subscriptions: Fix Enable: false subs not ignored

* Bitfinex: Fix WsAuth test failing on DataHandler

DataHandler is eaten by dataMonitor now, so we need to use ToRoutine

* Deribit: Subscription Pairs support

* Websocket: Accept nil lists for checkSubscriptions

If the user passes in a nil (implicitly empty) list, we would not panic.
Therefore the burden of correctness about that data lies with them.
The list of subscriptions is empty, and that's okay, and possibly
convenient

* Websocket: Add context to NilPointer errors

* Subscriptions: Add context to nil errors

* Exchange: Fix error expectations in UnsubToWSChans
2024-06-07 11:54:08 +10:00
Ryan O'Hara-Reid
f6a95da536 exchanges/request: abstract and consolidate rate limiting code to request package (#1477)
* initial consolidation of rate limiting code to request package to reduce bespoke code implementation

* continued

* finish abstraction

* lint

* exchanges: fix tests

* linter: fix

* poloniex: fix auth rate limit not being set

* ratelimiter: convert from token to weight

* glorious: nits addressed with fire

* linter: rip

* change func name set -> get

* fix test

* derbit: impl

---------

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
2024-06-03 11:57:31 +10:00
Samuael A
aeb4a87913 exchanges: Add Deribit exchange support (#1082)
* deribit implementation

* add ws impll

* cleanup

* Update deribit_wrapper.go

* Add missing endpoints

* Fix config file

* asset type update

* Update code structure

* Update authenticated private endpoints unit tests

* Updating websocket

* Updating websocket connection and subscription handling

* Finishing up adding subscription push data

* Adding websocket public endpoint

* Adding WS endpoints

* Adding websocket unit tests

* Minor clean-up

* Integrating websocket endpoints into the wrapper funcs

* Updating exchange documentations

* Fixing test issues

* Code cleaning-up

* fix test issues

* Updating validations and logic errors

* Updating wrapper issues

* fix test issues

* Slight test update

* Unit test and code structure update

* Update websocket tempos

* Slight update on code structure

* Minor update on unit tests

* Update depending on review comments

* Minor code fix and doc re-generating

* Update on Candlestick wrapper functions

* Minor updates

* minor unit test updates

* Minor updates on weboscket and unit tests

* minor linter fix

* codespell and rate limiter issues

* single linter issue fix

* adding rate limiter

* Add ratelimiter to websocket conn and overall code update

* fix websocket push data issue

* Implementing missing wrapper function

* Websocket fix

* Minor update on missing endpoint and other

* fixing websocket issues and cleaningup

* Minor tempo fix

* Minor linter issues

* unit test update

* Indexing error fix

* Websocket connection fix

* string formatting fix

* Small fix on unit tests

* fix minor json conversion issue

* websocket and documentation update

* websocket, wrapper and unit test updates

* Documentation and unit tests update

* Fix unit tests

* wrapper fix for new change

* Unit test fix

* timestamp conversion and unit tests update

* Minor instrument ID conversion fix

* instrument formats and unit test update

* formatting and unit test fix

* config update

* Updating websocket and adding the Spot support

* Add small unit test fix

* unit test and websocket handlers update

* Linter issues fix

* minor documentation and code update

* Minor fix

* added a wrapper func GetLatestFundingRates

* Types, wrapper update, and unit tests

* Minor config update

* fix wrapper unit tests

* Resolve all panic and wrapper test issues

* minor unit test fix

* fix issues and adding newly added endpoints

* updates and added remaining endpoints with unit tests

* Update unit tests using assert

* Added missing endpoints and unit tests

* Minor updates and clean-ups

* Resolve tradable pair fetching  panic

* Mutex fix

* Added Options assets test and minor fixes

* subscription mothod updated

* Remove misadded code lines

* resolve websocket

* Updating tests, types, endpoint methods and others

* Added GetFuturesContractDetails and minor fix

* fix linter issue

* revert change on candlestic time

* Added filters to candles

* minor unit test and wrapper fix

* Minor unit tests update

* cahnge param key for GetOrderMarginByID

* updating unit tests and resolve issues

* Update websocket unit tests

* Minor fix based on review

* Revert unit test change

* fix pair config issue

* Added missing wrapper functions

* Fix missing review changes

* Fix options request pair formatting

* fix AllExchangeWrappers test issue

* Changes with unit test and wrapper based on the review

* Fix to options reg-exp

* wrapper functions fix

* Update MaximumFundingRateHistory filter and minor fixes

* Fix besed on review comment

* Fix issues on review comment

* linter fix

* fix minor unit test issue

* Fix unit test issues

* Update trade order cancellation responses

* fix config files issue

* lint update config files

* Update unit tests

* Update return values and response handling

* added missing endpoint and fixes based on review comment

* toggle useTestNet back

* Update cancel by label and other fix

* fix forgotten cancel all response type

* update CancelResp type

* Fix unmarshaling error

* updated websocket orderbook load issue

* fix websocket lock and groups

* Change Items to Tranche and fix linter issues

* Fix orderbook issue

* Update unit tests offline error handling, and endpoints argument and  error handling

* Contributors documentation update and change error return type

* Updated unit tests based on review comment

* Update unit tests and removed password change endpoint

* Fix race condition

* Update on tests, test pairs, and wrapper config

* Update test tradable pairs loading

* Update unit tests, fix linter issues, and update wrapper functions

* remove credentials

* Update test and fix authentication method and few authenticated endpoints

* fix codespell issue

* group the repeated currency code check to a func

* added unit test for repeated pair check func

* Added a base coin and related updates

---------

Co-authored-by: E Sequeira <earncef@earncef.com>
2024-05-29 10:28:03 +10:00
Gareth Kirwan
c37a115fc7 Linter: Spelling fixes (#1554) 2024-05-28 11:19:36 +10:00
Ryan O'Hara-Reid
ca9efe7275 currency/manager: remove deadlock potential in concurrent operations (#1545)
* currency/manager: remove deadlock potential in concurrent operations

* fix kucoin test, lets see what this does shalllllll weeeeee.

* another fix in the deep dark depths

---------

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
2024-05-17 15:16:02 +10:00
Gareth Kirwan
7d1eecfa7e Tests: Various race fixes and move TestFixtureToDataHandler (#1534)
* Tests: Move and simplify TestFixtureToDataHandler

* Currency: Fix PairsManager.Load breaking matcher

* Tests: Add multi-instance cache to UpdatePairsOnce

* Kraken: Fix TestUpdateTickers race error

Calling StorePairs on global instance can lead to race

* Bitfinex: Fix TestUpdateTickers racing intermittently

* Currency: Fix concurrent access to PM formats

* Currency: Fix SupportsAsset implementation

This should delegate entirely to PairManager's IsAssetSupported

* Okx: Fix PM intrusion, rm GetPairFromInstrumentID

* Exchange: Fix SetGlobalPairsManager to set asset enabled

* Bitflyer: Fix race on set TestGetCurrURL

TestGetCurrencyTradeURL would fail sometimes due to sequencing of
enabling futures but not having pairs for it.

* Tests: Simplify usage pattern for FixtureToDH
2024-05-16 14:09:26 +10:00
Gareth Kirwan
44d50a3617 Kraken: Fix GetLatestFundingRates requiring pair enabled (#1522)
* Kraken: Fix GetLatestFundingRates requiring pair enabled

* Tests: Add latestRateRequests to all asset tests
2024-04-19 17:16:17 +10:00
Gareth Kirwan
fdf6014dca Currency: Remove Pair Index formatting/parsing (#1520)
* Currency: Remove Pair Index formatting/parsing

This feature was originally for exchanges with only one pair (e.g. KRW) which made parsing easier.
However there's no examples of this left, and we can reduce complexity
overall by removing it.

* Exchange: Partial assertify tests and fixes

* Currency: Fix panic on a delimiterless small currency
2024-04-19 17:03:12 +10:00
Ryan O'Hara-Reid
6ccb0e0c2f gateio/kucoin: assortment of fixes (#1404)
* gateio: fix unmarshal bug and update fields

* gateio: fix wrapper function function, add helper methods

* update order types and add kucoin wrapper fix

* currency pairs

* Add tests

* gateio; inspect error and continue for no funds in account, kucoin: fetch all settlement amounts

* futures: order fixit

* finish off gateio updates for market orders

* cute line

* Update exchanges/kucoin/kucoin_wrapper.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/kucoin/kucoin_wrapper.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/gateio/gateio.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/gateio/gateio_wrapper.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/gateio/gateio_wrapper.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* glorious: nits - filter by pair match and fix bug where the endpoint returns details instead of message

* Add fix for leverage check (non-merge) my ip has been blocked from gateio still... scammmmmmmm

* glorious: nitters

* Update exchanges/gateio/gateio_test.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update exchanges/gateio/gateio_test.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update exchanges/gateio/gateio_test.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

---------

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
2024-03-06 13:06:13 +11:00
Adrian Gallagher
08da42ddb7 CI/build: Update Go version, linters and fix minor issues (#1473)
* CI/build: Update Go version, linters and fix minor issues

* Bump golangci-lint to v1.56.1

* BinanceUS: Make uint usage consistent

* Throw blank identifiers into the trash
2024-02-14 11:02:06 +11:00
Gareth Kirwan
e007f69f7c exchanges/websocket: Implement subscription configuration (#1394)
* Websockets: Move Subscription to its own package

This allows the small type to be imported from both `config` and from
`stream` without an import cycle, so we don't have to repeat ourselves

* Subs: Renamed Currency to Pair

This was being mis-used through much of the code, and since we're
already touching everything, we might as well fix it

* Websockets: Add Subscription configuration

* Binance: Add subscription configuration

* Kucoin: Subscription configuration

* Simplify GenerateDefaultSubs
* Improve TestGenSubs coverage
* Test Candle Sub generation
* Support Candle intervals
* Full responsibility for formatting Channel name on GenerateDefaultSubs
  OR consumer of Subscribe
* Simplify generatePayloads as a result
* Fix test coverage of asset types in processMarketSnapshot

* Exchanges: Abstract ParallelChanOp

* Tests: Generic ws mock instances

* Kucoin: Fix intermittent conflict in test currs

Use isolated test instance for `TestGetOpenInterest`.

`TestGetOpenInterest` would occassionally change pairs before
GenerateDefault Subs.
2024-01-24 15:54:07 +11:00
Scott
b71bf1f3d1 exchanges/futures: Implement open interest (#1417)
* adds open interest to exchanges

* ADDS TESTING YEAH

* New endpoints, BTSE, RPCS, cached

* slight design change, begin gateio

You will need to get cached for
each exchange that supports it

* gateio, huobi, rpc

* fix up kraken, cache retrieval

* okx, gateio

* finalising all implementations and tests

* definitely my final ever commit on this

* Well, well, well

* final v2

* quick fix of bug

* test coverage, assert notempty, test helper

Added a new testhelper for currency
management because its very annoying
in a parallel test setting which wastes
so much space otherwise

* minimises REST requests for Open Interest

* types.Number merge misses

* Minimises Kraken REST calls

* len change, value -> pointer receiver

* further fixup

* fixes gateio, batch calculates open interest

* single gateio, lint const fixes

* rejig and more thorough oi for huobi

* formatting expansion

* minor fix for handling expiring contracts

* rm unused Binance strings

* add bybit support, fix bybit issues

* oopsie doopsie, dont look at my whoopsie

* Fix issue, remove feature

* move an irrelevant function for the pr

* mini bybit upgrades

* fixes cli request bug
2024-01-12 15:27:35 +11:00
Samuael A
fb6d12ac69 exchanges: Update Bybit exchange to V5 (#1301)
* Adding Bybit public endpoints

* Completed adding market endpoints

* Added trade endpoints

* Adding position endpoints

* completing position endpoints

* Adding Pre-upgrade endpoints

* Completed adding Pre-upgrade and Account endpoints

* Added asset endpoints

* Added user endpoints and unit tests

* Adding spot leverage and margin trade endpoints

* spot margin trade added

* added spot-margin-trade, institutional lending, c2c lending, and broker

* Adding wrapper funnctions

* Working on wrapper public methods

* Added wrapper functions and unit tests

* Added websocket support with unit tests

* Update websocket handlers and added rate-limiter

* wrapper function, websocket handlers, and linter issues fixe

* unit tests fixes and codespell correction

* Update documentation

* Minor websocket handling fix and URL consts merging

* types, unit test other updates

* Updated websocket and methods based on review

* Added GetFeeByType method with unit test and fixes

* add filter for Unified and Normal endpoints

* Mock recording and unit tests update

* minor linter issue fix

* websocket and rest tests and fix

* change asset types and wrapper methods update

* rm: forgotten panic message

* endpoints, websocket  and unit tests update

* Added and updated endpoints and unit test

* linter and spell fix

* unit test and orders update

* Update on endpoints, fields, config pairs and formating, and unit tests

* minor update on responses

* Fix unit test and types

* Unit tests, models, and wrapper issues fix and mock test recording

* rm print statement

* Fix issue, add FundingRate wrapper func, mock record

* minor type and unit test update

* Update on order handling and unit test

* Minor test

* Minor fix in wrapper

* unit tests update, recording, and documentation update

* Unit tests and minor wrapper function update

* minor unit test fix

* Added newly added endpoints, unit tests, and mock recording

* Rename GetInstruments -> GetInstrumentInfo

* doc update

* Minor unit tests update

* Minor unit test and wrapper update

* Fix linter issue

* Add unit test and minor updates

* Revert websocket error declaration

* Revert websocket error declaration

* Balace --> Balance

* Fix config issues

* Added next funding time minor fix

* Update GetLatestFundingRates and record mock test data

* Added LatestFundingRate time

* Fix test issue of TestAllExchangeWrappers

* config pairs update

* configtest spot pairs update

* Minor update on options UpdateOrderExecutionLimits wrapper func

* Linter issue fix and added new currency codes

* Added new currency codes

* Update bybit pairs in config_example

* config assets pair format update
2024-01-11 12:35:46 +11:00
Gareth Kirwan
37b1121bbd BTSE: Fix duplicate pair errors on Million pairs (M_*) (#1401)
* BTSE: Fix duplicate error on Million pairs (M_*)

BTSE has listed Pitbull token with two symbols:
PIT-USD and M_PIT-USD for millons of PIT / USD.
The native token is not tradable, so we ignore them and
get a base of M_PIT because that's what later APIs will accept

* BTSE: Fix test errors on locked market

* Common: Improve AppendError and ExcludeError

This change switches from a stateful multiError to caring more about the
Unwrap() []error interface, the same as [go standard
lib](https://github.com/golang/go/blob/go1.21.4/src/errors/wrap.go#L54-L68)

Notably, if we implement Unwrap() []error and do NOT implement Is() then
we get free compatibility with the core functions.

The only distateful thing here is needing to deeply unwrap fmt.Errorf
errors, since they don't flatten. I can't see any way around that

* Pairs: Fix exchange config Pairs loading

When a pair string contained two punctuation runes, the first one is used,
and the configFormat is ignored.

This fix checks the list and corrects any with the wrong delimiter, or
errors if the format is inconsistent.

* BTSE: Fix all tickers retrieved by GetTicker

PR #764 introduced GetTickers, but it wasn't rolled out to BTSE.
This fix ensures that when one ticker is a locked market, the rest continue to
function. Particularly important if the locked market wasn't even
enabled anyway.

* Kucoin: Fix test config future pairs

* BTSE: Remove PIT tests; Token removed

BTSE have removed the PIT token pairs

All these changes stand, and this just removes the test

* ITBit: Fix fatal error on second run

This fix removes incorrect config pair delimiter, because it would be
re-inserted into config the first run, and then error the second time.

This delimiter doesn't match the config we have.
There's no implementation of fetching pairs, so what's in config files
now is all that matters

* Engine: Fix TestConfigAllJsonResponse

* Clarity of non-matching json improved
* Handling for fixing pair delimiters
2023-12-19 14:40:13 +11:00
Gareth Kirwan
88182ec414 Engine: Remove Default Forex Provider (#1395)
* Currency: Do not use a default forex provider

exchangerate.host now requires an API key.
Instead of finding a new Free (for now) default, this change simply
disables the currency exchange when nothing is enabled.

* SyncManager: Report ?.?? for an unknown forex amount

In a situation where we thought forex was available but we got an error,
this avoids showing 0.00 when there was actually an error.

* Currency: Tests for no default forex

* Currency: Use mock provider for tests

* Currency: Add API key to exchangerate.host

* Currency: Remove Exchangerate.host

Exchangerate.host was bought by apilayer, the old API deprecated, and
replaced with a proxy to the apilayer api.
We already have currencylayer support, so ther's no reason to keep exh.

Worth noting: New ERH keys actually work on currencylayer

* Currencies: Add test coverage for currency layer

* fixup! Currency: Tests for no default forex

Remove duplicate assignment

Fixes [review comment](https://github.com/thrasher-corp/gocryptotrader/pull/1395#discussion_r1395178513)

* fixup! Currency: Add API key to exchangerate.host

Remove unused ErrVar

Fixes [review
comment](https://github.com/thrasher-corp/gocryptotrader/pull/1395#discussion_r1396647418)

* fixup! Currency: Tests for no default forex

Fix spelling of override in test

Fixes [review comment](https://github.com/thrasher-corp/gocryptotrader/pull/1395#discussion_r1396701476)

* fixup! SyncManager: Report ?.?? for an unknown forex amount

Fix display of non-positive currency conversions.

Fixes [review comment](https://github.com/thrasher-corp/gocryptotrader/pull/1395/files#r1398527134)
2023-11-24 13:50:01 +11:00
Scott
70690d9a04 futures: Implement GetLatestFundingRates across exchanges (#1339)
* adds funding rate implementations and improvements

* merge fixes x1

* lint

* kucoin funding rates func make

* migrate sync-manager to keys

* some kucoin work

* adds some kucoin wrapper funcs

* ehhh, todo

* kucoin position

* start of orders

* adds the kucoin tests yay

* multiplier

* nits, EWS includes order limits

* NotYetImplemented, IsPerp improvements, cleaning

* lint, test fix, huobi time

* fixes issues, improves testing

* fixes linters I WRECKED

* local lint but remote lint, lint, lint, lint

* fixes err

* skip CI

* lint

* Supported rates, binance endpoints

* fixes weird mocktest problems

* no, CZ is invalid

* fixes some new EWS test errors
2023-11-03 11:01:32 +11:00
Ryan O'Hara-Reid
ceef7a14e0 currency: Adds matching lookup table built from available pairs (#1312)
* currency: Add pair matching update (cherry-pick)

* exchange/currency: Add tests and update func

* linter fix, also if using json unmarshal functionality stop usage of string conversion without delimiter

* gemini: fix test

* currency/manager: potential optimisation

* exchanges: purge derive from wrapper cases and add warning comment

* glorious: nits

* glorious: nits

* linter: fix

* glorious: nits

* whoops

* whoops

* glorious: nits continued

* glorious: diff THANKS!

* hitbtc: fix update tradable pairs strings splitting. continue if not enabled tickers update pair.

* glorious: nits

* linter: fix

* Update exchanges/exmo/exmo_wrapper.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* bitstamp: fix test when 32 biterinos architecturinos

* capture more strings for speed

* swapsies because whos running 32bit \0/?

---------

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2023-10-18 11:57:27 +11:00
Gareth Kirwan
d86da76b1f Bitstamp: Add auth channel subscription handling (#1333)
* Bitstamp: Add auth channel subscription handling

* Bitstamp: Avoid searching for asset type

We've hardcoded asset.Spot in order to find the pair.
Looking the asset up from the pair makes no sense.

* Bitstamp: Add type for wsOrders

* Bitstamp: Working test of Generic DataHandler

* Bitstamp: WS Order chan tests and remove type

orderType could be derived from status == New and Buy & price == 9+e9 or Sell & price == 0
But it would only be true for the first update and it really doesn't feel worth the risk
Consumers are going to have to merge to original request anyway

* Bitstamp: Linter fixes

* Kraken: Switch to shared fixture test

* Bitstamp: Fix lint on TestFixtureToDataHandler

* Engine: Add Clone for PairsManager

go-vet highlighted that the mutex here is a value when we copied the
PairsManager in a test.
Options to fix:
* Add a deep clone method to PairsManager
* Add a shallow clone method with a disclaimer
* Make the mutex a pointer
* Make the PairsManager itself a pointer

Options 3 and 4 are too invasive to justify changing at this point.
There's an inherent risk of PM being passed by value, but govet should
catch the copylock.
There's more risk in changing everything to use a pointer at this stage.

* Engine: Fix linter again, ironically

* Bitstamp: Rename OHLC const

* Bitstamp: Minor fixes to syntax

* Bitstamp: Simplify chanSymb=>pair

* Bitstamp: Still process order updates without ID

If there's a ClientOrderID we'll still process the order.
It doesn't seem likely we'd have this happen, but if it does we still
want consumers to get something.

* Bitstamp: Replace Clone with Lock methods

* Engine: Expose PairsManager's Mutex

Makes more sense than wrapping functions

* Bitstamp: Fix linter copylock (again)

* fixup! Engine: Expose PairsManager's Mutex

Omit Mutex from Json

* fixup! Bitstamp: Add auth channel subscription handling

Remove unused wsAuthToken

* Bitstamp: Simplify OrderData Unmarshal

* Bitstamp: Remove unused contexts

I added these following best practices, but the reality is that when/if
we get context awareness in GCT, there will be a lot more to fix and
this will be a drop in the ocean anyway.

* Bitstamp: Only call handleWSOrder for  MyOrders

* Bitstamp: Avoid allocating again in handleWSOrder

* CurrencyPairs: Remove public mutex

Simplified to a Load method to avoid making mutex public

* Tests: Improve test readability and clarity

* Bitstamp: Wrap errWSPairParsingError

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Bitstamp: FetchWSAuth mock and live test

---------

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2023-10-05 16:13:05 +11:00
Ryan O'Hara-Reid
ecf0a5bcb3 spellcheck: fix and update (#1359)
* spellcheck: fix and update

* spellcheck: add exclude line for gateio_types
2023-10-03 16:07:03 +11:00
Scott
7f0faf7850 futures: add GetFuturesContractDetails wrapper function (#1274)
* all in a days work

* cleanup

* cleanup for real, also stop it binance.json

* minor coverage

* adds gateio to the slurry

* cleanup of types

* verbose verbose verbose verbose verbose verbose

* fixes huobi parsing issue

* fix bybit contract identification

* cleanup

* merge fixes

* addresses many big problems raised by SHAZ

* tracking errors and fixes

* funding rate if avail, fixes currency formatting

* Addresses nits and sneaks in extra fixes

* lint

* minor fixes after rebase

* better contract splitter for currencies like T-USDT

* forgot to add the exchange name like a fool

* merge fixes x1

* kucoin, direction, contract size

* rn direction, fix kucoin time

* WHOOPS

* Update exchanges/kucoin/kucoin_wrapper.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* misdirection

---------

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
2023-10-03 16:04:45 +11:00
Samuael A
6105071114 exchanges: Add Kucoin support (#1102)
* init

* updates config

* wrapper configuration

* updates exchange readme

* adds SendAuthHTTPRequest and SendHTTPRequest

* adds ratelimit file

* adds test case and minor fixes

* improve error handling

* update testcases and improve GetSymbols API

* adds SPOT API's

* minor fix

* WIP

* WIP

* adds test case

* adds check in test case

* fixes in Auth. HTTP

* improvements

* adds trade, kline support and testcases

* adds SPOT API and testcases for same

* adds SPOT API and testcases

* adds SPOT API and testcase

* WIP

* adds API's

* adds API's

* adds test cases

* adds comment to exported data types

* adds API and test cases

* adds API

* adds API

* rearrange functions

* WIP: adds API

* adds API for Post Order SPOT

* adds API and few fixes

* fixes

* WIP

* WIP

* add PostBulkOrder API and its test case

* fix issues

* adds cancel order APIs and test cases for same

* add minor test fixes

* add API

* adds API

* fixes

* add API

* adds API and test cases

* fix test

* adds API

* adds test

* fix test

* adds API and test

* adds deposit API and test cases

* WIP

* adds API and test cases

* WIP

* WIP

* add public future API and test cases

* WIP

* remove v2 API and replace them with v1

* update test cases

* adds future order API and test cases

* adds futures order API

* adds API

* add API and test cases

* adds API and test cases

* adds API and test cases

* adds API and test cases

* Adding wrapper functions

* Fix on wrapper function

* Adding websocket support

* Complete addressing WS push datas

* Adding spot push data unit tests

* adding futures websocket push data handlers

* Adding futures websocket push data handlers

* Added unit tests

* Updating unit tests

* Updating wrapper and unit test functions

* Adding missing wrapper functions and code cleaning up

* Resolved linter issues

* Fixing websocket issues

* Fixing websocket issues

* Slight fix on config_example file

* Minor update

* Basic nits updates

* Fix minor linter issues

* Minor update

* Minor unit test update

* Minor unit test update

* Code update and linter issues fix

* Removed unnecessary type conversion codes

* Monor update based on review comment

* Fix based on review comments

* Adding rate-limiter

* Websocket update and overall minor fixes

* Removed IsAssetTypeEnabled method implementation

* Fix connection and formatting issues

* Updating orderbook issues

* Very minor label fix

* Minor error returning fix

* code cleaning up and minor spelling fix

* Updates on unit test

* Update on unit tests and slight code structure

* unit test update

* orderbook update and minor fix

* fix on race

* Mini linter fix

* fix minor parameter and unit test issues

* handler funcs and models update

* Fixing websocket and unit test issues

* order side string for active orders

* Fix on websocket and unit tests

* Minor type changes

* Minor Orderbook fix and unit test update

* Small fix on orderbook

* Updating orderbook functionality

* FIx on websocket orderbook handlers

* Small update on kucoin websocket

* fix missed review comments

* fix based on review comments

* Updating websocket orderbook and fixing unit tests

* Minor fixes

* unit test update

* Updating unit test according to enabled asset type

* toggle canManipulateRealOrders const

* Unit test update

* Fix minor issues

* minor fix

* documentation fix

* wrapper coverage and unused params fix

* testing and minor changes

* documentation, websocket and unit test update

* minor linter fix

* Websocket spot/margin subscription update

* minor ticker update fix

* minor fixes on endpoints

* timestamp and number convert method and unit tests

* timestamp convert minor update

* minor type and conversion fix

* create a common timestamp convert and fix minor issues

* linter and ticker fix

* Updating unit tests and order placing endpoint methods

* Added a pairs check

* Fix config test error

* rm unused error variable

* Fix source of linter issue

* code update: convert, wrapper and websocket fix

* minor code update

* Websocket code and unit tests update

* Websocket ticker ask/bid type change and small error msg fix

* docs update

* fix: websocket orderbook handling

* change orderbook channel to marketOrderbookLevel2Channels and fix websocket orderbook update

* Minor func rename and reciever change

* Minor orderbook unit test issue fix

* comment: about why we used a random delimiter '-' for futures

* update config files and FetchTradablePair func for futures pairs

* futures config pairs update

* remove ConnextionMonitorDelay from websocket setup

* fix on types and futures pair conversion

* updating config pairs

* change NewPairFromString to DeriveFrom

* unit tests update

* unit tests update

* Added TickerBatching

* added GetStandardConfig to GetDefaultConfig

---------

Co-authored-by: Jaydeep Rajpurohit <jaydeeppurohit1996@gmail.com>
2023-09-27 15:09:38 +10:00
Scott
5f2f6f884b Binance,Okx: Add Leverage, MarginType, Positions and CollateralMode support (#1220)
* init

* surprise train commit

* basic distinctions

* the terms of binance are confusing

* renames and introduction of allocatedMargin

* add new margin funcs

* pulling out wires

* implement proper getposition stuff

* bad coding day

* investigate order manager next

* a broken mess, but a progressing one

* finally completes some usdtmargined stuff

* coinMfutures eludes me

* expand to okx

* imports fix

* completes okx wrapper implementations

* cleans and polishes before rpc implementations

* rpc setup, order manager features, exch features

* more rpc, collateral and margin things

* mini test

* looking at rpc response, expansion of features

* reorganising before the storm

* changing how futures requests work

* cleanup and tests of cli usage

* remove silly client side logic

* cleanup

* collateral package, typo fix, margin err, rpc derive

* uses convert.StringToFloat ONLY ON STRUCTS FROM THIS PR

* fix binance order history bug

* niteroos

* adds new funcs to exchange standards testing

* more post merge fixes

* fix binance

* replace simepletimeformat

* fix for merge

* merge fixes

* micro fixes

* order side now required for leverage

* fix up the rest

* global -> portfolio collateral

* Update exchanges/collateral/collateral_test.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* adds fields and todos

* rm field redundancy

* lint fix oopsie daisy

* fixes panic, expands error and cli explanations (sorry shaz)

* ensures casing is appropriate for underlying

* Adds a shiny TODO

---------

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
2023-09-26 16:16:31 +10:00
Samuael A
9c83231696 exchanges: Update OKCoin V5 exchange support (#1206)
* starting public endpoints

* Adding public endpoints

* added public spot market endpoints

* websocket subscriptions updates

* websocket push data handlers completing

* linter fix

* Added funding private endpoints

* Adding authenticated account endpoints

* Added fiat and OTC-RFQ authenticated endpoints

* trading authenticated endpoints

* completing trade endpoints and add public wrapper endpoints

* Authenticated wrapper functions and corresponding unit test

* Adding authenticated websocket endpoint and fixing wrapper functions

* Documentation and exchange websocket update

* update websocket orderbook checksum handling

* linter issues fix and unit test update

* remove invalid orderbook endpoint and unit test

* Documentation, handlers, and model types update

* minot fix

* Minor fixes

* Updating unit tests and added missing endpoints

* Add missing credential check

* Minor unit test fixes

* fix minor linter issue

* add snaphot test unit test

* Fix on update checksum and documentation update

* update exchange, add UpdateOrderExecutionLimits, and update documentation

* Minor fix on tickers fetching

* Minor websocket fix and smaill unit tests

* Minor websocket and naming fixes

* uncomment default channels

* Fix type and unit test issues

* websocket channels and data handling update

* Update Advanced-Algo websocket handling and minor fixes

* documentation and minor code fixes

* Fix name changes

* documentation contribution update

* intervalToString method update

* fix exchange_wrapper_standard tests

* Fix minor issues based on exchange_wrapper_standards_test

* Fix wrapper extended candlestick check

* websocket orders fetching error check method update

* Exchange name check and change

* docs: Add missing contributors

---------

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
2023-08-22 09:44:39 +10:00
Scott
c7b3ace78c testing: bybit references, bybit mock testing, pair formatting, standards improvement (#1322)
* fixes tests

* pair formats and extra fixes

* quick change before shazbert sees

* sneaky lint

* adds bybit mock testing and fixes test

* whoops

* error response instead

* classic forgetting to lint

* bybit live test no longer auto-records results

* ty thrasher- Update exchanges/bybit/bybit_wrapper.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

---------

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
2023-08-21 14:48:56 +10:00
Ryan O'Hara-Reid
43a8044b44 gateio: websocket updates (#1282)
* gateio: websocket updates

* Update exchanges/gateio/gateio_websocket.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* revert that trick

* glorious:nits

* Update exchanges/gateio/gateio_ws_futures.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update exchanges/gateio/gateio_ws_futures.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update exchanges/gateio/gateio_websocket.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update exchanges/gateio/gateio_websocket.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update exchanges/gateio/gateio_ws_option.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update exchanges/gateio/gateio_ws_option.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update exchanges/gateio/gateio_websocket.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update exchanges/gateio/gateio_websocket.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* thrasher-nits

* Update exchanges/gateio/gateio_ws_futures.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update exchanges/gateio/gateio_ws_futures.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* thrasher: nits rides again chapter 2 volume 1

* rm unmarshaljson method for orderbook

* use gateio time where it can and update tests

* math.trunc and lower time frame on big books

* :eagle

---------

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
2023-08-04 17:39:38 +10:00
Scott
471f4f21c4 Binance,OKx: Implement fetching funding rates (#1239)
* adds basic groundwork for rates on binance

* more into rates on binance

* rm redudant redundancy, add payments

* mini commit before merging with testnet ability branch

* changes function signature and fixes resulting build

* gets billing data too

* funding rates package, features use, testnet reimpl

* new endpoint, refinements and tests

* cli fix, rpc impl, testing, payments

* fixups from looking at code

* typo fix

* niteroos

* merge fixes

* adds test, fixes cli issues

* woah nelly
2023-07-26 14:25:43 +10:00
Scott
fcc5ad4551 exchanges/qa: Add exchange wrapper testing suite (#1159)
* initial concept of a nice validation tester for exchanges

* adds some datahandler design

* expand testing

* more tests and fixes

* minor end of day fix for bithumb

* fixes implementation issues

* more test coverage and improvements, but not sure if i should continue

* fix more wrapper implementations

* adds error type, more fixes

* changes signature, fixes implementations

* fixes more wrapper implementations

* one more bit

* more cleanup

* WOW things work?

* lintle 1/1337

* mini bump

* fixes all linting

* neaten

* GetOrderInfo+ asset pair fixes+improvements

* adds new websocket test

* expand ws testing

* fix bug, expand tests, improve implementation

* code coverage of a lot of new codes

* fixes everything

* reverts accidental changes

* minor fixes from reviewing code

* removes Bitfinex cancelBatchOrder implementation

* fixes dumb baby typo for babies

* mini nit fixes

* so many nits to address

* addresses all the nits

* Titlecase

* switcheroo

* removes websocket testing for now

* fix appveyor, minor test fix

* fixes typo, re-kindles killed kode

* skip binance wrapper tests when running CI

* expired context, huobi okx fixes

* kodespull

* fix ordering

* time fix because why not

* fix exmo, others

* hopefully this fixes all of my life's problems

* last thing today

* huobi, more like hypotrophy

* golangci-lint, more like mypooroldknee-splint

* fix huobi times by removing them

* should fix okx currency issues

* blocks the application

* adds last little contingency for pairs

* addresses most nits and new problems

* lovely fixed before seeing why okx sucks

* fixes issues with okx websocket

* the classic receieieivaier

* lintle

* adds test and fixes existing tests

* expands error handling messages during setup

* fixes dumb okx bugs introduced

* quick fix for lint and exmo

* fixes nixes

* fix exmo deposit issue

* lint

* fixes issue with extra asset runs missing

* fix surprise race

* all the lint and merge fixes

* fixes surprise bugs in OKx

* fixes issues with times and chains

* fixing all the merge stuff

* merge fix

* rm logs and a panic potential

* lovely lint lament

* an easy demonstration of scenario, but not of initial purpose

* put it in the bin

* Revert "put it in the bin"

This reverts commit 15c6490f713233d43f10957367fcbf18e3818bdd.

* re-add after immediate error popup

* fix mini poor test design

* okx okay

* merge fixes

* fixes issues discovered in lovely test

* I FORGOT TO COMMIT THIS

* nit fixaroonaboo

* forgoetten test fix

* revert old okx asset intrument work

* fixes

* revert problems I didnt understand. update bybit

* fix merge bugs

* test cleanup

* further improvements

* reshuffle and lint

* rm redundant CI_TEST by rm the CI_TEST field that is redundant

* path fix

* move to its own section, dont run on 32 bit + appveyor

* lint

* fix lbank

* address nits

* let it rip

* fix failing test time range

* niteroo boogaloo

* mod tidy, use common.SimpleTimeFormat
2023-07-03 11:09:43 +10:00
Ryan O'Hara-Reid
92839ebaa2 currency: Add Pair method IsAssociated (#1230)
* currency: pair method add IsAssociated()

* Add pair into Order params, export error

* glorious: nits

---------

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
2023-06-16 12:34:44 +10:00
Ryan O'Hara-Reid
9e4ea6c62b currency: Add OrderParameters helpers for currency pair (#1217)
* currency: Add order aspect from pair helper method.

* Update currency/pair_methods.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update currency/pair_types.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* glorious: nit cont. rm pair field and used IsPopulated method instead of IsEmpty

* thrasher: nits

* ch test name

* glorious: nits

---------

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2023-06-09 10:07:04 +10:00
Samuael A
3eac6d12bd exchanges: Update GateIO exchange to V4 (#1058)
* Adding Public Endpoints and test functions

* Adding public endpoints and test functions

* Adding private spot endpoints

* Adding private endpoints and corresponding tests for margin

* Adding Margin Private endpoints

* Adding cross margin and flash swap endpoints

* Adding futures private endpoints

* Adding futures private endpoints and corresponding tests

* Adding Options and SubAccount endpoints and their unit tests

* Adding Wrapper functions

* Complete wrapper functions and corresponding unit test functions

* Fixing wrapper issues and adding websocket functions

* Update of Spot websocket and adding futures websocket handlers

* completed futures WS push data endpoints

* Completed Options websocket endpoints

* Adding websocket support for delivery futures and slight update on endpoint funcs

* Added Delivery websocket support and fix linter issues

* Update on Unit tests

* fix slight currency format error

* Fix slight endpoint tempos

* Update on conditional statements and unit tests issues

* fixing slight tempos

* Slight model and websocket data push method change

* Fix unit test tempos and updating models

* Fix on code structures and update on unit tests

* Slight code fix

* Remove print statements

* Update on tradable pairs fetch eps

* Fix websocket tempos

* Adding types to websocket routine manager

* Fix slight issues

* Slight fixes

* Updating wrapper funcs and models

* Slight update

* Update on test

* Update on tradable pairs

* update conditional statements

* Fixing slight issues

* Updating unit tests

* Minor fixes depending review comments

* Remove redundant method declaration

* Adding missing intervals

* Updating fetch tradable pairs

* update tradable pairs issues

* Addressing small tempos

* Slight fix on ticker

* Minor Fixes

* Minor review comment fixes

* Unit test and minor code updates

* Slight code updates

* Minor updates depending review comments

* Fixes

* Updating incoming message matcher

* Fix missing merge issue

* Fix minor wrapper issues

* Updating ratelimit and other issues

* Updating endpoint models and adding missing eps

* Update on code structure and models

* Minor codespell fixes

* Minor update on models

* fix unit test panic

* Minor race fix

* Fix issues in generating signature and unit tests

* Minor update on wrapper and unit tests

* Minor fix on wrapper

* Mini linter issues fix

* Minor fix

* endpoint fixes and slight update

* Minor fixes

* Updating exchange functions and unit tests

* Unit test and wrapper updates

* Remove options candlestick support

* Minor unit test and wrapper fix

* Unit test update

* minor fix on unit test and wrapper

* endpoints constants name change

* Add minor wrapper issues

* endpoint constants update

* endpoint url updates

* Updating subscriptions

* fixing dual mode endpoint methods

* minor fix

* rm small tempo

* Update on websocket orderbook handling

* Orderbook and currency pair update

* fix linter and test issues

* minor helper function update

* Fix wrapper coverage and wrapper issues

* delete unused variables

* Minor fix on ReadData() call

* separating websocket handlers

* separating websocket handlers

* Minor fix on enabled pair

* minor fix

* check instrument availability in spot

* create a separate subscriber for sake of multiple websocket connection

* linter fix

* minor websocket and gateio endpoints fix

* fix nil pointer exception

* minor fixes

* spelling fix decerializes -> deserializes

* fix Bitfinex unit test issues

* minor unknown currency pair labling fix

* minor currency pair handling fix

* slight update on GetDepositAddress wrapper unit test

* setting max request job to 200

* fixing numerical and timestamp type convert

* fix value overflow error

* change method of parsing orderbook price

* unifying timestamp conversion types to gateioTime

---------

Co-authored-by: Samuael Adnew <samuaelad@Samuaels-MacBook-Air.local>
2023-05-30 14:03:53 +10:00
Ryan O'Hara-Reid
db8735ec99 log: Add structured logging (#1171)
* basic implementation

* log: deprecate duplicate function, add tests and refine calls.

* linter: fixes

* linter: update struct

* linter and new type

* log tests: update to not lint issue

* linter: stop complaining please

* glorious: nits

* log: rm comment code

* glorious: nits

* glorious: nits

* glorious: nits

* glorious: nits missed

---------

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
2023-05-10 17:52:53 +10:00
Samuael A
969669cb0d linters: Bump golangci-lint version to v1.52.2 and fix issues (#1169)
* fix linter issues

* linter issue suppressing and fixing

* change linter version and linter issues fix

* Bump version
2023-04-27 10:13:46 +10:00
Scott
80b2136ce9 GCT: fix spelling (#1164)
* kodespull

* fix my spelling

* silly sausage, rm sneaky pointer
2023-04-02 10:21:13 +10:00
Adrian Gallagher
7e08e483fb CI: Bump go version, linters and fix minor issues (#1130)
* CI: Bump go version, linters and fix minor issues

* Bump version, fix loop variables

* Revert

* Rid TODOs now that 1.51 has been released
2023-02-03 15:56:59 +11:00
Adrian Gallagher
a79e0d2b3e qa/spelling: Add Codespell support (#1121)
* Add codespell support

* Fix paths

* Add HTML files to exclusion list
2023-01-30 12:36:56 +11:00
Scott
017cdf1384 Backtester: Live trading upgrades (#1023)
* Modifications for a smoother live run

* Fixes data appending

* Successfully allows multi-currency live trading. Adds multiple currencies to live DCA strategy

* Attempting to get cash and carry working

* Poor attempts at sorting out data and appending it properly with USD in mind

* =designs new live data handler

* Updates cash and carry strat to work

* adds test coverage. begins closeallpositions function

* Updates cash and carry to work live

* New kline.Event type. Cancels orders on close. Rn types

* =Fixes USD funding issue

* =fixes tests

* fixes tests AGAIN

* adds coverage to close all orders

* crummy tests, should override

* more tests

* more tests

* more coverage

* removes scourge of currency.Pair maps. More tests

* missed currency stuff

* Fixes USD data issue & collateral issue. Needs to close ALL orders

* Now triggers updates on the very first data entry

* All my problems are solved now????

* fixes tests, extends coverage

* there is some really funky candle stuff going on

* my brain is melting

* better shutdown management, fixes freezing bug

* fixes data duplication issues, adds retries to requests

* reduces logging, adds verbose options

* expands coverage over all new functionality

* fixes fun bug from curr == curr to curr.Equal(curr)

* fixes setup issues and tests

* starts adding external wallet amounts for funding

* more setup for assets

* setup live fund calcs and placing orders

* successfully performs automated cash and carry

* merge fixes

* funding properly set at all times

* fixes some bugs, need to address currencystatistics still

* adds 'appeneded' trait, attempts to fix some stats

* fixes stat bugs, adds cool new fetchfees feature

* fixes terrible processing bugs

* tightens realorder stats, sadly loses some live stats

* this actually sets everything correctly for bothcd ..cd ..cd ..cd ..cd ..!

* fix tests

* coverage

* beautiful new test coverage

* docs

* adds new fee getter delayer

* commits from the correct directory

* Lint

* adds verbose to fund manager

* Fix bug in t2b2 strat. Update dca live config. Docs

* go mod tidy

* update buf

* buf + test improvement

* Post merge fixes

* fixes surprise offset bug

* fix sizing restrictions for cash and carry

* fix server lints

* merge fixes

* test fixesss

* lintle fixles

* slowloris

* rn run to task, bug fixes, close all on close

* rpc lint and fixes

* bugfix: order manager not processing orders properly

* somewhat addresses nits

* absolutely broken end of day commit

* absolutely massive knockon effects from nits

* massive knockon effects continue

* fixes things

* address remaining nits

* jk now fixes things

* addresses the easier nits

* more nit fixers

* more niterinos addressederinos

* refactors holdings and does some nits

* so buf

* addresses some nits, fixes holdings bugs

* cleanup

* attempts to fix alert chans to prevent many chans waiting?

* terrible code, will revert

* to be reviewed in detail tomorrow

* Fixes up channel system

* smashes those nits

* fixes extra candles, fixes collateral bug, tests

* fixes data races, introduces reflection

* more checks n tests

* Fixes cash and carry issues. Fixes more cool bugs

* fixes ~typer~ typo

* replace spot strats from ftx to binance

* fixes all the tests I just destroyed

* removes example path, rm verbose

* 1) what 2) removes FTX references from the Backtester

* renamed, non-working strategies

* Removes FTX references almost as fast as sbf removes funds

* regen docs, add contrib names,sort contrib names

* fixes merge renamings

* Addresses nits. Fixes setting API credentials. Fixes Binance limit retrieval

* Fixes live order bugs with real orders and without

* Apply suggestions from code review

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update backtester/engine/live.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update backtester/engine/live.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update backtester/config/strategyconfigbuilder/main.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* updates docs

* even better docs

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
2023-01-05 13:03:17 +11:00
Samuael A
2ac165a477 exchanges: Add OKX support (#1005)
* public endpoints methods added

* Completing mapping REST endpoints

* Binanceus Wrapper methods -Partially

* Binanceus Wrapper methods -Partially

* BinaWra functions with test funs; Not Completed

* Finalizing wrapper methods & test

* Fix & Complete wrapper functions

* Finalizing wrapper methods & test

* Adding Stream Datas

* WS Test functions

* CI: Fix golangci-lint linter issues

* CI: Fix reverting unnessesary changes and type conversion issues

* CI: Fix reverting unnessesary changes and type conversion issues

* Adding Public endpoints and tests

* Adding Market and Public Endpoints

* Adding Public endoints

* Public Trading Endpoints & Authenticated Trade order methods

* Adding Authenticated Methods and Tests

* Adding algo and Funding Authenticated endpoints

* Adding funding trading endpoints and correspondint tests

* adding authenticated endpoints

* Completing Block Trading endpoints and added subaccount endpoints

* Completing sub account and grid Trading endpoints

* Adding Rate Limit and missing endpoints

* Wrapper and Websocket handlers

* Fixing Websocket Test and Push Data Handler Issues

* Fixing Websocket Test and Push Data Handler Issues

* Fixing linter issues, package dependency, and other slight tempos

* Fixing linter and slight tempos

* Update on test functions, and Rest and Websocket Endpoint handlers

* Remove okex, adding comments, and slight fixes on endpoints.

* Fixing linter issues and adding comments

* Slight code changes, updating documentation, and n and linter issues

* Fix context and configuration endpoint issues

* slight fixes on config and test files

* adding some missing test and fix linter issues

* fix linter issue

* Slight fixes on code structure, shorthand exp,and ot and other

* Fix slight linter issue

* Slight code fixes and fixing linter issues

* fixing linter iissues

* fixing linter iissues

* slight linter issue fix

* slight linter issue fix

* Fix on models, type convert funcs and endpoints

* Adding Error messages map and update of models

* Fix on error message string and linter issues

* Fix slight linter issue

* Fix slight linter issue

* Fixing type converts, models, and linter issues

* Adding Ws fixes

* Slight fix on websocket and other issues

* Adding slight websocket fixes

* Remove 'books5' channel default subscription

* Small changes on default subscription and checksum

* Fix slight websocket tempos

* Fix Wrapper function tempost and linter issues

* Resolving slight naming and other issues

* Resolve slight pointer issues

* resolve slight linter issues

* Resolve config files issue

* Update websocket and wrapper funcs with test and docs

* fixs on websocket multiplexer, types, and other slight issues

* fix slight linter issues

* slight update on web-socket orderbook and tickers

* fix slight issues and websocket runtime errors

* Slight unit test fix and assing simple semaphore

* FIx race issue

* Update on authenticated endpoints

* Fix wsSetupRun check in websocket 'setupWsAuth' func

* Update wsSetupRun check in websocket 'setupWsAuth' func

* Slight update on websocket handling

* Fix some race conditions

* fix slight tempos

* fix authenticated test issues

* Update on conditional statements

* slight update on unit test

* fix unit test tempos

* Fix slight tempos

* Change check map from struct valued to bool valued

* slight fix trial

* Slight unit test update

* Fix websocket timeout error

* Updating websocket subscription endpoints, and unit tests

* update unit tests

* Slight issue on wrapper method 'GetActiveOrders'

* Overall code update

* Addressing missing review comments

* Fix unit test tempo and linter issue

* Monor fix

* Slight update

* Slight unit test fix

* Slight fixes

* Slight fixes

* Fixing on missing review comments

* Adding WS Fixes

* slight fix

* Monor fix on unit test

* Minor convert issue

* Minor change on WS

* Monor logic fix

* Fix code structure and logic issues

* Fixing small typos

* fix slight data format issue

* Update on trade and order wrapper methods

* Adding slight update

* fix on order detail

* Slight update on FetchTradablePairs wrapper method

* Slight update on wrapper

* Update on deserialization and other slight issues

* Final update

* Resolve missing review comments

* Slight update on config and unit test

* minor fix on GetDepositAddress param

* Minor fix
2022-12-09 11:44:29 +11:00
Ryan O'Hara-Reid
4e135c9590 logger: reduce go routine generation (#992)
* logger: reduce go routine generation

* logger: shift most of processing and prep work to the worker pool, add pool for fields because each log we are pushing the struct to the heap, has better segregation now and includes a buffer in scope instead of relying on a pool

* logger: shift fmt package calls to worker pool

* logger: conform tests to new design

* linter: fix issues

* Update log/logger_test.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update log/logger_test.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* UN-GLORIOUS: nits

* logger: Handle config variable

* logger: NITERINOS BY GLORIOUS CODE

* logger: revert

* glorious: nits

* Panic at the disco: fix

* Panic at the disco: fix

* logger: make sure logger closed and job channel emptied on start up error

* fix tests

* logger: reduce globals

* logger: finished reduces globals, reduce workers to one too keep everything in line.

* logger: remove comments

* logger/exhchange: linter issues

* db/test: fix linter

* logger: add tests shift wait before unlock

* logger: consolidate worker code; fix linter issue and make sure we can sustain writing for external testing.

* logger: fix race and warn for conflict in config

* logger: fix name and add to tests

* logger: remove zero value field

* glorious: panic fix and removal of code

* logger: reinstate channels in close

* logger: shift reinstate processing to SetupGlobalLogger

* logger: segregate config.json from internal log.Config

* logger: fix silly mistake that is silly

* engine: Add protection for nil issues and implement new constructor in tests

* logger: Force singular mutex usage throughout package, throw away funcs that are not used outside of this package, unexport a bunch. Fix tests.

* logger: actually set advanced settings

* Update log/loggers.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update log/loggers.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update log/loggers.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update log/loggers.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update log/logger_multiwriter.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* logger: test issue when not purging temp file and contents

* loggertest: add more protections for the panics

* linter: fix

* glorious: nits

* cleanup

* logger: linter fix

* linter: fix(?) :/

* linter: revert change

* linter: fix

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
2022-10-24 11:46:18 +11:00
E Sequeira
625020c6e8 ftx: fix null currency.Pair UnmarshalJSON (#1049)
* fix ftx unmarshal

* add tests

* fix ineffectual assignment to err

* Update currency/pair_test.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
2022-09-29 10:01:18 +10:00
Ryan O'Hara-Reid
f843b7d277 exchange: upgrade UpdatePair method (#991)
* exchange: upgrade update pair

* exchanges: Add enabled string matching and format handling if discrepency is found.

* linter: fixes

* bithumb: fix tests

* BTSE: api change fix ordering

* huobi: fix tests

* gloriousnits: stage 1

* gloriousnits: stage 2

* currency: more nits

* bitmex: add spot and process pairs before currency package call.

* bitmex: finished correct orderbook matching and other implementations

* linter: fix issue

* currency: Fix linter

* currency: segregate and protect pair store, update tests

* currency/manager: clean code, rm log output

* currency: Add store method and make sure formatting stays nil if not stored.

* gct: check errors

* engine/websocketroutineman: fix tests

* bybit: fix duplication bug

* huobi: fix test

* btse: fix tests?

* ob/buffer: fix tests

* Update currency/manager.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* glorious: nits strikes again.

* exchange: add bypassConfigFormatUpgrades to stop formatting

* GLORIOUS LINTER

* Update exchanges/bithumb/bithumb_wrapper.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* exchange: fix pair upgrade issue when duplications are in both avail and enabled pairs

* linter: fix shadow dec

* config: fix test

* Update currency/pair_test.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2022-09-16 08:59:27 +10:00