139 Commits

Author SHA1 Message Date
Gareth Kirwan
73e200e4e7 accounts: Move to instance methods, fix races and isolate tests (#1923)
* Bybit: Fix race in TestUpdateAccountInfo and  TestWSHandleData

* DriveBy rename TestWSHandleData
* This doesn't address running with -race=2+ due to the singleton

* Accounts: Add account.GetService()

* exchange: Assertify TestSetupDefaults

* Exchanges: Add account.Service override for testing

* Exchanges: Remove duplicate IsWebsocketEnabled test from TestSetupDefaults

* Dispatch: Replace nil checks with NilGuard

* Engine: Remove deprecated printAccountHoldingsChangeSummary

* Dispatcher: Add EnsureRunning method

* Accounts: Move singleton accounts service to exchange Accounts

* Move singleton accounts service to exchange Accounts

This maintains the concept of a global store, whilst allowing exchanges
to override it when needed, particularly for testing.

APIServer:

* Remove getAllActiveAccounts from apiserver

Deprecated apiserver only thing using this, so remove it instead of
updating it

* Update comment for UpdateAccountBalances everywhere

* Docs: Add punctuation to function comments

* Bybit: Coverage for wsProcessWalletPushData Save
2025-10-28 13:52:45 +11:00
Ryan O'Hara-Reid
dcf98ec700 exchanges/order, GateIO: Update USDT margined futures pathway for cancel all orders (#2021)
* order/gateio: update USDT margined futures pathway for cancel all orders and drop count field

* gateio: add and expand tests for CancelAllOrders and getExchangeSide

* Add test for load

* linter: fix

* Update exchanges/kraken/kraken_wrapper.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* Update exchanges/order/orders.go

Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>

* gk: nits

* glorious: nits

* reverted change for options

* Update exchanges/gateio/gateio_wrapper_test.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/gateio/gateio_wrapper.go

Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>

* Update exchanges/deribit/deribit_wrapper.go

Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>

* Update exchanges/gateio/gateio_wrapper_test.go

Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>

* Update exchanges/gateio/gateio_wrapper_test.go

Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>

* Add consts for cancel side references

* Update exchanges/gateio/gateio_wrapper.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update exchanges/gateio/gateio_wrapper.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update exchanges/gateio/gateio_websocket_request_futures.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* thrasher-: nits

---------

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
2025-10-02 15:50:51 +10:00
Ryan O'Hara-Reid
9f8b783c20 exchanges: Refactor UpdateTradablePairs method to remove forceUpdate parameter (#2043)
* Refactor UpdateTradablePairs method to remove forceUpdate parameter

- Updated the signature of UpdateTradablePairs in multiple exchange wrappers to remove the forceUpdate boolean parameter.
- Adjusted related test cases to reflect the change in method signature.
- Ensured that the UpdatePairs method calls within UpdateTradablePairs no longer reference the removed parameter.

* update exchange wrapper template

* linter: fix

* glorious: nits

* thrasher/glorious: nits

* Update exchanges/exchange_test.go

Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>

* Update exchanges/exchange_test.go

Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>

* Update exchanges/exchange_test.go

Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>

* fix things

* misc: fix

* Update exchanges/exchange_test.go

Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>

---------

Co-authored-by: shazbert <ryan.oharareid@thrasher.io>
Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>
2025-09-22 12:20:23 +10:00
cranktakular
fd9aaf00a2 Coinbase: Update exchange implementation (#1480)
* Slight enhance of Coinbase tests

Continual enhance of Coinbase tests

The revamp continues

Oh jeez the Orderbook part's unfinished don't look

Coinbase revamp, Orderbook still unfinished

* Coinbase revamp; CreateReport is still WIP

* More coinbase improvements; onto sandbox testing

* Coinbase revamp continues

* Coinbase revamp continues

* Coinbasepro revamp is ceaseless

* Coinbase revamp, starting on advanced trade API

* Coinbase Advanced Trade Starts in Ernest

V3 done, onto V2

Coinbase revamp nears completion

Coinbase revamp nears completion

Test commit should fail

Coinbase revamp nears completion

* Coinbase revamp stage wrapper

* Coinbase wrapper coherence continues

* Coinbase wrapper continues writhing

* Coinbase wrapper & codebase cleanup

* Coinbase updates & wrap progress

* More Coinbase wrapper progress

* Wrapper is wrapped, kinda

* Test & type checking

* Coinbase REST revamp finished

* Post-merge fix

* WS revamp begins

* WS Main Revamp Done?

* CB websocket tidying up

* Coinbase WS wrapperupperer

* Coinbase revamp done??

* Linter progress

* Continued lint cleanup

* Further lint cleanup

* Increased lint coverage

* Does this fix all sloppy reassigns & shadowing?

* Undoing retry policy change

* Documentation regeneration

* Coinbase code improvements

* Providing warning about known issue

* Updating an error to new format

* Making gocritic happy

* Review adherence

* Endpoints moved to V3 & nil pointer fixes

* Removing seemingly superfluous constant

* Glorious improvements

* Removing unused error

* Partial public endpoint addition

* Slight improvements

* Wrapper improvements; still a few errors left in other packages

* A lil Coinbase progress

* Json cleaning

* Lint appeasement

* Config repair

* Config fix (real)

* Little fix

* New public endpoint incorporation

* Additional fixes

* Improvements & Appeasements

* LineSaver

* Additional fixes

* Another fix

* Fixing picked nits

* Quick fixies

* Lil fixes

* Subscriptions: Add List.Enabled

* CoinbasePro: Add subscription templating

* fixup! CoinbasePro: Add subscription templating

* fixup! CoinbasePro: Add subscription templating

* Comment fix

* Subsequent fixes

* Issues hopefully fixed

* Lint fix

* Glorious fixes

* Json formatting

* ShazNits

* (L/N)i(n/)t

* Adding a test

* Tiny test improvement

* Template patch testing

* Fixes

* Further shaznits

* Lint nit

* JWT move and other fixes

* Small nits

* Shaznit, singular

* Post-merge fix

* Post-merge fixes

* Typo fix

* Some glorious nits

* Required changes

* Stop going

* Alias attempt

* Alias fix & test cleanup

* Test fix

* GetDepositAddress logic improvement

* Status update: Fixed

* Lint fix

* Happy birthday to PR 1480

* Cleanups

* Necessary nit corrections

* Fixing sillybug

* As per request

* Programming progress

* Order fixes

* Further fixies

* Test fix

* Pre-merge fixes

* More shaznits

* Context

* Sonic error handling

* Import fix

* Better Sonic error handling

* Perfect Sonic error handling?

* F purge

* Coinbase improvements

* API Update Conformity

* Coinbase continuation

* Coinbase order improvements

* Coinbase order improvements

* CreateOrderConfig improvements

* Managing API updates

* Coinbase API update progression

* jwt rename

* Comment link fix

* Coinbase v2 cleanup

* Post-merge fixes

* Review fixes

* GK's suggestions

* Linter fix

* Minor gbjk fixes

* Nit fixes

* Merge fix

* Lint fixes

* Coinbase rename stage 1

* Coinbase rename stage 2

* Coinbase rename stage 3

* Coinbase rename stage 4

* Coinbase rename final fix

* Coinbase: PoC on converting to request structs

* Applying requested changes

* Many review fixes, handled

* Thrashed by nits

* More minor modifications

* The last nit!?

---------

Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>
2025-09-16 13:37:00 +10:00
Scott
85403fe801 exchange/order/limits: Migrate to new package and integrate with exchanges (#1860)
* move limits, transition to key gen

* rollout NewExchangePairAssetKey everywhere

* test improvements

* self-review fixes

* ok, lets go

* fix merge issue

* slower value func,assertify,drop IsValidPairString

* remove binance reference for backtesting test

* Redundant nil checks removed due to redundancy

* Update order_test.go

* Move limits back into /exchanges/

* puts limits in a different box again

* SHAZBERT SPECIAL SUGGESTIONS

* Update gateio_wrapper.go

* fixes all build issues

* Many niteroos!

* something has gone awry

* bugfix

* gk's everywhere nits

* lint

* extra lint

* re-remove IsValidPairString

* lint fix

* standardise test

* revert some bads

* dupe rm

* another revert 360 mcgee

* un-in-revertify

* Update exchange/order/limits/levels_test.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* fix

* Update exchanges/binance/binance_test.go

HERE'S HOPING GITHUB FORMATS THIS CORRECTLY!

Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>

* update text

* rn func, same line err gk4202000

---------

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>
2025-08-26 12:30:21 +10:00
Adrian Gallagher
7ebc392532 build/linters: Bump Go to v1.25 and golangci-lint to v2.4.0 (#2005)
* build/linters: Bump Go to v1.25 and golangci-lint to v2.4.0

* refactor: Update TODO comments for net.Listen and net.DialTimeout; improve variable naming in websocket and exchange methods

* refactor: Rename massageMissingData to backfillMissingData for clarity and update references in RSI and MFI calculations

* fix: Correct typo in TODO comment for net.Listen in RPC server
2025-08-20 11:55:15 +10:00
Adrian Gallagher
d5ba674fc4 codebase: Cleanup various things (#1935)
* codebase: Rid base64/hex to string common funcs

* codebase: Rid local scope variable usage and other improvements

* codebase: Refactor currency pair usage across multiple exchanges

- Updated HitBTC tests to use the new currency pair format.
- Modified Kraken futures types to use currency.Pair instead of string for Symbol.
- Adjusted Kraken wrapper methods to handle currency pairs correctly.
- Refined OKX tests and types to utilize currency.Pair for instrument IDs.
- Enhanced Poloniex tests to consistently use predefined currency pairs.
- Streamlined order and orderbook tests to replace string pairs with currency.NewBTCUSD().
- Improved Yobit tests to utilize a standardized currency pair format.
- Updated validator wrapper to use currency pairs directly instead of string conversions.

* codebase: Use types.Number where possible

* refactor: update PayoutFee type to types.Number for consistency

* Refactor: Remove crypto functions to use standard library and other minor changes

- Removed custom crypto functions for SHA256, SHA512, and MD5 from the common/crypto package.
- Replaced usages of removed functions with standard library implementations in various files including:
  - cmd/websocket_client/main.go
  - engine/apiserver.go
  - exchanges/kraken/kraken.go
  - exchanges/lbank/lbank.go
  - exchanges/okx/okx_business_websocket.go
  - exchanges/kucoin/kucoin_websocket.go
  - gctscript/vm/vm.go
- Updated tests to reflect changes in the crypto functions.
- Renamed several functions for clarity, particularly in the context of order book updates across multiple exchanges.

* refactor: replace assert with require for consistency in test assertions

* refactor: Improve Binance futures candlestick test, standardise orderbook update function names and improve test parallelism

* refactor: Replace require.Len with require.Equal for better output in TestGetFuturesKlineData
2025-06-12 14:12:36 +10:00
Adrian Gallagher
a5b638bfb7 GHA: Add additional checks for common issues (#1922)
* GHA, tests: Add additional checks for common issues

These checks include:
- Ensuring that all testify funcs use their formatted variants (e.g., `assert.Equalf(t, expected, actual)` instead of `assert.Equal(t, expected, actual)`).
- Replacing `%s` with %q
- Enforcing consistent usage of should/must wording for testify assert/require messages

* Add support for checking backticked string format specifiers and fix issues

* tests: Fix error comparisons

* tests: Replace errors.Is(err, nil) usage with testify and automate check

* refactor: Rename ExtractPort to ExtractPortOrDefault

* tests: Replace assert with require for error handling in multiple test files

* tests: Replace assert with require for error handling and improve assertions in data tests

* tests: Fix typo in assertion message for StreamVol test

* OKX: Fix GetOpenInterestAndVolumeStrike test with instrument selection and improved assertions

* OKX: Revert intentional error check

* Improve error message for expiry time check in GetOpenInterestAndVolumeStrike test
2025-05-28 12:26:51 +10:00
Gareth Kirwan
c2d876d8b0 Exchanges: Remove WalletAddress (#1882)
* Exchanges: Remove cancel order walletAddress

* Order: Refactor TestMatchFilter

TestMatchFilter had inconsistent testing of empty values, and was
painful when a field was removed due to index methodology.
This should provide equal test clarity, but improve maintainability and
improve coverage on empty values.
2025-04-30 17:10:36 +10:00
dependabot[bot]
179519a3c3 golangci-lint: Bump to v2.0.2 and fix issues (#1871)
* build(deps): Bump golangci/golangci-lint-action from 6 to 7

Bumps [golangci/golangci-lint-action](https://github.com/golangci/golangci-lint-action) from 6 to 7.
- [Release notes](https://github.com/golangci/golangci-lint-action/releases)
- [Commits](https://github.com/golangci/golangci-lint-action/compare/v6...v7)

---
updated-dependencies:
- dependency-name: golangci/golangci-lint-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* golangci-lint: Bump to v2 and fix issues

* refactor: remove no longer need ifshort nolint directives, fix test grammar and improve for loop logic

* nits: update order pair handling in tests and improve string replacement logic

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
2025-04-02 15:36:05 +11:00
Jun
1b7fa2259a account: refactor for save holdings (#1846)
* account: refactor for save holdings

Signed-off-by: Ye Sijun <junnplus@gmail.com>

* Update exchanges/account/account.go

Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>

* Update exchanges/account/account.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* account: minor changes

Signed-off-by: Ye Sijun <junnplus@gmail.com>

---------

Signed-off-by: Ye Sijun <junnplus@gmail.com>
Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2025-04-02 12:32:23 +11:00
Adrian Gallagher
4651af5767 modernise: Run new gopls modernise tool against the codebase and fix minor issues (#1826)
* modernise: Run new gopls modernise tool against codebase

* Address shazbert's nits

* apichecker, gctcli: Simplify HTML scraping functions and improve depth limit handling

* refactor: Create minSyncInterval const and update order book limit handling for binance and binanceUS

* refactor: Various slice usage improvements and rename TODO

* tranches: Revert deleteByID changes due to performance decrease

Shazbert was a F1 driver in a past lifetime 🏎️

* tranches: Simply retrieve copy

Thanks to shazbert

* documentation: Sort contributors list by contributions

* tranches: Remove deadcode in deleteByID
2025-03-21 09:17:10 +11:00
Jun
4474278053 exchanges/account: add UpdatedAt field to Balance and ProtectedBalance (#1827)
* add updatedAt for account balance

Signed-off-by: Ye Sijun <junnplus@gmail.com>

* add nil pointer test for load

Signed-off-by: Ye Sijun <junnplus@gmail.com>

* account: set default updatedAt to balance for Update

Signed-off-by: Ye Sijun <junnplus@gmail.com>

* engine: add UpdatedAt for account info

Signed-off-by: Ye Sijun <junnplus@gmail.com>

* account: force updatedAt for load balance

Signed-off-by: Ye Sijun <junnplus@gmail.com>

* minor change for test

Signed-off-by: Ye Sijun <junnplus@gmail.com>

---------

Signed-off-by: Ye Sijun <junnplus@gmail.com>
2025-03-13 13:57:14 +11:00
Adrian Gallagher
d64d56f77c build/ci: Update Go to v1.24, golangci-lint to v1.64.6 and fix issues (#1804)
* build/ci: Update Go to v1.24, golangci-lint to v1.64.5 and fix issues

* Address shazbert's nitters

* linter/config: Fix new linter issue and use versionSize const

* Address gk's nitters and fix additional linter issue after rebase

* Address glorious nits

* staticcheck: Fix additional linter issues after upgrading to Go 1.24.1 and golangci-lint v1.64.6

Also addresses nits

* Improve testing, assertify usage and use common.ErrParsingWSField

* TestCreateNewStrategy: Replace must > should wording
2025-03-10 16:33:55 +11:00
Ryan O'Hara-Reid
e99adca86f encoding/json: Add custom JSON package with build tag support for Sonic (#1623)
* tag optional sonic and allow full library conversion

* Add workflow and disallow arm and darwin usage

* Add basic hotswap benchmark

* linter: fix

* use bash

* linter: fix?

* Fix whoopsie, add to make file, also add mention in features list.

* test enforcement

* actually read documentation see if this works

* linter: fix

* linter: fix

* sonic: bump tagged version

* encoding/json: drop build tag arch and os filters

* encoding/json: consolidate tests

* encoding/json: log build tag usage

* rm superfluous builds

* glorious/nits: add template change and regen docs

* glorious/nits: update commentary on nolint directive

* glorious/nits: rm init func and log results in main.go

* Test to actually pull flag in

* linter: fix

* thrasher: nits

* gk: nits 4 goflags goooooooooo!

* gk: nits rn

* make sonic default json implementation

* screen 386

* linter: fix

* Add commentary

* glorious: nits Makefile not working

* gk: nits

* gk: nits whoops

* whoopsirino

* mention 32bit systems won't be sonic

* gk: super-duper nit of extremes

---------

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
2025-02-20 16:05:55 +11:00
Ryan O'Hara-Reid
08e015a125 exchanges/wrappers: Refactor fetch orderbook/ticker/account info funcs (#1440)
* acrost: Pull thread, examine

* fix tests

* linter

* fix_linter

* revert rm ctx param to limit breakages when merging usptream

* linter fix

* Add in priority update grouping so that tests pass

* Update cmd/exchange_wrapper_standards/exchange_wrapper_standards_test.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious nits

* fixed spelling

* whoopsie

* aanother whoops

* glorious: NITTERS!

* glorious: further nitters

* srry linter gods

* glorious: nits continued

* sub test p ara lel

* drop main t.Parallel

* fix whoops

* wrappertests: use context with cancel (test)

* linter: fix

* ensure primary execution

* kucoin test fix

* revert standards test changes and bypass non critical errors

* rm single override

* wrap exported error for accounts

* thrasher: nits ch name

* gk: nits

* gk: nits FetchTickerCached -> GetCachedTicker

* gk: nits rn FetchOrderbookCached -> GetCachedOrderbook

* gk: nits rn FetchAccountInfoCached -> GetCachedAccountInfo

* linter: fix

* gk: nits

* thrasher: nitters 1

* thrasher: nitters tmpls

* gk: nitter

---------

Co-authored-by: shazbert <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2025-02-19 10:47:10 +11:00
Gareth Kirwan
4c7f48ae0e GateIO: Fix GetFuturesContractDetails for delivery futures and minor other fixes (#1766)
* GateIO: Fix GetFuturesContractDetails for Deliveries

Was returning the product of all the contracts, so 1444 instead of 38
contracts.

* GateIO: Fix GetOpenInterest returning asset.ErrNotEnabled

Using wrong error for pair not enabled

* GateIO: Rename GetSingleContract and GetSingleDeliveryContracts

Especially fixes GetSingleContract, which seems misleading to not say
Futures.
There's a load of `GetSingle*` here that should probably also be fixed,
but these two justified a dyno

* GateIO: Rename GateIOGetPersonalTradingHistory to GetMySpotTradingHistory

* GateIO: Rename GetMyPersonalTradingHistory to GetMyFuturesTradingHistory

* GateIO: Remove duplicate DeliveryTradingHistory

* GateIO: Rename Get*PersonalTradingHistory to GetMy*TradingHistory

* Linter: Disable shadow linting for err

It's been a year, and I'm still getting caught out by govet demanding I
don't shadow a var I was deliberately shadowing.
Made worse by an increase in clashes with stylecheck when they both want
opposite things on the same line.

* GateIO: Add missing Futures and tradinghistory fields

* GateIO: Improve WS Header parsing

This unifies handling for time_ms and time in response headers, since
options and delivery have only time, but spot has time_ms as well.
We use the better of the two results.

Also [improves performance 2x](https://gist.github.com/gbjk/7cacb63b9a256e745534bb05ca853c48)

* GateIO: Use time_ms WS fields where available

Removes the deprecated _time json fields and populates our Time fields
with the time_ms values
2025-01-14 15:19:17 +11:00
Ryan O'Hara-Reid
ac731ce283 websocket/gateio: Support multi connection management and integrate with GateIO (#1580)
* gateio: Add multi asset websocket support WIP.

* meow

* Add tests and shenanigans

* integrate flushing and for enabling/disabling pairs from rpc shenanigans

* some changes

* linter: fixes strikes again.

* Change name ConnectionAssociation -> ConnectionCandidate for better clarity on purpose. Change connections map to point to candidate to track subscriptions for future dynamic connections holder and drop struct ConnectionDetails.

* Add subscription tests (state functional)

* glorious:nits + proxy handling

* Spelling

* linter: fixerino

* instead of nil, dont do nil.

* clean up nils

* cya nils

* don't need to set URL or check if its running

* stop ping handler routine leak

* * Fix bug where reader routine on error that is not a disconnection error but websocket frame error or anything really makes the reader routine return and then connection never cycles and the buffer gets filled.
* Handle reconnection via an errors.Is check which is simpler and in that scope allow for quick disconnect reconnect without waiting for connection cycle.
* Dial now uses code from DialContext but just calls context.Background()
* Don't allow reader to return on parse binary response error. Just output error and return a non nil response

* Allow rollback on connect on any error across all connections

* fix shadow jutsu

* glorious/gk: nitters - adds in ws mock server

* linter: fix

* fix deadlock on connection as the previous channel had no reader and would hang connection reader for eternity.

* gk: nits

* Leak issue and edge case

* gk: nits

* gk: drain brain

* glorious: nits

* Update exchanges/stream/websocket.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* add tests

* linter: fix

* After merge

* Add error connection info

* Fix edge case where it does not reconnect made by an already closed connection

* stream coverage

* glorious: nits

* glorious: nits removed asset error handling in stream package

* linter: fix

* rm block

* Add basic readme

* fix asset enabled flush cycle for multi connection

* spella: fix

* linter: fix

* Add glorious suggestions, fix some race thing

* reinstate name before any routine gets spawned

* stop on error in mock tests

* glorious: nits

* glorious: nits found in CI build

* Add test for drain, bumped wait times as there seems to be something happening on macos CI builds, used context.WithTimeout because its instant.

* mutex across shutdown and connect for protection

* lint: fix

* test time withoffset, reinstate stop

* fix whoops

* const trafficCheckInterval; rm testmain

* y

* fix lint

* bump time check window

* stream: fix intermittant test failures while testing routines and remove code that is not needed.

* spells

* cant do what I did

* protect race due to routine.

* update testURL

* use mock websocket connection instead of test URL's

* linter: fix

* remove url because its throwing errors on CI builds

* connections drop all the time, don't need to worry about not being able to echo back ws data as it can be easily reviewed _test file side.

* remove another superfluous url thats not really set up for this

* spawn overwatch routine when there is no errors, inline checker instead of waiting for a time period, add sleep inline with echo handler as this is really quick and wanted to ensure that latency is handing correctly

* linter: fixerino uperino

* glorious: panix

* linter: things

* whoops

* defer lock and use functions that don't require locking in SetProxyAddress

* lint: fix

* thrasher: nits

---------

Co-authored-by: shazbert <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2024-10-10 15:09:52 +11:00
Adrian Gallagher
225429bda6 CI/build: Update Go version, linters and fix minor issues (#1612)
* CI/build: Update Go version, linters and fix minor issues

* linters: Add intrange, copyloopvar, additional go vet linters to match gopls and fix issues
2024-08-16 17:41:11 +10:00
Gareth Kirwan
1199f38546 subscriptions: Encapsulate, replace Pair with Pairs and refactor; improve exchange support
* Websocket: Use ErrSubscribedAlready

instead of errChannelAlreadySubscribed

* Subscriptions: Replace Pair with Pairs

Given that some subscriptions have multiple pairs, support that as the
standard.

* Docs: Update subscriptions in add new exch

* RPC: Update Subscription Pairs

* Linter: Disable testifylint.Len

We deliberately use Equal over Len to avoid spamming the contents of large Slices

* Websocket: Add suffix to state consts

* Binance: Subscription Pairs support

* Bitfinex: Subscription Pairs support

* Bithumb: Subscription Pairs support

* Bitmex: Subscription Pairs support

* Bitstamp: Subscription Pairs support

* BTCMarkets: Subscription Pairs support

* BTSE: Subscription Pairs support

* Coinbase: Subscription Pairs support

* Coinut: Subscription Pairs support

* GateIO: Subscription Pairs support

* Gemini: Subscription Pairs support and improvement

* Hitbtc: Subscription Pairs support

* Huboi: Subscription Pairs support

* Kucoin: Subscription Pairs support

* Okcoin: Subscription Pairs support

* Poloniex: Subscription Pairs support

* Kraken: Add subscription Pairs support

Note: This is a naieve implementation because we want to rebase the
kraken websocket rewrite on top of this

* Bybit: Subscription Pairs support

* Okx: Subscription Pairs support

* Bitmex: Subsription configuration

* Fixes unauthenticated websocket left as CanUseAuth
* Fixes auth subs happening privately

* CoinbasePro: Subscription Configuration

* Consolidate ProductIDs when all subscriptions are for the same list

* Websocket: Log actual sent message when Verbose

* Subscriptions: Improve clarity of which key is which in Match

* Subscriptions: Lint fix for HugeParam

* Subscriptions: Add AddPairs and move keys from test

* Subscriptions: Simplify subscription keys and add key types

* Subscriptions: Add List.GroupPairs Rename sub.AddPairs

* Subscription: Fix ExactKey not matching 0 pairs

* Subscriptions: Remove unused IdentityKey and HasPairKey

* Subscriptions: Fix GetKey test

* Subscriptions: Test coverage improvements

* Websocket: Change State on Add/Remove

* Subscriptions: Improve error context

* Subscriptions: Fix Enable: false subs not ignored

* Bitfinex: Fix WsAuth test failing on DataHandler

DataHandler is eaten by dataMonitor now, so we need to use ToRoutine

* Deribit: Subscription Pairs support

* Websocket: Accept nil lists for checkSubscriptions

If the user passes in a nil (implicitly empty) list, we would not panic.
Therefore the burden of correctness about that data lies with them.
The list of subscriptions is empty, and that's okay, and possibly
convenient

* Websocket: Add context to NilPointer errors

* Subscriptions: Add context to nil errors

* Exchange: Fix error expectations in UnsubToWSChans
2024-06-07 11:54:08 +10:00
Scott
f1ff951199 engine/exchanges: Add GetCurrencyTradeURL wrapper func/gRPC endpoint (#1521)
* beginning the concept

* expands testing and implementations

* test standardisation, expansion

* more

* finish remainder, add rpc func

* lint

* grammar? I barely know her!

* wow i forgot something wow wow wow wow

* rm SendPayload

* enFixio!

* improve binance long-dated support

* update test design, update wrapper funcs

* adds kraken futures, adds bybit support

* fixes SIMPLE bugs

* s is for sucks

* fixed option url x2

* fixed silly test
2024-05-03 17:01:17 +10:00
Adrian Gallagher
08da42ddb7 CI/build: Update Go version, linters and fix minor issues (#1473)
* CI/build: Update Go version, linters and fix minor issues

* Bump golangci-lint to v1.56.1

* BinanceUS: Make uint usage consistent

* Throw blank identifiers into the trash
2024-02-14 11:02:06 +11:00
Adrian Gallagher
d57fefbcfc engine/gRPC proxy: Fix mux regression and add test coverage (#1456)
* engine/gRPC proxy: Fix mux regression and enhance test coverage

* Use a temp dir for TLS creds and add credentials test tables

* Update GetRPCEndpoints grpcProxyName ListenAddr field

* Log unauthorised access attempts
2024-02-05 15:22:54 +11:00
Gareth Kirwan
d7818ea956 Exchanges: Remove bespoke pair upgrade handling and abstract Start/Run (#1424)
* Exchanges: Remove Pair upgrade handling

Now redundant behind #1401. These paths should never be met.

Several legacy coin upgrade paths being deprecated as well: ZUSD and CNY
Expecting any users with bad config from 3+ years ago would have to
reset anyway.

Also: At the time the intention of this was to upgrade the config
format.
However now, instead, it'd mostly serve to reset enabled pairs if
there's a config mistake, which doesn't feel right.

* Kraken: Fix typo in Kraken type struct

* Exchanges: Abstract exchange Start() and Run()

* Exchanges: Add test for abstracted Start

* Exchanges: Move Start to Bootstrap

* Simplify waitgroup usage
* Add call to exchange.Bootstrap to allow overide or supplementation

* Exchanges: Concurrent common bootstap actions

* Gateio: Remove incorrect Run in test

* GateIO: Fix pair dependencies in tests

This ensures that the pairs are initialised no more than needed and
kind-of just-in-time.
Better pattern might be to use a function to get these pairs when we
need them.

* Exchanges: Complete UpdatePairs before ExecLims

If we're going to update pairs, it needs to complete before we check for
limits to avoid errors on old pairs

* Exchanges: Remove Start and Run from tmpl

Since they're replaced by bootstrap now and shouldn't need customisation
normally

* Alphapoint: Move Start to Bootstrap

* GateIO: Fix linter shadow var
2024-01-31 19:29:36 +11:00
Gareth Kirwan
e007f69f7c exchanges/websocket: Implement subscription configuration (#1394)
* Websockets: Move Subscription to its own package

This allows the small type to be imported from both `config` and from
`stream` without an import cycle, so we don't have to repeat ourselves

* Subs: Renamed Currency to Pair

This was being mis-used through much of the code, and since we're
already touching everything, we might as well fix it

* Websockets: Add Subscription configuration

* Binance: Add subscription configuration

* Kucoin: Subscription configuration

* Simplify GenerateDefaultSubs
* Improve TestGenSubs coverage
* Test Candle Sub generation
* Support Candle intervals
* Full responsibility for formatting Channel name on GenerateDefaultSubs
  OR consumer of Subscribe
* Simplify generatePayloads as a result
* Fix test coverage of asset types in processMarketSnapshot

* Exchanges: Abstract ParallelChanOp

* Tests: Generic ws mock instances

* Kucoin: Fix intermittent conflict in test currs

Use isolated test instance for `TestGetOpenInterest`.

`TestGetOpenInterest` would occassionally change pairs before
GenerateDefault Subs.
2024-01-24 15:54:07 +11:00
Scott
b71bf1f3d1 exchanges/futures: Implement open interest (#1417)
* adds open interest to exchanges

* ADDS TESTING YEAH

* New endpoints, BTSE, RPCS, cached

* slight design change, begin gateio

You will need to get cached for
each exchange that supports it

* gateio, huobi, rpc

* fix up kraken, cache retrieval

* okx, gateio

* finalising all implementations and tests

* definitely my final ever commit on this

* Well, well, well

* final v2

* quick fix of bug

* test coverage, assert notempty, test helper

Added a new testhelper for currency
management because its very annoying
in a parallel test setting which wastes
so much space otherwise

* minimises REST requests for Open Interest

* types.Number merge misses

* Minimises Kraken REST calls

* len change, value -> pointer receiver

* further fixup

* fixes gateio, batch calculates open interest

* single gateio, lint const fixes

* rejig and more thorough oi for huobi

* formatting expansion

* minor fix for handling expiring contracts

* rm unused Binance strings

* add bybit support, fix bybit issues

* oopsie doopsie, dont look at my whoopsie

* Fix issue, remove feature

* move an irrelevant function for the pr

* mini bybit upgrades

* fixes cli request bug
2024-01-12 15:27:35 +11:00
Scott
70690d9a04 futures: Implement GetLatestFundingRates across exchanges (#1339)
* adds funding rate implementations and improvements

* merge fixes x1

* lint

* kucoin funding rates func make

* migrate sync-manager to keys

* some kucoin work

* adds some kucoin wrapper funcs

* ehhh, todo

* kucoin position

* start of orders

* adds the kucoin tests yay

* multiplier

* nits, EWS includes order limits

* NotYetImplemented, IsPerp improvements, cleaning

* lint, test fix, huobi time

* fixes issues, improves testing

* fixes linters I WRECKED

* local lint but remote lint, lint, lint, lint

* fixes err

* skip CI

* lint

* Supported rates, binance endpoints

* fixes weird mocktest problems

* no, CZ is invalid

* fixes some new EWS test errors
2023-11-03 11:01:32 +11:00
Ryan O'Hara-Reid
ceef7a14e0 currency: Adds matching lookup table built from available pairs (#1312)
* currency: Add pair matching update (cherry-pick)

* exchange/currency: Add tests and update func

* linter fix, also if using json unmarshal functionality stop usage of string conversion without delimiter

* gemini: fix test

* currency/manager: potential optimisation

* exchanges: purge derive from wrapper cases and add warning comment

* glorious: nits

* glorious: nits

* linter: fix

* glorious: nits

* whoops

* whoops

* glorious: nits continued

* glorious: diff THANKS!

* hitbtc: fix update tradable pairs strings splitting. continue if not enabled tickers update pair.

* glorious: nits

* linter: fix

* Update exchanges/exmo/exmo_wrapper.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* bitstamp: fix test when 32 biterinos architecturinos

* capture more strings for speed

* swapsies because whos running 32bit \0/?

---------

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2023-10-18 11:57:27 +11:00
Scott
7f0faf7850 futures: add GetFuturesContractDetails wrapper function (#1274)
* all in a days work

* cleanup

* cleanup for real, also stop it binance.json

* minor coverage

* adds gateio to the slurry

* cleanup of types

* verbose verbose verbose verbose verbose verbose

* fixes huobi parsing issue

* fix bybit contract identification

* cleanup

* merge fixes

* addresses many big problems raised by SHAZ

* tracking errors and fixes

* funding rate if avail, fixes currency formatting

* Addresses nits and sneaks in extra fixes

* lint

* minor fixes after rebase

* better contract splitter for currencies like T-USDT

* forgot to add the exchange name like a fool

* merge fixes x1

* kucoin, direction, contract size

* rn direction, fix kucoin time

* WHOOPS

* Update exchanges/kucoin/kucoin_wrapper.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* misdirection

---------

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
2023-10-03 16:04:45 +11:00
Scott
5f2f6f884b Binance,Okx: Add Leverage, MarginType, Positions and CollateralMode support (#1220)
* init

* surprise train commit

* basic distinctions

* the terms of binance are confusing

* renames and introduction of allocatedMargin

* add new margin funcs

* pulling out wires

* implement proper getposition stuff

* bad coding day

* investigate order manager next

* a broken mess, but a progressing one

* finally completes some usdtmargined stuff

* coinMfutures eludes me

* expand to okx

* imports fix

* completes okx wrapper implementations

* cleans and polishes before rpc implementations

* rpc setup, order manager features, exch features

* more rpc, collateral and margin things

* mini test

* looking at rpc response, expansion of features

* reorganising before the storm

* changing how futures requests work

* cleanup and tests of cli usage

* remove silly client side logic

* cleanup

* collateral package, typo fix, margin err, rpc derive

* uses convert.StringToFloat ONLY ON STRUCTS FROM THIS PR

* fix binance order history bug

* niteroos

* adds new funcs to exchange standards testing

* more post merge fixes

* fix binance

* replace simepletimeformat

* fix for merge

* merge fixes

* micro fixes

* order side now required for leverage

* fix up the rest

* global -> portfolio collateral

* Update exchanges/collateral/collateral_test.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* adds fields and todos

* rm field redundancy

* lint fix oopsie daisy

* fixes panic, expands error and cli explanations (sorry shaz)

* ensures casing is appropriate for underlying

* Adds a shiny TODO

---------

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
2023-09-26 16:16:31 +10:00
Scott
bcabf44b8c gctcli: Add colourful exchange-style rendering to orderbook fetching commands (optional) (#1348)
* fancybook

* fix bug

* oopsie-doodle

* now I remember why we don't use required
2023-09-14 10:10:22 +10:00
Scott
471f4f21c4 Binance,OKx: Implement fetching funding rates (#1239)
* adds basic groundwork for rates on binance

* more into rates on binance

* rm redudant redundancy, add payments

* mini commit before merging with testnet ability branch

* changes function signature and fixes resulting build

* gets billing data too

* funding rates package, features use, testnet reimpl

* new endpoint, refinements and tests

* cli fix, rpc impl, testing, payments

* fixups from looking at code

* typo fix

* niteroos

* merge fixes

* adds test, fixes cli issues

* woah nelly
2023-07-26 14:25:43 +10:00
TaltaM
f48993dfdc [gRPC] Add stream authentication to gRPC server (#1273) 2023-07-20 09:25:34 +10:00
Ryan O'Hara-Reid
81a8b4a575 common: remove SimpleTimeFormat const for time package layout const DateTime (#1246)
* switch over to package defined const for time layout

* bump appveyor playa

* bumperino to latest while setting patherino

* whoooops

* bump VS version set GOROOT

* puge build cache

* Revert "puge build cache"

This reverts commit 315bb578afc19529457f435e52af2172f5143bc5.

* bumperino to test

* purge setting of golang directory for version and allow default

* purge cache state when file change

* whoops

* thrasher: nits

* don't need to flusherino the cacherino

---------

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
2023-07-06 11:17:43 +10:00
Scott
fcc5ad4551 exchanges/qa: Add exchange wrapper testing suite (#1159)
* initial concept of a nice validation tester for exchanges

* adds some datahandler design

* expand testing

* more tests and fixes

* minor end of day fix for bithumb

* fixes implementation issues

* more test coverage and improvements, but not sure if i should continue

* fix more wrapper implementations

* adds error type, more fixes

* changes signature, fixes implementations

* fixes more wrapper implementations

* one more bit

* more cleanup

* WOW things work?

* lintle 1/1337

* mini bump

* fixes all linting

* neaten

* GetOrderInfo+ asset pair fixes+improvements

* adds new websocket test

* expand ws testing

* fix bug, expand tests, improve implementation

* code coverage of a lot of new codes

* fixes everything

* reverts accidental changes

* minor fixes from reviewing code

* removes Bitfinex cancelBatchOrder implementation

* fixes dumb baby typo for babies

* mini nit fixes

* so many nits to address

* addresses all the nits

* Titlecase

* switcheroo

* removes websocket testing for now

* fix appveyor, minor test fix

* fixes typo, re-kindles killed kode

* skip binance wrapper tests when running CI

* expired context, huobi okx fixes

* kodespull

* fix ordering

* time fix because why not

* fix exmo, others

* hopefully this fixes all of my life's problems

* last thing today

* huobi, more like hypotrophy

* golangci-lint, more like mypooroldknee-splint

* fix huobi times by removing them

* should fix okx currency issues

* blocks the application

* adds last little contingency for pairs

* addresses most nits and new problems

* lovely fixed before seeing why okx sucks

* fixes issues with okx websocket

* the classic receieieivaier

* lintle

* adds test and fixes existing tests

* expands error handling messages during setup

* fixes dumb okx bugs introduced

* quick fix for lint and exmo

* fixes nixes

* fix exmo deposit issue

* lint

* fixes issue with extra asset runs missing

* fix surprise race

* all the lint and merge fixes

* fixes surprise bugs in OKx

* fixes issues with times and chains

* fixing all the merge stuff

* merge fix

* rm logs and a panic potential

* lovely lint lament

* an easy demonstration of scenario, but not of initial purpose

* put it in the bin

* Revert "put it in the bin"

This reverts commit 15c6490f713233d43f10957367fcbf18e3818bdd.

* re-add after immediate error popup

* fix mini poor test design

* okx okay

* merge fixes

* fixes issues discovered in lovely test

* I FORGOT TO COMMIT THIS

* nit fixaroonaboo

* forgoetten test fix

* revert old okx asset intrument work

* fixes

* revert problems I didnt understand. update bybit

* fix merge bugs

* test cleanup

* further improvements

* reshuffle and lint

* rm redundant CI_TEST by rm the CI_TEST field that is redundant

* path fix

* move to its own section, dont run on 32 bit + appveyor

* lint

* fix lbank

* address nits

* let it rip

* fix failing test time range

* niteroo boogaloo

* mod tidy, use common.SimpleTimeFormat
2023-07-03 11:09:43 +10:00
TaltaM
db7441c723 dispatch: channel reuse fix (#1237)
* Add test for dispatch channel reuse

* Dispatcher - make chans bidirectional

* No need to to keep the type assertion separate from the Get()

* Unexport Pipe's channel and add getter
2023-06-21 12:33:40 +10:00
Ryan O'Hara-Reid
db8735ec99 log: Add structured logging (#1171)
* basic implementation

* log: deprecate duplicate function, add tests and refine calls.

* linter: fixes

* linter: update struct

* linter and new type

* log tests: update to not lint issue

* linter: stop complaining please

* glorious: nits

* log: rm comment code

* glorious: nits

* glorious: nits

* glorious: nits

* glorious: nits missed

---------

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
2023-05-10 17:52:53 +10:00
Samuael A
969669cb0d linters: Bump golangci-lint version to v1.52.2 and fix issues (#1169)
* fix linter issues

* linter issue suppressing and fixing

* change linter version and linter issues fix

* Bump version
2023-04-27 10:13:46 +10:00
Ryan O'Hara-Reid
d2561402c4 common: update Errors type (#1129)
* common: adjust common error slice to allow multi errors.Is matching and conform to interface better

* zb: forgot to save?

* linties: fixies

* linties: word change as well.

* nitters: glorious

* buts

* nitters: fix glorious bug

* Update common/common.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* nitters: shifty

---------

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2023-02-20 10:48:24 +11:00
Adrian Gallagher
7e08e483fb CI: Bump go version, linters and fix minor issues (#1130)
* CI: Bump go version, linters and fix minor issues

* Bump version, fix loop variables

* Revert

* Rid TODOs now that 1.51 has been released
2023-02-03 15:56:59 +11:00
Adrian Gallagher
a79e0d2b3e qa/spelling: Add Codespell support (#1121)
* Add codespell support

* Fix paths

* Add HTML files to exclusion list
2023-01-30 12:36:56 +11:00
Ryan O'Hara-Reid
83cfefa45c kline/exchanges: automatic creation of unsupported candle intervals (#1091)
* kline: Add builder and testing

* Ideas

* kline: deploy builder functionality across GCT

* exchanges: implement across gct

* exchanges: Add tests and fix implementations before kline package testing and veri.

* kline: Add tests and start to fix ConvertToNewInterval

* kline: fix ConvertToNewInterval add tests

* kline: complete overarching tests now on to exchanges

* kline: finish exchange tests and implement limits

* exchanges: more fixes

* linter: fix

* engine: fix tests

* kraken: fix recent trades and other fixes

* zb: fix tests

* bithumb: fix empty insertion

* kline: refactor/optimize CreateKline function

* kline: remove the mooos!

* kline: prealloc CalculateCandleDateRanges

* linter: fix

* exchanges: prealloc extended

* fix whoopsie

* reverse fix because this is a whoopsie

* okx: fix risidual issues

* linter: fix

* kline: initial nits from @gloriouscode

* kline: rename builder -> request and cascade change

* linter: fix + test

* kline: update forced alignment on start and end times when CreateKlineRequest is called.

* nits: more more more

* NITS: Addressed

* tests: fix race issue

* Update exchanges/kline/request.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* kline: add method AddPadding() to automatically fill in holes in kline.Request functionality and reject if missing data when converting

* kline: Add params start and end to addPadding() to insert blanks in between block

* kline: remove test comment code as it's not needed anymore

* kline: fix lint and test

* kline: sort slice without extra bool check every iteration

* okx: fix issues with timeing and candles and such from niterinos & address typo

* Update exchanges/kline/kline.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: niterinos

* Update exchanges/poloniex/poloniex_wrapper.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits now onto conflicts YAYA!!!

* Update exchanges/exchange_test.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits again

* thrasher: nitters

* thrasher: niterinos - adds partial flag for incomplete recent candles and fetching.

* kline: rm fmtizzle packageizzle

* glorious: nitters

* glorious: more niterinos

* fix last niterinos

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2023-01-17 16:22:33 +11:00
Scott
017cdf1384 Backtester: Live trading upgrades (#1023)
* Modifications for a smoother live run

* Fixes data appending

* Successfully allows multi-currency live trading. Adds multiple currencies to live DCA strategy

* Attempting to get cash and carry working

* Poor attempts at sorting out data and appending it properly with USD in mind

* =designs new live data handler

* Updates cash and carry strat to work

* adds test coverage. begins closeallpositions function

* Updates cash and carry to work live

* New kline.Event type. Cancels orders on close. Rn types

* =Fixes USD funding issue

* =fixes tests

* fixes tests AGAIN

* adds coverage to close all orders

* crummy tests, should override

* more tests

* more tests

* more coverage

* removes scourge of currency.Pair maps. More tests

* missed currency stuff

* Fixes USD data issue & collateral issue. Needs to close ALL orders

* Now triggers updates on the very first data entry

* All my problems are solved now????

* fixes tests, extends coverage

* there is some really funky candle stuff going on

* my brain is melting

* better shutdown management, fixes freezing bug

* fixes data duplication issues, adds retries to requests

* reduces logging, adds verbose options

* expands coverage over all new functionality

* fixes fun bug from curr == curr to curr.Equal(curr)

* fixes setup issues and tests

* starts adding external wallet amounts for funding

* more setup for assets

* setup live fund calcs and placing orders

* successfully performs automated cash and carry

* merge fixes

* funding properly set at all times

* fixes some bugs, need to address currencystatistics still

* adds 'appeneded' trait, attempts to fix some stats

* fixes stat bugs, adds cool new fetchfees feature

* fixes terrible processing bugs

* tightens realorder stats, sadly loses some live stats

* this actually sets everything correctly for bothcd ..cd ..cd ..cd ..cd ..!

* fix tests

* coverage

* beautiful new test coverage

* docs

* adds new fee getter delayer

* commits from the correct directory

* Lint

* adds verbose to fund manager

* Fix bug in t2b2 strat. Update dca live config. Docs

* go mod tidy

* update buf

* buf + test improvement

* Post merge fixes

* fixes surprise offset bug

* fix sizing restrictions for cash and carry

* fix server lints

* merge fixes

* test fixesss

* lintle fixles

* slowloris

* rn run to task, bug fixes, close all on close

* rpc lint and fixes

* bugfix: order manager not processing orders properly

* somewhat addresses nits

* absolutely broken end of day commit

* absolutely massive knockon effects from nits

* massive knockon effects continue

* fixes things

* address remaining nits

* jk now fixes things

* addresses the easier nits

* more nit fixers

* more niterinos addressederinos

* refactors holdings and does some nits

* so buf

* addresses some nits, fixes holdings bugs

* cleanup

* attempts to fix alert chans to prevent many chans waiting?

* terrible code, will revert

* to be reviewed in detail tomorrow

* Fixes up channel system

* smashes those nits

* fixes extra candles, fixes collateral bug, tests

* fixes data races, introduces reflection

* more checks n tests

* Fixes cash and carry issues. Fixes more cool bugs

* fixes ~typer~ typo

* replace spot strats from ftx to binance

* fixes all the tests I just destroyed

* removes example path, rm verbose

* 1) what 2) removes FTX references from the Backtester

* renamed, non-working strategies

* Removes FTX references almost as fast as sbf removes funds

* regen docs, add contrib names,sort contrib names

* fixes merge renamings

* Addresses nits. Fixes setting API credentials. Fixes Binance limit retrieval

* Fixes live order bugs with real orders and without

* Apply suggestions from code review

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update backtester/engine/live.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update backtester/engine/live.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update backtester/config/strategyconfigbuilder/main.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* updates docs

* even better docs

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
2023-01-05 13:03:17 +11:00
Scott
d92ffe6e9e cli: rpc time parsing (#1107)
* fix rpc time parsing

* Update cmd/gctcli/helpers.go

Beautifully handled

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Fixes combined commit

* uses ParseInLocation, parse with timezone, remove helper functions

* lint

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
2022-12-28 12:23:59 +11:00
Ryan O'Hara-Reid
9acbdbf203 depth: Add methods to derive liquidity allowances on orderbooks by volume and slippage (#962)
* depth: methods to derive liquidity impact details

* depth: fix comments on link list methods

* depth: fix whoopsie

* Update exchanges/orderbook/linked_list_test.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* orderbook: standardise methods to GCT math package

* averagePrice: implementation (WIP)

* ll: hmmmmmm

* continued

* orderbook: reworked functions

* WIP

* orderbook: add tests link up with RPC

* orderbook: refined calculations, add tests (WIP)

* orderbook: add tests finalise/verify remove state until next PR if needed

* rpcserver/orderbook: remove redundant type and change wording

* linter: fix

* Update exchanges/orderbook/linked_list.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* gctcli: noobed it up

* orderbook: work work work (yesterday)

* depth: WIP and testing

* orderbook: improve calculations for bids traversal

* orderbook: adjust tests

* orderbook: linters/nits

* orderbook: fix error returns and add asset to whalebomb

* orderbook: drop error when full book is potentially consumed

* Update cmd/gctcli/orderbook.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* orderbook: Add tests and nits

* glorious: nits

* backtester: handle new errors

* grpc: linter

* orderbook: remove pesky t.Log()s

* glorious: nits (yesterday)

* depth/gctcli: Add in purchase requirements into orderbook movement, will also standardize in next commit after tests are fixed.

* orderbook: standardize and overhaul

* orderbook: update comments, update tests

* rpcserver: add average ordercost and amounts

* depth: add spread and imbalance methods

* linter: fix

* glorious: nits

* orderbook: don't purge price, rn test.

* glorious: codes nits

* linter: fix

* Update exchanges/orderbook/linked_list.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update exchanges/orderbook/linked_list.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update exchanges/orderbook/linked_list.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update exchanges/orderbook/linked_list.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update exchanges/orderbook/linked_list.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* thrasher: nits

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
2022-10-14 16:43:37 +11:00
Adrian Gallagher
b50b2e1b21 binance/btse/rpcserver: Specify AnySide/AnyType for order retrieval and fix order index bugs (#1054) 2022-10-10 12:33:28 +11:00
Ryan O'Hara-Reid
f843b7d277 exchange: upgrade UpdatePair method (#991)
* exchange: upgrade update pair

* exchanges: Add enabled string matching and format handling if discrepency is found.

* linter: fixes

* bithumb: fix tests

* BTSE: api change fix ordering

* huobi: fix tests

* gloriousnits: stage 1

* gloriousnits: stage 2

* currency: more nits

* bitmex: add spot and process pairs before currency package call.

* bitmex: finished correct orderbook matching and other implementations

* linter: fix issue

* currency: Fix linter

* currency: segregate and protect pair store, update tests

* currency/manager: clean code, rm log output

* currency: Add store method and make sure formatting stays nil if not stored.

* gct: check errors

* engine/websocketroutineman: fix tests

* bybit: fix duplication bug

* huobi: fix test

* btse: fix tests?

* ob/buffer: fix tests

* Update currency/manager.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* glorious: nits strikes again.

* exchange: add bypassConfigFormatUpgrades to stop formatting

* GLORIOUS LINTER

* Update exchanges/bithumb/bithumb_wrapper.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* exchange: fix pair upgrade issue when duplications are in both avail and enabled pairs

* linter: fix shadow dec

* config: fix test

* Update currency/pair_test.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2022-09-16 08:59:27 +10:00
Scott
1461cba363 backtester: standalone application (#988)
* Ramshackle early leads to GRPC backtester

* Adds GRPC server, default config generation

* Partial support for GRPC backtester config

* Update to use Buf, merge fixes

* Full config for GRPC

* Adds new commands, causes big panic

* Fixes panics

* Setup for the future

* Docs update

* test

* grpc tests

* Fix merge issues. Lint and test

* minor fixes after rebase

* Docs, formatting and main fixes

* Change buf owner

* shazNits

* test-123

* rpc fixes

* string fixes

* Removes --singlerun flag and just relies on --singlerunstrategypath

* fixes test

* initial post merge compatability fixes

* this actually all seems to work? unexpected

* adds pluginpath to config

* rm unused func. add gitignore

* rm unused func. add gitignore

* lintle

* tITLE cASE lOG fIX,rm auth package, gitignore, tmpdir fix

* buf updates + gen. go mod tidy

* x2

* Update default port, update error text
2022-09-08 16:22:30 +10:00
Scott
46cadd6f15 FTX: Funding rates, payments & stats + order manager tracking (#976)
* Adds basic PoC for calculating/retrieving position data

* A very unfortunate day of miscalculations

* Adds position summary and funding rate details to RPC

* Offline funding rate calculations

* More helpers, more stats, refining data, automated retrieval

* Adds new rpc server commands and attempts some organisation

* lower string, lower stress

* Adds ordermanager config. Fleshes outcli. Tracks positions automatically

* Adds new separation for funding payments/rates

* Combines funding rates and payments

* Fun test coverage

* ALL THE TESTS... I hope

* Fixes

* polishes ftx tests. improves perp check. Loops rates

* Final touches before nit attax

* buff 💪

* Stops NotYetImplemented spam with one simple trick!

* Some lovely little niteroos

* linteroo

* Clarifies a couple of errors to help narrow likely end user problems

* Fixes asset type bug, fixes closed position order return, fixes unset status bug

* Fixes order manager handling when no rates are available yet

* Continues on no funding rates instead. Removes err

* Don't show predicted rate if the time is zero

* Addresses scenario with no funding rate payments

* Bug fixes and commentary before updating maps to use *currency.Item

* Adds a pair key type

* Polishes pKey, fixes map order bug

* key is not a property in the event someone changes the base/quote

* Adds improvements to order processing...Breaks it all

* Shakes up the design of things by removing a function

* Fixes issues with order manager positions. Limits update range

* Fixes build issues. Identification of bad tests.

* Merges and fixes features from master and this branch

* buff linter 💪

* re-gen

* proto regen

* Addresses some nits. But not all of them.

* Fixes issue where funding rates weren't returned 🎉

* completes transition futures tracking to map[*currency.Item]map[*currency.Item]

* who did that? not me

* removes redundant check on account of being redundant and unnecessary

* so buf

* addresses nits: duplications, startTime, loops, go tidy, typos

* fixes minor mistakes

* fixes 🍣 🐻 changes to int64
2022-08-23 12:16:50 +10:00
Adrian Gallagher
68588560e3 CI: Bump go version, linters and fix minor issues (#1010)
* Bump golang, golangci-lint versions and fix issues

* Add -fno-stack-protector

* Fix AppVeyor golangci-lint ver

* Nitters

* Nitters round 2
2022-08-17 11:37:22 +10:00