Commit Graph

270 Commits

Author SHA1 Message Date
findfluctuate
a2d90c0ab9 chore: drop legacy build tags (#2125)
Signed-off-by: findfluctuate <findfluctuate@outlook.com>
2025-12-15 12:40:41 +11:00
Adrian Gallagher
bb122dcafa codebase: Remove web frontend and related services (#2067)
* codebase: Remove web frontend and related services

* refactor: Update StartPPROF to accept context and adjust related tests

* refactor: Simplify SetIfZero functions and update related tests

* config: Clarify DowngradeConfig method documentation regarding permanent removal of deprecated fields

* refactor: Rename setIfZeroAndWarn to setDefaultIfZeroWarn for clarity and update related calls

* refactor: Update error handling in DataHistoryManager and remove redundant error variable
2025-09-30 13:32:09 +10:00
cranktakular
fd9aaf00a2 Coinbase: Update exchange implementation (#1480)
* Slight enhance of Coinbase tests

Continual enhance of Coinbase tests

The revamp continues

Oh jeez the Orderbook part's unfinished don't look

Coinbase revamp, Orderbook still unfinished

* Coinbase revamp; CreateReport is still WIP

* More coinbase improvements; onto sandbox testing

* Coinbase revamp continues

* Coinbase revamp continues

* Coinbasepro revamp is ceaseless

* Coinbase revamp, starting on advanced trade API

* Coinbase Advanced Trade Starts in Ernest

V3 done, onto V2

Coinbase revamp nears completion

Coinbase revamp nears completion

Test commit should fail

Coinbase revamp nears completion

* Coinbase revamp stage wrapper

* Coinbase wrapper coherence continues

* Coinbase wrapper continues writhing

* Coinbase wrapper & codebase cleanup

* Coinbase updates & wrap progress

* More Coinbase wrapper progress

* Wrapper is wrapped, kinda

* Test & type checking

* Coinbase REST revamp finished

* Post-merge fix

* WS revamp begins

* WS Main Revamp Done?

* CB websocket tidying up

* Coinbase WS wrapperupperer

* Coinbase revamp done??

* Linter progress

* Continued lint cleanup

* Further lint cleanup

* Increased lint coverage

* Does this fix all sloppy reassigns & shadowing?

* Undoing retry policy change

* Documentation regeneration

* Coinbase code improvements

* Providing warning about known issue

* Updating an error to new format

* Making gocritic happy

* Review adherence

* Endpoints moved to V3 & nil pointer fixes

* Removing seemingly superfluous constant

* Glorious improvements

* Removing unused error

* Partial public endpoint addition

* Slight improvements

* Wrapper improvements; still a few errors left in other packages

* A lil Coinbase progress

* Json cleaning

* Lint appeasement

* Config repair

* Config fix (real)

* Little fix

* New public endpoint incorporation

* Additional fixes

* Improvements & Appeasements

* LineSaver

* Additional fixes

* Another fix

* Fixing picked nits

* Quick fixies

* Lil fixes

* Subscriptions: Add List.Enabled

* CoinbasePro: Add subscription templating

* fixup! CoinbasePro: Add subscription templating

* fixup! CoinbasePro: Add subscription templating

* Comment fix

* Subsequent fixes

* Issues hopefully fixed

* Lint fix

* Glorious fixes

* Json formatting

* ShazNits

* (L/N)i(n/)t

* Adding a test

* Tiny test improvement

* Template patch testing

* Fixes

* Further shaznits

* Lint nit

* JWT move and other fixes

* Small nits

* Shaznit, singular

* Post-merge fix

* Post-merge fixes

* Typo fix

* Some glorious nits

* Required changes

* Stop going

* Alias attempt

* Alias fix & test cleanup

* Test fix

* GetDepositAddress logic improvement

* Status update: Fixed

* Lint fix

* Happy birthday to PR 1480

* Cleanups

* Necessary nit corrections

* Fixing sillybug

* As per request

* Programming progress

* Order fixes

* Further fixies

* Test fix

* Pre-merge fixes

* More shaznits

* Context

* Sonic error handling

* Import fix

* Better Sonic error handling

* Perfect Sonic error handling?

* F purge

* Coinbase improvements

* API Update Conformity

* Coinbase continuation

* Coinbase order improvements

* Coinbase order improvements

* CreateOrderConfig improvements

* Managing API updates

* Coinbase API update progression

* jwt rename

* Comment link fix

* Coinbase v2 cleanup

* Post-merge fixes

* Review fixes

* GK's suggestions

* Linter fix

* Minor gbjk fixes

* Nit fixes

* Merge fix

* Lint fixes

* Coinbase rename stage 1

* Coinbase rename stage 2

* Coinbase rename stage 3

* Coinbase rename stage 4

* Coinbase rename final fix

* Coinbase: PoC on converting to request structs

* Applying requested changes

* Many review fixes, handled

* Thrashed by nits

* More minor modifications

* The last nit!?

---------

Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>
2025-09-16 13:37:00 +10:00
Adrian Gallagher
ba92ba3254 docs: Add CODING_GUIDELINES.md and other adjustments (#1988)
* docs: Create new CODING_GUIDELINES doc

Rids excess contribution instructions from other packages
Adds AGENTS.md file for the AI overlords
Rids unused templates
Updates ADD_NEW_EXCHANGE.md with minor fixes

* docs: Fix linter issues and minor adjustments based on Copilot feedback

* docs: Update coding guidelines for API parameters and testing practices

* docs: Remove redundant GoDoc references

Adds copilot-instructions.md

* docs: Update CODING_GUIDELINES with export recommendations and test commentary

* docs: Fix formatting inconsistencies in ADD_NEW_EXCHANGE.md links

* docs: Update struct naming conventions for request and response types

* docs: Improve clarity and consistency in ADD_NEW_EXCHANGE.md and CODING_GUIDELINES.md

* refactor: Simplify error handling in QueryOrder method
2025-08-08 12:10:53 +10:00
Adrian Gallagher
16c1ff3bdf Slack: Update shared invite link (#1967)
* Slack: Update shared invite link

* fix: add rel="noopener noreferrer" to Slack invite links for security
2025-07-08 13:19:46 +10:00
Adrian Gallagher
19b8957f3f codebase: Replace !errors.Is(err, target) with testify (#1931)
* tests: Replace !errors.Is(err, target) with testify equivalents

* codebase: Manual !errors.Is(err, target) replacements

* typo: Replace errMisMatchedEvent with errMismatchedEvent

* tests: Enhance error messages for better output

* tests: Refactor error assertions in various test cases to use require and improve clarity

* misc linter: Fix assert should wording

* tests: Simplify assertions in TestCreateSignals for clarity and conciseness

* tests: Enhance assertion message in TestCreateSignals
2025-06-10 16:29:57 +10:00
Adrian Gallagher
98a390b181 config: Fix TestPromptForConfigEncryption race (#1929)
* Here's how I resolved a race condition in the encryption prompt tests:

I've refactored `promptForConfigEncryption` to accept an `io.Reader`. This allows tests to use `strings.NewReader` instead of relying on the global `os.Stdin`. This change isolates the input source for `TestPromptForConfigEncryption`, preventing concurrent access conflicts with `TestPromptForConfigKey` which uses `withInteractiveResponse` to manipulate `os.Stdin`.

The original issue manifested as a data race detected by `go test -race` between these two test functions due to their parallel execution (`t.Parallel()`) and their interaction with the shared `os.Stdin` resource.

Here are the changes I made:
- `config/config_encryption.go`:
    - Modified `promptForConfigEncryption` to `promptForConfigEncryption(r io.Reader) (bool, error)`.
    - Introduced `PromptForConfigEncryption()` as a public wrapper that calls the refactored function with `os.Stdin` for application use.
- `config/config.go`:
    - Updated the call site for prompting config encryption to use the new `PromptForConfigEncryption()` wrapper.
- `config/config_encryption_test.go`:
    - Updated `TestPromptForConfigEncryption` to call the (now unexported) `promptForConfigEncryption` with `strings.NewReader` for various input scenarios.
    - `t.Parallel()` was maintained for `TestPromptForConfigEncryption`.

To verify, I confirmed that running `go test -race ./config/...` shows the previously reported race condition is no longer present. All tests in the `config` package now pass with the race detector enabled.

* Refactor(config): Remove redundant loop variable capture in test

I've removed the explicit `tc := tc` line in `TestPromptForConfigEncryption`
as it is no longer necessary for Go versions 1.22 and later.
The project's Go version (1.24.3 as per CI) handles loop variable
scoping correctly for parallel subtests, making this capture redundant.

This change is a follow-up to the fix for the race condition in issue #1928,
addressing your feedback on code style. No functional changes are introduced
by this commit.

* Style(config): Remove explicit empty string in test case

Refactors the "input_empty_eof" test case in
`TestPromptForConfigEncryption` to remove the explicit
assignment of `input: ""`. This relies on Go's default
behavior for struct field initialization (a string field
defaults to an empty string), making the code more concise.

This change is a follow-up to previous refactorings for issue #1928,
addressing your feedback on code style. No functional changes
are introduced by this commit.

* Update config/config_encryption_test.go

Co-authored-by: Ryan O'Hara-Reid <oharareid.ryan@gmail.com>

* linter: Make indenting a happy bing

* config: Rid PromptForConfigEncryption

---------

Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
Co-authored-by: Ryan O'Hara-Reid <oharareid.ryan@gmail.com>
2025-06-07 12:46:25 +10:00
Adrian Gallagher
a5b638bfb7 GHA: Add additional checks for common issues (#1922)
* GHA, tests: Add additional checks for common issues

These checks include:
- Ensuring that all testify funcs use their formatted variants (e.g., `assert.Equalf(t, expected, actual)` instead of `assert.Equal(t, expected, actual)`).
- Replacing `%s` with %q
- Enforcing consistent usage of should/must wording for testify assert/require messages

* Add support for checking backticked string format specifiers and fix issues

* tests: Fix error comparisons

* tests: Replace errors.Is(err, nil) usage with testify and automate check

* refactor: Rename ExtractPort to ExtractPortOrDefault

* tests: Replace assert with require for error handling in multiple test files

* tests: Replace assert with require for error handling and improve assertions in data tests

* tests: Fix typo in assertion message for StreamVol test

* OKX: Fix GetOpenInterestAndVolumeStrike test with instrument selection and improved assertions

* OKX: Revert intentional error check

* Improve error message for expiry time check in GetOpenInterestAndVolumeStrike test
2025-05-28 12:26:51 +10:00
Gareth Kirwan
c2bb050eac Bitmex: Fix deprecated API endpoints and add config migration support (#1901)
* Bitmex: Fix configured WS url ignored

* Bitmex: Replace deprecated WS api endpoint

* [Bitmex deprecated the old WS multiplexing endpoint](https://blog.bitmex.com/api_announcement/api-update-remove-support-realtimemd/)
* [Bitmex deprecated the www WS endpoint in 2021](https://blog.bitmex.com/api_announcement/change-of-websocket-endpoint/). Apparently still in service though.

Fixes #1894
2025-05-14 13:56:53 +10:00
Gareth Kirwan
3caa149d8e Tests: Use currency.NewBTCUSD and NewBTCUSDT (#1895)
* Tests: Use currency.NewUSD and NewUSDT

Simple refactor to use the provided shortcut methods

* Github: Add CI check to ensure NewPair not used

Add a step to ensure NewPair(BTC, USD*) isn't used
2025-05-07 11:32:06 +10:00
Adrian Gallagher
bea16af380 golangci-lint: Enable usetesting and unused linters (#1893)
* golangci-lint: Enable usetesting and unused linters

* tests: Improve assertions in various test cases for clarity and accuracy

* tests: Enhance error assertions in TestExecuteStrategyFromFile for improved clarity

* tests: Update assertions for improved clarity and accuracy

* tests: Replace assert with require for task count checks

* config/versions/v7: Replace context.Background() with t.Context()

* Bithumb: Centralise and consoliate testPair, relax UpdateTickers check

with some glorious Doom Eternal music

* Bithumb: Use UpdatePairsOnce and update remaining pair string

* Bithumb: Add UpdatePairsOnce to TestUpdateTickers
2025-05-01 14:44:29 +10:00
Gareth Kirwan
88ac5274c9 GateIO: Split futures into USDTM and CoinM futures (#1786)
* Config: v5 Split GateIO futures into CoinM and USDT

* GateIO: Split asset.Futures into CoinM and USDT

* Fix CancelBatchOrders using wrong endpoint for CoinMarginedFutures
* Fix TestGetActiveOrders expecting currency.ErrCurrencyPairsEmpty

* Config: Add config version continuity step to CI

* GateIO: Pin CoinM futures to just BTC/USD

Right now we only have a /btc endpoint available, and only BTCUSD is
available.
If GateIO offers more, we'll need to add a settlement currencies list
again
2025-04-30 15:39:39 +10:00
Gareth Kirwan
1bf3433d61 Config: Refactor version packages (#1887)
* Config: Move config versions to separate pacakges

* Config: Move version tests to blackbox texts

* Config: Protect registerVersion from overflow

* Config: Protect against version already registered
2025-04-22 12:13:01 +10:00
dependabot[bot]
179519a3c3 golangci-lint: Bump to v2.0.2 and fix issues (#1871)
* build(deps): Bump golangci/golangci-lint-action from 6 to 7

Bumps [golangci/golangci-lint-action](https://github.com/golangci/golangci-lint-action) from 6 to 7.
- [Release notes](https://github.com/golangci/golangci-lint-action/releases)
- [Commits](https://github.com/golangci/golangci-lint-action/compare/v6...v7)

---
updated-dependencies:
- dependency-name: golangci/golangci-lint-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* golangci-lint: Bump to v2 and fix issues

* refactor: remove no longer need ifshort nolint directives, fix test grammar and improve for loop logic

* nits: update order pair handling in tests and improve string replacement logic

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
2025-04-02 15:36:05 +11:00
Gareth Kirwan
563ae7883b Config: Eliminate shared config version package vars (#1867)
* Config: Replace consts for Version2 from/to

Could have moved them to v2/consts.go but doesn't seem enough RoI

* Config: Use generic error in Version4

* Config: Fix Version5 pkg vars

* Config: Fix Version6 test func names and shared test var
2025-04-01 19:51:38 +11:00
Adrian Gallagher
cc05f7e6fd portfolio: Fix CryptoID balance issue and assertify tests (#1861)
* portfolio: Fix CryptoID balance issue and assertify tests

* portfolio: Expand context usage, enhance tests and a few other minor improvements

* portfolio: Further improvements and enhance common.IsValidCryptoAddress

* config, portfolio: Use v6.DefaultConfig, switch to context.WithCancel
2025-03-28 12:41:01 +11:00
Adrian Gallagher
4651af5767 modernise: Run new gopls modernise tool against the codebase and fix minor issues (#1826)
* modernise: Run new gopls modernise tool against codebase

* Address shazbert's nits

* apichecker, gctcli: Simplify HTML scraping functions and improve depth limit handling

* refactor: Create minSyncInterval const and update order book limit handling for binance and binanceUS

* refactor: Various slice usage improvements and rename TODO

* tranches: Revert deleteByID changes due to performance decrease

Shazbert was a F1 driver in a past lifetime 🏎️

* tranches: Simply retrieve copy

Thanks to shazbert

* documentation: Sort contributors list by contributions

* tranches: Remove deadcode in deleteByID
2025-03-21 09:17:10 +11:00
Gareth Kirwan
ade3ac5c59 Config: OrderManager upgrade (#1737)
* Config: Allow missing Versions

This allows easier development of non-stacked version upgrades.
Though the PRs still need to be merged sequentially, or renumbered right
before merging

* Config: Move OrderManager upgrade to Version management
2025-03-19 16:11:28 +11:00
Adrian Gallagher
f5faca2eb2 linter: Enable gofumpt and run against codebase (#1848)
* linter: Enable gofumpt and run against codebase

* Address shazbert's nits

* gofumpt: Fix issues after rebase
2025-03-18 10:23:16 +11:00
Gareth Kirwan
16d2d9f35a Config: AssetEnabled upgrade (#1735)
* Config: Move assetEnabled upgrade to Version management

* Assets: Do not error on asset not enabled, or disabled

This became more messy with Disabling something that's defaulted to
disabled.
Taking an idealogical stance against erroring that what you want to have
done is already done.

* CurrencyManager: Set AssetEnabled when StorePairs(enabled)

* RPCServer: Fix tests expecting StoreAssetPairFormat to enable the asset

Also assertifies

* Bitfinex: Fix tests for MarginFunding subs

* GCTWrapper: Improve TestMain clarity

* BTSE: Add futures to testconfig

* Exchanges: Rename StoreAssetPairStore

Previously we were calling it "Format", but accepting everything from
the PairStore.
We were also defaulting to turning the Asset on.

Now callers need to get their AssetEnabled set as they want it, so
there's no magic

This change also moves responsibility for error wrapping outside to the
caller.

* Config: AssetEnabled upgrade should respect assetTypes

Previously we ignored the field and just turned on everything.
I think that was because we couldn't get at the old value.
In either case, we have the option to do better, and respect the
assetEnabled value

* Config: Improve exchange config version upgrade error messages
2025-03-17 21:47:37 +11:00
Adrian Gallagher
d64d56f77c build/ci: Update Go to v1.24, golangci-lint to v1.64.6 and fix issues (#1804)
* build/ci: Update Go to v1.24, golangci-lint to v1.64.5 and fix issues

* Address shazbert's nitters

* linter/config: Fix new linter issue and use versionSize const

* Address gk's nitters and fix additional linter issue after rebase

* Address glorious nits

* staticcheck: Fix additional linter issues after upgrading to Go 1.24.1 and golangci-lint v1.64.6

Also addresses nits

* Improve testing, assertify usage and use common.ErrParsingWSField

* TestCreateNewStrategy: Replace must > should wording
2025-03-10 16:33:55 +11:00
Gareth Kirwan
9fcaa9130b Config: Tighten config version handling as uint16 (#1825)
* Config: Tighten config version handling as uint16

This constrains the versions to uint16 and improves error handling.
LatestVersion becomes literally that.
Fixes handling for negative or overflowing versions in config

* Config: Rename LatestVersion to UseLatestVersion
2025-03-06 12:12:57 +11:00
Ryan O'Hara-Reid
e99adca86f encoding/json: Add custom JSON package with build tag support for Sonic (#1623)
* tag optional sonic and allow full library conversion

* Add workflow and disallow arm and darwin usage

* Add basic hotswap benchmark

* linter: fix

* use bash

* linter: fix?

* Fix whoopsie, add to make file, also add mention in features list.

* test enforcement

* actually read documentation see if this works

* linter: fix

* linter: fix

* sonic: bump tagged version

* encoding/json: drop build tag arch and os filters

* encoding/json: consolidate tests

* encoding/json: log build tag usage

* rm superfluous builds

* glorious/nits: add template change and regen docs

* glorious/nits: update commentary on nolint directive

* glorious/nits: rm init func and log results in main.go

* Test to actually pull flag in

* linter: fix

* thrasher: nits

* gk: nits 4 goflags goooooooooo!

* gk: nits rn

* make sonic default json implementation

* screen 386

* linter: fix

* Add commentary

* glorious: nits Makefile not working

* gk: nits

* gk: nits whoops

* whoopsirino

* mention 32bit systems won't be sonic

* gk: super-duper nit of extremes

---------

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
2025-02-20 16:05:55 +11:00
Gareth Kirwan
3748c97b12 Config: Fix config version downgrade (#1770)
* Config: Rename DecryptConfigFile to DecryptConfigData

Because this isn't really a file, it's a byte slice

* Config: Rename EncryptConfigFile to EncryptConfigData

Because it's not actually a file

* Config: Fix config version downgrade

Fixes #1769
2025-02-20 09:27:52 +11:00
Ryan O'Hara-Reid
6613c56738 stream/buffer: remove time.ticker publish limiter for websocket orderbook updates (#1681)
* stream/buffer: remove publish period for for signalling change through data handler

* Add config upgrade and downgrade

* linter: fix

* GK: nits

* Update config/versions/v3.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update config/versions/v3.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update config/versions/v3.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update config/versions/v3.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update config/versions/v3.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* config: fix up commited suggestion

---------

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
2025-02-10 16:15:33 +11:00
Gareth Kirwan
219ed903bc Config: Add versioning (#1671)
* Config: Version Management

* Engine: Improve visibility of TestConfigAllJsonResponse failures

* Config: Update cmd/config to allow upgrades

* Config: Add Version2 to rename GDAX

* Config: Restructure versioning to share types

This restructure allows us to share types between versions, avoids
needing to import the versions, and puts the test fixtures in same
package.
It's a win on all fronts

* Config: Fix SetNTPCheck using log

Called from engine before logger is inited, and also just wrong to use
log to communicate with user

* Config: Improve TestMigrateConfig

* Config: Drop requirement for versions to be registered in sequence

Checking the versions at Deploy is much saner.

* Config: Fix file encrypted but flag not set

* Config: Add -edit and encryption upgrade to cmd/config

This simplifies the handling for encryption prompts by moving it to a
field on config, allowing us to simplify all the places were were
passing around config

Also moves password entry to being secure (echo-off)

* Tests: Fix inconsistent should/must assertions
2024-12-09 15:04:16 +11:00
Adrian Gallagher
69bb645fc2 docs: Migrate development tracking from Trello to GitHub Projects (#1696)
* docs: Migrate development tracking from Trello to GitHub Projects

* docs: Replace non-template based references and correct templates
2024-11-08 10:10:49 +11:00
Adrian Gallagher
b8e836d74f common: Replace StringDataCompare with slices.Contains and cleanup string funcs (#1631)
* common: Replace StringDataCompare with slices.Contains and cleanup string funcs

* common/docs: Update SliceDifference and remove outdated steps from ADD_NEW_EXCHANGE.md

* common: Improve SliceDifference
2024-09-13 10:43:20 +10:00
Adrian Gallagher
15e0eedc8b tests: Fix various issues (#1634)
* GateIO: Align struct for int64 usage

* CI/build: Skip specific tests when run under Docker

* slippage: Fix TestRandomSlippage intermittent issue
2024-08-27 12:54:11 +10:00
Adrian Gallagher
74f4df6348 okcoin: Remove exchange implementation (#1626)
* okcoin: Remove exchange implementation

* exchanges/order: Rid TradeMode from Submit type
2024-08-24 12:19:04 +10:00
Adrian Gallagher
225429bda6 CI/build: Update Go version, linters and fix minor issues (#1612)
* CI/build: Update Go version, linters and fix minor issues

* linters: Add intrange, copyloopvar, additional go vet linters to match gopls and fix issues
2024-08-16 17:41:11 +10:00
Gareth Kirwan
1199f38546 subscriptions: Encapsulate, replace Pair with Pairs and refactor; improve exchange support
* Websocket: Use ErrSubscribedAlready

instead of errChannelAlreadySubscribed

* Subscriptions: Replace Pair with Pairs

Given that some subscriptions have multiple pairs, support that as the
standard.

* Docs: Update subscriptions in add new exch

* RPC: Update Subscription Pairs

* Linter: Disable testifylint.Len

We deliberately use Equal over Len to avoid spamming the contents of large Slices

* Websocket: Add suffix to state consts

* Binance: Subscription Pairs support

* Bitfinex: Subscription Pairs support

* Bithumb: Subscription Pairs support

* Bitmex: Subscription Pairs support

* Bitstamp: Subscription Pairs support

* BTCMarkets: Subscription Pairs support

* BTSE: Subscription Pairs support

* Coinbase: Subscription Pairs support

* Coinut: Subscription Pairs support

* GateIO: Subscription Pairs support

* Gemini: Subscription Pairs support and improvement

* Hitbtc: Subscription Pairs support

* Huboi: Subscription Pairs support

* Kucoin: Subscription Pairs support

* Okcoin: Subscription Pairs support

* Poloniex: Subscription Pairs support

* Kraken: Add subscription Pairs support

Note: This is a naieve implementation because we want to rebase the
kraken websocket rewrite on top of this

* Bybit: Subscription Pairs support

* Okx: Subscription Pairs support

* Bitmex: Subsription configuration

* Fixes unauthenticated websocket left as CanUseAuth
* Fixes auth subs happening privately

* CoinbasePro: Subscription Configuration

* Consolidate ProductIDs when all subscriptions are for the same list

* Websocket: Log actual sent message when Verbose

* Subscriptions: Improve clarity of which key is which in Match

* Subscriptions: Lint fix for HugeParam

* Subscriptions: Add AddPairs and move keys from test

* Subscriptions: Simplify subscription keys and add key types

* Subscriptions: Add List.GroupPairs Rename sub.AddPairs

* Subscription: Fix ExactKey not matching 0 pairs

* Subscriptions: Remove unused IdentityKey and HasPairKey

* Subscriptions: Fix GetKey test

* Subscriptions: Test coverage improvements

* Websocket: Change State on Add/Remove

* Subscriptions: Improve error context

* Subscriptions: Fix Enable: false subs not ignored

* Bitfinex: Fix WsAuth test failing on DataHandler

DataHandler is eaten by dataMonitor now, so we need to use ToRoutine

* Deribit: Subscription Pairs support

* Websocket: Accept nil lists for checkSubscriptions

If the user passes in a nil (implicitly empty) list, we would not panic.
Therefore the burden of correctness about that data lies with them.
The list of subscriptions is empty, and that's okay, and possibly
convenient

* Websocket: Add context to NilPointer errors

* Subscriptions: Add context to nil errors

* Exchange: Fix error expectations in UnsubToWSChans
2024-06-07 11:54:08 +10:00
Gareth Kirwan
fdf6014dca Currency: Remove Pair Index formatting/parsing (#1520)
* Currency: Remove Pair Index formatting/parsing

This feature was originally for exchanges with only one pair (e.g. KRW) which made parsing easier.
However there's no examples of this left, and we can reduce complexity
overall by removing it.

* Exchange: Partial assertify tests and fixes

* Currency: Fix panic on a delimiterless small currency
2024-04-19 17:03:12 +10:00
Gareth Kirwan
68a6f5828f Binance: Subscribe/unsubscribe response handling (#1444)
* Binance: Fix subscription failures ignored

* Testing: Fix race on shared config singleton

* Config: Privatise Global config var

We should *either* use a private var *or* use an accessor, but it
doesn't make sense to mix paradigms.
Since GetConfig() is well established this instead removes the limited uses of direct public access and adds a Setter

* Zip: Fix test failure on http mocks
2024-02-02 19:27:17 +11:00
Gareth Kirwan
e007f69f7c exchanges/websocket: Implement subscription configuration (#1394)
* Websockets: Move Subscription to its own package

This allows the small type to be imported from both `config` and from
`stream` without an import cycle, so we don't have to repeat ourselves

* Subs: Renamed Currency to Pair

This was being mis-used through much of the code, and since we're
already touching everything, we might as well fix it

* Websockets: Add Subscription configuration

* Binance: Add subscription configuration

* Kucoin: Subscription configuration

* Simplify GenerateDefaultSubs
* Improve TestGenSubs coverage
* Test Candle Sub generation
* Support Candle intervals
* Full responsibility for formatting Channel name on GenerateDefaultSubs
  OR consumer of Subscribe
* Simplify generatePayloads as a result
* Fix test coverage of asset types in processMarketSnapshot

* Exchanges: Abstract ParallelChanOp

* Tests: Generic ws mock instances

* Kucoin: Fix intermittent conflict in test currs

Use isolated test instance for `TestGetOpenInterest`.

`TestGetOpenInterest` would occassionally change pairs before
GenerateDefault Subs.
2024-01-24 15:54:07 +11:00
Gareth Kirwan
37b1121bbd BTSE: Fix duplicate pair errors on Million pairs (M_*) (#1401)
* BTSE: Fix duplicate error on Million pairs (M_*)

BTSE has listed Pitbull token with two symbols:
PIT-USD and M_PIT-USD for millons of PIT / USD.
The native token is not tradable, so we ignore them and
get a base of M_PIT because that's what later APIs will accept

* BTSE: Fix test errors on locked market

* Common: Improve AppendError and ExcludeError

This change switches from a stateful multiError to caring more about the
Unwrap() []error interface, the same as [go standard
lib](https://github.com/golang/go/blob/go1.21.4/src/errors/wrap.go#L54-L68)

Notably, if we implement Unwrap() []error and do NOT implement Is() then
we get free compatibility with the core functions.

The only distateful thing here is needing to deeply unwrap fmt.Errorf
errors, since they don't flatten. I can't see any way around that

* Pairs: Fix exchange config Pairs loading

When a pair string contained two punctuation runes, the first one is used,
and the configFormat is ignored.

This fix checks the list and corrects any with the wrong delimiter, or
errors if the format is inconsistent.

* BTSE: Fix all tickers retrieved by GetTicker

PR #764 introduced GetTickers, but it wasn't rolled out to BTSE.
This fix ensures that when one ticker is a locked market, the rest continue to
function. Particularly important if the locked market wasn't even
enabled anyway.

* Kucoin: Fix test config future pairs

* BTSE: Remove PIT tests; Token removed

BTSE have removed the PIT token pairs

All these changes stand, and this just removes the test

* ITBit: Fix fatal error on second run

This fix removes incorrect config pair delimiter, because it would be
re-inserted into config the first run, and then error the second time.

This delimiter doesn't match the config we have.
There's no implementation of fetching pairs, so what's in config files
now is all that matters

* Engine: Fix TestConfigAllJsonResponse

* Clarity of non-matching json improved
* Handling for fixing pair delimiters
2023-12-19 14:40:13 +11:00
Gareth Kirwan
88182ec414 Engine: Remove Default Forex Provider (#1395)
* Currency: Do not use a default forex provider

exchangerate.host now requires an API key.
Instead of finding a new Free (for now) default, this change simply
disables the currency exchange when nothing is enabled.

* SyncManager: Report ?.?? for an unknown forex amount

In a situation where we thought forex was available but we got an error,
this avoids showing 0.00 when there was actually an error.

* Currency: Tests for no default forex

* Currency: Use mock provider for tests

* Currency: Add API key to exchangerate.host

* Currency: Remove Exchangerate.host

Exchangerate.host was bought by apilayer, the old API deprecated, and
replaced with a proxy to the apilayer api.
We already have currencylayer support, so ther's no reason to keep exh.

Worth noting: New ERH keys actually work on currencylayer

* Currencies: Add test coverage for currency layer

* fixup! Currency: Tests for no default forex

Remove duplicate assignment

Fixes [review comment](https://github.com/thrasher-corp/gocryptotrader/pull/1395#discussion_r1395178513)

* fixup! Currency: Add API key to exchangerate.host

Remove unused ErrVar

Fixes [review
comment](https://github.com/thrasher-corp/gocryptotrader/pull/1395#discussion_r1396647418)

* fixup! Currency: Tests for no default forex

Fix spelling of override in test

Fixes [review comment](https://github.com/thrasher-corp/gocryptotrader/pull/1395#discussion_r1396701476)

* fixup! SyncManager: Report ?.?? for an unknown forex amount

Fix display of non-positive currency conversions.

Fixes [review comment](https://github.com/thrasher-corp/gocryptotrader/pull/1395/files#r1398527134)
2023-11-24 13:50:01 +11:00
Ryan O'Hara-Reid
a13a6c3248 Kucoin: Add order execution limits (#1369)
* cmd/exchange_template: Add wrapper function

* exchanges: force UpdateOrderExecutionLimits to wrappers as this is important for order deployment

* kucoin: Add spot order execution limits

* linter: fix

* glorious: nits

* kucoin: change from name to symbol for correct fee fetching and order deployment

* WHAAAAAAAT says Vitalik

* kucoin: Add futures limit support, fix insertion of non margin tradable pairs, update naming and add comments.

* kucoin: implement master branch changes

* Update exchanges/kucoin/kucoin_test.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* kucoin/test: update commentary

---------

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2023-11-14 11:34:38 +11:00
Ryan O'Hara-Reid
ecf0a5bcb3 spellcheck: fix and update (#1359)
* spellcheck: fix and update

* spellcheck: add exclude line for gateio_types
2023-10-03 16:07:03 +11:00
Scott
5f2f6f884b Binance,Okx: Add Leverage, MarginType, Positions and CollateralMode support (#1220)
* init

* surprise train commit

* basic distinctions

* the terms of binance are confusing

* renames and introduction of allocatedMargin

* add new margin funcs

* pulling out wires

* implement proper getposition stuff

* bad coding day

* investigate order manager next

* a broken mess, but a progressing one

* finally completes some usdtmargined stuff

* coinMfutures eludes me

* expand to okx

* imports fix

* completes okx wrapper implementations

* cleans and polishes before rpc implementations

* rpc setup, order manager features, exch features

* more rpc, collateral and margin things

* mini test

* looking at rpc response, expansion of features

* reorganising before the storm

* changing how futures requests work

* cleanup and tests of cli usage

* remove silly client side logic

* cleanup

* collateral package, typo fix, margin err, rpc derive

* uses convert.StringToFloat ONLY ON STRUCTS FROM THIS PR

* fix binance order history bug

* niteroos

* adds new funcs to exchange standards testing

* more post merge fixes

* fix binance

* replace simepletimeformat

* fix for merge

* merge fixes

* micro fixes

* order side now required for leverage

* fix up the rest

* global -> portfolio collateral

* Update exchanges/collateral/collateral_test.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* adds fields and todos

* rm field redundancy

* lint fix oopsie daisy

* fixes panic, expands error and cli explanations (sorry shaz)

* ensures casing is appropriate for underlying

* Adds a shiny TODO

---------

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
2023-09-26 16:16:31 +10:00
Ryan O'Hara-Reid
1adbc99526 exchanges: improves GetDefaultConfig method (#1245)
* exchanges: Add function to get standard config

* exchanges: add tests (cherry-pick here and above)

* after pick stuff

* cleanup

---------

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
2023-09-18 11:45:02 +10:00
Scott
d3102a08dc egnine/sync-manager: add config support (#1326)
* allows sync manager customisation for values and logs

* config-example add

* who doesnt like more coverage?

* ensures you can actually disable it via config el oh el

* less ifs, better control

* fix verbose

* sync trades default false

* fix summary being printed when not enabled

* fixes config checker and output

* nits

* I can put this behind me now

* Fixed logCaSiNg

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* combines if statements

---------

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
2023-08-29 15:48:41 +10:00
Samuael A
9c83231696 exchanges: Update OKCoin V5 exchange support (#1206)
* starting public endpoints

* Adding public endpoints

* added public spot market endpoints

* websocket subscriptions updates

* websocket push data handlers completing

* linter fix

* Added funding private endpoints

* Adding authenticated account endpoints

* Added fiat and OTC-RFQ authenticated endpoints

* trading authenticated endpoints

* completing trade endpoints and add public wrapper endpoints

* Authenticated wrapper functions and corresponding unit test

* Adding authenticated websocket endpoint and fixing wrapper functions

* Documentation and exchange websocket update

* update websocket orderbook checksum handling

* linter issues fix and unit test update

* remove invalid orderbook endpoint and unit test

* Documentation, handlers, and model types update

* minot fix

* Minor fixes

* Updating unit tests and added missing endpoints

* Add missing credential check

* Minor unit test fixes

* fix minor linter issue

* add snaphot test unit test

* Fix on update checksum and documentation update

* update exchange, add UpdateOrderExecutionLimits, and update documentation

* Minor fix on tickers fetching

* Minor websocket fix and smaill unit tests

* Minor websocket and naming fixes

* uncomment default channels

* Fix type and unit test issues

* websocket channels and data handling update

* Update Advanced-Algo websocket handling and minor fixes

* documentation and minor code fixes

* Fix name changes

* documentation contribution update

* intervalToString method update

* fix exchange_wrapper_standard tests

* Fix minor issues based on exchange_wrapper_standards_test

* Fix wrapper extended candlestick check

* websocket orders fetching error check method update

* Exchange name check and change

* docs: Add missing contributors

---------

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
2023-08-22 09:44:39 +10:00
Scott
fcc5ad4551 exchanges/qa: Add exchange wrapper testing suite (#1159)
* initial concept of a nice validation tester for exchanges

* adds some datahandler design

* expand testing

* more tests and fixes

* minor end of day fix for bithumb

* fixes implementation issues

* more test coverage and improvements, but not sure if i should continue

* fix more wrapper implementations

* adds error type, more fixes

* changes signature, fixes implementations

* fixes more wrapper implementations

* one more bit

* more cleanup

* WOW things work?

* lintle 1/1337

* mini bump

* fixes all linting

* neaten

* GetOrderInfo+ asset pair fixes+improvements

* adds new websocket test

* expand ws testing

* fix bug, expand tests, improve implementation

* code coverage of a lot of new codes

* fixes everything

* reverts accidental changes

* minor fixes from reviewing code

* removes Bitfinex cancelBatchOrder implementation

* fixes dumb baby typo for babies

* mini nit fixes

* so many nits to address

* addresses all the nits

* Titlecase

* switcheroo

* removes websocket testing for now

* fix appveyor, minor test fix

* fixes typo, re-kindles killed kode

* skip binance wrapper tests when running CI

* expired context, huobi okx fixes

* kodespull

* fix ordering

* time fix because why not

* fix exmo, others

* hopefully this fixes all of my life's problems

* last thing today

* huobi, more like hypotrophy

* golangci-lint, more like mypooroldknee-splint

* fix huobi times by removing them

* should fix okx currency issues

* blocks the application

* adds last little contingency for pairs

* addresses most nits and new problems

* lovely fixed before seeing why okx sucks

* fixes issues with okx websocket

* the classic receieieivaier

* lintle

* adds test and fixes existing tests

* expands error handling messages during setup

* fixes dumb okx bugs introduced

* quick fix for lint and exmo

* fixes nixes

* fix exmo deposit issue

* lint

* fixes issue with extra asset runs missing

* fix surprise race

* all the lint and merge fixes

* fixes surprise bugs in OKx

* fixes issues with times and chains

* fixing all the merge stuff

* merge fix

* rm logs and a panic potential

* lovely lint lament

* an easy demonstration of scenario, but not of initial purpose

* put it in the bin

* Revert "put it in the bin"

This reverts commit 15c6490f713233d43f10957367fcbf18e3818bdd.

* re-add after immediate error popup

* fix mini poor test design

* okx okay

* merge fixes

* fixes issues discovered in lovely test

* I FORGOT TO COMMIT THIS

* nit fixaroonaboo

* forgoetten test fix

* revert old okx asset intrument work

* fixes

* revert problems I didnt understand. update bybit

* fix merge bugs

* test cleanup

* further improvements

* reshuffle and lint

* rm redundant CI_TEST by rm the CI_TEST field that is redundant

* path fix

* move to its own section, dont run on 32 bit + appveyor

* lint

* fix lbank

* address nits

* let it rip

* fix failing test time range

* niteroo boogaloo

* mod tidy, use common.SimpleTimeFormat
2023-07-03 11:09:43 +10:00
Adrian Gallagher
400bcb6b56 telegram: Fix pushing events to an authorised client list (#1208)
* fix_communications_authorised_clients

* Telegram: Link config to authorised clients list

* Telegram: Prevent multiple spam messages from unauthed user

* Telegram: Improve command handling for authenticated users

Telegram doesn't allow you to easily fetch the user ID of a user unless they have previously sent you a message and is currently waiting to be processed, or if they message you on the fly once the bot is connected. This ensures that the user ID is stored for future usage upon a single successful auth command.

It also fixes the offset as the previous code wouldn't be able to process incoming messages once connected and instead only relay them.

* Bump docs

* default to UTC time in case bot is run on a server with diff time zones

* Enhance config for already upgraded configs

---------

Co-authored-by: shanhuhai5739 <shanhu5739@gmail.com>
2023-06-05 10:37:13 +10:00
Ryan O'Hara-Reid
db8735ec99 log: Add structured logging (#1171)
* basic implementation

* log: deprecate duplicate function, add tests and refine calls.

* linter: fixes

* linter: update struct

* linter and new type

* log tests: update to not lint issue

* linter: stop complaining please

* glorious: nits

* log: rm comment code

* glorious: nits

* glorious: nits

* glorious: nits

* glorious: nits missed

---------

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
2023-05-10 17:52:53 +10:00
Adrian Gallagher
7e08e483fb CI: Bump go version, linters and fix minor issues (#1130)
* CI: Bump go version, linters and fix minor issues

* Bump version, fix loop variables

* Revert

* Rid TODOs now that 1.51 has been released
2023-02-03 15:56:59 +11:00
Donald Adu-Poku
82c79a9287 stream: set connection monitor delay. (#1120)
* stream: set connection monitor delay.

- this fixes a bug where the connection monitor delay config value does not
get set to the websocket on intialization.

* multi: add connection monitor delay to exchange config.

- this adds the connection monitor delay config option to the exchange type.
- the validate function of the exchange type has been updated to validate the
connection monitor delay value as well.

* multi: resolve review issues.
2023-01-31 09:21:51 +11:00
Adrian Gallagher
a79e0d2b3e qa/spelling: Add Codespell support (#1121)
* Add codespell support

* Fix paths

* Add HTML files to exclusion list
2023-01-30 12:36:56 +11:00
Samuael A
2ac165a477 exchanges: Add OKX support (#1005)
* public endpoints methods added

* Completing mapping REST endpoints

* Binanceus Wrapper methods -Partially

* Binanceus Wrapper methods -Partially

* BinaWra functions with test funs; Not Completed

* Finalizing wrapper methods & test

* Fix & Complete wrapper functions

* Finalizing wrapper methods & test

* Adding Stream Datas

* WS Test functions

* CI: Fix golangci-lint linter issues

* CI: Fix reverting unnessesary changes and type conversion issues

* CI: Fix reverting unnessesary changes and type conversion issues

* Adding Public endpoints and tests

* Adding Market and Public Endpoints

* Adding Public endoints

* Public Trading Endpoints & Authenticated Trade order methods

* Adding Authenticated Methods and Tests

* Adding algo and Funding Authenticated endpoints

* Adding funding trading endpoints and correspondint tests

* adding authenticated endpoints

* Completing Block Trading endpoints and added subaccount endpoints

* Completing sub account and grid Trading endpoints

* Adding Rate Limit and missing endpoints

* Wrapper and Websocket handlers

* Fixing Websocket Test and Push Data Handler Issues

* Fixing Websocket Test and Push Data Handler Issues

* Fixing linter issues, package dependency, and other slight tempos

* Fixing linter and slight tempos

* Update on test functions, and Rest and Websocket Endpoint handlers

* Remove okex, adding comments, and slight fixes on endpoints.

* Fixing linter issues and adding comments

* Slight code changes, updating documentation, and n and linter issues

* Fix context and configuration endpoint issues

* slight fixes on config and test files

* adding some missing test and fix linter issues

* fix linter issue

* Slight fixes on code structure, shorthand exp,and ot and other

* Fix slight linter issue

* Slight code fixes and fixing linter issues

* fixing linter iissues

* fixing linter iissues

* slight linter issue fix

* slight linter issue fix

* Fix on models, type convert funcs and endpoints

* Adding Error messages map and update of models

* Fix on error message string and linter issues

* Fix slight linter issue

* Fix slight linter issue

* Fixing type converts, models, and linter issues

* Adding Ws fixes

* Slight fix on websocket and other issues

* Adding slight websocket fixes

* Remove 'books5' channel default subscription

* Small changes on default subscription and checksum

* Fix slight websocket tempos

* Fix Wrapper function tempost and linter issues

* Resolving slight naming and other issues

* Resolve slight pointer issues

* resolve slight linter issues

* Resolve config files issue

* Update websocket and wrapper funcs with test and docs

* fixs on websocket multiplexer, types, and other slight issues

* fix slight linter issues

* slight update on web-socket orderbook and tickers

* fix slight issues and websocket runtime errors

* Slight unit test fix and assing simple semaphore

* FIx race issue

* Update on authenticated endpoints

* Fix wsSetupRun check in websocket 'setupWsAuth' func

* Update wsSetupRun check in websocket 'setupWsAuth' func

* Slight update on websocket handling

* Fix some race conditions

* fix slight tempos

* fix authenticated test issues

* Update on conditional statements

* slight update on unit test

* fix unit test tempos

* Fix slight tempos

* Change check map from struct valued to bool valued

* slight fix trial

* Slight unit test update

* Fix websocket timeout error

* Updating websocket subscription endpoints, and unit tests

* update unit tests

* Slight issue on wrapper method 'GetActiveOrders'

* Overall code update

* Addressing missing review comments

* Fix unit test tempo and linter issue

* Monor fix

* Slight update

* Slight unit test fix

* Slight fixes

* Slight fixes

* Fixing on missing review comments

* Adding WS Fixes

* slight fix

* Monor fix on unit test

* Minor convert issue

* Minor change on WS

* Monor logic fix

* Fix code structure and logic issues

* Fixing small typos

* fix slight data format issue

* Update on trade and order wrapper methods

* Adding slight update

* fix on order detail

* Slight update on FetchTradablePairs wrapper method

* Slight update on wrapper

* Update on deserialization and other slight issues

* Final update

* Resolve missing review comments

* Slight update on config and unit test

* minor fix on GetDepositAddress param

* Minor fix
2022-12-09 11:44:29 +11:00