Commit Graph

190 Commits

Author SHA1 Message Date
Ryan O'Hara-Reid
f843b7d277 exchange: upgrade UpdatePair method (#991)
* exchange: upgrade update pair

* exchanges: Add enabled string matching and format handling if discrepency is found.

* linter: fixes

* bithumb: fix tests

* BTSE: api change fix ordering

* huobi: fix tests

* gloriousnits: stage 1

* gloriousnits: stage 2

* currency: more nits

* bitmex: add spot and process pairs before currency package call.

* bitmex: finished correct orderbook matching and other implementations

* linter: fix issue

* currency: Fix linter

* currency: segregate and protect pair store, update tests

* currency/manager: clean code, rm log output

* currency: Add store method and make sure formatting stays nil if not stored.

* gct: check errors

* engine/websocketroutineman: fix tests

* bybit: fix duplication bug

* huobi: fix test

* btse: fix tests?

* ob/buffer: fix tests

* Update currency/manager.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* glorious: nits strikes again.

* exchange: add bypassConfigFormatUpgrades to stop formatting

* GLORIOUS LINTER

* Update exchanges/bithumb/bithumb_wrapper.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* exchange: fix pair upgrade issue when duplications are in both avail and enabled pairs

* linter: fix shadow dec

* config: fix test

* Update currency/pair_test.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2022-09-16 08:59:27 +10:00
Scott
1461cba363 backtester: standalone application (#988)
* Ramshackle early leads to GRPC backtester

* Adds GRPC server, default config generation

* Partial support for GRPC backtester config

* Update to use Buf, merge fixes

* Full config for GRPC

* Adds new commands, causes big panic

* Fixes panics

* Setup for the future

* Docs update

* test

* grpc tests

* Fix merge issues. Lint and test

* minor fixes after rebase

* Docs, formatting and main fixes

* Change buf owner

* shazNits

* test-123

* rpc fixes

* string fixes

* Removes --singlerun flag and just relies on --singlerunstrategypath

* fixes test

* initial post merge compatability fixes

* this actually all seems to work? unexpected

* adds pluginpath to config

* rm unused func. add gitignore

* rm unused func. add gitignore

* lintle

* tITLE cASE lOG fIX,rm auth package, gitignore, tmpdir fix

* buf updates + gen. go mod tidy

* x2

* Update default port, update error text
2022-09-08 16:22:30 +10:00
Scott
46cadd6f15 FTX: Funding rates, payments & stats + order manager tracking (#976)
* Adds basic PoC for calculating/retrieving position data

* A very unfortunate day of miscalculations

* Adds position summary and funding rate details to RPC

* Offline funding rate calculations

* More helpers, more stats, refining data, automated retrieval

* Adds new rpc server commands and attempts some organisation

* lower string, lower stress

* Adds ordermanager config. Fleshes outcli. Tracks positions automatically

* Adds new separation for funding payments/rates

* Combines funding rates and payments

* Fun test coverage

* ALL THE TESTS... I hope

* Fixes

* polishes ftx tests. improves perp check. Loops rates

* Final touches before nit attax

* buff 💪

* Stops NotYetImplemented spam with one simple trick!

* Some lovely little niteroos

* linteroo

* Clarifies a couple of errors to help narrow likely end user problems

* Fixes asset type bug, fixes closed position order return, fixes unset status bug

* Fixes order manager handling when no rates are available yet

* Continues on no funding rates instead. Removes err

* Don't show predicted rate if the time is zero

* Addresses scenario with no funding rate payments

* Bug fixes and commentary before updating maps to use *currency.Item

* Adds a pair key type

* Polishes pKey, fixes map order bug

* key is not a property in the event someone changes the base/quote

* Adds improvements to order processing...Breaks it all

* Shakes up the design of things by removing a function

* Fixes issues with order manager positions. Limits update range

* Fixes build issues. Identification of bad tests.

* Merges and fixes features from master and this branch

* buff linter 💪

* re-gen

* proto regen

* Addresses some nits. But not all of them.

* Fixes issue where funding rates weren't returned 🎉

* completes transition futures tracking to map[*currency.Item]map[*currency.Item]

* who did that? not me

* removes redundant check on account of being redundant and unnecessary

* so buf

* addresses nits: duplications, startTime, loops, go tidy, typos

* fixes minor mistakes

* fixes 🍣 🐻 changes to int64
2022-08-23 12:16:50 +10:00
Ryan O'Hara-Reid
e93ee83563 script: implementation of error insertion on return (#986)
* exchanges/account: shift credentials to account package and segregate funds to keys

* merge: fixes

* linter: fix

* Update exchanges/account/account.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits + protection for string panic

* glorious_suggestion: add method for matching keys

* linter: fix tests

* account: add protected method for credentials minimizing access, display full account details to rpc.

* linter: spelling kweeeeeeen

* accounts/portfolio: clean/check portfolio code and quickly check balances from change. Add protected method for future matching.

* accounts: theres no point in pointerising everything

* linter: ok pointerise this then...

* exchanges: fix regression add in little notes.

* glorious: nits

* Update exchanges/account/credentials.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/account/credentials_test.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/account/credentials_test.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* gloriously: fix glorious glorious test gloriously

* script: initial implementation of error insertion on return

* script: make script context aware(ish) and update error handle in examples

* script: add tests

* script: add syntax highlighting to readme

* Update gctscript/vm/vm.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update gctscript/vm/vm.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update gctscript/examples/exchange/account_info.gct

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update gctscript/examples/exchange/cancel_order.gct

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update gctscript/examples/verbose.gct

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update gctscript/modules/gct/gct_test.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* rm: bros

* scripts: handle errors in examples when they are going to use the data after fetching

* linter: fix rides again

* SCOTT_SPELL_CHECK_LINTER: fix

* gctscript: fix tests

* glorious: niiiiiiiiiiiiits

* scriptmodules/gct: standardize runtime errors

* Update gctscript/modules/gct/exchange.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update gctscript/modules/gct/exchange.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update gctscript/modules/gct/exchange.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update gctscript/modules/gct/exchange.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update gctscript/modules/gct/gct.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update gctscript/modules/gct/gct.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update gctscript/modules/gct/gct.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update gctscript/modules/gct/gct.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update gctscript/modules/gct/gct.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update gctscript/modules/gct/gct.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update gctscript/modules/gct/gct.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits/reverts

* go mod: tidy

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2022-08-17 14:18:53 +10:00
Adrian Gallagher
68588560e3 CI: Bump go version, linters and fix minor issues (#1010)
* Bump golang, golangci-lint versions and fix issues

* Add -fno-stack-protector

* Fix AppVeyor golangci-lint ver

* Nitters

* Nitters round 2
2022-08-17 11:37:22 +10:00
samuael
ace5e64c10 exchanges: Add Binance.us support (#949)
* public endpoints methods added

* Completing mapping REST endpoints

* Binanceus Wrapper methods -Partially

* BinaWra functions with test funs; Not Completed

* Finalizing wrapper methods & test

* Finalizing wrapper methods & test

* Fix & Complete wrapper functions

* Adding Stream Datas

* WS Test functions

* CI: Fix golangci-lint linter issues

* CI: Fix reverting unnessesary changes and type conversion issues

* CI: Fix reverting unnessesary changes and type conversion issues

* CI: Fix comment, method use, error handling, and code handling issues

* build(deps): bump github.com/urfave/cli/v2 from 2.4.0 to 2.4.8 (#932)

Bumps [github.com/urfave/cli/v2](https://github.com/urfave/cli) from 2.4.0 to 2.4.8.
- [Release notes](https://github.com/urfave/cli/releases)
- [Changelog](https://github.com/urfave/cli/blob/main/docs/CHANGELOG.md)
- [Commits](https://github.com/urfave/cli/compare/v2.4.0...v2.4.8)

---
updated-dependencies:
- dependency-name: github.com/urfave/cli/v2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* build(deps): bump google.golang.org/grpc from 1.45.0 to 1.46.0 (#931)

Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.45.0 to 1.46.0.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.45.0...v1.46.0)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* asset: bitmask type optimisation (#922)

* asset: basic optim. bitmask

* glorious: nits

* currency: forgot parralel in testttttt

* ticker/orderbook: test fixes

* engine/rpcserver: fix and expand tests

* test: use `T.TempDir` to create temporary test directory (#934)

* test: use `T.TempDir` to create temporary test directory

This commit replaces `ioutil.TempDir` with `t.TempDir` in tests. The
directory created by `t.TempDir` is automatically removed when the test
and all its subtests complete.

Prior to this commit, temporary directory created using `ioutil.TempDir`
needs to be removed manually by calling `os.RemoveAll`, which is omitted
in some tests. The error handling boilerplate e.g.
	defer func() {
		if err := os.RemoveAll(dir); err != nil {
			t.Fatal(err)
		}
	}
is also tedious, but `t.TempDir` handles this for us nicely.

Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>

* test: fix TestEncryptTwiceReusesSaltButNewCipher on Windows

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>

* test: fix TestCheckConnection on Windows

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>

* test: fix TestRPCServer_GetTicker_LastUpdatedNanos on Windows

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>

* test: cleanup TestGenerateReport

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>

* account: storage, processing and method on balances update (#916)

* account: update account storage, retrieval and implement alert functionality when a currency change occurs.

* account: Add cancel channel

* account: remove old code

* account: don't embed mutex

* Update exchanges/account/account.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/account/account.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/account/account.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* account: addr nits

* account: Pull out test into indiv.

* account: Add test for update method

* account: add no change to test

* Update exchanges/account/account.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* account/portfolio: differentiate between asset type segregation and default to spot holdings.

* glorious: nits

* thrasher: nit

* ticker: fix spelling

* Update engine/portfolio_manager.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* thrasher: nits

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* build(deps): bump github.com/urfave/cli/v2 from 2.4.8 to 2.5.1 (#936)

Bumps [github.com/urfave/cli/v2](https://github.com/urfave/cli) from 2.4.8 to 2.5.1.
- [Release notes](https://github.com/urfave/cli/releases)
- [Changelog](https://github.com/urfave/cli/blob/main/docs/CHANGELOG.md)
- [Commits](https://github.com/urfave/cli/compare/v2.4.8...v2.5.1)

---
updated-dependencies:
- dependency-name: github.com/urfave/cli/v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* orderbook/buffer: data integrity and resubscription pass (#910)

* orderbook/buffer: data integrity and resubscription pass

* btcmarkets: REMOVE THAT LIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIINE!!!!!!!!!!!!!!!!!

* buffer: reinstate publish, refaactor, invalidate more and comments

* buffer/orderbook: improve update and snapshot performance. Move Update type to orderbook package to util. pointer through entire function calls. (cleanup). Change action string to uint8 for easier comparison. Add parsing helper. Update current test benchmark comments.

* dispatch: change publish func to variadic id param

* dispatch: remove sender receiver wait time as this adds overhead and complexity. update tests.

* dispatch: don't create pointers for every job container

* rpcserver: fix assertion issues with data publishing change

* linter: fixes

* glorious: nits addr

* depth: change validation handling to incorporate and store err

* linter: fix more issues

* dispatch: fix race

* travis: update before fetching

* depth: wrap and return wrapped error in invalidate call and fix tests

* btcmarkets: fix commenting

* workflow: check

* workflow: check

* orderbook: check error

* buffer/depth: return invalidation error and fix tests

* gctcli: display errors on orderbook streams

* buffer: remove unused types

* orderbook/bitmex: shift function to bitmex

* orderbook: Add specific comments to unexported functions that don't have locking require locking.

* orderbook: restrict published data functionality to orderbook.Outbound interface

* common: add assertion failure helper for error

* dispatch: remove atomics, add mutex protection, remove add/remove worker, redo main tests

* dispatch: export function

* engine: revert and change sub logger to manager

* engine: remove old test

* dispatch: add common variable ;)

* btcmarket: don't overflow int in tests on 32bit systems

* ci: force 1.17.7 usage for go

* Revert "ci: force 1.17.7 usage for go"

This reverts commit af2f95563bf218cf2b9f36a9fcf3258e2c6a2d91.

* golangci: bump version add and remove linter items

* Revert "golangci: bump version add and remove linter items"

This reverts commit 3c98bffc9d030e39faca0387ea40c151df2ab06b.

* dispatch: remove unsused mutex from mux

* order: slight optimizations

* nits: glorious

* dispatch: fix regression on uuid generation and input inline with master

* linter: fix

* linter: fix

* glorious: nit - rm slice segration

* account: fix test after merge

* coinbasepro: revert change

* account: close channel instead of needing a receiver, push alert in routine to prepare for waiter.

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>

* exchange/wrapper: add GetServerTime() for exchange analytics (#938)

* exchange/wrapper: add GetServerTime() for exchange analytics

* binance: fix linter issue

* glorious: nits

* glorious: nits rides again

* thrasher: nits implement huobi

* thrasher: nits add to exchange_wrapper_issues cmd

* order: slight optimizations (#917)

* order: slight optimizations

* orders: add benchmarks, small optimize and change order side to uin8 for comparitive optimizations.

* orders: continue to convert string type -> uint

* orders/backtester: interim move type to orders package, later can expand or deprecate.

* orders: handle errors

* orders: optimize filters and remove error returns when its clearly not needed

* orders: remove log call

* backtester: zero value check

* orders/futures: zero value -> flag

* linter: fix

* linter: more fixes

* linters: rides again

* glorious: nits

* common: Add zero value unix check for time values; also addresses glorious nits

* glorious scott: nits

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>

* btcm: add order execution limit wrapper support (#941)

* btcm: add in order execution limit fetching

* btcm/test: add t.Parrrrrralllleeellllllllll

* btcm/wrapper: add update on startup

* glorious: nits

* thrasher: nit add status field

* build(deps): bump github.com/urfave/cli/v2 from 2.5.1 to 2.6.0 (#944)

Bumps [github.com/urfave/cli/v2](https://github.com/urfave/cli) from 2.5.1 to 2.6.0.
- [Release notes](https://github.com/urfave/cli/releases)
- [Changelog](https://github.com/urfave/cli/blob/main/docs/CHANGELOG.md)
- [Commits](https://github.com/urfave/cli/compare/v2.5.1...v2.6.0)

---
updated-dependencies:
- dependency-name: github.com/urfave/cli/v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* proto/lint: Add protobuf GitHub action and linter (#943)

* Buf upgrades

* Buf format and basic endpoint fixes

* gRPC linter fixes

* Amend buf.yaml linter exceptions

* Update README

* Freshly generated gRPC code after depends update

* Nitterinos

* ordermanager: fix test error introduced in #917  (#942)

* ordermanager: fix residual test issue from #917 and reduce some racey action

* glorious: nits; also removed functions that weren't being used and were unexported

* rm: pew

* linter: fix issues

* glourious: nits

* credentials: fix test issue with racey racey horse basey

* engine: Add websocket data handler register function (#935)

* engine: Add websocket interceptor register function

* Update engine/engine.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update engine/websocketroutine_manager_types.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* engine/websock: switch to data handler function register and range over handlers to still include default gct handling

* engine/websocket: change name

* glorious: nits

* linter: fix

* glorious: nits

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* btcm: add modify order functionality, change return to pointer (#940)

* btcm: add modify order functionality, change return to pointer

* glorious: nits

* glorious: nits

* btcm: Adjust function name

* thrasher: nits

* thrasher: nits cont...

* request: adds WithVerbose function to package to add verbosity to request context (#950)

* request: adds WithVerbose function to package to add verbosity to request context

* request: add t.Parr....

* thrasher: nits

* build(deps): bump google.golang.org/grpc from 1.46.0 to 1.46.2 (#951)

Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.46.0 to 1.46.2.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.46.0...v1.46.2)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* exchange: expose GetCredentials() and split GetAuthenticatedAPISupport() (#954)

* exchange/wrapper: expose GetCredentials func to IBotInterface

* exchanges: split up GetAuthenticatedAPISupport into specific function calls, organize IBotExchange functionality getter functions

* interface: change name - RPCSercer: rm GetBase func call.

* glorious: nits (fix panic)

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>

* CI: merge fixes

* CI: fixing github generated lint issues

* orders: Add method for creating cancel struct from order details (#947)

* orders: Add method for creating cancel struct from order details

* orders: remove uneeded fields

* glorious: nit

* grpc: add shutdown call for external management (#957)

* grpc: add shutdown call for external management

* go mod: tidy

* glorious: suggestion

* Update engine/engine.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update engine/rpcserver.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update main.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update engine/rpcserver.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* build(deps): bump github.com/lib/pq from 1.10.5 to 1.10.6 (#960)

Bumps [github.com/lib/pq](https://github.com/lib/pq) from 1.10.5 to 1.10.6.
- [Release notes](https://github.com/lib/pq/releases)
- [Commits](https://github.com/lib/pq/compare/v1.10.5...v1.10.6)

---
updated-dependencies:
- dependency-name: github.com/lib/pq
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* build(deps): bump github.com/urfave/cli/v2 from 2.6.0 to 2.8.0 (#958)

Bumps [github.com/urfave/cli/v2](https://github.com/urfave/cli) from 2.6.0 to 2.8.0.
- [Release notes](https://github.com/urfave/cli/releases)
- [Changelog](https://github.com/urfave/cli/blob/main/docs/CHANGELOG.md)
- [Commits](https://github.com/urfave/cli/compare/v2.6.0...v2.8.0)

---
updated-dependencies:
- dependency-name: github.com/urfave/cli/v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* build(deps): bump github.com/grpc-ecosystem/grpc-gateway/v2 (#963)

Bumps [github.com/grpc-ecosystem/grpc-gateway/v2](https://github.com/grpc-ecosystem/grpc-gateway) from 2.10.0 to 2.10.2.
- [Release notes](https://github.com/grpc-ecosystem/grpc-gateway/releases)
- [Changelog](https://github.com/grpc-ecosystem/grpc-gateway/blob/master/.goreleaser.yml)
- [Commits](https://github.com/grpc-ecosystem/grpc-gateway/compare/v2.10.0...v2.10.2)

---
updated-dependencies:
- dependency-name: github.com/grpc-ecosystem/grpc-gateway/v2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* orders: adds method to retrieve snapshot of order execution limit values (#946)

* orders: add method to Limit to retrieve order execution limit snapshots

* currency/btcmarkets: add error and update field name to standard

* linter: fix

* limts: don't return pointer

* limit: Add notes

* glorious: nits

* linter: fix

* limit: reinstate nil check

* exchanges: change field names to be more consistent (@thrasher-) suggestion

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>

* orders: Add derive modify struct method from order.Detail  (#948)

* orders: Add derive modify struct method to order.Detail and then subsequent method to derive and standardize response details

* exchanges: call modify method in wrappers

* linter: fixes

* engine/wsroutineman: remove print summary

* glorious: nits, removed modifyOrder functionality for Bithumb. There are not docs to support this.

* Update exchanges/order/orders.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* build(deps): bump github.com/spf13/viper from 1.11.0 to 1.12.0 (#965)

Bumps [github.com/spf13/viper](https://github.com/spf13/viper) from 1.11.0 to 1.12.0.
- [Release notes](https://github.com/spf13/viper/releases)
- [Commits](https://github.com/spf13/viper/compare/v1.11.0...v1.12.0)

---
updated-dependencies:
- dependency-name: github.com/spf13/viper
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* build(deps): bump github.com/urfave/cli/v2 from 2.8.0 to 2.8.1 (#964)

Bumps [github.com/urfave/cli/v2](https://github.com/urfave/cli) from 2.8.0 to 2.8.1.
- [Release notes](https://github.com/urfave/cli/releases)
- [Changelog](https://github.com/urfave/cli/blob/main/docs/CHANGELOG.md)
- [Commits](https://github.com/urfave/cli/compare/v2.8.0...v2.8.1)

---
updated-dependencies:
- dependency-name: github.com/urfave/cli/v2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* orders: Add methods to derive SubmitResponse and Detail types (#955)

* orders: deprecate SubmitResponse return and change to *order.Detail construct detail from order.Submit struct

* orders: add coverage, fix tests

* coinut: rm test for checking

* orders: revert change for return and change field ID to a more explicit name OrderID

* orders: Add method to see if the order was placed

* order: change field name in Cancel type to be more explicit

* orders: standardize field -> OrderID

* backtester: populate change

* orders: add test

* gctscript: fix field name

* linter: fix issues

* linter: more fixes

* linter: forever

* exchanges_tests: populate order.Submit field exchange name

* Update exchanges/order/order_types.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/order/orders.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* glorious: nits

* thrasher: nits

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* build(deps): bump bufbuild/buf-setup-action from 1.4.0 to 1.5.0 (#973)

Bumps [bufbuild/buf-setup-action](https://github.com/bufbuild/buf-setup-action) from 1.4.0 to 1.5.0.
- [Release notes](https://github.com/bufbuild/buf-setup-action/releases)
- [Commits](https://github.com/bufbuild/buf-setup-action/compare/v1.4.0...v1.5.0)

---
updated-dependencies:
- dependency-name: bufbuild/buf-setup-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* build(deps): bump google.golang.org/grpc from 1.46.2 to 1.47.0 (#972)

Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.46.2 to 1.47.0.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.46.2...v1.47.0)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* build(deps): bump github.com/grpc-ecosystem/grpc-gateway/v2 (#971)

Bumps [github.com/grpc-ecosystem/grpc-gateway/v2](https://github.com/grpc-ecosystem/grpc-gateway) from 2.10.2 to 2.10.3.
- [Release notes](https://github.com/grpc-ecosystem/grpc-gateway/releases)
- [Changelog](https://github.com/grpc-ecosystem/grpc-gateway/blob/master/.goreleaser.yml)
- [Commits](https://github.com/grpc-ecosystem/grpc-gateway/compare/v2.10.2...v2.10.3)

---
updated-dependencies:
- dependency-name: github.com/grpc-ecosystem/grpc-gateway/v2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* build(deps): bump github.com/d5/tengo/v2 from 2.10.1 to 2.11.2 (#975)

Bumps [github.com/d5/tengo/v2](https://github.com/d5/tengo) from 2.10.1 to 2.11.2.
- [Release notes](https://github.com/d5/tengo/releases)
- [Changelog](https://github.com/d5/tengo/blob/master/.goreleaser.yml)
- [Commits](https://github.com/d5/tengo/compare/v2.10.1...v2.11.2)

---
updated-dependencies:
- dependency-name: github.com/d5/tengo/v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* build(deps): bump github.com/urfave/cli/v2 from 2.8.1 to 2.10.1 (#979)

Bumps [github.com/urfave/cli/v2](https://github.com/urfave/cli) from 2.8.1 to 2.10.1.
- [Release notes](https://github.com/urfave/cli/releases)
- [Changelog](https://github.com/urfave/cli/blob/main/docs/CHANGELOG.md)
- [Commits](https://github.com/urfave/cli/compare/v2.8.1...v2.10.1)

---
updated-dependencies:
- dependency-name: github.com/urfave/cli/v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* build(deps): bump github.com/d5/tengo/v2 from 2.11.2 to 2.12.0 (#978)

Bumps [github.com/d5/tengo/v2](https://github.com/d5/tengo) from 2.11.2 to 2.12.0.
- [Release notes](https://github.com/d5/tengo/releases)
- [Changelog](https://github.com/d5/tengo/blob/master/.goreleaser.yml)
- [Commits](https://github.com/d5/tengo/compare/v2.11.2...v2.12.0)

---
updated-dependencies:
- dependency-name: github.com/d5/tengo/v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* CI: adding WS and REST tests, and minor fixes

* CI: fixes on available asset and related minor issues

* CI: fixes on endpoint function, test functions, and types

* CI: updating templates and slight fixes

* CI: updating slight fixes on tests and withdraws Request model

* build(deps): bump styfle/cancel-workflow-action from 0.9.1 to 0.10.0 (#985)

Bumps [styfle/cancel-workflow-action](https://github.com/styfle/cancel-workflow-action) from 0.9.1 to 0.10.0.
- [Release notes](https://github.com/styfle/cancel-workflow-action/releases)
- [Commits](https://github.com/styfle/cancel-workflow-action/compare/0.9.1...0.10.0)

---
updated-dependencies:
- dependency-name: styfle/cancel-workflow-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* build(deps): bump bufbuild/buf-setup-action from 1.5.0 to 1.6.0 (#984)

Bumps [bufbuild/buf-setup-action](https://github.com/bufbuild/buf-setup-action) from 1.5.0 to 1.6.0.
- [Release notes](https://github.com/bufbuild/buf-setup-action/releases)
- [Commits](https://github.com/bufbuild/buf-setup-action/compare/v1.5.0...v1.6.0)

---
updated-dependencies:
- dependency-name: bufbuild/buf-setup-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* backtester: Futures handling & FTX Cash and Carry example strategy (#930)

* implements futures functions and GRPC functions on new branch

* lint and test fixes

* Fix uneven split pnl. Adds collateral weight test. docs. New clear func

* Test protection if someone has zero collateral

* Uses string instead of double for accuracy

* Fixes old code panic

* context, match, docs

* Addresses Shazniterinos, var names, expanded tests

* Returns subaccount name, provides USD values when offlinecalc

* Fixes oopsie

* Fixes cool bug which allowed made up subaccount results

* Subaccount override on FTX, subaccount results for collateral

* Strenghten collateral account info checks. Improve FTX test

* English is my first language

* Fixes oopsies

* Adds some conceptual futures order details to track PNL

* Initial design of future order processing in the backtester

* Introduces futures concept for collateral and spot/futures config diffs

* Fixes most tests

* Simple designs for collateral funding pair concept

* Expands interface use so much it hurts

* Implements more collateral interfaces

* Adds liquidation, adds strategy, struggles with Binance

* Attempts at getting FTX to work

* Adds calculatePNL as a wrapper function and adds an `IsFutures` asset check

* Successfully loads backtester with collateral currency

* Fails to really get much going for supporting futures

* Merges master changes

* Fleshes out how FTX processes collateral

* Further FTX collateral workings

* hooks up more ftx collateral and pnl calculations

* more funcs to flesh out handling

* Adds more links, just can't fit the pieces together :(

* Greatly expands futures order processing

* Fleshes out position tracker to also handle asset and exchange +testing

* RM linkedOrderID. rn positioncontroller, unexport

* Successfully tracks futures order positions

* Fails to calculate PNL

* Calculates pnl from orders accurately with exception to flipping orders

* Calculates PNL from orders

* Adds another controller layer to make it ez from orderstore

* Backtester now compiles. Adds test coverage

* labels things add scaling collateral test

* Calculates pnl in line with fees

* Mostly accurate PNL, with exception to appending with diff prices

* Adds locks, adds rpc function

* grpc implementations

* Gracefully handles rpc function

* beautiful tests!

* rejiggles tests to polish

* Finishes FTX testing, adds comments

* Exposes collateral calculations to rpc

* Adds commands and testing for rpcserver.go functions

* Increase testing and fix up backtester code

* Returns cool changes to original branch

* end of day fixes

* Fixing some tests

* Fixing tests 🎉

* Fixes all the tests

* Splits the backtester setup and running into different files

* Merge, minor fixes

* Messing with some strategy updates

* Failed understanding at collateral usage

* Begins the creation of cash and carry strategy

* Adds underlying pair, adds filldependentevent for futures

* Completes fill prerequsite event implementation. Can't short though

* Some bug fixes

* investigating funds

* CAN NOW CREATE A SHORT ORDER

* Minor change in short size

* Fixes for unrealised PNL & collateral rendering

* Fixes lint and tests

* Adds some verbosity

* Updates to pnl calc

* Tracks pnl for short orders, minor update to strategy

* Close and open event based on conditions

* Adds pnl data for currency statistics

* Working through PNL calculation automatically. Now panics

* Adds tracking, is blocked from design

* Work to flesh out closing a position

* vain attempts at tracking zeroing out bugs

* woww, super fun new subloggers 🎉

* Begins attempt at automatically handling contracts and collateral based on direction

* Merge master + fixes

* Investigating issues with pnl and holdings

* Minor pnl fixes

* Fixes future position sizing, needs contract sizing

* Can render pnl results, focussing on funding statistics

* tracking candles for futures, but why not btc

* Improves funding statistics

* Colours and stats

* Fixes collateral and snapshot bugs

* Completes test

* Fixes totals bug

* Fix double buy, expand stats, fixes usd totals, introduce interface

* Begins report formatting and calculations

* Appends pnl to receiving curr. Fixes map[time]. accurate USD

* Improves report output rendering

* PNL stats in report. New tests for futures

* Fixes existing tests before adding new coverage

* Test coverage

* Completes portfolio coverage

* Increase coverage exchange, portfolio. fix size bug. NEW CHART

* WHAT IS GOING ON WITH PNL

* Fixes PNL calculation. Adds ability to skip om futures tracking

* minor commit before merge

* Adds basic liquidation to backtester

* Changes liquidation to order based

* Liquidationnnnnn

* Further fleshes out liquidations

* Completes liquidations in a honorable manner. Adds AppendReasonf

* Beginnings of spot futures gap chart. Needs to link currencies to render difference

* Removes fake liquidation. Adds cool new chart

* Fixes somet tests,allows for zero fee value v nil distinction,New tests

* Some annoying test fixes that took too long

* portfolio coverage

* holding coverage, privatisation funding

* Testwork

* boring tests

* engine coverage

* More backtesting coverage

* Funding, strategy, report test coverage

* Completes coverage of report package

* Documentation, fixes some assumptions on asset errors

* Changes before master merge

* Lint and Tests

* defaults to non-coloured rendering

* Chart rendering

* Fixes surprise non-local-lints

* Niterinos to the extremeos

* Fixes merge problems

* The linter splintered across the glinting plinths

* Many nits addressed. Now sells spot position on final candle

* Adds forgotten coverage

* Adds ability to size futures contracts to match spot positions.

* fixes order sell sizing

* Adds tests to sizing. Fixes charting issue

* clint splintered the linters with flint

* Improves stats, stat rendering

* minifix

* Fixes tests and fee bug

* Merge fixeroos

* Microfixes

* Updates orderPNL on first Correctly utilises fees. Adds committed funds

* New base funcs. New order summary

* Fun test updates

* Fix logo colouring

* Fixes niteroonies

* Fix report

* BAD COMMIT

* Fixes funding issues.Updates default fee rates.Combines cashcarry case

* doc regen

* Now returns err

* Fixes sizing bug issue introduced in PR

* Fixes fun fee/total US value bug

* Fix chart bug. Show log charts with disclaimer

* sellside fee

* fixes fee and slippage view

* Fixed slippage price issue

* Fixes calculation and removes rendering

* Fixes stats and some rendering

* Merge fix

* Fixes merge issues

* go mod tidy, lint updates

* New linter attempt

* Version bump in appveyor and makefile

* Regex filename, config fixes, template h2 fixes

* Removes bad stats.

* neatens config builder. Moves filename generator

* Fixes issue where linter wants to fix my spelling

* Fixes pointers and starts

* build(deps): bump github.com/urfave/cli/v2 from 2.10.1 to 2.10.3 (#982)

Bumps [github.com/urfave/cli/v2](https://github.com/urfave/cli) from 2.10.1 to 2.10.3.
- [Release notes](https://github.com/urfave/cli/releases)
- [Changelog](https://github.com/urfave/cli/blob/main/docs/CHANGELOG.md)
- [Commits](https://github.com/urfave/cli/compare/v2.10.1...v2.10.3)

---
updated-dependencies:
- dependency-name: github.com/urfave/cli/v2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* alert: Add optimizations (#939)

* alert: Add optimizations

* alert: add basic benchmarks

* alert: fix linter issue

* documentation: change to text/template as html/template escapes to protect against code injection. Add readme.md for alert.

* README: Add package name

* alert: link up with engine settings

* request: isVerbose refactor

* Update exchanges/alert/alert_test.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/alert/alert.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* glorious: fun police

* documentation: regen

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* technical_analysis: TWAP & VWAP + TA methods to candles and link to existing RPC server for GCTCLI prototyping  (#970)

* kline: add weighted price helpers for candles

* twap/vwap: basic implementation and hook to rpc for protype

* ta: cont implementation. (WIP)

* kline: Add tests

* kline: add helpers

* ta: full impl.

* kline: remove support for macd and add in correlation-coefficient handling

* rpc: change naming convention

* linter: fix

* protolinter: fix

* linter: ++

* kline: reinstate macd handling after adding in check

* glorious: nits

* gctcl: linter

* Update exchanges/kline/weighted_price.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* glorious: nits v2.0

* kline: fix test

* huobi-tests: shift from next quarter to this weeks contracts as they were erroring out in tests.

* btcmarkets: update supported kline intervals

* zb: fix test

* rpcserver: fix bug and tests

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* build(deps): bump github.com/urfave/cli/v2 from 2.10.3 to 2.11.0 (#993)

Bumps [github.com/urfave/cli/v2](https://github.com/urfave/cli) from 2.10.3 to 2.11.0.
- [Release notes](https://github.com/urfave/cli/releases)
- [Changelog](https://github.com/urfave/cli/blob/main/docs/CHANGELOG.md)
- [Commits](https://github.com/urfave/cli/compare/v2.10.3...v2.11.0)

---
updated-dependencies:
- dependency-name: github.com/urfave/cli/v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* FTX: Margin lending/borrow rate history (#981)

* Adds lending rates/borrows to FTX and the command

* Movements, renames, rpc test

* Fleshing out rpc response

* Allows rpcserver to calculate offline (but not gctcli). Expands tests

* rn structs. add exchange_wrapper_issues support

* Adds a nice yearly rate

* Surprise yearly borrow rate!

* Rn+Mv to margin package. Fixes some serious whoopsies

* Adds average lend/borrow rates instead of sum

* rm oopsie whoopsie

* This is what the linter was having an issue with

* re-gen

* lintl

* niteroos

* build(deps): bump google.golang.org/grpc from 1.47.0 to 1.48.0 (#995)

Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.47.0 to 1.48.0.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.47.0...v1.48.0)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* account: segregate holdings by credentials for future multi-key management (#956)

* exchanges/account: shift credentials to account package and segregate funds to keys

* merge: fixes

* linter: fix

* Update exchanges/account/account.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits + protection for string panic

* glorious_suggestion: add method for matching keys

* linter: fix tests

* account: add protected method for credentials minimizing access, display full account details to rpc.

* linter: spelling kweeeeeeen

* accounts/portfolio: clean/check portfolio code and quickly check balances from change. Add protected method for future matching.

* accounts: theres no point in pointerising everything

* linter: ok pointerise this then...

* exchanges: fix regression add in little notes.

* glorious: nits

* Update exchanges/account/credentials.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/account/credentials_test.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/account/credentials_test.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* gloriously: fix glorious glorious test gloriously

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* CI: fixing linter issue and conflicts

* CI: fixing ratelimit and other slight issues

* build(deps): bump github.com/grpc-ecosystem/grpc-gateway/v2 (#998)

Bumps [github.com/grpc-ecosystem/grpc-gateway/v2](https://github.com/grpc-ecosystem/grpc-gateway) from 2.10.3 to 2.11.0.
- [Release notes](https://github.com/grpc-ecosystem/grpc-gateway/releases)
- [Changelog](https://github.com/grpc-ecosystem/grpc-gateway/blob/master/.goreleaser.yml)
- [Commits](https://github.com/grpc-ecosystem/grpc-gateway/compare/v2.10.3...v2.11.0)

---
updated-dependencies:
- dependency-name: github.com/grpc-ecosystem/grpc-gateway/v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* build(deps): bump github.com/d5/tengo/v2 from 2.12.0 to 2.12.1 (#997)

Bumps [github.com/d5/tengo/v2](https://github.com/d5/tengo) from 2.12.0 to 2.12.1.
- [Release notes](https://github.com/d5/tengo/releases)
- [Changelog](https://github.com/d5/tengo/blob/master/.goreleaser.yml)
- [Commits](https://github.com/d5/tengo/compare/v2.12.0...v2.12.1)

---
updated-dependencies:
- dependency-name: github.com/d5/tengo/v2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* build(deps): bump github.com/urfave/cli/v2 from 2.11.0 to 2.11.1 (#996)

Bumps [github.com/urfave/cli/v2](https://github.com/urfave/cli) from 2.11.0 to 2.11.1.
- [Release notes](https://github.com/urfave/cli/releases)
- [Changelog](https://github.com/urfave/cli/blob/main/docs/CHANGELOG.md)
- [Commits](https://github.com/urfave/cli/compare/v2.11.0...v2.11.1)

---
updated-dependencies:
- dependency-name: github.com/urfave/cli/v2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* CI: fixing type and other slight issues

* Cleanup after merge

* Endpoints rate limit update

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Ryan O'Hara-Reid <oharareid.ryan@gmail.com>
Co-authored-by: Eng Zer Jun <engzerjun@gmail.com>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
2022-08-12 15:30:11 +10:00
Jaydeep Rajpurohit
247da918a8 exchanges: Add ByBit support (#887)
* few fixes and add ratelimiter

* adds test

* revert configtest.json changes

* configtest updated

* WIP: adds public endpoint support

* WIP: adds public endpoint support

* adds public endpoint support

* WIP: adds auth. endpoint support

* adds test for auth. endpoint

* fixes

* adds auth. endpoint support

* WIP: ws support

* WIP

* WIP

* WIP

* WIP

* WIP

* WIP

* WIP

* Testing

* Complete WS spot testing

* adds support for ws events

* minor change

* WIP: adds REST support for CoinMarginedFutures

* Fixes

* WIP: adds REST support for CoinMarginedFutures

* Fixes

* improvement in SPOT REST

* Typo fix

* WIP: add REST support for CMF Account API

* minor fixes

* WIP: add support for CMF conditional orders and few minor fixes

* complete support for CMF conditional orders

* adds support for public CMF endpoint

* adds support for CMF position API

* Complete REST CMF support

* WIP

* Testing REST CMF support

* Testing REST CMF support

* Testing REST CMF support completed

* WIP: add support for UMF

* completed non-auth UMF

* WIP: add support for REST Auth. UMF

* WIP: add support for REST Auth. UMF and some improvements

* WIP

* WIP

* WIP

* completed REST UMF

* renaming

* adds REST support for futures

* add testcases for UMF and some optimizations

* add testcases for futures

* Testing UMF, futures and its changes

* Fixes

* Fixes after testing

* WIP

* WIP

* WIP

* completed ws USDT futures support

* WIP: ws support for futures

* fixes in WS futures

* fixes in WS support

* roll back changes made for WS CMF, USDT and Futures

* fixes

* WIP

* WIP

* fixes

* Steps for new PR

* WIP

* WIP

* WIP

* WIP

* complete PR setup

* fixes for successfully running tests

* update in symbol for futures pair in test file

* WIP

* Fixes in test file and other minor fix

* fix testdata/configtest.json

* reset CONTRIBUTORS file

* review changes

* remove unwanted file

* remove redundant code

* improvisation

* adds comment for exported functions

* remove unwanted TODO and commented code

* fix

* improvisation

* fix

* defined errors

* improvisation

* improvisation

* improvisation

* updates test

* adds comment for exported types

* review changes

* review changes

* fix

* fixes

* Changes for making BYBIT compatible with existing code base

* Test file changes

* Changes for making BYBIT compatible with existing code base

* Changes for making BYBIT compatible with existing code base

* fix lint issues

* fix

* review changes

* review changes

* review changes

* review changes

* review changes

* review changes

* review changes

* review changes

* review changes

* review changes

* WIP

* add test cases for new API's

* minor improvements

* add missing API and their tests

* minor fixes

* add bybitTime

* add bybitTimeSec, bybitTimeMilliSec, bybitTimeNanoSec and necessary support

* fix GetTradeHistory function

* error handling

* test fixes

* add GetServerTime API

* adds GetHistoricCandlesExtended and review changes

* test fixes

* minor fix

* integrating CMF Bybit recent change log

* minor fixes

* adds extractCurrencyPair

* minor fixes

* minor fix

* review changes

* adds variable declaration of error

* review commit

* adds embeddable type in API response for all API and integrate it

* fixes

* adds authentication WS connection

* review changes

* review changes

* compatible changes

* adds asset to GetWithdrawalsHistory

* adds asset_type in rpc.proto

* adds asset argument in gctcli withdrawal request command

* improve error handling in exchange API error

* web socket fix

* review changes

* improvements

* improvements

* minor fix

* review changes

* fixing wrapper issues

* fixes

* fixes

* review changes

* add test cases

* fix for GetActiveOrders

* lint fixes

* fixes in websocket

* adds wrapper testcases

* adds wrapper testcases

* adds wrapper testcases

* fixes

* fix issue with GetHistoricCandlesExtended

* fix merge issues

* improving error reporting

* adds wrapper testcases and a minor fix

* gctrpc changes

* adds test cases
fixes in websocket

* review changes for ws

* review changes in WS

* fix gctrpc

* merge fixes

* review changes

* WIP

* updates pair in configs

* adds new asset USDCMarginedFutures

* adds URL const for USDCMarginedFutures

* adds API support

* minor fixes

* adds kline API

* minor fix

* adds API

* adds API

* adds API

* WIP

* WIP

* WIP

* adds support for USDC auth requests to SendAuthHTTPRequest

* adds SendUSDCAuthHTTPRequest

* run test and fix them

* rollback support added for Auth. USDC request inside SendAuthHTTPRequest

* adds API and test cases

* adds API and test cases

* adds APIs and test cases

* adds APIs

* adds rate limit for USDC

* adds USDCMarginedFutures to wrapper

* adds USDC testcases in wrapper and fix few issues

* minor test fixes

* minor test fixes

* fix lint issues

* WIP

* Merge changes

* minor fixes

* remove "else" and optimize

* review changes

* review changes

* review changes

* fix lint issue

* merge fix

* fix test

* fix templates and run them

* changes after merge

* review changes and improvements

* code improvement

* fixes with respect to changes in API response in documentation

* fixed review change in test

* adds check in CancelExistingOrder

* update exchange template

* review changes

* adds GetDepositAddress API

* WIP: adds GetOrderHistory

* complete GetOrderHistory

* fixes

* adds test case

* fixes and add WithdrawFund API

* WIP

* WIP

* updating all SendAuthHTTPRequest call

* adds WithdrawCryptocurrencyFunds

* update test cases

* fix lint issues

* fixes after merge

* adds GetAvailableTransferChains and few fixes

* minor fix in GetDepositAddress

* minor fix with WS ping/pong handling

* add ping handler for WS Auth.

* fix typo mistake

* update doc
2022-08-08 11:29:43 +10:00
Ryan O'Hara-Reid
663e753f52 account: segregate holdings by credentials for future multi-key management (#956)
* exchanges/account: shift credentials to account package and segregate funds to keys

* merge: fixes

* linter: fix

* Update exchanges/account/account.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits + protection for string panic

* glorious_suggestion: add method for matching keys

* linter: fix tests

* account: add protected method for credentials minimizing access, display full account details to rpc.

* linter: spelling kweeeeeeen

* accounts/portfolio: clean/check portfolio code and quickly check balances from change. Add protected method for future matching.

* accounts: theres no point in pointerising everything

* linter: ok pointerise this then...

* exchanges: fix regression add in little notes.

* glorious: nits

* Update exchanges/account/credentials.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/account/credentials_test.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/account/credentials_test.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* gloriously: fix glorious glorious test gloriously

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2022-07-21 15:05:31 +10:00
Scott
bed9425a08 FTX: Margin lending/borrow rate history (#981)
* Adds lending rates/borrows to FTX and the command

* Movements, renames, rpc test

* Fleshing out rpc response

* Allows rpcserver to calculate offline (but not gctcli). Expands tests

* rn structs. add exchange_wrapper_issues support

* Adds a nice yearly rate

* Surprise yearly borrow rate!

* Rn+Mv to margin package. Fixes some serious whoopsies

* Adds average lend/borrow rates instead of sum

* rm oopsie whoopsie

* This is what the linter was having an issue with

* re-gen

* lintl

* niteroos
2022-07-12 14:27:35 +10:00
Ryan O'Hara-Reid
7da745120f technical_analysis: TWAP & VWAP + TA methods to candles and link to existing RPC server for GCTCLI prototyping (#970)
* kline: add weighted price helpers for candles

* twap/vwap: basic implementation and hook to rpc for protype

* ta: cont implementation. (WIP)

* kline: Add tests

* kline: add helpers

* ta: full impl.

* kline: remove support for macd and add in correlation-coefficient handling

* rpc: change naming convention

* linter: fix

* protolinter: fix

* linter: ++

* kline: reinstate macd handling after adding in check

* glorious: nits

* gctcl: linter

* Update exchanges/kline/weighted_price.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* glorious: nits v2.0

* kline: fix test

* huobi-tests: shift from next quarter to this weeks contracts as they were erroring out in tests.

* btcmarkets: update supported kline intervals

* zb: fix test

* rpcserver: fix bug and tests

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2022-07-08 15:21:56 +10:00
Ryan O'Hara-Reid
68db4155bf alert: Add optimizations (#939)
* alert: Add optimizations

* alert: add basic benchmarks

* alert: fix linter issue

* documentation: change to text/template as html/template escapes to protect against code injection. Add readme.md for alert.

* README: Add package name

* alert: link up with engine settings

* request: isVerbose refactor

* Update exchanges/alert/alert_test.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/alert/alert.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* glorious: fun police

* documentation: regen

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2022-07-01 15:35:28 +10:00
Scott
f929b4d51e backtester: Futures handling & FTX Cash and Carry example strategy (#930)
* implements futures functions and GRPC functions on new branch

* lint and test fixes

* Fix uneven split pnl. Adds collateral weight test. docs. New clear func

* Test protection if someone has zero collateral

* Uses string instead of double for accuracy

* Fixes old code panic

* context, match, docs

* Addresses Shazniterinos, var names, expanded tests

* Returns subaccount name, provides USD values when offlinecalc

* Fixes oopsie

* Fixes cool bug which allowed made up subaccount results

* Subaccount override on FTX, subaccount results for collateral

* Strenghten collateral account info checks. Improve FTX test

* English is my first language

* Fixes oopsies

* Adds some conceptual futures order details to track PNL

* Initial design of future order processing in the backtester

* Introduces futures concept for collateral and spot/futures config diffs

* Fixes most tests

* Simple designs for collateral funding pair concept

* Expands interface use so much it hurts

* Implements more collateral interfaces

* Adds liquidation, adds strategy, struggles with Binance

* Attempts at getting FTX to work

* Adds calculatePNL as a wrapper function and adds an `IsFutures` asset check

* Successfully loads backtester with collateral currency

* Fails to really get much going for supporting futures

* Merges master changes

* Fleshes out how FTX processes collateral

* Further FTX collateral workings

* hooks up more ftx collateral and pnl calculations

* more funcs to flesh out handling

* Adds more links, just can't fit the pieces together :(

* Greatly expands futures order processing

* Fleshes out position tracker to also handle asset and exchange +testing

* RM linkedOrderID. rn positioncontroller, unexport

* Successfully tracks futures order positions

* Fails to calculate PNL

* Calculates pnl from orders accurately with exception to flipping orders

* Calculates PNL from orders

* Adds another controller layer to make it ez from orderstore

* Backtester now compiles. Adds test coverage

* labels things add scaling collateral test

* Calculates pnl in line with fees

* Mostly accurate PNL, with exception to appending with diff prices

* Adds locks, adds rpc function

* grpc implementations

* Gracefully handles rpc function

* beautiful tests!

* rejiggles tests to polish

* Finishes FTX testing, adds comments

* Exposes collateral calculations to rpc

* Adds commands and testing for rpcserver.go functions

* Increase testing and fix up backtester code

* Returns cool changes to original branch

* end of day fixes

* Fixing some tests

* Fixing tests 🎉

* Fixes all the tests

* Splits the backtester setup and running into different files

* Merge, minor fixes

* Messing with some strategy updates

* Failed understanding at collateral usage

* Begins the creation of cash and carry strategy

* Adds underlying pair, adds filldependentevent for futures

* Completes fill prerequsite event implementation. Can't short though

* Some bug fixes

* investigating funds

* CAN NOW CREATE A SHORT ORDER

* Minor change in short size

* Fixes for unrealised PNL & collateral rendering

* Fixes lint and tests

* Adds some verbosity

* Updates to pnl calc

* Tracks pnl for short orders, minor update to strategy

* Close and open event based on conditions

* Adds pnl data for currency statistics

* Working through PNL calculation automatically. Now panics

* Adds tracking, is blocked from design

* Work to flesh out closing a position

* vain attempts at tracking zeroing out bugs

* woww, super fun new subloggers 🎉

* Begins attempt at automatically handling contracts and collateral based on direction

* Merge master + fixes

* Investigating issues with pnl and holdings

* Minor pnl fixes

* Fixes future position sizing, needs contract sizing

* Can render pnl results, focussing on funding statistics

* tracking candles for futures, but why not btc

* Improves funding statistics

* Colours and stats

* Fixes collateral and snapshot bugs

* Completes test

* Fixes totals bug

* Fix double buy, expand stats, fixes usd totals, introduce interface

* Begins report formatting and calculations

* Appends pnl to receiving curr. Fixes map[time]. accurate USD

* Improves report output rendering

* PNL stats in report. New tests for futures

* Fixes existing tests before adding new coverage

* Test coverage

* Completes portfolio coverage

* Increase coverage exchange, portfolio. fix size bug. NEW CHART

* WHAT IS GOING ON WITH PNL

* Fixes PNL calculation. Adds ability to skip om futures tracking

* minor commit before merge

* Adds basic liquidation to backtester

* Changes liquidation to order based

* Liquidationnnnnn

* Further fleshes out liquidations

* Completes liquidations in a honorable manner. Adds AppendReasonf

* Beginnings of spot futures gap chart. Needs to link currencies to render difference

* Removes fake liquidation. Adds cool new chart

* Fixes somet tests,allows for zero fee value v nil distinction,New tests

* Some annoying test fixes that took too long

* portfolio coverage

* holding coverage, privatisation funding

* Testwork

* boring tests

* engine coverage

* More backtesting coverage

* Funding, strategy, report test coverage

* Completes coverage of report package

* Documentation, fixes some assumptions on asset errors

* Changes before master merge

* Lint and Tests

* defaults to non-coloured rendering

* Chart rendering

* Fixes surprise non-local-lints

* Niterinos to the extremeos

* Fixes merge problems

* The linter splintered across the glinting plinths

* Many nits addressed. Now sells spot position on final candle

* Adds forgotten coverage

* Adds ability to size futures contracts to match spot positions.

* fixes order sell sizing

* Adds tests to sizing. Fixes charting issue

* clint splintered the linters with flint

* Improves stats, stat rendering

* minifix

* Fixes tests and fee bug

* Merge fixeroos

* Microfixes

* Updates orderPNL on first Correctly utilises fees. Adds committed funds

* New base funcs. New order summary

* Fun test updates

* Fix logo colouring

* Fixes niteroonies

* Fix report

* BAD COMMIT

* Fixes funding issues.Updates default fee rates.Combines cashcarry case

* doc regen

* Now returns err

* Fixes sizing bug issue introduced in PR

* Fixes fun fee/total US value bug

* Fix chart bug. Show log charts with disclaimer

* sellside fee

* fixes fee and slippage view

* Fixed slippage price issue

* Fixes calculation and removes rendering

* Fixes stats and some rendering

* Merge fix

* Fixes merge issues

* go mod tidy, lint updates

* New linter attempt

* Version bump in appveyor and makefile

* Regex filename, config fixes, template h2 fixes

* Removes bad stats.

* neatens config builder. Moves filename generator

* Fixes issue where linter wants to fix my spelling

* Fixes pointers and starts
2022-06-30 15:43:41 +10:00
Ryan O'Hara-Reid
a12cd3d733 orders: Add methods to derive SubmitResponse and Detail types (#955)
* orders: deprecate SubmitResponse return and change to *order.Detail construct detail from order.Submit struct

* orders: add coverage, fix tests

* coinut: rm test for checking

* orders: revert change for return and change field ID to a more explicit name OrderID

* orders: Add method to see if the order was placed

* order: change field name in Cancel type to be more explicit

* orders: standardize field -> OrderID

* backtester: populate change

* orders: add test

* gctscript: fix field name

* linter: fix issues

* linter: more fixes

* linter: forever

* exchanges_tests: populate order.Submit field exchange name

* Update exchanges/order/order_types.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/order/orders.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* glorious: nits

* thrasher: nits

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2022-06-06 11:52:15 +10:00
Ryan O'Hara-Reid
a63aa6b616 orders: Add derive modify struct method from order.Detail (#948)
* orders: Add derive modify struct method to order.Detail and then subsequent method to derive and standardize response details

* exchanges: call modify method in wrappers

* linter: fixes

* engine/wsroutineman: remove print summary

* glorious: nits, removed modifyOrder functionality for Bithumb. There are not docs to support this.

* Update exchanges/order/orders.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2022-05-30 14:38:21 +10:00
Ryan O'Hara-Reid
ee9c35d5ee grpc: add shutdown call for external management (#957)
* grpc: add shutdown call for external management

* go mod: tidy

* glorious: suggestion

* Update engine/engine.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update engine/rpcserver.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update main.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update engine/rpcserver.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2022-05-20 11:20:39 +10:00
Ryan O'Hara-Reid
c492e6600e orders: Add method for creating cancel struct from order details (#947)
* orders: Add method for creating cancel struct from order details

* orders: remove uneeded fields

* glorious: nit
2022-05-19 17:53:58 +10:00
Ryan O'Hara-Reid
14cde7b786 exchange: expose GetCredentials() and split GetAuthenticatedAPISupport() (#954)
* exchange/wrapper: expose GetCredentials func to IBotInterface

* exchanges: split up GetAuthenticatedAPISupport into specific function calls, organize IBotExchange functionality getter functions

* interface: change name - RPCSercer: rm GetBase func call.

* glorious: nits (fix panic)

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
2022-05-18 10:52:16 +10:00
Ryan O'Hara-Reid
16a93b49a4 btcm: add modify order functionality, change return to pointer (#940)
* btcm: add modify order functionality, change return to pointer

* glorious: nits

* glorious: nits

* btcm: Adjust function name

* thrasher: nits

* thrasher: nits cont...
2022-05-16 10:55:23 +10:00
Ryan O'Hara-Reid
ccde38d25a engine: Add websocket data handler register function (#935)
* engine: Add websocket interceptor register function

* Update engine/engine.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update engine/websocketroutine_manager_types.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* engine/websock: switch to data handler function register and range over handlers to still include default gct handling

* engine/websocket: change name

* glorious: nits

* linter: fix

* glorious: nits

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2022-05-16 09:04:17 +10:00
Ryan O'Hara-Reid
5cb26e7ecf ordermanager: fix test error introduced in #917 (#942)
* ordermanager: fix residual test issue from #917 and reduce some racey action

* glorious: nits; also removed functions that weren't being used and were unexported

* rm: pew

* linter: fix issues

* glourious: nits

* credentials: fix test issue with racey racey horse basey
2022-05-11 14:18:21 +10:00
Adrian Gallagher
61212fb8ea proto/lint: Add protobuf GitHub action and linter (#943)
* Buf upgrades

* Buf format and basic endpoint fixes

* gRPC linter fixes

* Amend buf.yaml linter exceptions

* Update README

* Freshly generated gRPC code after depends update

* Nitterinos
2022-05-10 15:38:50 +10:00
Ryan O'Hara-Reid
cdcc9630de order: slight optimizations (#917)
* order: slight optimizations

* orders: add benchmarks, small optimize and change order side to uin8 for comparitive optimizations.

* orders: continue to convert string type -> uint

* orders/backtester: interim move type to orders package, later can expand or deprecate.

* orders: handle errors

* orders: optimize filters and remove error returns when its clearly not needed

* orders: remove log call

* backtester: zero value check

* orders/futures: zero value -> flag

* linter: fix

* linter: more fixes

* linters: rides again

* glorious: nits

* common: Add zero value unix check for time values; also addresses glorious nits

* glorious scott: nits

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
2022-05-06 12:27:21 +10:00
Ryan O'Hara-Reid
c6ad429827 orderbook/buffer: data integrity and resubscription pass (#910)
* orderbook/buffer: data integrity and resubscription pass

* btcmarkets: REMOVE THAT LIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIINE!!!!!!!!!!!!!!!!!

* buffer: reinstate publish, refaactor, invalidate more and comments

* buffer/orderbook: improve update and snapshot performance. Move Update type to orderbook package to util. pointer through entire function calls. (cleanup). Change action string to uint8 for easier comparison. Add parsing helper. Update current test benchmark comments.

* dispatch: change publish func to variadic id param

* dispatch: remove sender receiver wait time as this adds overhead and complexity. update tests.

* dispatch: don't create pointers for every job container

* rpcserver: fix assertion issues with data publishing change

* linter: fixes

* glorious: nits addr

* depth: change validation handling to incorporate and store err

* linter: fix more issues

* dispatch: fix race

* travis: update before fetching

* depth: wrap and return wrapped error in invalidate call and fix tests

* btcmarkets: fix commenting

* workflow: check

* workflow: check

* orderbook: check error

* buffer/depth: return invalidation error and fix tests

* gctcli: display errors on orderbook streams

* buffer: remove unused types

* orderbook/bitmex: shift function to bitmex

* orderbook: Add specific comments to unexported functions that don't have locking require locking.

* orderbook: restrict published data functionality to orderbook.Outbound interface

* common: add assertion failure helper for error

* dispatch: remove atomics, add mutex protection, remove add/remove worker, redo main tests

* dispatch: export function

* engine: revert and change sub logger to manager

* engine: remove old test

* dispatch: add common variable ;)

* btcmarket: don't overflow int in tests on 32bit systems

* ci: force 1.17.7 usage for go

* Revert "ci: force 1.17.7 usage for go"

This reverts commit af2f95563bf218cf2b9f36a9fcf3258e2c6a2d91.

* golangci: bump version add and remove linter items

* Revert "golangci: bump version add and remove linter items"

This reverts commit 3c98bffc9d030e39faca0387ea40c151df2ab06b.

* dispatch: remove unsused mutex from mux

* order: slight optimizations

* nits: glorious

* dispatch: fix regression on uuid generation and input inline with master

* linter: fix

* linter: fix

* glorious: nit - rm slice segration

* account: fix test after merge

* coinbasepro: revert change

* account: close channel instead of needing a receiver, push alert in routine to prepare for waiter.

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
2022-05-03 12:37:08 +10:00
Ryan O'Hara-Reid
6a02097431 account: storage, processing and method on balances update (#916)
* account: update account storage, retrieval and implement alert functionality when a currency change occurs.

* account: Add cancel channel

* account: remove old code

* account: don't embed mutex

* Update exchanges/account/account.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/account/account.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/account/account.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* account: addr nits

* account: Pull out test into indiv.

* account: Add test for update method

* account: add no change to test

* Update exchanges/account/account.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* account/portfolio: differentiate between asset type segregation and default to spot holdings.

* glorious: nits

* thrasher: nit

* ticker: fix spelling

* Update engine/portfolio_manager.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* thrasher: nits

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
2022-05-02 14:01:03 +10:00
Eng Zer Jun
21b3d6a6c9 test: use T.TempDir to create temporary test directory (#934)
* test: use `T.TempDir` to create temporary test directory

This commit replaces `ioutil.TempDir` with `t.TempDir` in tests. The
directory created by `t.TempDir` is automatically removed when the test
and all its subtests complete.

Prior to this commit, temporary directory created using `ioutil.TempDir`
needs to be removed manually by calling `os.RemoveAll`, which is omitted
in some tests. The error handling boilerplate e.g.
	defer func() {
		if err := os.RemoveAll(dir); err != nil {
			t.Fatal(err)
		}
	}
is also tedious, but `t.TempDir` handles this for us nicely.

Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>

* test: fix TestEncryptTwiceReusesSaltButNewCipher on Windows

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>

* test: fix TestCheckConnection on Windows

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>

* test: fix TestRPCServer_GetTicker_LastUpdatedNanos on Windows

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>

* test: cleanup TestGenerateReport

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-04-28 12:01:15 +10:00
Ryan O'Hara-Reid
b87a0791b0 asset: bitmask type optimisation (#922)
* asset: basic optim. bitmask

* glorious: nits

* currency: forgot parralel in testttttt

* ticker/orderbook: test fixes

* engine/rpcserver: fix and expand tests
2022-04-28 09:49:43 +10:00
Adrian Gallagher
9a4eb9de84 CI: Fix golangci-lint linter issues, add prealloc linter and bump version depends for Go 1.18 (#915)
* Bump CI versions

* Specifically set go version as 1.17.x bumps it to 1.18

* Another

* Adjust AppVeyor

* Part 1 of linter issues

* Part 2

* Fix various linters and improvements

* Part 3

* Finishing touches

* Tests and EqualFold

* Fix nitterinos plus bonus requester jobs bump for exchanges with large number of tests

* Fix nitterinos and bump golangci-lint timeout for AppVeyor

* Address nits, ensure all books are returned on err due to syncer regression

* Fix the wiggins

* Fix duplication

* Fix nitterinos
2022-04-20 13:45:15 +10:00
Ryan O'Hara-Reid
022001fecf btcmarkets: update order submission (#908)
* btcm/order: update order submission

* btcmarkets: addr nits

* btcmarkets: reinstate new order test
2022-03-25 15:05:24 +11:00
Ryan O'Hara-Reid
09fa2f236a context: Add authenticated HTTP credentials (#892)
* gRPC: context overide

* exchanges: continue update

* exchange: Update context handling
*Add setter methods for API credentials
*Shift credentials functionality to its own file in exchanges package
*Add tests
*Refactor function DeployCredentialsToContext for library usage
*Add function to process credential metadata from API boundary to internal use context value.
*Add OTP rpc handling

* exchanges: reverts to old style in GetFeeByType, reverts some code I accidently deleted. Plus things and other. XD

* template: update

* exchanges: fix linter issues

* REMOVE THAT AWESOME NEW LINE!

* gct: fix some tests

* I cant spell :(

* exchanges/gctscript: fix more tests

* coinnut: fix tests

* Update exchanges/credentials.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/credentials.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/credentials.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/credentials.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/credentials.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* exchanges/gctcli: stop applying empty credentials

* fix linters

* exchanges: add test

* rpceserver: actually check error for errors

* rpcserver: fix up tests

* Update exchanges/credentials.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* exchanges/creds: move tests to corresponding files, add protection and segration for Credentials struct & ptr values

* exchanges/creds: allow subaccount to override default credentials via gRPC

* exchanges/credentials: don't return nil in GetCredentials

* creds: spelling

* exchanges: fix glorious NITS!

* credentials: Add in test and refactor IsEmpty method.

* credentials: change type positioning (glorious)

* exchange_template: Fix template changes

* DOCS: Refresh

* docs: fix spelling

* DOCS: fix alignment and add package

* DOCS: ALIGN!

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2022-03-21 13:58:08 +11:00
Scott
86d3724507 Futures order position tracking & FTX scaled collateral calculation (#868)
* implements futures functions and GRPC functions on new branch

* lint and test fixes

* Fix uneven split pnl. Adds collateral weight test. docs. New clear func

* Test protection if someone has zero collateral

* Uses string instead of double for accuracy

* Fixes old code panic

* context, match, docs

* Addresses Shazniterinos, var names, expanded tests

* Returns subaccount name, provides USD values when offlinecalc

* Fixes oopsie

* Fixes cool bug which allowed made up subaccount results

* Subaccount override on FTX, subaccount results for collateral

* Strenghten collateral account info checks. Improve FTX test

* English is my first language

* Fixes oopsies

* Fixes for unrealised PNL & collateral rendering

* Fixes lint and tests

* Shaznit fixes

* Secret Shaznit

* Updates account information across wrappers to include more fields

* Updates online collateral calculations. Updates RPC data

* Accurately calculates collateral offline and online minus testing

* Tests and lint chocolate

* Simplifies accountinfo results

* Fixes shaznits

* Adds new func

* Increases collateral accuracy again again again x 200

* Increases accuracy of collateral rendering

* Fixes minor merge/test issues

* Linterino

* Fixes ws test. Improves collateral calculations and rendering

* Make it prettier

* Removes the lock I put on 👀

* Adds `additional_collateral_used` field, renders orig currency

* Fixes unrelated test

* Fix test

* Correctly calculate spot margin borrow collateral

* Address fun lint surprise

See https://github.com/golangci/golangci-lint/issues/741#issuecomment-1017014331

* Strange lint fixing x2

* Continued lint journey

* Nolint the nolint to not lint the lint

* Adds two new fields to response

* More linting issues arising

* fIX3s_c4s|NG

* Fixes command flags' incorrect numbering

* FairMarket = Won
2022-02-28 16:39:36 +11:00
Ryan O'Hara-Reid
6127e2ab73 Request: Fix http.Client race issue when setting transport layer proxy and timeouts (#885)
* backtester/request: trying to fix panic (WIP)

* request: fix race for transport layer

* request: linter issue fix

* request: more linter issues

* requester: Add function to remove the tracking of underlying http client and add to engine unload exchange.

* request: add more context to error return

* request: Fix after cherry pick issues

* request: fix niterinos

* exchanges: change return to package variable

* request: changed named

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
2022-02-18 09:22:10 +11:00
Ryan O'Hara-Reid
11da520dc8 Currency: Add additional functionality, refactor and improvements (#881)
* currency: Add method to derive pair

* currency: Add method to lower entire charset but used the slice copy and returned that. This will change the original, just gotta see if this is an issue, but the slice usually goes out of scope anyway.

* currency/pairs: add filter method

* currency: add function to derive select currencies from currency pairs

* currency/engine: slight adjustments

* currency: fix linter issue also shift burden of proof to caller instead of repair, more performant.

* currency: more linter

* pairs: optimize; reduce allocs/op and B/op

* currency: Add in function 'NewPairsFromString' for testing purposes

* currency: don't suppress error

* currency: stop panic on empty currency code

* currency: Add helper method to match currencies between exchanges

* currency: fixed my bad spelling

* currency: Implement stable coin checks, refactored base code methods, optimized upper and lower case strings for currency code/pairs

* currency: add pairs method to derive stable coins from internal list.

* Currency: Cleanup, fix tests.

* engine/exchanges/currency: fix whoops

* Currency: force govet no copy on Item datatype

* Currency: fix naughty linter issues

* exchange: revert change

* currency/config: fix config upgrade mistake

* currency: re-implement currency sub-systems

* *RetrieveConfigCurrencyPairs removed
*CheckCurrencyConfigValues to only provide warnings, add additional support when, disable when support is lost or not available and set default values.
*Drop Cryptocurrencies from configuration as this is not needed.
*Drop REST Poll delay field as this was unused.
*Update default values for currencyFileUpdateDuration & foreignExchangeUpdateDuration.
*Allow Role to be marshalled for file type.
*Refactor RunUpdater to verify and check config values and set default running foreign exchange provider.

* currency: cleanup

* currency: change match -> equal for comparison which is more of a standard and little easier to find

* currency: address nits

* currency: fix whoops

* currency: Add some more pairs methods

* currency: linter issues

* currency: RM unused field

* currency: rm verbose

* currency: fix word

* currency: gocritic

* currency: fix another whoopsie

* example_config: default to show log system name

* Currency: Force all support packages to use Equal method for comparison as there is a small comparison bug when checking upper and lower casing, this has a more of a pronounced impact between exchanges and client instances of currency generation

* currency: fix log name

* ordermanager: fix potential panic

* currency: small optim.

* engine: display correct bool and force shutdown

* currency: add function and fix regression
* Change ConvertCurrency -> ConvertFiat to be more precise
* ADD GetForeignExchangeRate to get specific exchange rate for fiat pair
* Fix currency display and formatting regression and tied in with config.Currency fields

* engine: fix tests

* currency: return the amount when no conversion needs to take place

* currency: reduce method name

* currency: Address nits glorious nits

* currency: fix linter

* currency: addr nits

* currency: check underlying role in test

* gct: change to EMPTYCODE and EMPTYPAIR across codebase

* currency: fix nits

* currency: this fixes test race but this issue has not been resolved. Please see: https://trello.com/c/54eizOIo/143-currency-package-upgrades

* currency: Add temp dir for testing

* Update engine/engine.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* documentation: update and regen

* currency: Address niterinos

* currency: Add test case for config upgrade when falling over to exchange rate host as default from exchangeRates provider

* currency: addr nits

* currency: fix whoops

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
2022-02-17 16:24:57 +11:00
Ryan O'Hara-Reid
19b76bf2e2 engine/okgroup: fix panic when calling GetDepositAddress() (#858)
* engine/okgroup: fix caller panic and patched for re-occurance

* engine/helpers: remove nil check as it was not neccessary.
2021-12-16 09:43:50 +11:00
Florent Clairambault
d1df77171a tiny fixes: Path separator + typos (#853)
When we are on a *NIX system, the path is incorrect and we can't copy-paste it.

Added some tiny typo fixes.
2021-12-06 09:23:47 +11:00
Ryan O'Hara-Reid
ac692b04f4 log: fix bug, expand test coverage and slightly optimize (#847)
* log: fix bugs expand coverage and optimise

* log: fix linter issues

* log: fix linter issue and pack methods in same file

* log: drop defer

* logger: move global check inside getfields  and remove unused test function

* logger: Increase note thanks @gloriouscode

* logger: wrap error with writer type

* logger: change variable name

* logger: change variable names and remove validsublogger func as it doesn't add functionality over a standard map call

* logs: error when unsupported output is applied on setup calls

* logs: add glorious suggestion

* logger: add protection to reduce olympic gold medal races

* logger: fix linter issues

* log: glorious niterinos
2021-11-30 16:43:27 +11:00
Adrian Gallagher
02afa1e98b exchanges: remove Coinbene exchange support (#849)
* exchanges: remove Coinbene exchange support

* RM Coinbene from apichecker backup
2021-11-29 11:34:35 +11:00
Ryan O'Hara-Reid
da3402476e tool/exchange_wrapper_coverage: fix regression and implement reflection (#837)
* cmd/tools/exchange: fix regression and implement reflection so as this can dynamically scale to our interface

* exchanges: add comment and fix whoopsie

* exchanges: fix linter issues

* wrapper_cov_tool: add actual full interface method count to get a better perceived deployment

* exchanges/tool: addr glorious nits

* kraken: remove string in test

* exchange_template_tool: fix tmpl issue
2021-11-17 15:41:21 +11:00
Scott
6eaa2e4073 Backtester: USD tracking (#818)
* Initial concept for creating price tracking pairs

* Completes coverage, even with a slow test

* I dont know what point to hook this stuff up

* Bit of a broken way of handling tracking pairs

* Correctly calculates USD rates against all currencies

* Removes dependency on GCT config

* Failed currency statistics redesign

* initial Update chart to use highcharts

* Minor changes to stats

* Creats funding stats to handle the stat calculations. Needs more work

* tracks USD snapshots and BREAKS THINGS FURTHER

* Fixed!

* Adds ratio calculations and such, but its WRONG. do it at totals level dummy

* End of day basic lint

* Remaining lints

* USD totals statistics

* Minor panic fixes

* Printing of funding stats, but its bad

* Properly calculates overall benchmark, moves funding stat output

* Adds some template charge, removes duplicate fields

* New charts!

* Darkcharts. funding protection when disabled

* Now works with usd tracking/funding disabled!

* Attempting to only show working stats based on settings.

* Spruces up the goose/reporting

* Completes report HTML rendering

* lint and test fixes

* funding statistics testing

* slightly more test coverage

* Test coverage

* Initial documentation

* Fixes tests

* Database testing and rendering improvements and breakages

* report and cmd rendering, linting. fix comma output. rm gct cfg

* PR mode 🎉 Path field, config builder support,testing,linting,docs

* minor calculation improvement

* Secret lint that did not show up locally

* Disable USD tracking for example configs

* ShazNitNoScope

* Forgotten errors

* ""

* literally Logarithmically logically renders the date 👀

* Fixes typos, fixes parallel test, fixes chart gui and exporting
2021-11-08 12:10:15 +11:00
Luis Rascão
4531fdcb4a exchanges/websocket: Expose Trades/Fills feed through data channel (#814)
* Expose trade feed websocket exchange data through data channel

Most relevant to applications that import GCT as a lib, this allows
them to (through configuration, disabled by default) receive trade data
through the data channel similarly to the orderbook feed.

* exchanges: allow exposure of trade websocket feed through data channel

* Expose fill feed websocket abstracted exchange data through data channel

* exchanges: allow exposure of fill websocket feed through data channel
2021-10-28 10:25:15 +11:00
Ryan O'Hara-Reid
099ffa1a60 stream/websocket: Consolidate fields by using exchange config pointer (#809)
* stream: add exchange config pointer to setup WebsocketSetup struct to reduce and consolidate setting of variables.

* config: reduce stutter

* config: reduce minor stutter

* glorious: nits addr.

* Update exchanges/stream/websocket.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* websocket: implement fix

* engine/helpers: fix test

* exchanges: fix after merge issues

* exchange_template: fix output

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2021-10-20 15:45:06 +11:00
Adrian Gallagher
0c00b7e1df exchanges/engine: Add multichain deposit/withdrawal support (#794)
* Add exchange multichain support

* Start tidying up

* Add multichain transfer support for Bitfinex and fix poloniex bug

* Add Coinbene multichain support

* Start adjusting the deposit address manager

* Fix deposit tests and further enhancements

* Cleanup

* Add bypass flag, expand tests plus error coverage for Huobi

Adjust helpers

* Address nitterinos

* BFX wd changes

* Address nitterinos

* Minor fixes rebasing on master

* Fix BFX acceptableMethods test

* Add some TO-DOs for 2 tests WRT races

* Fix acceptableMethods test round 2

* Address nitterinos
2021-10-15 15:55:38 +11:00
Adrian Gallagher
f0d45aa1d2 golangci-lint/CI: Bump versions and introduce new linters (#798)
* golangci-lint/CI: Bump versions

Fix remaining linter issues

* Specifically set AppVeyor version

* Fix the infamous typos 👀

* Add go env cmd to AppVeyor

* Add go version cmd to AppVeyor

* Specify AppVeyor image, adjust linters

* Update go get to go install due to deprecation

* Bump golangci-lint timeout time for AppVeyor

* Change NW contract to NQ

* Address nitters

* GetRandomPair -> Pair{}

* Address nits

* Address time nitterinos plus additional tweaks

* More time inception upgrades!

* Bending time and space
2021-10-14 16:38:53 +11:00
Scott
adf7659e95 backtester: shared exchange level funding, decimal implementation (#783)
* Better designed backtester funding concept

* Fleshes out funding concepts further to allow two funding types

* Adds types, finishes adding to portfolio and adds to exchange

* Fixes a bug to reveal another

* Fixes issues with purchasing

* A partial conversion to using decimal.decimal for the backtester

* Further decimal rollout. Can compile and output report

* More cleanup

* Fix rendering and initial funds issue.

* Adds new concept for trading using the exchange level funding to see what happens

* Fixes a bug in funding not being found

* New strat config to test RSI and discover issues

* Can run with pairs that contain 0 funding

* Finally fixes the arrangement to share funds

* Adds testing and funding transfer

* end of day

* More comments, more tests!

* Improves item comparisons and completes testing

* Initial attempt at new strategy which utilisies shared funding and transfers

* end of day broken

* Chronological output. Fixes output bug where multi currency.

* End of day commit

* Fixes bug where events were being overwritten in a simultaneous context

* Begins transitioning from portfolio holdings to funding holdings. Am I doing the right thing

* End of day run around

* Likely fix for holding calculations

* Improvement to template. Improvement to holdings

* DARK MODE. Report upgrades. Even handling with funds. Fix output

* Output funding to cmd

* Add new trasnferred funds "side"

* Fixing test run 1

* Test updates

* Test updating

* More test fixing

* Fixes portfolio tests

* More test fixes

* Fixes remaining tests and lints

* Fixes currencystatistics tests. Adds decimal math implementations

* Fixes hilarious bug where there could only be on holding

* Adds funding support for config. Minor fixes

* Adds documentation

* Finishes config builder support for funding

* Logs inexact conversions, updates tests. adds config validation

* The quest to understand a new funding bug begins. New strategy

* Fixes bug where wrong funding was retrieved. Expands t2b2 strat

* End of the day commit. Gotta revert the nulldecimal stuff

* Fixes tests, adds extra funding transfer feature

* Fixes initial total values, tries to add a grand total value

* Rebase fixes, documentation updates, tests for strategy

* Swaps the err statement for tests. Regenerates tests. Math warnings

* Attempts to solve Live data problems. Fixes volume

* Fixes live data missing

* can trade at any interval. skip volume sizing. volume colours.

* config regen. display fixes

* test fixes, lint fixes

* Anti-funky errors

* docs

* Rmbad

* docs

* docs update

* Simplifies err handling. Updates readmes. Data type checks

* docs. new field initial-base-funds. comment errs. config test coverage

* minMaxing

* testfix

* Fixes fee calculation, re-bans minMax being equal

* Crazy concepts to attempt to solve totals. Addresses nits

* Adds in totals calculation for exchange level funding.Uses external API

In future, this will be replaced by proper pricing supplied by the same
exchange that is requested. This is an unknown price

* rm dollar signs in cmd and report. rm bad error. fix chart decimal. padding

* re-run docs post merge

* Fixes oopsie for fee parsing

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
2021-09-27 16:01:23 +10:00
Ryan O'Hara-Reid
5dfbbf84de engine/exchanges: Add exchange currency state subsystem (#774)
* state: Add management system (init)

* linter: fix

* engine: gofmt

* gct: after merge fixup

* documentation: add

* rpc: implement services for testing

* gctcli: gofmt state_management.go

* documentation: reinstate lost information

* state: Add pair check to determine trading operation

* exchanges: add interface for specific state scoped subsystem functionality

* engine/order_man: reduce code footprint using new method

* RPC: implement pair trading request and change exported name to something specific to state

* engine: add tests

* engine: Add to withdraw manager

* documentation: reinstate soxipy in contrib. list

* engine: const fake name

* Glorious: NITERINOS

* merge: fix issues

* engine: csm incorporate service name into log output

* engine: fix linter issues

* gct: fix tests

* currencystate: remove management type

* rpc: fix tests

* backtester: fix tests

* Update engine/currency_state_manager.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update engine/currency_state_manager.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/currencystate/currency_state.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/alert/alert.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/alert/alert.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* config: integrate with config and remove flag delay adjustment

* gctcli: fix issues after name changes

* engine: gofmt manager file

* Update engine/rpcserver.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* engine: Add enable/disable manager functions, add default popoulation for potential assets

* linter: fix

* engine/test: bump subsystem count

* Update engine/currency_state_manager.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/bithumb/bithumb.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits addressed

* alert: fix commenting for its generalized purpose

* glorious: nits

* engine: use standard string in log output

* bitfinex: apply patch, thanks @thrasher-

* bitfinex: fix spelling

* engine/currencystate: Add logs/fix logs

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2021-09-27 13:33:49 +10:00
TaltaM
fd600972ba engine/order manager: Initial REST managed order updating (resolves #772) (#775)
* Initial REST managed order updating

* Apply gloriousCode's changes.go patch

* Update internal order ID handling

* Check error

* Replace string with string pointer

* Avoid nil pointers in upsert

* Update test for UpdateOrderFromDetail()

* Add tests for orders.go

* Remove unnecessary newline

* Address comments

* Add missing nil check

* Add tests for new functions in order_manager.go

* Remove empty line

* Change log level for updates from Info to Debug (keep added orders at Info)

* Initialize orders before running the timer

* [TEMP] Add verbosity for debugging

* Nil checking on exchangeManager in GetExchanges()

- exchangeManager.GetExchanges() and iExchangeManager.GetExchanges() return an error on nil
- bot.GetExchanges() wraps exchangeManager.GetExchanges() and returns an empty slice

* Revert b5afe1a46b

* Do not start the order manager runner thread

Instead, mark the order manager as running

* Remove redundant error.Is() and remove print wrapper on msg

* Add atomic blocker and waitgroup on processOrders()

* Disable unnecessary orderManager runner thread for rpcserver_test

* Remove redundant err from orderStore.getActiveOrders()

* [FIX] Populate requiresProcessing using UpsertResponse data instead of REST return data

.. because the data returned by the REST calls do not include the internal user ID's

* [TEST] Verify that processOrders() actually processes queried order data

* Remove leftover warning and add nil check on wg.Done()

* Apply suggestions from code review

Log category changes - as suggested

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Return when no exchanges available

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
2021-09-15 09:28:31 +10:00
Ryan O'Hara-Reid
d636049fb2 exchanges: Initial context propagation (#744)
* gct: phase one context awareness pass

* exchanges: context propagation pass

* common/requester: force context requirement

* gctcli/exchanges: linter fix

* rpcserver: fix test using dummy rpc server

* backtester: fix comments

* grpc: add correct cancel and timeout for commands

* rpcserver_test: add comment on dummy server

* common: deprecated SendHTTPGetRequest

* linter: fix

* linter: turn on no context check

* apichecker: fix context linter issue

* binance: use param context

* common: remove checks as this gets executed before main

* common: change mutex to RW as clients can be used by multiple go routines.

* common: remove init and JIT default client. Unexport global variables and add protection.

* common: Add comments

* bithumb: after dinner mints fix
2021-09-11 13:52:07 +10:00
Ryan O'Hara-Reid
a54c5107f4 engine: GetSubsystemsStatus fix (#773)
* engine: GetSubsystemsStatus fix

* engine: force map literal to stop doubling up on keys, expanded test coverage

* engine: Deploy default for migration requirement.

* glorious: nits addr

* glorious: suggestion

* tests: fix
2021-09-03 11:59:52 +10:00
Ryan O'Hara-Reid
8020e1ec6a linter: Enable error checking linter (#766)
* golangci: Enable err checking linter to expose unchecked errors.

* gct: handle errors across the board

* gct: handle errors NOTE: Found bug in FTX (WIP)

* linter: fix issues

* ftx/exchanges: fix bug where error was being returned when setting pair management variables to an already enabled state

* bitmex: fix bug where a dangly supported asset in config danglied up the place.

* linter: fix more linter issues

* linter: fix my terrible spelling.

* currency: fix test

* exchanges: fix tests

* logger: fix test

* exchanges: fix tests

* glorious: nits

* vm: revert rm variable and instigate test
2021-08-30 14:06:40 +10:00
Luis Rascão
c9ab0b1164 exchanges: Add UpdateTickers method and reserve UpdateTicker for single ticker symbol update requests (if supported) (#764)
* exchanges: add an UpdateTickers method to the main exchange interface

This method will fetch all currency pair tickers of a given asset type
and update them internally, does nothing for now.

* exchanges: refactor UpdateTicker on all exchanges

Keep the exact previous behaviour but implement the UpdateTickers
method and refactor UpdateTicker by using it where applicable.

* sync_manager: update all tickers when batching is enabled

* binance: UpdateTicker to fetch single ticker symbol

* ftx: UpdateTicker to fetch single ticker symbol
2021-08-27 12:31:34 +10:00
Ryan O'Hara-Reid
4851e94eba Engine/ExchangeManager: Return error for method GetExchangeByName (#760)
* engine: Add error returns

* engine: after merge fixes

* engine: remove interface

* linter: fix shadow declarations

* engine: fix tests

* niterinos: fixed

* GLORIOUS NITS!
2021-08-26 13:09:14 +10:00