26 Commits

Author SHA1 Message Date
Ryan O'Hara-Reid
ac91fabcd5 orderbook: consolidate slice array types to orderbook package (#1992)
* orderbook: consolidate slice array types to orderbook package

* Update exchanges/bybit/bybit_types.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* linter: fix and add test

* cranktakular: nits

* cranktakular: nits

* Update exchanges/orderbook/orderbook_types.go

Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>

* Update exchanges/gateio/gateio_test.go

Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>

* gk: nits consolidation

* gk: rm unifySpotOrderbook func

* gk: nit but different

* linter: fix

* gk: nits

* glorious: nits

* Update exchanges/binance/binance.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update exchanges/binance/binance_cfutures.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update exchanges/binanceus/binanceus.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* thrasher-:nits

* thrasher-: more nit

---------

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
2025-10-02 14:22:20 +10:00
Adrian Gallagher
d5b2cf1759 linters: Add modernise tool check and fix issues (#2012)
* linters: Add modernise tool check and fix issues

* engine: Simplify exch.SetDefaults call and remove localWG

* CI: Revert config versions lint workflow
2025-08-26 12:45:13 +10:00
Samuael A.
3f534a15f1 cmd/exchange_template, exchanges: Update templates and propogate to exchanges (#1777)
* Added TimeInForce type and updated related files

* Linter issue fix and minor coinbasepro type update

* Bitrex consts update

* added unit test and minor changes in bittrex

* Unit tests update

* Fix minor linter issues

* Update TestStringToTimeInForce unit test

* Exchange test template change

* A different approach

* fix conflict with gateio timeInForce

* minor exchange template update

* Minor fix to test_files template

* Update order tests

* Complete updating the order unit tests

* Updating exchange wrapper and test template files

* update kucoin and deribit wrapper to match the time in force change

* minor comment update

* fix time-in-force related test errors

* linter issue fix

* ADD_NEW_EXCHANGE documentation update

* time in force constants, functions and unit tests update

* shift tif policies to TimeInForce

* Update time-in-force, related functions, and unit tests

* fix linter issue and time-in-force processing

* added a good till crossing tif value

* order type fix and fix related tim-in-force entries

* update time-in-force unmarshaling and unit test

* consistency guideline added

* fix time-in-force error in gateio

* linter issue fix

* update based on review comments

* add unit test and fix missing issues

* minor fix and added benchmark unit test

* change GTT to GTC for limit

* fix linter issue

* added time-in-force value to place order param

* fix minor issues based on review comment and move tif code to separate files

* update on exchanges linked to time-in-force

* resolve missing review comments

* minor linter issues fix

* added time-in-force handler and update timeInForce parametered endpoint

* minor fixes based on review

* nits fix

* update based on review

* linter fix

* rm getTimeInForce func and minor change to time-in-force

* minor change

* update based on review comments

* wrappers and time-in-force calling approach

* minor change

* update gateio string to timeInForce conversion and unit test

* update exchange template

* update wrapper template file

* policy comments, and template files update

* rename all exchange types name to Exchange

* update on template files and template generation

* templates and generation code and other updates

* linter issue fix

* added subscriptions and websocket templates

* update ADD_NEW_EXCHANGE.md with recent binance functions and implementations

* rename template files and update unit tests

* minor template and unit test fix

* rename templates and fix on unit tests

* update on template files and documentation

* removed unnecessary tag fix and update templates

* fix Add_NEW_EXCHANGE.md doc file

* formatting, comments, and error checks update on template files

* rename exchange receivers to e and ex for consistency

* rename unit test exchange receiver and minor updates

* linter issues fix

* fix deribit issue and minor style update

* fix test issues caused by receiver change

* raname local variables exchange declaration variables

* update templates comments

* update templates and related comments

* renamed ex to e

* update template comments

* toggle WS to false to improve coverage

* template comments update

* added test coverage to Ws enabled and minor changes

---------

Co-authored-by: Samuel Reid <43227667+cranktakular@users.noreply.github.com>
2025-07-17 10:46:36 +10:00
Ryan O'Hara-Reid
c892f492a9 buffer/orderbook: shift orderbook update logic from buffer package to orderbook package (#1908)
* buffer/orderbook: shift orderbook update logic from buffer package to orderbook package

* Update exchanges/orderbook/depth.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* linter: fixes

* spelling: fix

* samboss: add in some todos

* sammy nit: add unlock on error

* sammy nits: rm ptr to slice field buffer in orderbookHolder

* sammy nits: Add more coverage bro

* sammy nits: even more coverage

* gk: nits on commentary

* gk: nits change sort.Slice to slices.SortFunc

* gk: fix commentary on buffer clearing

* gk: nits fin

* linter: fix

* Update exchange/websocket/buffer/buffer.go

Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>

* Update exchange/websocket/buffer/buffer.go

Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>

* Update exchanges/orderbook/tranches.go

Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>

* Update exchanges/orderbook/orderbook.go

Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>

* Update exchange/websocket/buffer/buffer_test.go

Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>

* Update exchange/websocket/buffer/buffer_test.go

Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>

* Update exchanges/orderbook/incremental_updates.go

Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>

* gk: refresh action types and names

* gk nits: consolidate error vars and naming

* gk nits: more name changes

* gk nits; buffer tests update

* gk nits: error var names change

* linter: FIX

* it gets inlined but there is an alloc

* rn field in TODO

* Update exchanges/binance/binance_websocket.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update exchanges/binance/binance_websocket.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* orderbook: shift verify/validate funcs to validate.go and rn Verify() -> Validate()

* orderbook: validate even in presence of checksum and allow cowboy mode

* buffer; fix test

* kraken: fix futures orderbook by reversing incoming bids

* okx: change default spread pair

* Update exchanges/orderbook/validate.go

Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>

* Update exchanges/orderbook/validate.go

Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>

* Update exchanges/orderbook/validate.go

Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>

* Update exchanges/orderbook/validate.go

Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>

* Update exchanges/orderbook/validate.go

Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>

* gk: initial nits

* rn fields V(v)erifyorderbook to V(v)alidateOrderbook

* buffer/orderbook: nilguard in validate and change method receiver w -> o

---------

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
2025-06-18 16:19:58 +10:00
Ryan O'Hara-Reid
3e80f1b9e5 websocket/exchanges: populate context before multi connection upgrade (#1933)
* websocket/exchanges: populate context before multi connection upgrade

* fix test

* linter: fix

* gk: dial

* gk: nits rm param names

---------

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
2025-06-17 13:43:00 +10:00
Ryan O'Hara-Reid
2958e64afe orderbook: change Base struct name to Book (#1914)
* orderbook: change Base struct name to Snapshot

* linter: fix

* Update exchanges/exchange.go

Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>

* Update exchanges/orderbook/depth.go

Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>

* Update exchanges/orderbook/orderbook_types.go

Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>

* Snapshot -> Book

* Tranche(s) -> Level(s)

* Tranche(s) -> Level(s)

* rm tranche ref

* linter: fix

* linter: rides again

* update tests

* Update exchange/websocket/buffer/buffer.go

Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>

* Update backtester/eventhandlers/exchange/slippage/slippage.go

Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>

* Update exchange/websocket/buffer/buffer.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update exchange/websocket/buffer/buffer.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update exchanges/orderbook/orderbook_test.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update exchanges/orderbook/orderbook_test.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update exchanges/orderbook/orderbook_test.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update exchanges/orderbook/orderbook_types.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update exchanges/orderbook/orderbook_types.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* fixup tests

* Update exchanges/orderbook/orderbook_test.go

Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>

* Update exchanges/orderbook/orderbook_test.go

Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>

* Update exchanges/orderbook/orderbook_test.go

Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>

* Update exchanges/orderbook/orderbook_test.go

Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>

* Update exchanges/orderbook/orderbook_test.go

Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>

* Update exchanges/orderbook/orderbook_test.go

Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>

* Update exchanges/orderbook/orderbook_test.go

Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>

* Update exchanges/orderbook/orderbook_test.go

Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>

* Update exchanges/orderbook/orderbook_types.go

Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>

* Update exchanges/orderbook/orderbook_types.go

Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>

* gk: nits and rm stuff that is not needed

* Update exchanges/orderbook/orderbook_test.go

Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>

* gk: nits

---------

Co-authored-by: shazbert <ryan.oharareid@thrasher.io>
Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
2025-06-16 17:09:25 +10:00
Adrian Gallagher
d5ba674fc4 codebase: Cleanup various things (#1935)
* codebase: Rid base64/hex to string common funcs

* codebase: Rid local scope variable usage and other improvements

* codebase: Refactor currency pair usage across multiple exchanges

- Updated HitBTC tests to use the new currency pair format.
- Modified Kraken futures types to use currency.Pair instead of string for Symbol.
- Adjusted Kraken wrapper methods to handle currency pairs correctly.
- Refined OKX tests and types to utilize currency.Pair for instrument IDs.
- Enhanced Poloniex tests to consistently use predefined currency pairs.
- Streamlined order and orderbook tests to replace string pairs with currency.NewBTCUSD().
- Improved Yobit tests to utilize a standardized currency pair format.
- Updated validator wrapper to use currency pairs directly instead of string conversions.

* codebase: Use types.Number where possible

* refactor: update PayoutFee type to types.Number for consistency

* Refactor: Remove crypto functions to use standard library and other minor changes

- Removed custom crypto functions for SHA256, SHA512, and MD5 from the common/crypto package.
- Replaced usages of removed functions with standard library implementations in various files including:
  - cmd/websocket_client/main.go
  - engine/apiserver.go
  - exchanges/kraken/kraken.go
  - exchanges/lbank/lbank.go
  - exchanges/okx/okx_business_websocket.go
  - exchanges/kucoin/kucoin_websocket.go
  - gctscript/vm/vm.go
- Updated tests to reflect changes in the crypto functions.
- Renamed several functions for clarity, particularly in the context of order book updates across multiple exchanges.

* refactor: replace assert with require for consistency in test assertions

* refactor: Improve Binance futures candlestick test, standardise orderbook update function names and improve test parallelism

* refactor: Replace require.Len with require.Equal for better output in TestGetFuturesKlineData
2025-06-12 14:12:36 +10:00
Samuael A.
0e9adcd1e1 exchanges: Types and unmarshalling methods update (#1899)
* Added TimeInForce type and updated related files

* Linter issue fix and minor coinbasepro type update

* Bitrex consts update

* added unit test and minor changes in bittrex

* Unit tests update

* Fix minor linter issues

* Update TestStringToTimeInForce unit test

* fix conflict with gateio timeInForce

* Update order tests

* Complete updating the order unit tests

* update kucoin and deribit wrapper to match the time in force change

* fix time-in-force related test errors

* linter issue fix

* time in force constants, functions and unit tests update

* shift tif policies to TimeInForce

* Update time-in-force, related functions, and unit tests

* fix linter issue and time-in-force processing

* added a good till crossing tif value

* order type fix and fix related tim-in-force entries

* update time-in-force unmarshaling and unit test

* fix time-in-force error in gateio

* linter issue fix

* update based on review comments

* add unit test and fix missing issues

* minor fix and added benchmark unit test

* change GTT to GTC for limit

* fix linter issue

* added time-in-force value to place order param

* fix minor issues based on review comment and move tif code to separate files

* update on exchanges linked to time-in-force

* resolve missing review comments

* minor linter issues fix

* added time-in-force handler and update timeInForce parametered endpoint

* minor fixes based on review

* nits fix

* update based on review

* linter fix

* rm getTimeInForce func and minor change to time-in-force

* minor change

* update based on review comments

* wrappers and time-in-force calling approach

* update slice data processing

* fix linter issues

* remove unnecessary Unmarshal methods and replace type delatration and added unit tests

* minor change

* minor changes to types

* update gateio string to timeInForce conversion and unit test

* removed unused timeInForceString func from kraken

* removed redundant parentheses in slice unmarshaling

* array to slice conversion and other updates

* reverted slice target to array

* Binanceus unit test NotNil check to Len check

* change NotNil to Len check

* rename unmarshal unit test funcs name

* wrap json strings with []byte
2025-06-02 13:54:49 +10:00
Ryan O'Hara-Reid
cad7586e98 exchange/websocket, gateio: Rename/export package again, add websocket request functions for futures trading (#1603)
* gateio: Add multi asset websocket support WIP.

* meow

* Add tests and shenanigans

* integrate flushing and for enabling/disabling pairs from rpc shenanigans

* some changes

* linter: fixes strikes again.

* Change name ConnectionAssociation -> ConnectionCandidate for better clarity on purpose. Change connections map to point to candidate to track subscriptions for future dynamic connections holder and drop struct ConnectionDetails.

* Add subscription tests (state functional)

* glorious:nits + proxy handling

* Spelling

* linter: fixerino

* instead of nil, dont do nil.

* clean up nils

* cya nils

* don't need to set URL or check if its running

* stream match update

* update tests

* linter: fix

* glorious: nits + handle context cancellations

* stop ping handler routine leak

* * Fix bug where reader routine on error that is not a disconnection error but websocket frame error or anything really makes the reader routine return and then connection never cycles and the buffer gets filled.
* Handle reconnection via an errors.Is check which is simpler and in that scope allow for quick disconnect reconnect without waiting for connection cycle.
* Dial now uses code from DialContext but just calls context.Background()
* Don't allow reader to return on parse binary response error. Just output error and return a non nil response

* Allow rollback on connect on any error across all connections

* fix shadow jutsu

* glorious/gk: nitters - adds in ws mock server

* linter: fix

* fix deadlock on connection as the previous channel had no reader and would hang connection reader for eternity.

* glorious: whooops

* gk: nits

* Leak issue and edge case

* Websocket: Add SendMessageReturnResponses

* whooooooopsie

* gk: nitssssss

* Update exchanges/stream/stream_match.go

Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>

* Update exchanges/stream/stream_match_test.go

Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>

* linter: appease the linter gods

* gk: nits

* gk: drain brain

* started

* more changes before merge match pr

* gateio: still building out

* gateio: finish spot

* fix up tests in gateio

* Add tests for stream package

* rm unused field

* glorious: nits

* rn files, specifically set function names to asset and offload routing to websocket type.

* linter: fix

* Add futures websocket request support

* gateio: integrate with IBOTExchange (cherry pick my nose)

* linter: fix

* glorious: nits

* add counter and update gateio

* fix collision issue

* Update exchanges/stream/websocket.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* add tests

* linter: fix

* After merge

* Add error connection info

* upgrade to upstream merge

* Fix edge case where it does not reconnect made by an already closed connection

* stream coverage

* glorious: nits

* glorious: nits removed asset error handling in stream package

* linter: fix

* rm block

* Add basic readme

* fix asset enabled flush cycle for multi connection

* spella: fix

* linter: fix

* Add glorious suggestions, fix some race thing

* reinstate name before any routine gets spawned

* stop on error in mock tests

* glorious: nits

* Set correct price

* glorious: nits found in CI build

* Add test for drain, bumped wait times as there seems to be something happening on macos CI builds, used context.WithTimeout because its instant.

* mutex across shutdown and connect for protection

* lint: fix

* test time withoffset, reinstate stop

* fix whoops

* const trafficCheckInterval; rm testmain

* y

* fix lint

* bump time check window

* stream: fix intermittant test failures while testing routines and remove code that is not needed.

* spells

* cant do what I did

* protect race due to routine.

* update testURL

* use mock websocket connection instead of test URL's

* linter: fix

* remove url because its throwing errors on CI builds

* connections drop all the time, don't need to worry about not being able to echo back ws data as it can be easily reviewed _test file side.

* remove another superfluous url thats not really set up for this

* spawn overwatch routine when there is no errors, inline checker instead of waiting for a time period, add sleep inline with echo handler as this is really quick and wanted to ensure that latency is handing correctly

* linter: fixerino uperino

* fix ID bug, why I do this, I don't know.

* glorious: panix

* linter: things

* whoops

* dont need to make consecutive Unix() calls

* websocket: fix potential panic on error and no responses and adding waitForResponses

* rm json parser and handle in json package instead

* in favour of json package unmarshalling

* linter: fix

* linter: fix again

* * change field name OutboundRequestSignature to WrapperDefinedConnectionSignature for agnostic inbound and outbound connections.
* change method name GetOutboundConnection to GetConnection for agnostic inbound and outbound connections.
* drop outbound field map for improved performance just using a range and field check (less complex as well)
* change field name connections to connectionToWrapper for better clarity

* spells and magic and wands

* glorious: nits

* comparable check for signature

* mv err var

* glorious: nits and stuff

* attempt to fix race

* glorious: nits

* gk: nits; engine log cleanup

* gk: nits; OCD

* gk: nits; move function change file names

* gk: nits; 🚀

* gk: nits; convert variadic function and message inspection to interface and include a specific function for that handling so as to not need nil on every call

* gk: nits; continued

* gk: engine nits; rm loaded exchange

* gk: nits; drop WebsocketLoginResponse

* stream: Add match method EnsureMatchWithData

* gk: nits; rn Inspect to IsFinal

* gk: nits; rn to MessageFilter

* linter: fix

* gateio: update rate limit definitions (cherry-pick)

* Add test and missing

* Shared REST rate limit definitions with Websocket service, set lookup item to nil for systems that do not require rate limiting; add glorious nit

* integrate rate limits for websocket trading spot

* conform to match upstream changes

* standardise names to upstream style

* fix wrapper standards test when sending a auth request through a websocket connection

* whoops

* Update exchanges/gateio/gateio_types.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* linter: fix

* linter: overload

* whoops

* spelling fixes on recent merge

* glorious: nits

* linter: fix?

* glorious: nits

* gk: assert errors touched

* gk: unexport derive functions

* gk: nitssssssss

* fix test

* gk: nitters v1

* gk: http status

* gk/nits: Add getAssetFromFuturesPair

* gk: nits single response when submitting

* gk: new pair with delimiter in tests

* gk: param update slice to slice of pointers

* gk: add asset type in params, includes t.Context() for tests

* linter: fix

* linter: fix

* fix merge whoopsie

* glorious: nits

* gk: nit

* shift over to websocket package error

* internal/exchange/websocket -> exchange/websocket

* PEAK OCD!

* appease the OCD gods

* thrasher: nits

---------

Co-authored-by: shazbert <ryan.oharareid@thrasher.io>
Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2025-04-11 16:47:33 +10:00
Gareth Kirwan
b4e45e9a1b Websocket: Restructure files and types (#1859)
* Websocket: Rename stream package

* Websocket: Rename Websocket to Manager

* Websocket: Replace explicit errs with common.NilGuard

* Websocket: Move websocket_types.go to types.go

* Websocket: Minor field comment and alignment in types

* Webosocket: Rename WebsocketConnection to Connection

* Alphapoint: Make gorilla ws import explicit

Just to avoid confusion with our own packages.

* Websocket: Move stream_match to match

* Websocket: Move websocket_connection to connection

* Websocket: Move websocket.go to manager.go

* Websocket: Break out all subscription methods into subscriptions.go

* Websocket: Move connection type into its file

* Websocket: Remove PositionUpdated

Type is not used anywhere

* Kraken: Use local constant for pong

Was the only use of websocket.Pong and doesn't really feel right to
represent kraken's api resp in one of our packages

* Websocket: Move connection sub-types to connection package

* Websocket: Move manager types into manager

* Websocket: Move ConnectionWrapper into manager

* Websocket: Move websocket_test to manager_test

* Websocket: Privatise connectionWrapper

* Websocket: Remaining types into types.go

These really belong somewhere else mostly, but this will do for now

* Websocket: Tidy up connection method vars

* Gofumpt: Moving package imports around

* Websocket: Rename errDuplicateConnectionSetup

* Websocket: Fix duplicate import of gws

* Websocket: Fix gofumpt -extra

* Websocket: Standardise import of gws across other pkgs

* Kraken: Remove unused sub conf consts

These were replaced by the generic Levels and Depth fields on all subs

* Websocket: Privitise ConnectioWrapper fields

* Websocket: inline single use var WebsocketNotAuthenticatedUsingRest

* Websocket: Move documentation to template

* Bithumb: Assertify TestWsHandleData
2025-04-10 16:25:02 +10:00
Adrian Gallagher
4651af5767 modernise: Run new gopls modernise tool against the codebase and fix minor issues (#1826)
* modernise: Run new gopls modernise tool against codebase

* Address shazbert's nits

* apichecker, gctcli: Simplify HTML scraping functions and improve depth limit handling

* refactor: Create minSyncInterval const and update order book limit handling for binance and binanceUS

* refactor: Various slice usage improvements and rename TODO

* tranches: Revert deleteByID changes due to performance decrease

Shazbert was a F1 driver in a past lifetime 🏎️

* tranches: Simply retrieve copy

Thanks to shazbert

* documentation: Sort contributors list by contributions

* tranches: Remove deadcode in deleteByID
2025-03-21 09:17:10 +11:00
Adrian Gallagher
f5faca2eb2 linter: Enable gofumpt and run against codebase (#1848)
* linter: Enable gofumpt and run against codebase

* Address shazbert's nits

* gofumpt: Fix issues after rebase
2025-03-18 10:23:16 +11:00
Ryan O'Hara-Reid
e99adca86f encoding/json: Add custom JSON package with build tag support for Sonic (#1623)
* tag optional sonic and allow full library conversion

* Add workflow and disallow arm and darwin usage

* Add basic hotswap benchmark

* linter: fix

* use bash

* linter: fix?

* Fix whoopsie, add to make file, also add mention in features list.

* test enforcement

* actually read documentation see if this works

* linter: fix

* linter: fix

* sonic: bump tagged version

* encoding/json: drop build tag arch and os filters

* encoding/json: consolidate tests

* encoding/json: log build tag usage

* rm superfluous builds

* glorious/nits: add template change and regen docs

* glorious/nits: update commentary on nolint directive

* glorious/nits: rm init func and log results in main.go

* Test to actually pull flag in

* linter: fix

* thrasher: nits

* gk: nits 4 goflags goooooooooo!

* gk: nits rn

* make sonic default json implementation

* screen 386

* linter: fix

* Add commentary

* glorious: nits Makefile not working

* gk: nits

* gk: nits whoops

* whoopsirino

* mention 32bit systems won't be sonic

* gk: super-duper nit of extremes

---------

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
2025-02-20 16:05:55 +11:00
Ryan O'Hara-Reid
ac731ce283 websocket/gateio: Support multi connection management and integrate with GateIO (#1580)
* gateio: Add multi asset websocket support WIP.

* meow

* Add tests and shenanigans

* integrate flushing and for enabling/disabling pairs from rpc shenanigans

* some changes

* linter: fixes strikes again.

* Change name ConnectionAssociation -> ConnectionCandidate for better clarity on purpose. Change connections map to point to candidate to track subscriptions for future dynamic connections holder and drop struct ConnectionDetails.

* Add subscription tests (state functional)

* glorious:nits + proxy handling

* Spelling

* linter: fixerino

* instead of nil, dont do nil.

* clean up nils

* cya nils

* don't need to set URL or check if its running

* stop ping handler routine leak

* * Fix bug where reader routine on error that is not a disconnection error but websocket frame error or anything really makes the reader routine return and then connection never cycles and the buffer gets filled.
* Handle reconnection via an errors.Is check which is simpler and in that scope allow for quick disconnect reconnect without waiting for connection cycle.
* Dial now uses code from DialContext but just calls context.Background()
* Don't allow reader to return on parse binary response error. Just output error and return a non nil response

* Allow rollback on connect on any error across all connections

* fix shadow jutsu

* glorious/gk: nitters - adds in ws mock server

* linter: fix

* fix deadlock on connection as the previous channel had no reader and would hang connection reader for eternity.

* gk: nits

* Leak issue and edge case

* gk: nits

* gk: drain brain

* glorious: nits

* Update exchanges/stream/websocket.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* add tests

* linter: fix

* After merge

* Add error connection info

* Fix edge case where it does not reconnect made by an already closed connection

* stream coverage

* glorious: nits

* glorious: nits removed asset error handling in stream package

* linter: fix

* rm block

* Add basic readme

* fix asset enabled flush cycle for multi connection

* spella: fix

* linter: fix

* Add glorious suggestions, fix some race thing

* reinstate name before any routine gets spawned

* stop on error in mock tests

* glorious: nits

* glorious: nits found in CI build

* Add test for drain, bumped wait times as there seems to be something happening on macos CI builds, used context.WithTimeout because its instant.

* mutex across shutdown and connect for protection

* lint: fix

* test time withoffset, reinstate stop

* fix whoops

* const trafficCheckInterval; rm testmain

* y

* fix lint

* bump time check window

* stream: fix intermittant test failures while testing routines and remove code that is not needed.

* spells

* cant do what I did

* protect race due to routine.

* update testURL

* use mock websocket connection instead of test URL's

* linter: fix

* remove url because its throwing errors on CI builds

* connections drop all the time, don't need to worry about not being able to echo back ws data as it can be easily reviewed _test file side.

* remove another superfluous url thats not really set up for this

* spawn overwatch routine when there is no errors, inline checker instead of waiting for a time period, add sleep inline with echo handler as this is really quick and wanted to ensure that latency is handing correctly

* linter: fixerino uperino

* glorious: panix

* linter: things

* whoops

* defer lock and use functions that don't require locking in SetProxyAddress

* lint: fix

* thrasher: nits

---------

Co-authored-by: shazbert <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2024-10-10 15:09:52 +11:00
Ryan O'Hara-Reid
b461c32a5e stream/okx: allow rate limit definitions to be used by the stream package (#1641)
* stream: rate limiter definitions

* Update exchanges/request/request_test.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

---------

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2024-09-13 13:56:46 +10:00
Ryan O'Hara-Reid
cb6b3421a7 exchanges/websocket: update websocket rate limiting to use requester rate limiting functionality (#1578)
* exchanges/websocket: update websocket rate limiting to use requester rate limiting functionality.

* glorious: nits

* rm unsused

* updoo

* glorious: purgerino

* reduce duplicate code

* thrasher: engrish

---------

Co-authored-by: shazbert <ryan.oharareid@thrasher.io>
2024-09-02 16:43:05 +10:00
Adrian Gallagher
225429bda6 CI/build: Update Go version, linters and fix minor issues (#1612)
* CI/build: Update Go version, linters and fix minor issues

* linters: Add intrange, copyloopvar, additional go vet linters to match gopls and fix issues
2024-08-16 17:41:11 +10:00
Gareth Kirwan
1199f38546 subscriptions: Encapsulate, replace Pair with Pairs and refactor; improve exchange support
* Websocket: Use ErrSubscribedAlready

instead of errChannelAlreadySubscribed

* Subscriptions: Replace Pair with Pairs

Given that some subscriptions have multiple pairs, support that as the
standard.

* Docs: Update subscriptions in add new exch

* RPC: Update Subscription Pairs

* Linter: Disable testifylint.Len

We deliberately use Equal over Len to avoid spamming the contents of large Slices

* Websocket: Add suffix to state consts

* Binance: Subscription Pairs support

* Bitfinex: Subscription Pairs support

* Bithumb: Subscription Pairs support

* Bitmex: Subscription Pairs support

* Bitstamp: Subscription Pairs support

* BTCMarkets: Subscription Pairs support

* BTSE: Subscription Pairs support

* Coinbase: Subscription Pairs support

* Coinut: Subscription Pairs support

* GateIO: Subscription Pairs support

* Gemini: Subscription Pairs support and improvement

* Hitbtc: Subscription Pairs support

* Huboi: Subscription Pairs support

* Kucoin: Subscription Pairs support

* Okcoin: Subscription Pairs support

* Poloniex: Subscription Pairs support

* Kraken: Add subscription Pairs support

Note: This is a naieve implementation because we want to rebase the
kraken websocket rewrite on top of this

* Bybit: Subscription Pairs support

* Okx: Subscription Pairs support

* Bitmex: Subsription configuration

* Fixes unauthenticated websocket left as CanUseAuth
* Fixes auth subs happening privately

* CoinbasePro: Subscription Configuration

* Consolidate ProductIDs when all subscriptions are for the same list

* Websocket: Log actual sent message when Verbose

* Subscriptions: Improve clarity of which key is which in Match

* Subscriptions: Lint fix for HugeParam

* Subscriptions: Add AddPairs and move keys from test

* Subscriptions: Simplify subscription keys and add key types

* Subscriptions: Add List.GroupPairs Rename sub.AddPairs

* Subscription: Fix ExactKey not matching 0 pairs

* Subscriptions: Remove unused IdentityKey and HasPairKey

* Subscriptions: Fix GetKey test

* Subscriptions: Test coverage improvements

* Websocket: Change State on Add/Remove

* Subscriptions: Improve error context

* Subscriptions: Fix Enable: false subs not ignored

* Bitfinex: Fix WsAuth test failing on DataHandler

DataHandler is eaten by dataMonitor now, so we need to use ToRoutine

* Deribit: Subscription Pairs support

* Websocket: Accept nil lists for checkSubscriptions

If the user passes in a nil (implicitly empty) list, we would not panic.
Therefore the burden of correctness about that data lies with them.
The list of subscriptions is empty, and that's okay, and possibly
convenient

* Websocket: Add context to NilPointer errors

* Subscriptions: Add context to nil errors

* Exchange: Fix error expectations in UnsubToWSChans
2024-06-07 11:54:08 +10:00
Ryan O'Hara-Reid
4cd4fb06b4 orderbook: Refactor package structure for simplicity and efficiency (#1465)
* initial purge and benchmarks proof before rn overhaul

* rn LinkedList -> Tranche(s) and purge references

* roll out acrost exchanges

* linterino

* rn silly billy label

* linter strikes AAAAAGAIN!

* fix some things

* rm comment

* Add actual comparison from master to branch benchmark for sorting algorithms

* lower case via git mv YAAY!

* drop code

* convert type name

* glorious: nits

---------

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
2024-05-14 15:51:34 +10:00
Gareth Kirwan
52c6b3bf0b Websocket: Various refactors and test improvements (#1466)
* Websocket: Remove IsInit and simplify SetProxyAddress

IsInit was basically the same as IsConnected.
Any time Connect was called both would be set to true.
Any time we had a disconnect they'd both be set to false
Shutdown() incorrectly didn't setInit(false)

SetProxyAddress simplified to only reconnect a connected Websocket.
Any other state means it hasn't been Connected, or it's about to
reconnect anyway.
There's no handling for IsConnecting previously, either, so I've wrapped
that behind the main mutex.

* Websocket: Expand and Assertify tests

* Websocket: Simplify state transistions

* Websocket: Simplify Connecting/Connected state

* Websocket: Tests and errors for websocket

* Websocket: Make WebsocketNotEnabled a real error

This allows for testing and avoids the repetition.
If each returned error is a error.New() you can never use errors.Is()

* Websocket: Add more testable errors

* Websocket: Improve GenerateMessageID test

Testing just the last id doesn't feel very robust

* Websocket: Protect Setup() from races

* Websocket: Use atomics instead of mutex

This was spurred by looking at the setState call in trafficMonitor and
the effect on blocking and efficiency.
With the new atomic types in Go 1.19, and the small types in use here,
atomics should be safe for our usage. bools should be truly atomic,
and uint32 is atomic when the accepted value range is less than one byte/uint8 since
that can be written atomicly by concurrent processors.
Maybe that's not even a factor any more, however we don't even have to worry enough to check.

* Websocket: Fix and simplify traffic monitor

trafficMonitor had a check throttle at the end of the for loop to stop it just gobbling the (blocking) trafficAlert channel non-stop.
That makes sense, except that nothing is sent to the trafficAlert channel if there's no listener.
So that means that it's out by one second on the trafficAlert, because any traffic received during the pause is doesn't try to send a traffic alert.

The unstopped timer is deliberately leaked for later GC when shutdown.
It won't delay/block anything, and it's a trivial memory leak during an infrequent event.

Deliberately Choosing to recreate the timer each time instead of using Stop, drain and reset

* Websocket: Split traficMonitor test on behaviours

* Websocket: Remove trafficMonitor connected status

trafficMonitor does not need to set the connection to be connected.
Connect() does that. Anything after that should result in a full
shutdown and restart. It can't and shouldn't become connected
unexpectedly, and this is most likely a race anyway.

Also dropped trafficCheckInterval to 100ms to mitigate races of traffic
alerts being buffered for too long.

* Websocket: Set disconnected earlier in Shutdown

This caused a possible race where state is still connected, but we start
to trigger interested actors via ShutdownC and Wait.
They may check state and then call Shutdown again, such as
trafficMonitor

* Websocket: Wait 5s for slow tests to pass traffic draining

Keep getting failures upstream on test rigs.
Think they can be very contended, so this pushes the boundary right out
to 5s
2024-02-23 18:39:25 +11:00
Gareth Kirwan
e007f69f7c exchanges/websocket: Implement subscription configuration (#1394)
* Websockets: Move Subscription to its own package

This allows the small type to be imported from both `config` and from
`stream` without an import cycle, so we don't have to repeat ourselves

* Subs: Renamed Currency to Pair

This was being mis-used through much of the code, and since we're
already touching everything, we might as well fix it

* Websockets: Add Subscription configuration

* Binance: Add subscription configuration

* Kucoin: Subscription configuration

* Simplify GenerateDefaultSubs
* Improve TestGenSubs coverage
* Test Candle Sub generation
* Support Candle intervals
* Full responsibility for formatting Channel name on GenerateDefaultSubs
  OR consumer of Subscribe
* Simplify generatePayloads as a result
* Fix test coverage of asset types in processMarketSnapshot

* Exchanges: Abstract ParallelChanOp

* Tests: Generic ws mock instances

* Kucoin: Fix intermittent conflict in test currs

Use isolated test instance for `TestGetOpenInterest`.

`TestGetOpenInterest` would occassionally change pairs before
GenerateDefault Subs.
2024-01-24 15:54:07 +11:00
Gareth Kirwan
f9437dbd08 Bitfinex: Websocket subscription improvements (#1353)
* Websockets: Add keys to websocket subscriptions

* This switches all RO uses of the mutex to use a RLock method.
* The mutex used for discrete field access has had scope drift from
  name 'connectionMutex' so rename to more appropriate fieldsMutex
* The mutex used for Set/CanUseAuthEndpoints moves from the
  subscriptions endpoint to the fieldsMutex
* Add GetSubscription by key
* Expose stream.Matcher type

* Bitfinex: Subscribe and Unsubscribe atomicly

* Fix Auth failures ignored
* This change makes it so that Subscribe and Unsubscribe wait for success
** Tells the DataHandler about errors
** Errors are returned to consumers
* Subscribes concurrently to the channels
* It also simplifies the chanId to stream mapping
* Removes unable to locate chanID: %d errors which are just noise
* Paves the way for unified channelSubscription id handling
* Adds support for subId for Book subscriptions, which is more robust

* Vastly simplifies what we need to test TestWsSubscribedResponse
This test was working to ensure that the various fancy key parsing
mechanisms all worked. Now that we use subId, we just need a thorough
test of that
* Expose Match.Set in order to capture websocket incoming data
Can't see another way of doing this. Doesn't seem too bad

* Allow tests to run with auth or WS
These flags made it difficult to run the tests whilst working on
websockets

* Enable API auth and WS in testconfig
This change minimises the changes requires for a full test run against
live endpoints, so that new contributors have a clearer testing path.
I cannot see any reason to turn WS off and Auth endpoints off when we're
not going to run API tests without Creds being set, and we're not going
to do live fire tests without canManipulateRealOrders

* TestWsSubscribe and various fixes
** Enables the websocket for live non-authed integration tests by default
** Adds an integration test for subscriptions
** Changes the Ws tests to respect canManipulateRealOrders
** Uses WsConnect instead of setupWs; fixes seqNo config not sent for WS tests
** Allows api creds to live in config/testdata.json which might be
  less likely to accidentally commit, and less obtrusive

* Bitfinex: Support period and timeframe for Candles

* Fixes manual Subscribe() symbol or key formatting
* Unifies handling of params for DefaultSubscriptions and manual
  subsrciptions

* Bitfinex: Handle conf and info WS channel events

* Bitfinex: Better tests for subscriptions

* fixup! Websockets: Add keys to websocket subscriptions

* fixup! Bitfinex: Subscribe and Unsubscribe atomicly

* fixup! Websockets: Add keys to websocket subscriptions

* Websockets: Add Pending subscription status

Add a status tracker so that Sub/Unsub can prevent duplicates,
and also fixes when first message comes before we have added the sub
to the tracker

* Websockets: Add State instead of pending

This change allows more clarity about the current state and
checks for specifically already Unsubing

* Bitfinex: Fix first sub message maybe lost

The only link we have between a sub req and the sub resp is the subID.
And the only link we have between a sub message and the sub is the chanID.
We can't derive a link using Pair or anything else.

This meant that by sending the resp and its chanID down the IncomingData
channel, we allowed the channel reader to maybe process the next
message, the first message on the channel, before the runtime executed
the switch back to subscribeToChan waiting on the chan.

To fix this, we key initially on subId.(string), and then replace it
with chanId.(int64) when we have it *inside* the wsHandleData so we
know we've procedurally handled it before the next message.

subscribeToChan is then free to remove the subId keyed Sub regardless of
error or not

If there's an error, we don't need to inline handling because there
won't be any second update.

Expands test coverage to make sure those subId keyed subscriptions are
removed.

* Websocket: Validate state in SetChanState

* fixup! Bitfinex: Fix first sub message maybe lost

* Websockets: Rename RemoveUnsuccessfulSubs

Implementation doesn't imply Unsuccessful or need to.
This change supports the registering of Pending subs

* Bitfinex: Fix race in Tests
2023-11-02 12:10:43 +11:00
Ryan O'Hara-Reid
ecf0a5bcb3 spellcheck: fix and update (#1359)
* spellcheck: fix and update

* spellcheck: add exclude line for gateio_types
2023-10-03 16:07:03 +11:00
Ryan O'Hara-Reid
ad9de19d47 orderbook: Check assignment of time values and reject if not set (#1318)
* orderbook: Check assignment of time values and reject if not set.

* linter: fix

* buffer: additional linter winter fixter

* Implement through pending exchanges

* finished push

* linty: minty

* gomod: tidy

* thrasher: nits

* glorious: nits

* orderbook: purge type now in favour of external call allocation

* orderbook: push last param

* orderbook: only 1 unlock call is needed

---------

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
2023-09-07 11:00:16 +10:00
Ryan O'Hara-Reid
d23898e63a engine: Adds shutdown method to exchange manager and unload all exchanges when engine is stopped (#1112)
* engine: shutdown and unload exchange when engine is stopped

* linter: fixes

* engine/exchMan: add nil check

* engine/exchanges: add shutdown method to exchanges, rm len check lock not needed, expanded code coverage, address some nits

* exchMan: report all failed shutdowns across exchanges, implement timer and monitoring routines.

* exchMan: improve shutdown sequence and aloc.

* further improvement

* exchman: log from warn to error

* websockconnection: Suppress error return when closure is caused by library

* linter: fix

* fix racies

* add note on why not parallel tests

* glorious: nits

* spelling kween

* thrasher: nits

* engine: change print of setting using reflection, I keep forgetting to implement this so program around forgetfulness

* engine/exchange_management: remove wait group and just rely on intermediary lock

* glorious: nits

* Update common/common.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update main.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

---------

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
2023-04-05 13:07:35 +10:00
samuael
ace5e64c10 exchanges: Add Binance.us support (#949)
* public endpoints methods added

* Completing mapping REST endpoints

* Binanceus Wrapper methods -Partially

* BinaWra functions with test funs; Not Completed

* Finalizing wrapper methods & test

* Finalizing wrapper methods & test

* Fix & Complete wrapper functions

* Adding Stream Datas

* WS Test functions

* CI: Fix golangci-lint linter issues

* CI: Fix reverting unnessesary changes and type conversion issues

* CI: Fix reverting unnessesary changes and type conversion issues

* CI: Fix comment, method use, error handling, and code handling issues

* build(deps): bump github.com/urfave/cli/v2 from 2.4.0 to 2.4.8 (#932)

Bumps [github.com/urfave/cli/v2](https://github.com/urfave/cli) from 2.4.0 to 2.4.8.
- [Release notes](https://github.com/urfave/cli/releases)
- [Changelog](https://github.com/urfave/cli/blob/main/docs/CHANGELOG.md)
- [Commits](https://github.com/urfave/cli/compare/v2.4.0...v2.4.8)

---
updated-dependencies:
- dependency-name: github.com/urfave/cli/v2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* build(deps): bump google.golang.org/grpc from 1.45.0 to 1.46.0 (#931)

Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.45.0 to 1.46.0.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.45.0...v1.46.0)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* asset: bitmask type optimisation (#922)

* asset: basic optim. bitmask

* glorious: nits

* currency: forgot parralel in testttttt

* ticker/orderbook: test fixes

* engine/rpcserver: fix and expand tests

* test: use `T.TempDir` to create temporary test directory (#934)

* test: use `T.TempDir` to create temporary test directory

This commit replaces `ioutil.TempDir` with `t.TempDir` in tests. The
directory created by `t.TempDir` is automatically removed when the test
and all its subtests complete.

Prior to this commit, temporary directory created using `ioutil.TempDir`
needs to be removed manually by calling `os.RemoveAll`, which is omitted
in some tests. The error handling boilerplate e.g.
	defer func() {
		if err := os.RemoveAll(dir); err != nil {
			t.Fatal(err)
		}
	}
is also tedious, but `t.TempDir` handles this for us nicely.

Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>

* test: fix TestEncryptTwiceReusesSaltButNewCipher on Windows

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>

* test: fix TestCheckConnection on Windows

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>

* test: fix TestRPCServer_GetTicker_LastUpdatedNanos on Windows

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>

* test: cleanup TestGenerateReport

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>

* account: storage, processing and method on balances update (#916)

* account: update account storage, retrieval and implement alert functionality when a currency change occurs.

* account: Add cancel channel

* account: remove old code

* account: don't embed mutex

* Update exchanges/account/account.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/account/account.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/account/account.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* account: addr nits

* account: Pull out test into indiv.

* account: Add test for update method

* account: add no change to test

* Update exchanges/account/account.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* account/portfolio: differentiate between asset type segregation and default to spot holdings.

* glorious: nits

* thrasher: nit

* ticker: fix spelling

* Update engine/portfolio_manager.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* thrasher: nits

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* build(deps): bump github.com/urfave/cli/v2 from 2.4.8 to 2.5.1 (#936)

Bumps [github.com/urfave/cli/v2](https://github.com/urfave/cli) from 2.4.8 to 2.5.1.
- [Release notes](https://github.com/urfave/cli/releases)
- [Changelog](https://github.com/urfave/cli/blob/main/docs/CHANGELOG.md)
- [Commits](https://github.com/urfave/cli/compare/v2.4.8...v2.5.1)

---
updated-dependencies:
- dependency-name: github.com/urfave/cli/v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* orderbook/buffer: data integrity and resubscription pass (#910)

* orderbook/buffer: data integrity and resubscription pass

* btcmarkets: REMOVE THAT LIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIINE!!!!!!!!!!!!!!!!!

* buffer: reinstate publish, refaactor, invalidate more and comments

* buffer/orderbook: improve update and snapshot performance. Move Update type to orderbook package to util. pointer through entire function calls. (cleanup). Change action string to uint8 for easier comparison. Add parsing helper. Update current test benchmark comments.

* dispatch: change publish func to variadic id param

* dispatch: remove sender receiver wait time as this adds overhead and complexity. update tests.

* dispatch: don't create pointers for every job container

* rpcserver: fix assertion issues with data publishing change

* linter: fixes

* glorious: nits addr

* depth: change validation handling to incorporate and store err

* linter: fix more issues

* dispatch: fix race

* travis: update before fetching

* depth: wrap and return wrapped error in invalidate call and fix tests

* btcmarkets: fix commenting

* workflow: check

* workflow: check

* orderbook: check error

* buffer/depth: return invalidation error and fix tests

* gctcli: display errors on orderbook streams

* buffer: remove unused types

* orderbook/bitmex: shift function to bitmex

* orderbook: Add specific comments to unexported functions that don't have locking require locking.

* orderbook: restrict published data functionality to orderbook.Outbound interface

* common: add assertion failure helper for error

* dispatch: remove atomics, add mutex protection, remove add/remove worker, redo main tests

* dispatch: export function

* engine: revert and change sub logger to manager

* engine: remove old test

* dispatch: add common variable ;)

* btcmarket: don't overflow int in tests on 32bit systems

* ci: force 1.17.7 usage for go

* Revert "ci: force 1.17.7 usage for go"

This reverts commit af2f95563bf218cf2b9f36a9fcf3258e2c6a2d91.

* golangci: bump version add and remove linter items

* Revert "golangci: bump version add and remove linter items"

This reverts commit 3c98bffc9d030e39faca0387ea40c151df2ab06b.

* dispatch: remove unsused mutex from mux

* order: slight optimizations

* nits: glorious

* dispatch: fix regression on uuid generation and input inline with master

* linter: fix

* linter: fix

* glorious: nit - rm slice segration

* account: fix test after merge

* coinbasepro: revert change

* account: close channel instead of needing a receiver, push alert in routine to prepare for waiter.

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>

* exchange/wrapper: add GetServerTime() for exchange analytics (#938)

* exchange/wrapper: add GetServerTime() for exchange analytics

* binance: fix linter issue

* glorious: nits

* glorious: nits rides again

* thrasher: nits implement huobi

* thrasher: nits add to exchange_wrapper_issues cmd

* order: slight optimizations (#917)

* order: slight optimizations

* orders: add benchmarks, small optimize and change order side to uin8 for comparitive optimizations.

* orders: continue to convert string type -> uint

* orders/backtester: interim move type to orders package, later can expand or deprecate.

* orders: handle errors

* orders: optimize filters and remove error returns when its clearly not needed

* orders: remove log call

* backtester: zero value check

* orders/futures: zero value -> flag

* linter: fix

* linter: more fixes

* linters: rides again

* glorious: nits

* common: Add zero value unix check for time values; also addresses glorious nits

* glorious scott: nits

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>

* btcm: add order execution limit wrapper support (#941)

* btcm: add in order execution limit fetching

* btcm/test: add t.Parrrrrralllleeellllllllll

* btcm/wrapper: add update on startup

* glorious: nits

* thrasher: nit add status field

* build(deps): bump github.com/urfave/cli/v2 from 2.5.1 to 2.6.0 (#944)

Bumps [github.com/urfave/cli/v2](https://github.com/urfave/cli) from 2.5.1 to 2.6.0.
- [Release notes](https://github.com/urfave/cli/releases)
- [Changelog](https://github.com/urfave/cli/blob/main/docs/CHANGELOG.md)
- [Commits](https://github.com/urfave/cli/compare/v2.5.1...v2.6.0)

---
updated-dependencies:
- dependency-name: github.com/urfave/cli/v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* proto/lint: Add protobuf GitHub action and linter (#943)

* Buf upgrades

* Buf format and basic endpoint fixes

* gRPC linter fixes

* Amend buf.yaml linter exceptions

* Update README

* Freshly generated gRPC code after depends update

* Nitterinos

* ordermanager: fix test error introduced in #917  (#942)

* ordermanager: fix residual test issue from #917 and reduce some racey action

* glorious: nits; also removed functions that weren't being used and were unexported

* rm: pew

* linter: fix issues

* glourious: nits

* credentials: fix test issue with racey racey horse basey

* engine: Add websocket data handler register function (#935)

* engine: Add websocket interceptor register function

* Update engine/engine.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update engine/websocketroutine_manager_types.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* engine/websock: switch to data handler function register and range over handlers to still include default gct handling

* engine/websocket: change name

* glorious: nits

* linter: fix

* glorious: nits

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* btcm: add modify order functionality, change return to pointer (#940)

* btcm: add modify order functionality, change return to pointer

* glorious: nits

* glorious: nits

* btcm: Adjust function name

* thrasher: nits

* thrasher: nits cont...

* request: adds WithVerbose function to package to add verbosity to request context (#950)

* request: adds WithVerbose function to package to add verbosity to request context

* request: add t.Parr....

* thrasher: nits

* build(deps): bump google.golang.org/grpc from 1.46.0 to 1.46.2 (#951)

Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.46.0 to 1.46.2.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.46.0...v1.46.2)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* exchange: expose GetCredentials() and split GetAuthenticatedAPISupport() (#954)

* exchange/wrapper: expose GetCredentials func to IBotInterface

* exchanges: split up GetAuthenticatedAPISupport into specific function calls, organize IBotExchange functionality getter functions

* interface: change name - RPCSercer: rm GetBase func call.

* glorious: nits (fix panic)

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>

* CI: merge fixes

* CI: fixing github generated lint issues

* orders: Add method for creating cancel struct from order details (#947)

* orders: Add method for creating cancel struct from order details

* orders: remove uneeded fields

* glorious: nit

* grpc: add shutdown call for external management (#957)

* grpc: add shutdown call for external management

* go mod: tidy

* glorious: suggestion

* Update engine/engine.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update engine/rpcserver.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update main.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update engine/rpcserver.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* build(deps): bump github.com/lib/pq from 1.10.5 to 1.10.6 (#960)

Bumps [github.com/lib/pq](https://github.com/lib/pq) from 1.10.5 to 1.10.6.
- [Release notes](https://github.com/lib/pq/releases)
- [Commits](https://github.com/lib/pq/compare/v1.10.5...v1.10.6)

---
updated-dependencies:
- dependency-name: github.com/lib/pq
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* build(deps): bump github.com/urfave/cli/v2 from 2.6.0 to 2.8.0 (#958)

Bumps [github.com/urfave/cli/v2](https://github.com/urfave/cli) from 2.6.0 to 2.8.0.
- [Release notes](https://github.com/urfave/cli/releases)
- [Changelog](https://github.com/urfave/cli/blob/main/docs/CHANGELOG.md)
- [Commits](https://github.com/urfave/cli/compare/v2.6.0...v2.8.0)

---
updated-dependencies:
- dependency-name: github.com/urfave/cli/v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* build(deps): bump github.com/grpc-ecosystem/grpc-gateway/v2 (#963)

Bumps [github.com/grpc-ecosystem/grpc-gateway/v2](https://github.com/grpc-ecosystem/grpc-gateway) from 2.10.0 to 2.10.2.
- [Release notes](https://github.com/grpc-ecosystem/grpc-gateway/releases)
- [Changelog](https://github.com/grpc-ecosystem/grpc-gateway/blob/master/.goreleaser.yml)
- [Commits](https://github.com/grpc-ecosystem/grpc-gateway/compare/v2.10.0...v2.10.2)

---
updated-dependencies:
- dependency-name: github.com/grpc-ecosystem/grpc-gateway/v2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* orders: adds method to retrieve snapshot of order execution limit values (#946)

* orders: add method to Limit to retrieve order execution limit snapshots

* currency/btcmarkets: add error and update field name to standard

* linter: fix

* limts: don't return pointer

* limit: Add notes

* glorious: nits

* linter: fix

* limit: reinstate nil check

* exchanges: change field names to be more consistent (@thrasher-) suggestion

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>

* orders: Add derive modify struct method from order.Detail  (#948)

* orders: Add derive modify struct method to order.Detail and then subsequent method to derive and standardize response details

* exchanges: call modify method in wrappers

* linter: fixes

* engine/wsroutineman: remove print summary

* glorious: nits, removed modifyOrder functionality for Bithumb. There are not docs to support this.

* Update exchanges/order/orders.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* build(deps): bump github.com/spf13/viper from 1.11.0 to 1.12.0 (#965)

Bumps [github.com/spf13/viper](https://github.com/spf13/viper) from 1.11.0 to 1.12.0.
- [Release notes](https://github.com/spf13/viper/releases)
- [Commits](https://github.com/spf13/viper/compare/v1.11.0...v1.12.0)

---
updated-dependencies:
- dependency-name: github.com/spf13/viper
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* build(deps): bump github.com/urfave/cli/v2 from 2.8.0 to 2.8.1 (#964)

Bumps [github.com/urfave/cli/v2](https://github.com/urfave/cli) from 2.8.0 to 2.8.1.
- [Release notes](https://github.com/urfave/cli/releases)
- [Changelog](https://github.com/urfave/cli/blob/main/docs/CHANGELOG.md)
- [Commits](https://github.com/urfave/cli/compare/v2.8.0...v2.8.1)

---
updated-dependencies:
- dependency-name: github.com/urfave/cli/v2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* orders: Add methods to derive SubmitResponse and Detail types (#955)

* orders: deprecate SubmitResponse return and change to *order.Detail construct detail from order.Submit struct

* orders: add coverage, fix tests

* coinut: rm test for checking

* orders: revert change for return and change field ID to a more explicit name OrderID

* orders: Add method to see if the order was placed

* order: change field name in Cancel type to be more explicit

* orders: standardize field -> OrderID

* backtester: populate change

* orders: add test

* gctscript: fix field name

* linter: fix issues

* linter: more fixes

* linter: forever

* exchanges_tests: populate order.Submit field exchange name

* Update exchanges/order/order_types.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/order/orders.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* glorious: nits

* thrasher: nits

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* build(deps): bump bufbuild/buf-setup-action from 1.4.0 to 1.5.0 (#973)

Bumps [bufbuild/buf-setup-action](https://github.com/bufbuild/buf-setup-action) from 1.4.0 to 1.5.0.
- [Release notes](https://github.com/bufbuild/buf-setup-action/releases)
- [Commits](https://github.com/bufbuild/buf-setup-action/compare/v1.4.0...v1.5.0)

---
updated-dependencies:
- dependency-name: bufbuild/buf-setup-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* build(deps): bump google.golang.org/grpc from 1.46.2 to 1.47.0 (#972)

Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.46.2 to 1.47.0.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.46.2...v1.47.0)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* build(deps): bump github.com/grpc-ecosystem/grpc-gateway/v2 (#971)

Bumps [github.com/grpc-ecosystem/grpc-gateway/v2](https://github.com/grpc-ecosystem/grpc-gateway) from 2.10.2 to 2.10.3.
- [Release notes](https://github.com/grpc-ecosystem/grpc-gateway/releases)
- [Changelog](https://github.com/grpc-ecosystem/grpc-gateway/blob/master/.goreleaser.yml)
- [Commits](https://github.com/grpc-ecosystem/grpc-gateway/compare/v2.10.2...v2.10.3)

---
updated-dependencies:
- dependency-name: github.com/grpc-ecosystem/grpc-gateway/v2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* build(deps): bump github.com/d5/tengo/v2 from 2.10.1 to 2.11.2 (#975)

Bumps [github.com/d5/tengo/v2](https://github.com/d5/tengo) from 2.10.1 to 2.11.2.
- [Release notes](https://github.com/d5/tengo/releases)
- [Changelog](https://github.com/d5/tengo/blob/master/.goreleaser.yml)
- [Commits](https://github.com/d5/tengo/compare/v2.10.1...v2.11.2)

---
updated-dependencies:
- dependency-name: github.com/d5/tengo/v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* build(deps): bump github.com/urfave/cli/v2 from 2.8.1 to 2.10.1 (#979)

Bumps [github.com/urfave/cli/v2](https://github.com/urfave/cli) from 2.8.1 to 2.10.1.
- [Release notes](https://github.com/urfave/cli/releases)
- [Changelog](https://github.com/urfave/cli/blob/main/docs/CHANGELOG.md)
- [Commits](https://github.com/urfave/cli/compare/v2.8.1...v2.10.1)

---
updated-dependencies:
- dependency-name: github.com/urfave/cli/v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* build(deps): bump github.com/d5/tengo/v2 from 2.11.2 to 2.12.0 (#978)

Bumps [github.com/d5/tengo/v2](https://github.com/d5/tengo) from 2.11.2 to 2.12.0.
- [Release notes](https://github.com/d5/tengo/releases)
- [Changelog](https://github.com/d5/tengo/blob/master/.goreleaser.yml)
- [Commits](https://github.com/d5/tengo/compare/v2.11.2...v2.12.0)

---
updated-dependencies:
- dependency-name: github.com/d5/tengo/v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* CI: adding WS and REST tests, and minor fixes

* CI: fixes on available asset and related minor issues

* CI: fixes on endpoint function, test functions, and types

* CI: updating templates and slight fixes

* CI: updating slight fixes on tests and withdraws Request model

* build(deps): bump styfle/cancel-workflow-action from 0.9.1 to 0.10.0 (#985)

Bumps [styfle/cancel-workflow-action](https://github.com/styfle/cancel-workflow-action) from 0.9.1 to 0.10.0.
- [Release notes](https://github.com/styfle/cancel-workflow-action/releases)
- [Commits](https://github.com/styfle/cancel-workflow-action/compare/0.9.1...0.10.0)

---
updated-dependencies:
- dependency-name: styfle/cancel-workflow-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* build(deps): bump bufbuild/buf-setup-action from 1.5.0 to 1.6.0 (#984)

Bumps [bufbuild/buf-setup-action](https://github.com/bufbuild/buf-setup-action) from 1.5.0 to 1.6.0.
- [Release notes](https://github.com/bufbuild/buf-setup-action/releases)
- [Commits](https://github.com/bufbuild/buf-setup-action/compare/v1.5.0...v1.6.0)

---
updated-dependencies:
- dependency-name: bufbuild/buf-setup-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* backtester: Futures handling & FTX Cash and Carry example strategy (#930)

* implements futures functions and GRPC functions on new branch

* lint and test fixes

* Fix uneven split pnl. Adds collateral weight test. docs. New clear func

* Test protection if someone has zero collateral

* Uses string instead of double for accuracy

* Fixes old code panic

* context, match, docs

* Addresses Shazniterinos, var names, expanded tests

* Returns subaccount name, provides USD values when offlinecalc

* Fixes oopsie

* Fixes cool bug which allowed made up subaccount results

* Subaccount override on FTX, subaccount results for collateral

* Strenghten collateral account info checks. Improve FTX test

* English is my first language

* Fixes oopsies

* Adds some conceptual futures order details to track PNL

* Initial design of future order processing in the backtester

* Introduces futures concept for collateral and spot/futures config diffs

* Fixes most tests

* Simple designs for collateral funding pair concept

* Expands interface use so much it hurts

* Implements more collateral interfaces

* Adds liquidation, adds strategy, struggles with Binance

* Attempts at getting FTX to work

* Adds calculatePNL as a wrapper function and adds an `IsFutures` asset check

* Successfully loads backtester with collateral currency

* Fails to really get much going for supporting futures

* Merges master changes

* Fleshes out how FTX processes collateral

* Further FTX collateral workings

* hooks up more ftx collateral and pnl calculations

* more funcs to flesh out handling

* Adds more links, just can't fit the pieces together :(

* Greatly expands futures order processing

* Fleshes out position tracker to also handle asset and exchange +testing

* RM linkedOrderID. rn positioncontroller, unexport

* Successfully tracks futures order positions

* Fails to calculate PNL

* Calculates pnl from orders accurately with exception to flipping orders

* Calculates PNL from orders

* Adds another controller layer to make it ez from orderstore

* Backtester now compiles. Adds test coverage

* labels things add scaling collateral test

* Calculates pnl in line with fees

* Mostly accurate PNL, with exception to appending with diff prices

* Adds locks, adds rpc function

* grpc implementations

* Gracefully handles rpc function

* beautiful tests!

* rejiggles tests to polish

* Finishes FTX testing, adds comments

* Exposes collateral calculations to rpc

* Adds commands and testing for rpcserver.go functions

* Increase testing and fix up backtester code

* Returns cool changes to original branch

* end of day fixes

* Fixing some tests

* Fixing tests 🎉

* Fixes all the tests

* Splits the backtester setup and running into different files

* Merge, minor fixes

* Messing with some strategy updates

* Failed understanding at collateral usage

* Begins the creation of cash and carry strategy

* Adds underlying pair, adds filldependentevent for futures

* Completes fill prerequsite event implementation. Can't short though

* Some bug fixes

* investigating funds

* CAN NOW CREATE A SHORT ORDER

* Minor change in short size

* Fixes for unrealised PNL & collateral rendering

* Fixes lint and tests

* Adds some verbosity

* Updates to pnl calc

* Tracks pnl for short orders, minor update to strategy

* Close and open event based on conditions

* Adds pnl data for currency statistics

* Working through PNL calculation automatically. Now panics

* Adds tracking, is blocked from design

* Work to flesh out closing a position

* vain attempts at tracking zeroing out bugs

* woww, super fun new subloggers 🎉

* Begins attempt at automatically handling contracts and collateral based on direction

* Merge master + fixes

* Investigating issues with pnl and holdings

* Minor pnl fixes

* Fixes future position sizing, needs contract sizing

* Can render pnl results, focussing on funding statistics

* tracking candles for futures, but why not btc

* Improves funding statistics

* Colours and stats

* Fixes collateral and snapshot bugs

* Completes test

* Fixes totals bug

* Fix double buy, expand stats, fixes usd totals, introduce interface

* Begins report formatting and calculations

* Appends pnl to receiving curr. Fixes map[time]. accurate USD

* Improves report output rendering

* PNL stats in report. New tests for futures

* Fixes existing tests before adding new coverage

* Test coverage

* Completes portfolio coverage

* Increase coverage exchange, portfolio. fix size bug. NEW CHART

* WHAT IS GOING ON WITH PNL

* Fixes PNL calculation. Adds ability to skip om futures tracking

* minor commit before merge

* Adds basic liquidation to backtester

* Changes liquidation to order based

* Liquidationnnnnn

* Further fleshes out liquidations

* Completes liquidations in a honorable manner. Adds AppendReasonf

* Beginnings of spot futures gap chart. Needs to link currencies to render difference

* Removes fake liquidation. Adds cool new chart

* Fixes somet tests,allows for zero fee value v nil distinction,New tests

* Some annoying test fixes that took too long

* portfolio coverage

* holding coverage, privatisation funding

* Testwork

* boring tests

* engine coverage

* More backtesting coverage

* Funding, strategy, report test coverage

* Completes coverage of report package

* Documentation, fixes some assumptions on asset errors

* Changes before master merge

* Lint and Tests

* defaults to non-coloured rendering

* Chart rendering

* Fixes surprise non-local-lints

* Niterinos to the extremeos

* Fixes merge problems

* The linter splintered across the glinting plinths

* Many nits addressed. Now sells spot position on final candle

* Adds forgotten coverage

* Adds ability to size futures contracts to match spot positions.

* fixes order sell sizing

* Adds tests to sizing. Fixes charting issue

* clint splintered the linters with flint

* Improves stats, stat rendering

* minifix

* Fixes tests and fee bug

* Merge fixeroos

* Microfixes

* Updates orderPNL on first Correctly utilises fees. Adds committed funds

* New base funcs. New order summary

* Fun test updates

* Fix logo colouring

* Fixes niteroonies

* Fix report

* BAD COMMIT

* Fixes funding issues.Updates default fee rates.Combines cashcarry case

* doc regen

* Now returns err

* Fixes sizing bug issue introduced in PR

* Fixes fun fee/total US value bug

* Fix chart bug. Show log charts with disclaimer

* sellside fee

* fixes fee and slippage view

* Fixed slippage price issue

* Fixes calculation and removes rendering

* Fixes stats and some rendering

* Merge fix

* Fixes merge issues

* go mod tidy, lint updates

* New linter attempt

* Version bump in appveyor and makefile

* Regex filename, config fixes, template h2 fixes

* Removes bad stats.

* neatens config builder. Moves filename generator

* Fixes issue where linter wants to fix my spelling

* Fixes pointers and starts

* build(deps): bump github.com/urfave/cli/v2 from 2.10.1 to 2.10.3 (#982)

Bumps [github.com/urfave/cli/v2](https://github.com/urfave/cli) from 2.10.1 to 2.10.3.
- [Release notes](https://github.com/urfave/cli/releases)
- [Changelog](https://github.com/urfave/cli/blob/main/docs/CHANGELOG.md)
- [Commits](https://github.com/urfave/cli/compare/v2.10.1...v2.10.3)

---
updated-dependencies:
- dependency-name: github.com/urfave/cli/v2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* alert: Add optimizations (#939)

* alert: Add optimizations

* alert: add basic benchmarks

* alert: fix linter issue

* documentation: change to text/template as html/template escapes to protect against code injection. Add readme.md for alert.

* README: Add package name

* alert: link up with engine settings

* request: isVerbose refactor

* Update exchanges/alert/alert_test.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/alert/alert.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* glorious: fun police

* documentation: regen

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* technical_analysis: TWAP & VWAP + TA methods to candles and link to existing RPC server for GCTCLI prototyping  (#970)

* kline: add weighted price helpers for candles

* twap/vwap: basic implementation and hook to rpc for protype

* ta: cont implementation. (WIP)

* kline: Add tests

* kline: add helpers

* ta: full impl.

* kline: remove support for macd and add in correlation-coefficient handling

* rpc: change naming convention

* linter: fix

* protolinter: fix

* linter: ++

* kline: reinstate macd handling after adding in check

* glorious: nits

* gctcl: linter

* Update exchanges/kline/weighted_price.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* glorious: nits v2.0

* kline: fix test

* huobi-tests: shift from next quarter to this weeks contracts as they were erroring out in tests.

* btcmarkets: update supported kline intervals

* zb: fix test

* rpcserver: fix bug and tests

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* build(deps): bump github.com/urfave/cli/v2 from 2.10.3 to 2.11.0 (#993)

Bumps [github.com/urfave/cli/v2](https://github.com/urfave/cli) from 2.10.3 to 2.11.0.
- [Release notes](https://github.com/urfave/cli/releases)
- [Changelog](https://github.com/urfave/cli/blob/main/docs/CHANGELOG.md)
- [Commits](https://github.com/urfave/cli/compare/v2.10.3...v2.11.0)

---
updated-dependencies:
- dependency-name: github.com/urfave/cli/v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* FTX: Margin lending/borrow rate history (#981)

* Adds lending rates/borrows to FTX and the command

* Movements, renames, rpc test

* Fleshing out rpc response

* Allows rpcserver to calculate offline (but not gctcli). Expands tests

* rn structs. add exchange_wrapper_issues support

* Adds a nice yearly rate

* Surprise yearly borrow rate!

* Rn+Mv to margin package. Fixes some serious whoopsies

* Adds average lend/borrow rates instead of sum

* rm oopsie whoopsie

* This is what the linter was having an issue with

* re-gen

* lintl

* niteroos

* build(deps): bump google.golang.org/grpc from 1.47.0 to 1.48.0 (#995)

Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.47.0 to 1.48.0.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.47.0...v1.48.0)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* account: segregate holdings by credentials for future multi-key management (#956)

* exchanges/account: shift credentials to account package and segregate funds to keys

* merge: fixes

* linter: fix

* Update exchanges/account/account.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits + protection for string panic

* glorious_suggestion: add method for matching keys

* linter: fix tests

* account: add protected method for credentials minimizing access, display full account details to rpc.

* linter: spelling kweeeeeeen

* accounts/portfolio: clean/check portfolio code and quickly check balances from change. Add protected method for future matching.

* accounts: theres no point in pointerising everything

* linter: ok pointerise this then...

* exchanges: fix regression add in little notes.

* glorious: nits

* Update exchanges/account/credentials.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/account/credentials_test.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/account/credentials_test.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* gloriously: fix glorious glorious test gloriously

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* CI: fixing linter issue and conflicts

* CI: fixing ratelimit and other slight issues

* build(deps): bump github.com/grpc-ecosystem/grpc-gateway/v2 (#998)

Bumps [github.com/grpc-ecosystem/grpc-gateway/v2](https://github.com/grpc-ecosystem/grpc-gateway) from 2.10.3 to 2.11.0.
- [Release notes](https://github.com/grpc-ecosystem/grpc-gateway/releases)
- [Changelog](https://github.com/grpc-ecosystem/grpc-gateway/blob/master/.goreleaser.yml)
- [Commits](https://github.com/grpc-ecosystem/grpc-gateway/compare/v2.10.3...v2.11.0)

---
updated-dependencies:
- dependency-name: github.com/grpc-ecosystem/grpc-gateway/v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* build(deps): bump github.com/d5/tengo/v2 from 2.12.0 to 2.12.1 (#997)

Bumps [github.com/d5/tengo/v2](https://github.com/d5/tengo) from 2.12.0 to 2.12.1.
- [Release notes](https://github.com/d5/tengo/releases)
- [Changelog](https://github.com/d5/tengo/blob/master/.goreleaser.yml)
- [Commits](https://github.com/d5/tengo/compare/v2.12.0...v2.12.1)

---
updated-dependencies:
- dependency-name: github.com/d5/tengo/v2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* build(deps): bump github.com/urfave/cli/v2 from 2.11.0 to 2.11.1 (#996)

Bumps [github.com/urfave/cli/v2](https://github.com/urfave/cli) from 2.11.0 to 2.11.1.
- [Release notes](https://github.com/urfave/cli/releases)
- [Changelog](https://github.com/urfave/cli/blob/main/docs/CHANGELOG.md)
- [Commits](https://github.com/urfave/cli/compare/v2.11.0...v2.11.1)

---
updated-dependencies:
- dependency-name: github.com/urfave/cli/v2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* CI: fixing type and other slight issues

* Cleanup after merge

* Endpoints rate limit update

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Ryan O'Hara-Reid <oharareid.ryan@gmail.com>
Co-authored-by: Eng Zer Jun <engzerjun@gmail.com>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
2022-08-12 15:30:11 +10:00