Commit Graph

234 Commits

Author SHA1 Message Date
Gareth Kirwan
51f73300bb Bitfinex: Improve websocket type assertion checks for order processing (#1292)
* Bitfinex: Error if ws data id/cid is not a float64

* Bitfinex: Add annotation to WS id/cid assest errs
2023-09-07 15:43:38 +10:00
Ryan O'Hara-Reid
ad9de19d47 orderbook: Check assignment of time values and reject if not set (#1318)
* orderbook: Check assignment of time values and reject if not set.

* linter: fix

* buffer: additional linter winter fixter

* Implement through pending exchanges

* finished push

* linty: minty

* gomod: tidy

* thrasher: nits

* glorious: nits

* orderbook: purge type now in favour of external call allocation

* orderbook: push last param

* orderbook: only 1 unlock call is needed

---------

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
2023-09-07 11:00:16 +10:00
Gareth Kirwan
6ab4c27d1c Bitfinex: Refactor websocket wsUpdate handling and various improvements (#1317)
* Bitfinex: Fix cancel/update order WS ack not seen

Fixes #1288

* Bitfinex: Fix ws Unsubscribe and Resubscribe

Unsubscribe needed to use the channel id.
Resubscribe needs to have the original subscription params.

* Bitfinex: Fix ws Trades Fees on te

The ws channel for authenticated Trades sends two types of update:
* te, Trade Executed
* tu, Trade Execution Update

Only the second one contains fee information.
[See the docs](https://docs.bitfinex.com/reference/ws-auth-trades)

This commit fixes:
`exchange Bitfinex websocket error - unable to type assert trade fee`
after an executed market trade on the te update

* Bitfinex: Fix error on ws auth ok

This fixes:
`Bitfinex Could not find an existing channel subscription: account Pair:
ChannelID: 0`
It's not clear from history why we'd want to store a reference to the
ubiquitous 0 channel like this, but it's definitely wrong, and anything
that attempts to get channel information about 0 chan needs to be fixed
anyway.

* Bitfinex: Refactor wsUpdate handling

This commit doesn't break out all the sub-updater, but attempts to do
something about the unmanagable size of ws update handling

* Binfinex: Fix linter issue on chanId casing

* Bitfinex: Fix linter outdent complaint

* Bitfinex: Fix linter issues on test

* Bitfinex: Fix TestWsTradingPairSnapshot chan lookup

* Bitfinex: Remove unnecessary WsAddSubs in test

* Bitfinex: Fix TestWsSubscribedResponse chan

* Bitfinex: Throw a specific error for bad event

* Bitfinex: WS Type assertions for positionSnapshots

* Bitfinex: tradeUpdate type assertion

* Bitfinex: Reinstate default subscriptions

* Bitfinex: Assert chan assetType is the same

* Bitfinex: Lowercase error string

* Bitfinex: Refactor WS eventType/chanId handling

* Bitfinex: Fix linter issues

* Bitfinex: Fix delimiter for pairs with more than 6 chars

* Bitfinex: Fix WS handling of subscribed symbols

This simplifies the handling of subscription symbols.
Now that we know the channel up front from handling the subscribed
response we can limit the parsing forms needed

* Bitfinex: Placate the linter

* Bitfinex: Disable margin assets for WS

Margin WS Currently not fully implemented and causes subscription collisions with spot

* Bitfinex: Fix parsing of 4 part funding keys

This improves overall handling and errors on a few current assumptions
about key structure

* Bitfinex: Linter fixes

* Bitfinex: Remove key parsing from assetPairFromSymbol

* Bitfinex: Use native error wrapping

* Bitfinex: Skip disabled assets in default ws subs
2023-09-01 16:46:00 +10:00
Gareth Kirwan
577817c46e bitfinex,okx: Fix websocket orderbook LastUpdated timestamps (#1315)
* Bitfinex: Fix WS orderbook LastUpdate empty

* Okx: Fix WS Orderbook LastUpdated unset by updates
2023-08-15 15:01:05 +10:00
Bea
26692669c6 exchanges/order: Improve order side handling in SubmitOrder (#1299)
* SubmitOrder- order.Buy replaced with IsLong

Progresses #1270

* Btcmarkets: order.Buy and order.Sell changed to order.Bid and order.Ask

* Bithumb: SubmitOrder- order.Buy replaced with IsLong

Progresses #1270

* Binanceus: SubmitOrder- order.Buy replaced with IsLong

Progresses #1720

* Bistamp: SubmitOrder- order.Buy replaced with IsLong

Progresses #1270

* Bitfinex: Changed order Side to IsLong

* Binance:switch added to SubmitOrder

* Binance: Replaced Buy and Sell in SubmitOrder with IsLong and IsShort

* Binance:SubmitOrder if statement for order side simplified
2023-08-11 14:05:43 +10:00
Gareth Kirwan
1f786f01f5 Bitfinex: Fix websocket new/modify order fields (#1280)
* Bitfinex: Fix ws new/modify Order Fields

Fixes Type, Amount and Symbol for new order, and IsShort for modify
order

* Bitfinex: Fix ws order type for margin

Also unifies the pair symbol method to standardised fixCasing

* Fix lint warning on shadow err

* Bitfinex: Remove CustomID duplication for ws
2023-08-04 15:49:03 +10:00
Bea
0cae2ad804 Bitfinex: Implement UpdateOrderExecutionLimits and tests (#1269)
* Bitfinex: Order Execution Limits and Testing

* Bitfinex: added proposed changes

* Bitfinex: lint fixes- removed the unnecessary whitespace and replaced the value with indexing

* Bitfinex: Moved data len check before the string conversion, moved the error for the array conversion before the info len check

* Bitfinex: Change GetSiteInfoConfigData to return slice, add tests

* Bitfinex: Fixed lint issue by preallocating pairs
2023-07-31 17:56:45 +10:00
Ryan O'Hara-Reid
b797934854 bitfinex: fix websocket subscribing issue caused by formatting (#1276)
* bitfinex: fix subscribing issue caused by formatting

* glorious: nits

---------

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
2023-07-24 13:26:50 +10:00
Scott
fcc5ad4551 exchanges/qa: Add exchange wrapper testing suite (#1159)
* initial concept of a nice validation tester for exchanges

* adds some datahandler design

* expand testing

* more tests and fixes

* minor end of day fix for bithumb

* fixes implementation issues

* more test coverage and improvements, but not sure if i should continue

* fix more wrapper implementations

* adds error type, more fixes

* changes signature, fixes implementations

* fixes more wrapper implementations

* one more bit

* more cleanup

* WOW things work?

* lintle 1/1337

* mini bump

* fixes all linting

* neaten

* GetOrderInfo+ asset pair fixes+improvements

* adds new websocket test

* expand ws testing

* fix bug, expand tests, improve implementation

* code coverage of a lot of new codes

* fixes everything

* reverts accidental changes

* minor fixes from reviewing code

* removes Bitfinex cancelBatchOrder implementation

* fixes dumb baby typo for babies

* mini nit fixes

* so many nits to address

* addresses all the nits

* Titlecase

* switcheroo

* removes websocket testing for now

* fix appveyor, minor test fix

* fixes typo, re-kindles killed kode

* skip binance wrapper tests when running CI

* expired context, huobi okx fixes

* kodespull

* fix ordering

* time fix because why not

* fix exmo, others

* hopefully this fixes all of my life's problems

* last thing today

* huobi, more like hypotrophy

* golangci-lint, more like mypooroldknee-splint

* fix huobi times by removing them

* should fix okx currency issues

* blocks the application

* adds last little contingency for pairs

* addresses most nits and new problems

* lovely fixed before seeing why okx sucks

* fixes issues with okx websocket

* the classic receieieivaier

* lintle

* adds test and fixes existing tests

* expands error handling messages during setup

* fixes dumb okx bugs introduced

* quick fix for lint and exmo

* fixes nixes

* fix exmo deposit issue

* lint

* fixes issue with extra asset runs missing

* fix surprise race

* all the lint and merge fixes

* fixes surprise bugs in OKx

* fixes issues with times and chains

* fixing all the merge stuff

* merge fix

* rm logs and a panic potential

* lovely lint lament

* an easy demonstration of scenario, but not of initial purpose

* put it in the bin

* Revert "put it in the bin"

This reverts commit 15c6490f713233d43f10957367fcbf18e3818bdd.

* re-add after immediate error popup

* fix mini poor test design

* okx okay

* merge fixes

* fixes issues discovered in lovely test

* I FORGOT TO COMMIT THIS

* nit fixaroonaboo

* forgoetten test fix

* revert old okx asset intrument work

* fixes

* revert problems I didnt understand. update bybit

* fix merge bugs

* test cleanup

* further improvements

* reshuffle and lint

* rm redundant CI_TEST by rm the CI_TEST field that is redundant

* path fix

* move to its own section, dont run on 32 bit + appveyor

* lint

* fix lbank

* address nits

* let it rip

* fix failing test time range

* niteroo boogaloo

* mod tidy, use common.SimpleTimeFormat
2023-07-03 11:09:43 +10:00
Ryan O'Hara-Reid
d086c45b0a Bitfinex: filter security asset type from spot trading pairs (#1191)
* bitfinex: filter security pairs from spot trading

* asset: add securities type

* bitfinex: remove print

* linter: fix

* gomod tidy

* glorious: nits

* bitfinex: just some orderbook fun and added a pass through for bitfinex funding rate when rate == 0

* bitfinex: move update function from testmain to localised test

* test: fix up

* glorious:nits

* bitfinex; fix casing test

---------

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
2023-06-05 11:53:07 +10:00
Samuael A
3eac6d12bd exchanges: Update GateIO exchange to V4 (#1058)
* Adding Public Endpoints and test functions

* Adding public endpoints and test functions

* Adding private spot endpoints

* Adding private endpoints and corresponding tests for margin

* Adding Margin Private endpoints

* Adding cross margin and flash swap endpoints

* Adding futures private endpoints

* Adding futures private endpoints and corresponding tests

* Adding Options and SubAccount endpoints and their unit tests

* Adding Wrapper functions

* Complete wrapper functions and corresponding unit test functions

* Fixing wrapper issues and adding websocket functions

* Update of Spot websocket and adding futures websocket handlers

* completed futures WS push data endpoints

* Completed Options websocket endpoints

* Adding websocket support for delivery futures and slight update on endpoint funcs

* Added Delivery websocket support and fix linter issues

* Update on Unit tests

* fix slight currency format error

* Fix slight endpoint tempos

* Update on conditional statements and unit tests issues

* fixing slight tempos

* Slight model and websocket data push method change

* Fix unit test tempos and updating models

* Fix on code structures and update on unit tests

* Slight code fix

* Remove print statements

* Update on tradable pairs fetch eps

* Fix websocket tempos

* Adding types to websocket routine manager

* Fix slight issues

* Slight fixes

* Updating wrapper funcs and models

* Slight update

* Update on test

* Update on tradable pairs

* update conditional statements

* Fixing slight issues

* Updating unit tests

* Minor fixes depending review comments

* Remove redundant method declaration

* Adding missing intervals

* Updating fetch tradable pairs

* update tradable pairs issues

* Addressing small tempos

* Slight fix on ticker

* Minor Fixes

* Minor review comment fixes

* Unit test and minor code updates

* Slight code updates

* Minor updates depending review comments

* Fixes

* Updating incoming message matcher

* Fix missing merge issue

* Fix minor wrapper issues

* Updating ratelimit and other issues

* Updating endpoint models and adding missing eps

* Update on code structure and models

* Minor codespell fixes

* Minor update on models

* fix unit test panic

* Minor race fix

* Fix issues in generating signature and unit tests

* Minor update on wrapper and unit tests

* Minor fix on wrapper

* Mini linter issues fix

* Minor fix

* endpoint fixes and slight update

* Minor fixes

* Updating exchange functions and unit tests

* Unit test and wrapper updates

* Remove options candlestick support

* Minor unit test and wrapper fix

* Unit test update

* minor fix on unit test and wrapper

* endpoints constants name change

* Add minor wrapper issues

* endpoint constants update

* endpoint url updates

* Updating subscriptions

* fixing dual mode endpoint methods

* minor fix

* rm small tempo

* Update on websocket orderbook handling

* Orderbook and currency pair update

* fix linter and test issues

* minor helper function update

* Fix wrapper coverage and wrapper issues

* delete unused variables

* Minor fix on ReadData() call

* separating websocket handlers

* separating websocket handlers

* Minor fix on enabled pair

* minor fix

* check instrument availability in spot

* create a separate subscriber for sake of multiple websocket connection

* linter fix

* minor websocket and gateio endpoints fix

* fix nil pointer exception

* minor fixes

* spelling fix decerializes -> deserializes

* fix Bitfinex unit test issues

* minor unknown currency pair labling fix

* minor currency pair handling fix

* slight update on GetDepositAddress wrapper unit test

* setting max request job to 200

* fixing numerical and timestamp type convert

* fix value overflow error

* change method of parsing orderbook price

* unifying timestamp conversion types to gateioTime

---------

Co-authored-by: Samuael Adnew <samuaelad@Samuaels-MacBook-Air.local>
2023-05-30 14:03:53 +10:00
Ryan O'Hara-Reid
b20cf75d13 sharedtestvalues: Add helper functions and implement throughout exchange tests (#1163)
* exchanges/sharedtestvalues: implement new functions to handle test skipping and announcements for standardising.

* exchanges: fin test impl.

* linter: fixes

* exchange_template: fix test

* allocate so it doesn't make a panic at the disco

* glorious: nits

* glorious: nits

* Update exchanges/sharedtestvalues/sharedtestvalues.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/sharedtestvalues/sharedtestvalues.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* linter: fix

* linter: shhhh

---------

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2023-04-28 13:05:42 +10:00
Samuael A
969669cb0d linters: Bump golangci-lint version to v1.52.2 and fix issues (#1169)
* fix linter issues

* linter issue suppressing and fixing

* change linter version and linter issues fix

* Bump version
2023-04-27 10:13:46 +10:00
Ryan O'Hara-Reid
42475bf2b8 exchanges: add setTimeWindow boolean to GetKlineRequest param (#1160)
* exchanges: add setTimeWindow boolean to GetKlineRequest params to differentiate between a set time period return from endpoint.

* glorious: nits

* exchange: conjugation

* Update exchanges/exchange.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits and an assortment of differences

* exchanges: remove some comments

* glorious: nits

* cleanup

* tests: fix

* Update exchanges/hitbtc/hitbtc_wrapper.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/kline/kline.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/kline/kline_test.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* kline: fix test

* rm unused variables

* almost: nits

* glorious: nits

* linter: fix

* rm unused variable

* Refactored comment in the okex tests to ensure that it accurately reflects the variable name and the issue related to the time window, as requested by GloriousCode. The previous comment did not align with the identifier assigned to the property, which could cause confusion and misunderstanding among other programmers or stakeholders. The updated comment will improve the clarity and readability of the codebase and make it easier to understand the intended purpose of the associated variables. The change was made with the aim of improving the overall quality and maintainability of the code.

---------

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2023-04-27 10:10:19 +10:00
Ryan O'Hara-Reid
7cadc5cffb cmd/wrapper_coverage: prevent non-needed REST requests via context timeout (#1154)
* wrapper_coverage/exchanges: cancel context to not send rest requests/ populate context through functions that do rest requests

* linter: fix

* exchange_template: fix test

---------

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
2023-04-02 11:20:25 +10:00
Ryan O'Hara-Reid
d2561402c4 common: update Errors type (#1129)
* common: adjust common error slice to allow multi errors.Is matching and conform to interface better

* zb: forgot to save?

* linties: fixies

* linties: word change as well.

* nitters: glorious

* buts

* nitters: fix glorious bug

* Update common/common.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* nitters: shifty

---------

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2023-02-20 10:48:24 +11:00
Adrian Gallagher
7e08e483fb CI: Bump go version, linters and fix minor issues (#1130)
* CI: Bump go version, linters and fix minor issues

* Bump version, fix loop variables

* Revert

* Rid TODOs now that 1.51 has been released
2023-02-03 15:56:59 +11:00
Donald Adu-Poku
82c79a9287 stream: set connection monitor delay. (#1120)
* stream: set connection monitor delay.

- this fixes a bug where the connection monitor delay config value does not
get set to the websocket on intialization.

* multi: add connection monitor delay to exchange config.

- this adds the connection monitor delay config option to the exchange type.
- the validate function of the exchange type has been updated to validate the
connection monitor delay value as well.

* multi: resolve review issues.
2023-01-31 09:21:51 +11:00
Adrian Gallagher
a79e0d2b3e qa/spelling: Add Codespell support (#1121)
* Add codespell support

* Fix paths

* Add HTML files to exclusion list
2023-01-30 12:36:56 +11:00
Ryan O'Hara-Reid
05558aabfb bitfinex: fix potential panic caused by wrapper issues cmd (#1118)
* bitfinex: fix potential panic caused by wrapper issues cmd

* bitfinex: add tests and disallow empty base, empty quote is fine.

* rm quote check

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
2023-01-24 16:50:59 +11:00
Scott
03a24b3ab1 Backtester: custom interval support (#1115)
* add backtester support

* Prevent live data custom candles, prevent nanosecond candles

* test coverage

* a more interesting rsi strategy result

* actual custom candle and proper strat date

* add test to old funk

* typos 🌞 🌞

* this was definitely worth failing linting for

* Adds stricter processing and adapts to it

* now compat with partial and absent candles

* test fixes, zb fixes

* fix more introduced bugeroos

* fix more introduced bugeroosx2

* linting for one space is so annoying

* addresseroos niteroos

* Update backtester/engine/setup.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
2023-01-24 16:05:46 +11:00
Ryan O'Hara-Reid
83cfefa45c kline/exchanges: automatic creation of unsupported candle intervals (#1091)
* kline: Add builder and testing

* Ideas

* kline: deploy builder functionality across GCT

* exchanges: implement across gct

* exchanges: Add tests and fix implementations before kline package testing and veri.

* kline: Add tests and start to fix ConvertToNewInterval

* kline: fix ConvertToNewInterval add tests

* kline: complete overarching tests now on to exchanges

* kline: finish exchange tests and implement limits

* exchanges: more fixes

* linter: fix

* engine: fix tests

* kraken: fix recent trades and other fixes

* zb: fix tests

* bithumb: fix empty insertion

* kline: refactor/optimize CreateKline function

* kline: remove the mooos!

* kline: prealloc CalculateCandleDateRanges

* linter: fix

* exchanges: prealloc extended

* fix whoopsie

* reverse fix because this is a whoopsie

* okx: fix risidual issues

* linter: fix

* kline: initial nits from @gloriouscode

* kline: rename builder -> request and cascade change

* linter: fix + test

* kline: update forced alignment on start and end times when CreateKlineRequest is called.

* nits: more more more

* NITS: Addressed

* tests: fix race issue

* Update exchanges/kline/request.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* kline: add method AddPadding() to automatically fill in holes in kline.Request functionality and reject if missing data when converting

* kline: Add params start and end to addPadding() to insert blanks in between block

* kline: remove test comment code as it's not needed anymore

* kline: fix lint and test

* kline: sort slice without extra bool check every iteration

* okx: fix issues with timeing and candles and such from niterinos & address typo

* Update exchanges/kline/kline.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: niterinos

* Update exchanges/poloniex/poloniex_wrapper.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits now onto conflicts YAYA!!!

* Update exchanges/exchange_test.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits again

* thrasher: nitters

* thrasher: niterinos - adds partial flag for incomplete recent candles and fetching.

* kline: rm fmtizzle packageizzle

* glorious: nitters

* glorious: more niterinos

* fix last niterinos

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2023-01-17 16:22:33 +11:00
Scott
017cdf1384 Backtester: Live trading upgrades (#1023)
* Modifications for a smoother live run

* Fixes data appending

* Successfully allows multi-currency live trading. Adds multiple currencies to live DCA strategy

* Attempting to get cash and carry working

* Poor attempts at sorting out data and appending it properly with USD in mind

* =designs new live data handler

* Updates cash and carry strat to work

* adds test coverage. begins closeallpositions function

* Updates cash and carry to work live

* New kline.Event type. Cancels orders on close. Rn types

* =Fixes USD funding issue

* =fixes tests

* fixes tests AGAIN

* adds coverage to close all orders

* crummy tests, should override

* more tests

* more tests

* more coverage

* removes scourge of currency.Pair maps. More tests

* missed currency stuff

* Fixes USD data issue & collateral issue. Needs to close ALL orders

* Now triggers updates on the very first data entry

* All my problems are solved now????

* fixes tests, extends coverage

* there is some really funky candle stuff going on

* my brain is melting

* better shutdown management, fixes freezing bug

* fixes data duplication issues, adds retries to requests

* reduces logging, adds verbose options

* expands coverage over all new functionality

* fixes fun bug from curr == curr to curr.Equal(curr)

* fixes setup issues and tests

* starts adding external wallet amounts for funding

* more setup for assets

* setup live fund calcs and placing orders

* successfully performs automated cash and carry

* merge fixes

* funding properly set at all times

* fixes some bugs, need to address currencystatistics still

* adds 'appeneded' trait, attempts to fix some stats

* fixes stat bugs, adds cool new fetchfees feature

* fixes terrible processing bugs

* tightens realorder stats, sadly loses some live stats

* this actually sets everything correctly for bothcd ..cd ..cd ..cd ..cd ..!

* fix tests

* coverage

* beautiful new test coverage

* docs

* adds new fee getter delayer

* commits from the correct directory

* Lint

* adds verbose to fund manager

* Fix bug in t2b2 strat. Update dca live config. Docs

* go mod tidy

* update buf

* buf + test improvement

* Post merge fixes

* fixes surprise offset bug

* fix sizing restrictions for cash and carry

* fix server lints

* merge fixes

* test fixesss

* lintle fixles

* slowloris

* rn run to task, bug fixes, close all on close

* rpc lint and fixes

* bugfix: order manager not processing orders properly

* somewhat addresses nits

* absolutely broken end of day commit

* absolutely massive knockon effects from nits

* massive knockon effects continue

* fixes things

* address remaining nits

* jk now fixes things

* addresses the easier nits

* more nit fixers

* more niterinos addressederinos

* refactors holdings and does some nits

* so buf

* addresses some nits, fixes holdings bugs

* cleanup

* attempts to fix alert chans to prevent many chans waiting?

* terrible code, will revert

* to be reviewed in detail tomorrow

* Fixes up channel system

* smashes those nits

* fixes extra candles, fixes collateral bug, tests

* fixes data races, introduces reflection

* more checks n tests

* Fixes cash and carry issues. Fixes more cool bugs

* fixes ~typer~ typo

* replace spot strats from ftx to binance

* fixes all the tests I just destroyed

* removes example path, rm verbose

* 1) what 2) removes FTX references from the Backtester

* renamed, non-working strategies

* Removes FTX references almost as fast as sbf removes funds

* regen docs, add contrib names,sort contrib names

* fixes merge renamings

* Addresses nits. Fixes setting API credentials. Fixes Binance limit retrieval

* Fixes live order bugs with real orders and without

* Apply suggestions from code review

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update backtester/engine/live.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update backtester/engine/live.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update backtester/config/strategyconfigbuilder/main.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* updates docs

* even better docs

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
2023-01-05 13:03:17 +11:00
Ryan O'Hara-Reid
e4d487e586 exchanges: Change wrapper function FetchTradablePairs to return currency.Pairs (#1093)
* exchanges: FetchTradablePairs currency pair slice return (1)

* exchanges: finish conversion

* thrasher: nits + linter

* exchanges: shift var dec into for loop

* exchanges: Apply changes thanks @thrasher-

* Update exchanges/alphapoint/alphapoint_wrapper.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* thrasher: nitters

* Update exchanges/bitflyer/bitflyer_wrapper.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* thrasher: fix more nitters

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
2022-12-09 15:40:34 +11:00
Ryan O'Hara-Reid
558a70e87f orders: add Filter() method to GetOrdersRequest and force FilteredOrders return type on wrapper functions (#1055)
* orders: filter options return on validate

* exchanges: force filter usage by wrapper return type and shift method functionality

* exchanges: update tests and fix err shadow

* exchanges: shadowland

* exchanges: more shadow land spookyness

* glorious: nits and ftx upgrade

* linter: fix

* orders: preserve unpopulated fields through filtering operation

* glorious: nits

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
2022-10-24 13:28:32 +11:00
Ryan O'Hara-Reid
f843b7d277 exchange: upgrade UpdatePair method (#991)
* exchange: upgrade update pair

* exchanges: Add enabled string matching and format handling if discrepency is found.

* linter: fixes

* bithumb: fix tests

* BTSE: api change fix ordering

* huobi: fix tests

* gloriousnits: stage 1

* gloriousnits: stage 2

* currency: more nits

* bitmex: add spot and process pairs before currency package call.

* bitmex: finished correct orderbook matching and other implementations

* linter: fix issue

* currency: Fix linter

* currency: segregate and protect pair store, update tests

* currency/manager: clean code, rm log output

* currency: Add store method and make sure formatting stays nil if not stored.

* gct: check errors

* engine/websocketroutineman: fix tests

* bybit: fix duplication bug

* huobi: fix test

* btse: fix tests?

* ob/buffer: fix tests

* Update currency/manager.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* glorious: nits strikes again.

* exchange: add bypassConfigFormatUpgrades to stop formatting

* GLORIOUS LINTER

* Update exchanges/bithumb/bithumb_wrapper.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* exchange: fix pair upgrade issue when duplications are in both avail and enabled pairs

* linter: fix shadow dec

* config: fix test

* Update currency/pair_test.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2022-09-16 08:59:27 +10:00
Jaydeep Rajpurohit
247da918a8 exchanges: Add ByBit support (#887)
* few fixes and add ratelimiter

* adds test

* revert configtest.json changes

* configtest updated

* WIP: adds public endpoint support

* WIP: adds public endpoint support

* adds public endpoint support

* WIP: adds auth. endpoint support

* adds test for auth. endpoint

* fixes

* adds auth. endpoint support

* WIP: ws support

* WIP

* WIP

* WIP

* WIP

* WIP

* WIP

* WIP

* Testing

* Complete WS spot testing

* adds support for ws events

* minor change

* WIP: adds REST support for CoinMarginedFutures

* Fixes

* WIP: adds REST support for CoinMarginedFutures

* Fixes

* improvement in SPOT REST

* Typo fix

* WIP: add REST support for CMF Account API

* minor fixes

* WIP: add support for CMF conditional orders and few minor fixes

* complete support for CMF conditional orders

* adds support for public CMF endpoint

* adds support for CMF position API

* Complete REST CMF support

* WIP

* Testing REST CMF support

* Testing REST CMF support

* Testing REST CMF support completed

* WIP: add support for UMF

* completed non-auth UMF

* WIP: add support for REST Auth. UMF

* WIP: add support for REST Auth. UMF and some improvements

* WIP

* WIP

* WIP

* completed REST UMF

* renaming

* adds REST support for futures

* add testcases for UMF and some optimizations

* add testcases for futures

* Testing UMF, futures and its changes

* Fixes

* Fixes after testing

* WIP

* WIP

* WIP

* completed ws USDT futures support

* WIP: ws support for futures

* fixes in WS futures

* fixes in WS support

* roll back changes made for WS CMF, USDT and Futures

* fixes

* WIP

* WIP

* fixes

* Steps for new PR

* WIP

* WIP

* WIP

* WIP

* complete PR setup

* fixes for successfully running tests

* update in symbol for futures pair in test file

* WIP

* Fixes in test file and other minor fix

* fix testdata/configtest.json

* reset CONTRIBUTORS file

* review changes

* remove unwanted file

* remove redundant code

* improvisation

* adds comment for exported functions

* remove unwanted TODO and commented code

* fix

* improvisation

* fix

* defined errors

* improvisation

* improvisation

* improvisation

* updates test

* adds comment for exported types

* review changes

* review changes

* fix

* fixes

* Changes for making BYBIT compatible with existing code base

* Test file changes

* Changes for making BYBIT compatible with existing code base

* Changes for making BYBIT compatible with existing code base

* fix lint issues

* fix

* review changes

* review changes

* review changes

* review changes

* review changes

* review changes

* review changes

* review changes

* review changes

* review changes

* WIP

* add test cases for new API's

* minor improvements

* add missing API and their tests

* minor fixes

* add bybitTime

* add bybitTimeSec, bybitTimeMilliSec, bybitTimeNanoSec and necessary support

* fix GetTradeHistory function

* error handling

* test fixes

* add GetServerTime API

* adds GetHistoricCandlesExtended and review changes

* test fixes

* minor fix

* integrating CMF Bybit recent change log

* minor fixes

* adds extractCurrencyPair

* minor fixes

* minor fix

* review changes

* adds variable declaration of error

* review commit

* adds embeddable type in API response for all API and integrate it

* fixes

* adds authentication WS connection

* review changes

* review changes

* compatible changes

* adds asset to GetWithdrawalsHistory

* adds asset_type in rpc.proto

* adds asset argument in gctcli withdrawal request command

* improve error handling in exchange API error

* web socket fix

* review changes

* improvements

* improvements

* minor fix

* review changes

* fixing wrapper issues

* fixes

* fixes

* review changes

* add test cases

* fix for GetActiveOrders

* lint fixes

* fixes in websocket

* adds wrapper testcases

* adds wrapper testcases

* adds wrapper testcases

* fixes

* fix issue with GetHistoricCandlesExtended

* fix merge issues

* improving error reporting

* adds wrapper testcases and a minor fix

* gctrpc changes

* adds test cases
fixes in websocket

* review changes for ws

* review changes in WS

* fix gctrpc

* merge fixes

* review changes

* WIP

* updates pair in configs

* adds new asset USDCMarginedFutures

* adds URL const for USDCMarginedFutures

* adds API support

* minor fixes

* adds kline API

* minor fix

* adds API

* adds API

* adds API

* WIP

* WIP

* WIP

* adds support for USDC auth requests to SendAuthHTTPRequest

* adds SendUSDCAuthHTTPRequest

* run test and fix them

* rollback support added for Auth. USDC request inside SendAuthHTTPRequest

* adds API and test cases

* adds API and test cases

* adds APIs and test cases

* adds APIs

* adds rate limit for USDC

* adds USDCMarginedFutures to wrapper

* adds USDC testcases in wrapper and fix few issues

* minor test fixes

* minor test fixes

* fix lint issues

* WIP

* Merge changes

* minor fixes

* remove "else" and optimize

* review changes

* review changes

* review changes

* fix lint issue

* merge fix

* fix test

* fix templates and run them

* changes after merge

* review changes and improvements

* code improvement

* fixes with respect to changes in API response in documentation

* fixed review change in test

* adds check in CancelExistingOrder

* update exchange template

* review changes

* adds GetDepositAddress API

* WIP: adds GetOrderHistory

* complete GetOrderHistory

* fixes

* adds test case

* fixes and add WithdrawFund API

* WIP

* WIP

* updating all SendAuthHTTPRequest call

* adds WithdrawCryptocurrencyFunds

* update test cases

* fix lint issues

* fixes after merge

* adds GetAvailableTransferChains and few fixes

* minor fix in GetDepositAddress

* minor fix with WS ping/pong handling

* add ping handler for WS Auth.

* fix typo mistake

* update doc
2022-08-08 11:29:43 +10:00
Ryan O'Hara-Reid
663e753f52 account: segregate holdings by credentials for future multi-key management (#956)
* exchanges/account: shift credentials to account package and segregate funds to keys

* merge: fixes

* linter: fix

* Update exchanges/account/account.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits + protection for string panic

* glorious_suggestion: add method for matching keys

* linter: fix tests

* account: add protected method for credentials minimizing access, display full account details to rpc.

* linter: spelling kweeeeeeen

* accounts/portfolio: clean/check portfolio code and quickly check balances from change. Add protected method for future matching.

* accounts: theres no point in pointerising everything

* linter: ok pointerise this then...

* exchanges: fix regression add in little notes.

* glorious: nits

* Update exchanges/account/credentials.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/account/credentials_test.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/account/credentials_test.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* gloriously: fix glorious glorious test gloriously

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2022-07-21 15:05:31 +10:00
Ryan O'Hara-Reid
a12cd3d733 orders: Add methods to derive SubmitResponse and Detail types (#955)
* orders: deprecate SubmitResponse return and change to *order.Detail construct detail from order.Submit struct

* orders: add coverage, fix tests

* coinut: rm test for checking

* orders: revert change for return and change field ID to a more explicit name OrderID

* orders: Add method to see if the order was placed

* order: change field name in Cancel type to be more explicit

* orders: standardize field -> OrderID

* backtester: populate change

* orders: add test

* gctscript: fix field name

* linter: fix issues

* linter: more fixes

* linter: forever

* exchanges_tests: populate order.Submit field exchange name

* Update exchanges/order/order_types.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/order/orders.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* glorious: nits

* thrasher: nits

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2022-06-06 11:52:15 +10:00
Ryan O'Hara-Reid
a63aa6b616 orders: Add derive modify struct method from order.Detail (#948)
* orders: Add derive modify struct method to order.Detail and then subsequent method to derive and standardize response details

* exchanges: call modify method in wrappers

* linter: fixes

* engine/wsroutineman: remove print summary

* glorious: nits, removed modifyOrder functionality for Bithumb. There are not docs to support this.

* Update exchanges/order/orders.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2022-05-30 14:38:21 +10:00
Ryan O'Hara-Reid
16a93b49a4 btcm: add modify order functionality, change return to pointer (#940)
* btcm: add modify order functionality, change return to pointer

* glorious: nits

* glorious: nits

* btcm: Adjust function name

* thrasher: nits

* thrasher: nits cont...
2022-05-16 10:55:23 +10:00
Ryan O'Hara-Reid
cdcc9630de order: slight optimizations (#917)
* order: slight optimizations

* orders: add benchmarks, small optimize and change order side to uin8 for comparitive optimizations.

* orders: continue to convert string type -> uint

* orders/backtester: interim move type to orders package, later can expand or deprecate.

* orders: handle errors

* orders: optimize filters and remove error returns when its clearly not needed

* orders: remove log call

* backtester: zero value check

* orders/futures: zero value -> flag

* linter: fix

* linter: more fixes

* linters: rides again

* glorious: nits

* common: Add zero value unix check for time values; also addresses glorious nits

* glorious scott: nits

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
2022-05-06 12:27:21 +10:00
Ryan O'Hara-Reid
c6ad429827 orderbook/buffer: data integrity and resubscription pass (#910)
* orderbook/buffer: data integrity and resubscription pass

* btcmarkets: REMOVE THAT LIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIINE!!!!!!!!!!!!!!!!!

* buffer: reinstate publish, refaactor, invalidate more and comments

* buffer/orderbook: improve update and snapshot performance. Move Update type to orderbook package to util. pointer through entire function calls. (cleanup). Change action string to uint8 for easier comparison. Add parsing helper. Update current test benchmark comments.

* dispatch: change publish func to variadic id param

* dispatch: remove sender receiver wait time as this adds overhead and complexity. update tests.

* dispatch: don't create pointers for every job container

* rpcserver: fix assertion issues with data publishing change

* linter: fixes

* glorious: nits addr

* depth: change validation handling to incorporate and store err

* linter: fix more issues

* dispatch: fix race

* travis: update before fetching

* depth: wrap and return wrapped error in invalidate call and fix tests

* btcmarkets: fix commenting

* workflow: check

* workflow: check

* orderbook: check error

* buffer/depth: return invalidation error and fix tests

* gctcli: display errors on orderbook streams

* buffer: remove unused types

* orderbook/bitmex: shift function to bitmex

* orderbook: Add specific comments to unexported functions that don't have locking require locking.

* orderbook: restrict published data functionality to orderbook.Outbound interface

* common: add assertion failure helper for error

* dispatch: remove atomics, add mutex protection, remove add/remove worker, redo main tests

* dispatch: export function

* engine: revert and change sub logger to manager

* engine: remove old test

* dispatch: add common variable ;)

* btcmarket: don't overflow int in tests on 32bit systems

* ci: force 1.17.7 usage for go

* Revert "ci: force 1.17.7 usage for go"

This reverts commit af2f95563bf218cf2b9f36a9fcf3258e2c6a2d91.

* golangci: bump version add and remove linter items

* Revert "golangci: bump version add and remove linter items"

This reverts commit 3c98bffc9d030e39faca0387ea40c151df2ab06b.

* dispatch: remove unsused mutex from mux

* order: slight optimizations

* nits: glorious

* dispatch: fix regression on uuid generation and input inline with master

* linter: fix

* linter: fix

* glorious: nit - rm slice segration

* account: fix test after merge

* coinbasepro: revert change

* account: close channel instead of needing a receiver, push alert in routine to prepare for waiter.

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
2022-05-03 12:37:08 +10:00
Ryan O'Hara-Reid
6a02097431 account: storage, processing and method on balances update (#916)
* account: update account storage, retrieval and implement alert functionality when a currency change occurs.

* account: Add cancel channel

* account: remove old code

* account: don't embed mutex

* Update exchanges/account/account.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/account/account.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/account/account.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* account: addr nits

* account: Pull out test into indiv.

* account: Add test for update method

* account: add no change to test

* Update exchanges/account/account.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* account/portfolio: differentiate between asset type segregation and default to spot holdings.

* glorious: nits

* thrasher: nit

* ticker: fix spelling

* Update engine/portfolio_manager.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* thrasher: nits

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
2022-05-02 14:01:03 +10:00
Adrian Gallagher
9a4eb9de84 CI: Fix golangci-lint linter issues, add prealloc linter and bump version depends for Go 1.18 (#915)
* Bump CI versions

* Specifically set go version as 1.17.x bumps it to 1.18

* Another

* Adjust AppVeyor

* Part 1 of linter issues

* Part 2

* Fix various linters and improvements

* Part 3

* Finishing touches

* Tests and EqualFold

* Fix nitterinos plus bonus requester jobs bump for exchanges with large number of tests

* Fix nitterinos and bump golangci-lint timeout for AppVeyor

* Address nits, ensure all books are returned on err due to syncer regression

* Fix the wiggins

* Fix duplication

* Fix nitterinos
2022-04-20 13:45:15 +10:00
Ryan O'Hara-Reid
1669f1c626 btcmarkets: Add websocket orderbook checksum validation (#900)
* btcmarkets: add websocket checksum, fetch different book via REST

* Update exchanges/btcmarkets/btcmarkets_test.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* buffer: add explicit type for buffer related variables and comments, do all checks buffer side and load in setup as per glorious recom.

* buffer: fix tests add error

* buffer: test re-add code cov

* depth/stream/ws: fix tests, change field name to be more specific.

* buffer: rm unused field and small comment fixes

* btcm: remove redundant field

* glorious: nits

* buffer: fix commenting

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2022-03-21 16:19:58 +11:00
Ryan O'Hara-Reid
09fa2f236a context: Add authenticated HTTP credentials (#892)
* gRPC: context overide

* exchanges: continue update

* exchange: Update context handling
*Add setter methods for API credentials
*Shift credentials functionality to its own file in exchanges package
*Add tests
*Refactor function DeployCredentialsToContext for library usage
*Add function to process credential metadata from API boundary to internal use context value.
*Add OTP rpc handling

* exchanges: reverts to old style in GetFeeByType, reverts some code I accidently deleted. Plus things and other. XD

* template: update

* exchanges: fix linter issues

* REMOVE THAT AWESOME NEW LINE!

* gct: fix some tests

* I cant spell :(

* exchanges/gctscript: fix more tests

* coinnut: fix tests

* Update exchanges/credentials.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/credentials.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/credentials.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/credentials.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/credentials.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* exchanges/gctcli: stop applying empty credentials

* fix linters

* exchanges: add test

* rpceserver: actually check error for errors

* rpcserver: fix up tests

* Update exchanges/credentials.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* exchanges/creds: move tests to corresponding files, add protection and segration for Credentials struct & ptr values

* exchanges/creds: allow subaccount to override default credentials via gRPC

* exchanges/credentials: don't return nil in GetCredentials

* creds: spelling

* exchanges: fix glorious NITS!

* credentials: Add in test and refactor IsEmpty method.

* credentials: change type positioning (glorious)

* exchange_template: Fix template changes

* DOCS: Refresh

* docs: fix spelling

* DOCS: fix alignment and add package

* DOCS: ALIGN!

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2022-03-21 13:58:08 +11:00
Scott
86d3724507 Futures order position tracking & FTX scaled collateral calculation (#868)
* implements futures functions and GRPC functions on new branch

* lint and test fixes

* Fix uneven split pnl. Adds collateral weight test. docs. New clear func

* Test protection if someone has zero collateral

* Uses string instead of double for accuracy

* Fixes old code panic

* context, match, docs

* Addresses Shazniterinos, var names, expanded tests

* Returns subaccount name, provides USD values when offlinecalc

* Fixes oopsie

* Fixes cool bug which allowed made up subaccount results

* Subaccount override on FTX, subaccount results for collateral

* Strenghten collateral account info checks. Improve FTX test

* English is my first language

* Fixes oopsies

* Fixes for unrealised PNL & collateral rendering

* Fixes lint and tests

* Shaznit fixes

* Secret Shaznit

* Updates account information across wrappers to include more fields

* Updates online collateral calculations. Updates RPC data

* Accurately calculates collateral offline and online minus testing

* Tests and lint chocolate

* Simplifies accountinfo results

* Fixes shaznits

* Adds new func

* Increases collateral accuracy again again again x 200

* Increases accuracy of collateral rendering

* Fixes minor merge/test issues

* Linterino

* Fixes ws test. Improves collateral calculations and rendering

* Make it prettier

* Removes the lock I put on 👀

* Adds `additional_collateral_used` field, renders orig currency

* Fixes unrelated test

* Fix test

* Correctly calculate spot margin borrow collateral

* Address fun lint surprise

See https://github.com/golangci/golangci-lint/issues/741#issuecomment-1017014331

* Strange lint fixing x2

* Continued lint journey

* Nolint the nolint to not lint the lint

* Adds two new fields to response

* More linting issues arising

* fIX3s_c4s|NG

* Fixes command flags' incorrect numbering

* FairMarket = Won
2022-02-28 16:39:36 +11:00
Ryan O'Hara-Reid
6127e2ab73 Request: Fix http.Client race issue when setting transport layer proxy and timeouts (#885)
* backtester/request: trying to fix panic (WIP)

* request: fix race for transport layer

* request: linter issue fix

* request: more linter issues

* requester: Add function to remove the tracking of underlying http client and add to engine unload exchange.

* request: add more context to error return

* request: Fix after cherry pick issues

* request: fix niterinos

* exchanges: change return to package variable

* request: changed named

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
2022-02-18 09:22:10 +11:00
Ryan O'Hara-Reid
da3402476e tool/exchange_wrapper_coverage: fix regression and implement reflection (#837)
* cmd/tools/exchange: fix regression and implement reflection so as this can dynamically scale to our interface

* exchanges: add comment and fix whoopsie

* exchanges: fix linter issues

* wrapper_cov_tool: add actual full interface method count to get a better perceived deployment

* exchanges/tool: addr glorious nits

* kraken: remove string in test

* exchange_template_tool: fix tmpl issue
2021-11-17 15:41:21 +11:00
khcchiu
8617b50ff6 Exchanges: enrich order history with avg executed price, cost, and more (#792)
* Exchanges: enrich order history with avg executed price, cost, and more

* Fix division by zero in order detail enrichment

* Remove DateCompleted from Bithumb OrderData and fix OrderDate parsing

* Fixes on order detail fields and rename EnrichOrderDetail to CalculateCostsAndAmounts

* BTSE order history populate name and id

* Calculate average executed price for market order or when order amount is zero

* Minor fixes on infer order amounts, costs, and times

* Attach InferAmountsCostsAndTimes to Order.Detail

* Binance: fix order status

* Always use order.StringToOrderStatus() and ensure order has at least one of executed/remaining amount set
2021-10-26 11:22:30 +11:00
Ryan O'Hara-Reid
099ffa1a60 stream/websocket: Consolidate fields by using exchange config pointer (#809)
* stream: add exchange config pointer to setup WebsocketSetup struct to reduce and consolidate setting of variables.

* config: reduce stutter

* config: reduce minor stutter

* glorious: nits addr.

* Update exchanges/stream/websocket.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* websocket: implement fix

* engine/helpers: fix test

* exchanges: fix after merge issues

* exchange_template: fix output

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2021-10-20 15:45:06 +11:00
Luis Rascão
a70224d123 exchanges/websocket: Allow configuration of orderbook publish period (#805)
* Allow configuration of orderbook publish period

For some applications that import GCT it's more interesting to be
immediately notified of an exchange orderbook update instead of
only getting notified every 10 seconds. This option allows that
to happen while keeping the previous default.

* exchanges: allow configuration of orderbook update period
2021-10-20 11:44:24 +11:00
Adrian Gallagher
0c00b7e1df exchanges/engine: Add multichain deposit/withdrawal support (#794)
* Add exchange multichain support

* Start tidying up

* Add multichain transfer support for Bitfinex and fix poloniex bug

* Add Coinbene multichain support

* Start adjusting the deposit address manager

* Fix deposit tests and further enhancements

* Cleanup

* Add bypass flag, expand tests plus error coverage for Huobi

Adjust helpers

* Address nitterinos

* BFX wd changes

* Address nitterinos

* Minor fixes rebasing on master

* Fix BFX acceptableMethods test

* Add some TO-DOs for 2 tests WRT races

* Fix acceptableMethods test round 2

* Address nitterinos
2021-10-15 15:55:38 +11:00
Adrian Gallagher
f0d45aa1d2 golangci-lint/CI: Bump versions and introduce new linters (#798)
* golangci-lint/CI: Bump versions

Fix remaining linter issues

* Specifically set AppVeyor version

* Fix the infamous typos 👀

* Add go env cmd to AppVeyor

* Add go version cmd to AppVeyor

* Specify AppVeyor image, adjust linters

* Update go get to go install due to deprecation

* Bump golangci-lint timeout time for AppVeyor

* Change NW contract to NQ

* Address nitters

* GetRandomPair -> Pair{}

* Address nits

* Address time nitterinos plus additional tweaks

* More time inception upgrades!

* Bending time and space
2021-10-14 16:38:53 +11:00
Ryan O'Hara-Reid
5dfbbf84de engine/exchanges: Add exchange currency state subsystem (#774)
* state: Add management system (init)

* linter: fix

* engine: gofmt

* gct: after merge fixup

* documentation: add

* rpc: implement services for testing

* gctcli: gofmt state_management.go

* documentation: reinstate lost information

* state: Add pair check to determine trading operation

* exchanges: add interface for specific state scoped subsystem functionality

* engine/order_man: reduce code footprint using new method

* RPC: implement pair trading request and change exported name to something specific to state

* engine: add tests

* engine: Add to withdraw manager

* documentation: reinstate soxipy in contrib. list

* engine: const fake name

* Glorious: NITERINOS

* merge: fix issues

* engine: csm incorporate service name into log output

* engine: fix linter issues

* gct: fix tests

* currencystate: remove management type

* rpc: fix tests

* backtester: fix tests

* Update engine/currency_state_manager.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update engine/currency_state_manager.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/currencystate/currency_state.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/alert/alert.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/alert/alert.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* config: integrate with config and remove flag delay adjustment

* gctcli: fix issues after name changes

* engine: gofmt manager file

* Update engine/rpcserver.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* engine: Add enable/disable manager functions, add default popoulation for potential assets

* linter: fix

* engine/test: bump subsystem count

* Update engine/currency_state_manager.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/bithumb/bithumb.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits addressed

* alert: fix commenting for its generalized purpose

* glorious: nits

* engine: use standard string in log output

* bitfinex: apply patch, thanks @thrasher-

* bitfinex: fix spelling

* engine/currencystate: Add logs/fix logs

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2021-09-27 13:33:49 +10:00
Ryan O'Hara-Reid
d636049fb2 exchanges: Initial context propagation (#744)
* gct: phase one context awareness pass

* exchanges: context propagation pass

* common/requester: force context requirement

* gctcli/exchanges: linter fix

* rpcserver: fix test using dummy rpc server

* backtester: fix comments

* grpc: add correct cancel and timeout for commands

* rpcserver_test: add comment on dummy server

* common: deprecated SendHTTPGetRequest

* linter: fix

* linter: turn on no context check

* apichecker: fix context linter issue

* binance: use param context

* common: remove checks as this gets executed before main

* common: change mutex to RW as clients can be used by multiple go routines.

* common: remove init and JIT default client. Unexport global variables and add protection.

* common: Add comments

* bithumb: after dinner mints fix
2021-09-11 13:52:07 +10:00
Ryan O'Hara-Reid
66fbd43cf0 websocket: fix deadlock when enabling/disabling via gctrpc (#754)
* websocket: select case error if no receiver, add in functionality to reset to initial sync for books on a new websocket connection

* websocket: fix tests

* websocket: log error instead of losing it

* websocket: fix whoopsie

* exchanges: fix test

* websocket: force requirement of specific functionality

* exchanges: fix tests

* exchanges/websocket: move waitgroup add before scheduling across exchanges

* gateio: add feature subscribe

* bithumb/bittrex: include connection state reset, fix reconnection bug for Bithumb

* huobi: Add listen to shutdown to routine so it actually returns and stops being a naughty boy.

* huobi: add missing waitgroup add.

* exchanges: bleed comms channels

* binance: fix reconnection bug with buffer

* bithumb: fix reconnection bug with ws orderbook when websocket is diabled/enabled

* bithumb/bittrex: add bleeders for ws orderbook jobs

* linter: fix

* kraken: reduce code block from double assertion

* This bug ruined my day.

* glorious: error checking

* zb: add correct path for websocket connection

* exchange: Add verbosity when config conflicts and overwrites default values

* zb: add https to path

* exchanges: glorious nits

* stream: Add checkAndSetMonitoring to reduce potential routine bundling, increase timeout and check state in tests

* stream: remove check that is not needed.

* glorious: nits addr.

* lint: test
2021-09-03 17:21:23 +10:00
Ryan O'Hara-Reid
8020e1ec6a linter: Enable error checking linter (#766)
* golangci: Enable err checking linter to expose unchecked errors.

* gct: handle errors across the board

* gct: handle errors NOTE: Found bug in FTX (WIP)

* linter: fix issues

* ftx/exchanges: fix bug where error was being returned when setting pair management variables to an already enabled state

* bitmex: fix bug where a dangly supported asset in config danglied up the place.

* linter: fix more linter issues

* linter: fix my terrible spelling.

* currency: fix test

* exchanges: fix tests

* logger: fix test

* exchanges: fix tests

* glorious: nits

* vm: revert rm variable and instigate test
2021-08-30 14:06:40 +10:00
Luis Rascão
c9ab0b1164 exchanges: Add UpdateTickers method and reserve UpdateTicker for single ticker symbol update requests (if supported) (#764)
* exchanges: add an UpdateTickers method to the main exchange interface

This method will fetch all currency pair tickers of a given asset type
and update them internally, does nothing for now.

* exchanges: refactor UpdateTicker on all exchanges

Keep the exact previous behaviour but implement the UpdateTickers
method and refactor UpdateTicker by using it where applicable.

* sync_manager: update all tickers when batching is enabled

* binance: UpdateTicker to fetch single ticker symbol

* ftx: UpdateTicker to fetch single ticker symbol
2021-08-27 12:31:34 +10:00