Commit Graph

202 Commits

Author SHA1 Message Date
Gareth Kirwan
f9437dbd08 Bitfinex: Websocket subscription improvements (#1353)
* Websockets: Add keys to websocket subscriptions

* This switches all RO uses of the mutex to use a RLock method.
* The mutex used for discrete field access has had scope drift from
  name 'connectionMutex' so rename to more appropriate fieldsMutex
* The mutex used for Set/CanUseAuthEndpoints moves from the
  subscriptions endpoint to the fieldsMutex
* Add GetSubscription by key
* Expose stream.Matcher type

* Bitfinex: Subscribe and Unsubscribe atomicly

* Fix Auth failures ignored
* This change makes it so that Subscribe and Unsubscribe wait for success
** Tells the DataHandler about errors
** Errors are returned to consumers
* Subscribes concurrently to the channels
* It also simplifies the chanId to stream mapping
* Removes unable to locate chanID: %d errors which are just noise
* Paves the way for unified channelSubscription id handling
* Adds support for subId for Book subscriptions, which is more robust

* Vastly simplifies what we need to test TestWsSubscribedResponse
This test was working to ensure that the various fancy key parsing
mechanisms all worked. Now that we use subId, we just need a thorough
test of that
* Expose Match.Set in order to capture websocket incoming data
Can't see another way of doing this. Doesn't seem too bad

* Allow tests to run with auth or WS
These flags made it difficult to run the tests whilst working on
websockets

* Enable API auth and WS in testconfig
This change minimises the changes requires for a full test run against
live endpoints, so that new contributors have a clearer testing path.
I cannot see any reason to turn WS off and Auth endpoints off when we're
not going to run API tests without Creds being set, and we're not going
to do live fire tests without canManipulateRealOrders

* TestWsSubscribe and various fixes
** Enables the websocket for live non-authed integration tests by default
** Adds an integration test for subscriptions
** Changes the Ws tests to respect canManipulateRealOrders
** Uses WsConnect instead of setupWs; fixes seqNo config not sent for WS tests
** Allows api creds to live in config/testdata.json which might be
  less likely to accidentally commit, and less obtrusive

* Bitfinex: Support period and timeframe for Candles

* Fixes manual Subscribe() symbol or key formatting
* Unifies handling of params for DefaultSubscriptions and manual
  subsrciptions

* Bitfinex: Handle conf and info WS channel events

* Bitfinex: Better tests for subscriptions

* fixup! Websockets: Add keys to websocket subscriptions

* fixup! Bitfinex: Subscribe and Unsubscribe atomicly

* fixup! Websockets: Add keys to websocket subscriptions

* Websockets: Add Pending subscription status

Add a status tracker so that Sub/Unsub can prevent duplicates,
and also fixes when first message comes before we have added the sub
to the tracker

* Websockets: Add State instead of pending

This change allows more clarity about the current state and
checks for specifically already Unsubing

* Bitfinex: Fix first sub message maybe lost

The only link we have between a sub req and the sub resp is the subID.
And the only link we have between a sub message and the sub is the chanID.
We can't derive a link using Pair or anything else.

This meant that by sending the resp and its chanID down the IncomingData
channel, we allowed the channel reader to maybe process the next
message, the first message on the channel, before the runtime executed
the switch back to subscribeToChan waiting on the chan.

To fix this, we key initially on subId.(string), and then replace it
with chanId.(int64) when we have it *inside* the wsHandleData so we
know we've procedurally handled it before the next message.

subscribeToChan is then free to remove the subId keyed Sub regardless of
error or not

If there's an error, we don't need to inline handling because there
won't be any second update.

Expands test coverage to make sure those subId keyed subscriptions are
removed.

* Websocket: Validate state in SetChanState

* fixup! Bitfinex: Fix first sub message maybe lost

* Websockets: Rename RemoveUnsuccessfulSubs

Implementation doesn't imply Unsuccessful or need to.
This change supports the registering of Pending subs

* Bitfinex: Fix race in Tests
2023-11-02 12:10:43 +11:00
Ryan O'Hara-Reid
ceef7a14e0 currency: Adds matching lookup table built from available pairs (#1312)
* currency: Add pair matching update (cherry-pick)

* exchange/currency: Add tests and update func

* linter fix, also if using json unmarshal functionality stop usage of string conversion without delimiter

* gemini: fix test

* currency/manager: potential optimisation

* exchanges: purge derive from wrapper cases and add warning comment

* glorious: nits

* glorious: nits

* linter: fix

* glorious: nits

* whoops

* whoops

* glorious: nits continued

* glorious: diff THANKS!

* hitbtc: fix update tradable pairs strings splitting. continue if not enabled tickers update pair.

* glorious: nits

* linter: fix

* Update exchanges/exmo/exmo_wrapper.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* bitstamp: fix test when 32 biterinos architecturinos

* capture more strings for speed

* swapsies because whos running 32bit \0/?

---------

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2023-10-18 11:57:27 +11:00
Bea
859c4512fb Bitstamp: Add new trading fees API endpoint; refine fee calcuations and test coverage (#1289)
* Bitstamp: WIP fixing trading fees

* Bitstamp/amended the TestGetFee test and currected the getTradingFee function

* Bitstamp: TestGetFee implemented for maker and taker

* Bitfinex: added a wrapped error

* Bitstamp: GetTradingFee and test updated, fetched from the API

Progresses #1271

* Bitstamp: minor changes- whitespace removal and comment added

* Bitstamp: fixed lint issues in TestGetBalance and TestGetTransactions

* Bitstamp:Typo in types TradingFee comment

* Bitstamp: GetAccountTradingFees at view the fees on all pairs

	modified:   exchanges/bitstamp/bitstamp.go

* Bitstamp: returning the TradingFee info instead of just MakerTakerFees

* Bitstamp:TestGetAccountTradingFees + TestGetAccountTradingFees added,data structure amended to match the outcome

* Bitstamp:error and a test for empty pair added in GetAccountTradingFee

* Bitstamp: RM the whitespace linter error
2023-10-13 15:34:21 +11:00
Gareth Kirwan
d86da76b1f Bitstamp: Add auth channel subscription handling (#1333)
* Bitstamp: Add auth channel subscription handling

* Bitstamp: Avoid searching for asset type

We've hardcoded asset.Spot in order to find the pair.
Looking the asset up from the pair makes no sense.

* Bitstamp: Add type for wsOrders

* Bitstamp: Working test of Generic DataHandler

* Bitstamp: WS Order chan tests and remove type

orderType could be derived from status == New and Buy & price == 9+e9 or Sell & price == 0
But it would only be true for the first update and it really doesn't feel worth the risk
Consumers are going to have to merge to original request anyway

* Bitstamp: Linter fixes

* Kraken: Switch to shared fixture test

* Bitstamp: Fix lint on TestFixtureToDataHandler

* Engine: Add Clone for PairsManager

go-vet highlighted that the mutex here is a value when we copied the
PairsManager in a test.
Options to fix:
* Add a deep clone method to PairsManager
* Add a shallow clone method with a disclaimer
* Make the mutex a pointer
* Make the PairsManager itself a pointer

Options 3 and 4 are too invasive to justify changing at this point.
There's an inherent risk of PM being passed by value, but govet should
catch the copylock.
There's more risk in changing everything to use a pointer at this stage.

* Engine: Fix linter again, ironically

* Bitstamp: Rename OHLC const

* Bitstamp: Minor fixes to syntax

* Bitstamp: Simplify chanSymb=>pair

* Bitstamp: Still process order updates without ID

If there's a ClientOrderID we'll still process the order.
It doesn't seem likely we'd have this happen, but if it does we still
want consumers to get something.

* Bitstamp: Replace Clone with Lock methods

* Engine: Expose PairsManager's Mutex

Makes more sense than wrapping functions

* Bitstamp: Fix linter copylock (again)

* fixup! Engine: Expose PairsManager's Mutex

Omit Mutex from Json

* fixup! Bitstamp: Add auth channel subscription handling

Remove unused wsAuthToken

* Bitstamp: Simplify OrderData Unmarshal

* Bitstamp: Remove unused contexts

I added these following best practices, but the reality is that when/if
we get context awareness in GCT, there will be a lot more to fix and
this will be a drop in the ocean anyway.

* Bitstamp: Only call handleWSOrder for  MyOrders

* Bitstamp: Avoid allocating again in handleWSOrder

* CurrencyPairs: Remove public mutex

Simplified to a Load method to avoid making mutex public

* Tests: Improve test readability and clarity

* Bitstamp: Wrap errWSPairParsingError

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Bitstamp: FetchWSAuth mock and live test

---------

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2023-10-05 16:13:05 +11:00
Ryan O'Hara-Reid
ecf0a5bcb3 spellcheck: fix and update (#1359)
* spellcheck: fix and update

* spellcheck: add exclude line for gateio_types
2023-10-03 16:07:03 +11:00
Scott
7f0faf7850 futures: add GetFuturesContractDetails wrapper function (#1274)
* all in a days work

* cleanup

* cleanup for real, also stop it binance.json

* minor coverage

* adds gateio to the slurry

* cleanup of types

* verbose verbose verbose verbose verbose verbose

* fixes huobi parsing issue

* fix bybit contract identification

* cleanup

* merge fixes

* addresses many big problems raised by SHAZ

* tracking errors and fixes

* funding rate if avail, fixes currency formatting

* Addresses nits and sneaks in extra fixes

* lint

* minor fixes after rebase

* better contract splitter for currencies like T-USDT

* forgot to add the exchange name like a fool

* merge fixes x1

* kucoin, direction, contract size

* rn direction, fix kucoin time

* WHOOPS

* Update exchanges/kucoin/kucoin_wrapper.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* misdirection

---------

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
2023-10-03 16:04:45 +11:00
Ryan O'Hara-Reid
1adbc99526 exchanges: improves GetDefaultConfig method (#1245)
* exchanges: Add function to get standard config

* exchanges: add tests (cherry-pick here and above)

* after pick stuff

* cleanup

---------

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
2023-09-18 11:45:02 +10:00
Gareth Kirwan
ec312ca0d3 Bitstamp: Add websocket heartbeat (#1329)
This prevents the frequent (5x / hour) disconn/reconns we're seeing with a
10s or even 20s traffic timeout.
I'd like to base the interval on the traffic timeout / 2, but that's
non-trivial right now, and 8s isn't an excessive default
2023-08-29 09:22:50 +10:00
Gareth Kirwan
4cfa6c4396 Bitstamp: Fix websocket orderbook snapshot blanking LastUpdate (#1319)
* Bitstamp: Fix orderbook snapshot blanking LastUpdate

Drive-By: Remove:
`Bitstamp BTC/USDT spot orderbook has zero bid price, filtering.`
It's unactionable and happens every snapshot right now

* Bitstamp: Use ob timestamps for LastUpdate

Obviously.
2023-08-15 14:22:40 +10:00
Bea
26692669c6 exchanges/order: Improve order side handling in SubmitOrder (#1299)
* SubmitOrder- order.Buy replaced with IsLong

Progresses #1270

* Btcmarkets: order.Buy and order.Sell changed to order.Bid and order.Ask

* Bithumb: SubmitOrder- order.Buy replaced with IsLong

Progresses #1270

* Binanceus: SubmitOrder- order.Buy replaced with IsLong

Progresses #1720

* Bistamp: SubmitOrder- order.Buy replaced with IsLong

Progresses #1270

* Bitfinex: Changed order Side to IsLong

* Binance:switch added to SubmitOrder

* Binance: Replaced Buy and Sell in SubmitOrder with IsLong and IsShort

* Binance:SubmitOrder if statement for order side simplified
2023-08-11 14:05:43 +10:00
Gareth Kirwan
d262166424 Bitstamp: Fix subscription_succeeded not handled (#1298) 2023-08-04 13:01:37 +10:00
Bea
123ec8c73d Bitstamp: Implement UpdateOrderExecutionLimits and testing (#1264)
* Bitstamp: OrderExecutionLimits and testing

* Bitstamp: Fix FetchTradablePairs Loading Limits

* Bitstamp: Limit for the step increment added, type convert replaced with regular expression

* Update exchanges/bitstamp/bitstamp_type_convert.go

Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>

* Bitstamp: Changed MinimumBaseAmount to MinimumQuoteAmount

* Bitstamp: changed riceStepIncrementSize and AmountStepIncrementSize to math.Pow(-value)

---------

Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>
2023-07-28 15:43:32 +10:00
Gareth Kirwan
4a8eedf3f7 Bitstamp: Fix USDT balance not loading (#1265)
* Bitstamp: Fix USDT balance not loading

* Bitstamp: Fix GetBalance losing Fees
2023-07-18 13:44:30 +10:00
Ryan O'Hara-Reid
da38b4bdf7 go.mod: bump go version to 1.20 from 1.17 (#1266)
* go.mod: bump to version 1.20

* go fix yourself

---------

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
2023-07-12 10:49:23 +10:00
Ryan O'Hara-Reid
81a8b4a575 common: remove SimpleTimeFormat const for time package layout const DateTime (#1246)
* switch over to package defined const for time layout

* bump appveyor playa

* bumperino to latest while setting patherino

* whoooops

* bump VS version set GOROOT

* puge build cache

* Revert "puge build cache"

This reverts commit 315bb578afc19529457f435e52af2172f5143bc5.

* bumperino to test

* purge setting of golang directory for version and allow default

* purge cache state when file change

* whoops

* thrasher: nits

* don't need to flusherino the cacherino

---------

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
2023-07-06 11:17:43 +10:00
Scott
fcc5ad4551 exchanges/qa: Add exchange wrapper testing suite (#1159)
* initial concept of a nice validation tester for exchanges

* adds some datahandler design

* expand testing

* more tests and fixes

* minor end of day fix for bithumb

* fixes implementation issues

* more test coverage and improvements, but not sure if i should continue

* fix more wrapper implementations

* adds error type, more fixes

* changes signature, fixes implementations

* fixes more wrapper implementations

* one more bit

* more cleanup

* WOW things work?

* lintle 1/1337

* mini bump

* fixes all linting

* neaten

* GetOrderInfo+ asset pair fixes+improvements

* adds new websocket test

* expand ws testing

* fix bug, expand tests, improve implementation

* code coverage of a lot of new codes

* fixes everything

* reverts accidental changes

* minor fixes from reviewing code

* removes Bitfinex cancelBatchOrder implementation

* fixes dumb baby typo for babies

* mini nit fixes

* so many nits to address

* addresses all the nits

* Titlecase

* switcheroo

* removes websocket testing for now

* fix appveyor, minor test fix

* fixes typo, re-kindles killed kode

* skip binance wrapper tests when running CI

* expired context, huobi okx fixes

* kodespull

* fix ordering

* time fix because why not

* fix exmo, others

* hopefully this fixes all of my life's problems

* last thing today

* huobi, more like hypotrophy

* golangci-lint, more like mypooroldknee-splint

* fix huobi times by removing them

* should fix okx currency issues

* blocks the application

* adds last little contingency for pairs

* addresses most nits and new problems

* lovely fixed before seeing why okx sucks

* fixes issues with okx websocket

* the classic receieieivaier

* lintle

* adds test and fixes existing tests

* expands error handling messages during setup

* fixes dumb okx bugs introduced

* quick fix for lint and exmo

* fixes nixes

* fix exmo deposit issue

* lint

* fixes issue with extra asset runs missing

* fix surprise race

* all the lint and merge fixes

* fixes surprise bugs in OKx

* fixes issues with times and chains

* fixing all the merge stuff

* merge fix

* rm logs and a panic potential

* lovely lint lament

* an easy demonstration of scenario, but not of initial purpose

* put it in the bin

* Revert "put it in the bin"

This reverts commit 15c6490f713233d43f10957367fcbf18e3818bdd.

* re-add after immediate error popup

* fix mini poor test design

* okx okay

* merge fixes

* fixes issues discovered in lovely test

* I FORGOT TO COMMIT THIS

* nit fixaroonaboo

* forgoetten test fix

* revert old okx asset intrument work

* fixes

* revert problems I didnt understand. update bybit

* fix merge bugs

* test cleanup

* further improvements

* reshuffle and lint

* rm redundant CI_TEST by rm the CI_TEST field that is redundant

* path fix

* move to its own section, dont run on 32 bit + appveyor

* lint

* fix lbank

* address nits

* let it rip

* fix failing test time range

* niteroo boogaloo

* mod tidy, use common.SimpleTimeFormat
2023-07-03 11:09:43 +10:00
Ryan O'Hara-Reid
b20cf75d13 sharedtestvalues: Add helper functions and implement throughout exchange tests (#1163)
* exchanges/sharedtestvalues: implement new functions to handle test skipping and announcements for standardising.

* exchanges: fin test impl.

* linter: fixes

* exchange_template: fix test

* allocate so it doesn't make a panic at the disco

* glorious: nits

* glorious: nits

* Update exchanges/sharedtestvalues/sharedtestvalues.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/sharedtestvalues/sharedtestvalues.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* linter: fix

* linter: shhhh

---------

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2023-04-28 13:05:42 +10:00
Samuael A
969669cb0d linters: Bump golangci-lint version to v1.52.2 and fix issues (#1169)
* fix linter issues

* linter issue suppressing and fixing

* change linter version and linter issues fix

* Bump version
2023-04-27 10:13:46 +10:00
Ryan O'Hara-Reid
42475bf2b8 exchanges: add setTimeWindow boolean to GetKlineRequest param (#1160)
* exchanges: add setTimeWindow boolean to GetKlineRequest params to differentiate between a set time period return from endpoint.

* glorious: nits

* exchange: conjugation

* Update exchanges/exchange.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits and an assortment of differences

* exchanges: remove some comments

* glorious: nits

* cleanup

* tests: fix

* Update exchanges/hitbtc/hitbtc_wrapper.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/kline/kline.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/kline/kline_test.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* kline: fix test

* rm unused variables

* almost: nits

* glorious: nits

* linter: fix

* rm unused variable

* Refactored comment in the okex tests to ensure that it accurately reflects the variable name and the issue related to the time window, as requested by GloriousCode. The previous comment did not align with the identifier assigned to the property, which could cause confusion and misunderstanding among other programmers or stakeholders. The updated comment will improve the clarity and readability of the codebase and make it easier to understand the intended purpose of the associated variables. The change was made with the aim of improving the overall quality and maintainability of the code.

---------

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2023-04-27 10:10:19 +10:00
Ryan O'Hara-Reid
7cadc5cffb cmd/wrapper_coverage: prevent non-needed REST requests via context timeout (#1154)
* wrapper_coverage/exchanges: cancel context to not send rest requests/ populate context through functions that do rest requests

* linter: fix

* exchange_template: fix test

---------

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
2023-04-02 11:20:25 +10:00
Ryan O'Hara-Reid
d2561402c4 common: update Errors type (#1129)
* common: adjust common error slice to allow multi errors.Is matching and conform to interface better

* zb: forgot to save?

* linties: fixies

* linties: word change as well.

* nitters: glorious

* buts

* nitters: fix glorious bug

* Update common/common.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* nitters: shifty

---------

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2023-02-20 10:48:24 +11:00
Adrian Gallagher
7e08e483fb CI: Bump go version, linters and fix minor issues (#1130)
* CI: Bump go version, linters and fix minor issues

* Bump version, fix loop variables

* Revert

* Rid TODOs now that 1.51 has been released
2023-02-03 15:56:59 +11:00
Donald Adu-Poku
82c79a9287 stream: set connection monitor delay. (#1120)
* stream: set connection monitor delay.

- this fixes a bug where the connection monitor delay config value does not
get set to the websocket on intialization.

* multi: add connection monitor delay to exchange config.

- this adds the connection monitor delay config option to the exchange type.
- the validate function of the exchange type has been updated to validate the
connection monitor delay value as well.

* multi: resolve review issues.
2023-01-31 09:21:51 +11:00
Adrian Gallagher
a79e0d2b3e qa/spelling: Add Codespell support (#1121)
* Add codespell support

* Fix paths

* Add HTML files to exclusion list
2023-01-30 12:36:56 +11:00
Scott
03a24b3ab1 Backtester: custom interval support (#1115)
* add backtester support

* Prevent live data custom candles, prevent nanosecond candles

* test coverage

* a more interesting rsi strategy result

* actual custom candle and proper strat date

* add test to old funk

* typos 🌞 🌞

* this was definitely worth failing linting for

* Adds stricter processing and adapts to it

* now compat with partial and absent candles

* test fixes, zb fixes

* fix more introduced bugeroos

* fix more introduced bugeroosx2

* linting for one space is so annoying

* addresseroos niteroos

* Update backtester/engine/setup.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
2023-01-24 16:05:46 +11:00
Ryan O'Hara-Reid
83cfefa45c kline/exchanges: automatic creation of unsupported candle intervals (#1091)
* kline: Add builder and testing

* Ideas

* kline: deploy builder functionality across GCT

* exchanges: implement across gct

* exchanges: Add tests and fix implementations before kline package testing and veri.

* kline: Add tests and start to fix ConvertToNewInterval

* kline: fix ConvertToNewInterval add tests

* kline: complete overarching tests now on to exchanges

* kline: finish exchange tests and implement limits

* exchanges: more fixes

* linter: fix

* engine: fix tests

* kraken: fix recent trades and other fixes

* zb: fix tests

* bithumb: fix empty insertion

* kline: refactor/optimize CreateKline function

* kline: remove the mooos!

* kline: prealloc CalculateCandleDateRanges

* linter: fix

* exchanges: prealloc extended

* fix whoopsie

* reverse fix because this is a whoopsie

* okx: fix risidual issues

* linter: fix

* kline: initial nits from @gloriouscode

* kline: rename builder -> request and cascade change

* linter: fix + test

* kline: update forced alignment on start and end times when CreateKlineRequest is called.

* nits: more more more

* NITS: Addressed

* tests: fix race issue

* Update exchanges/kline/request.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* kline: add method AddPadding() to automatically fill in holes in kline.Request functionality and reject if missing data when converting

* kline: Add params start and end to addPadding() to insert blanks in between block

* kline: remove test comment code as it's not needed anymore

* kline: fix lint and test

* kline: sort slice without extra bool check every iteration

* okx: fix issues with timeing and candles and such from niterinos & address typo

* Update exchanges/kline/kline.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: niterinos

* Update exchanges/poloniex/poloniex_wrapper.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits now onto conflicts YAYA!!!

* Update exchanges/exchange_test.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits again

* thrasher: nitters

* thrasher: niterinos - adds partial flag for incomplete recent candles and fetching.

* kline: rm fmtizzle packageizzle

* glorious: nitters

* glorious: more niterinos

* fix last niterinos

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2023-01-17 16:22:33 +11:00
Scott
017cdf1384 Backtester: Live trading upgrades (#1023)
* Modifications for a smoother live run

* Fixes data appending

* Successfully allows multi-currency live trading. Adds multiple currencies to live DCA strategy

* Attempting to get cash and carry working

* Poor attempts at sorting out data and appending it properly with USD in mind

* =designs new live data handler

* Updates cash and carry strat to work

* adds test coverage. begins closeallpositions function

* Updates cash and carry to work live

* New kline.Event type. Cancels orders on close. Rn types

* =Fixes USD funding issue

* =fixes tests

* fixes tests AGAIN

* adds coverage to close all orders

* crummy tests, should override

* more tests

* more tests

* more coverage

* removes scourge of currency.Pair maps. More tests

* missed currency stuff

* Fixes USD data issue & collateral issue. Needs to close ALL orders

* Now triggers updates on the very first data entry

* All my problems are solved now????

* fixes tests, extends coverage

* there is some really funky candle stuff going on

* my brain is melting

* better shutdown management, fixes freezing bug

* fixes data duplication issues, adds retries to requests

* reduces logging, adds verbose options

* expands coverage over all new functionality

* fixes fun bug from curr == curr to curr.Equal(curr)

* fixes setup issues and tests

* starts adding external wallet amounts for funding

* more setup for assets

* setup live fund calcs and placing orders

* successfully performs automated cash and carry

* merge fixes

* funding properly set at all times

* fixes some bugs, need to address currencystatistics still

* adds 'appeneded' trait, attempts to fix some stats

* fixes stat bugs, adds cool new fetchfees feature

* fixes terrible processing bugs

* tightens realorder stats, sadly loses some live stats

* this actually sets everything correctly for bothcd ..cd ..cd ..cd ..cd ..!

* fix tests

* coverage

* beautiful new test coverage

* docs

* adds new fee getter delayer

* commits from the correct directory

* Lint

* adds verbose to fund manager

* Fix bug in t2b2 strat. Update dca live config. Docs

* go mod tidy

* update buf

* buf + test improvement

* Post merge fixes

* fixes surprise offset bug

* fix sizing restrictions for cash and carry

* fix server lints

* merge fixes

* test fixesss

* lintle fixles

* slowloris

* rn run to task, bug fixes, close all on close

* rpc lint and fixes

* bugfix: order manager not processing orders properly

* somewhat addresses nits

* absolutely broken end of day commit

* absolutely massive knockon effects from nits

* massive knockon effects continue

* fixes things

* address remaining nits

* jk now fixes things

* addresses the easier nits

* more nit fixers

* more niterinos addressederinos

* refactors holdings and does some nits

* so buf

* addresses some nits, fixes holdings bugs

* cleanup

* attempts to fix alert chans to prevent many chans waiting?

* terrible code, will revert

* to be reviewed in detail tomorrow

* Fixes up channel system

* smashes those nits

* fixes extra candles, fixes collateral bug, tests

* fixes data races, introduces reflection

* more checks n tests

* Fixes cash and carry issues. Fixes more cool bugs

* fixes ~typer~ typo

* replace spot strats from ftx to binance

* fixes all the tests I just destroyed

* removes example path, rm verbose

* 1) what 2) removes FTX references from the Backtester

* renamed, non-working strategies

* Removes FTX references almost as fast as sbf removes funds

* regen docs, add contrib names,sort contrib names

* fixes merge renamings

* Addresses nits. Fixes setting API credentials. Fixes Binance limit retrieval

* Fixes live order bugs with real orders and without

* Apply suggestions from code review

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update backtester/engine/live.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update backtester/engine/live.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update backtester/config/strategyconfigbuilder/main.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* updates docs

* even better docs

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
2023-01-05 13:03:17 +11:00
Ryan O'Hara-Reid
e4d487e586 exchanges: Change wrapper function FetchTradablePairs to return currency.Pairs (#1093)
* exchanges: FetchTradablePairs currency pair slice return (1)

* exchanges: finish conversion

* thrasher: nits + linter

* exchanges: shift var dec into for loop

* exchanges: Apply changes thanks @thrasher-

* Update exchanges/alphapoint/alphapoint_wrapper.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* thrasher: nitters

* Update exchanges/bitflyer/bitflyer_wrapper.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* thrasher: fix more nitters

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
2022-12-09 15:40:34 +11:00
Ryan O'Hara-Reid
558a70e87f orders: add Filter() method to GetOrdersRequest and force FilteredOrders return type on wrapper functions (#1055)
* orders: filter options return on validate

* exchanges: force filter usage by wrapper return type and shift method functionality

* exchanges: update tests and fix err shadow

* exchanges: shadowland

* exchanges: more shadow land spookyness

* glorious: nits and ftx upgrade

* linter: fix

* orders: preserve unpopulated fields through filtering operation

* glorious: nits

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
2022-10-24 13:28:32 +11:00
Ryan O'Hara-Reid
f843b7d277 exchange: upgrade UpdatePair method (#991)
* exchange: upgrade update pair

* exchanges: Add enabled string matching and format handling if discrepency is found.

* linter: fixes

* bithumb: fix tests

* BTSE: api change fix ordering

* huobi: fix tests

* gloriousnits: stage 1

* gloriousnits: stage 2

* currency: more nits

* bitmex: add spot and process pairs before currency package call.

* bitmex: finished correct orderbook matching and other implementations

* linter: fix issue

* currency: Fix linter

* currency: segregate and protect pair store, update tests

* currency/manager: clean code, rm log output

* currency: Add store method and make sure formatting stays nil if not stored.

* gct: check errors

* engine/websocketroutineman: fix tests

* bybit: fix duplication bug

* huobi: fix test

* btse: fix tests?

* ob/buffer: fix tests

* Update currency/manager.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* glorious: nits strikes again.

* exchange: add bypassConfigFormatUpgrades to stop formatting

* GLORIOUS LINTER

* Update exchanges/bithumb/bithumb_wrapper.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* exchange: fix pair upgrade issue when duplications are in both avail and enabled pairs

* linter: fix shadow dec

* config: fix test

* Update currency/pair_test.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2022-09-16 08:59:27 +10:00
Adrian Gallagher
68588560e3 CI: Bump go version, linters and fix minor issues (#1010)
* Bump golang, golangci-lint versions and fix issues

* Add -fno-stack-protector

* Fix AppVeyor golangci-lint ver

* Nitters

* Nitters round 2
2022-08-17 11:37:22 +10:00
Jaydeep Rajpurohit
247da918a8 exchanges: Add ByBit support (#887)
* few fixes and add ratelimiter

* adds test

* revert configtest.json changes

* configtest updated

* WIP: adds public endpoint support

* WIP: adds public endpoint support

* adds public endpoint support

* WIP: adds auth. endpoint support

* adds test for auth. endpoint

* fixes

* adds auth. endpoint support

* WIP: ws support

* WIP

* WIP

* WIP

* WIP

* WIP

* WIP

* WIP

* Testing

* Complete WS spot testing

* adds support for ws events

* minor change

* WIP: adds REST support for CoinMarginedFutures

* Fixes

* WIP: adds REST support for CoinMarginedFutures

* Fixes

* improvement in SPOT REST

* Typo fix

* WIP: add REST support for CMF Account API

* minor fixes

* WIP: add support for CMF conditional orders and few minor fixes

* complete support for CMF conditional orders

* adds support for public CMF endpoint

* adds support for CMF position API

* Complete REST CMF support

* WIP

* Testing REST CMF support

* Testing REST CMF support

* Testing REST CMF support completed

* WIP: add support for UMF

* completed non-auth UMF

* WIP: add support for REST Auth. UMF

* WIP: add support for REST Auth. UMF and some improvements

* WIP

* WIP

* WIP

* completed REST UMF

* renaming

* adds REST support for futures

* add testcases for UMF and some optimizations

* add testcases for futures

* Testing UMF, futures and its changes

* Fixes

* Fixes after testing

* WIP

* WIP

* WIP

* completed ws USDT futures support

* WIP: ws support for futures

* fixes in WS futures

* fixes in WS support

* roll back changes made for WS CMF, USDT and Futures

* fixes

* WIP

* WIP

* fixes

* Steps for new PR

* WIP

* WIP

* WIP

* WIP

* complete PR setup

* fixes for successfully running tests

* update in symbol for futures pair in test file

* WIP

* Fixes in test file and other minor fix

* fix testdata/configtest.json

* reset CONTRIBUTORS file

* review changes

* remove unwanted file

* remove redundant code

* improvisation

* adds comment for exported functions

* remove unwanted TODO and commented code

* fix

* improvisation

* fix

* defined errors

* improvisation

* improvisation

* improvisation

* updates test

* adds comment for exported types

* review changes

* review changes

* fix

* fixes

* Changes for making BYBIT compatible with existing code base

* Test file changes

* Changes for making BYBIT compatible with existing code base

* Changes for making BYBIT compatible with existing code base

* fix lint issues

* fix

* review changes

* review changes

* review changes

* review changes

* review changes

* review changes

* review changes

* review changes

* review changes

* review changes

* WIP

* add test cases for new API's

* minor improvements

* add missing API and their tests

* minor fixes

* add bybitTime

* add bybitTimeSec, bybitTimeMilliSec, bybitTimeNanoSec and necessary support

* fix GetTradeHistory function

* error handling

* test fixes

* add GetServerTime API

* adds GetHistoricCandlesExtended and review changes

* test fixes

* minor fix

* integrating CMF Bybit recent change log

* minor fixes

* adds extractCurrencyPair

* minor fixes

* minor fix

* review changes

* adds variable declaration of error

* review commit

* adds embeddable type in API response for all API and integrate it

* fixes

* adds authentication WS connection

* review changes

* review changes

* compatible changes

* adds asset to GetWithdrawalsHistory

* adds asset_type in rpc.proto

* adds asset argument in gctcli withdrawal request command

* improve error handling in exchange API error

* web socket fix

* review changes

* improvements

* improvements

* minor fix

* review changes

* fixing wrapper issues

* fixes

* fixes

* review changes

* add test cases

* fix for GetActiveOrders

* lint fixes

* fixes in websocket

* adds wrapper testcases

* adds wrapper testcases

* adds wrapper testcases

* fixes

* fix issue with GetHistoricCandlesExtended

* fix merge issues

* improving error reporting

* adds wrapper testcases and a minor fix

* gctrpc changes

* adds test cases
fixes in websocket

* review changes for ws

* review changes in WS

* fix gctrpc

* merge fixes

* review changes

* WIP

* updates pair in configs

* adds new asset USDCMarginedFutures

* adds URL const for USDCMarginedFutures

* adds API support

* minor fixes

* adds kline API

* minor fix

* adds API

* adds API

* adds API

* WIP

* WIP

* WIP

* adds support for USDC auth requests to SendAuthHTTPRequest

* adds SendUSDCAuthHTTPRequest

* run test and fix them

* rollback support added for Auth. USDC request inside SendAuthHTTPRequest

* adds API and test cases

* adds API and test cases

* adds APIs and test cases

* adds APIs

* adds rate limit for USDC

* adds USDCMarginedFutures to wrapper

* adds USDC testcases in wrapper and fix few issues

* minor test fixes

* minor test fixes

* fix lint issues

* WIP

* Merge changes

* minor fixes

* remove "else" and optimize

* review changes

* review changes

* review changes

* fix lint issue

* merge fix

* fix test

* fix templates and run them

* changes after merge

* review changes and improvements

* code improvement

* fixes with respect to changes in API response in documentation

* fixed review change in test

* adds check in CancelExistingOrder

* update exchange template

* review changes

* adds GetDepositAddress API

* WIP: adds GetOrderHistory

* complete GetOrderHistory

* fixes

* adds test case

* fixes and add WithdrawFund API

* WIP

* WIP

* updating all SendAuthHTTPRequest call

* adds WithdrawCryptocurrencyFunds

* update test cases

* fix lint issues

* fixes after merge

* adds GetAvailableTransferChains and few fixes

* minor fix in GetDepositAddress

* minor fix with WS ping/pong handling

* add ping handler for WS Auth.

* fix typo mistake

* update doc
2022-08-08 11:29:43 +10:00
Ryan O'Hara-Reid
663e753f52 account: segregate holdings by credentials for future multi-key management (#956)
* exchanges/account: shift credentials to account package and segregate funds to keys

* merge: fixes

* linter: fix

* Update exchanges/account/account.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits + protection for string panic

* glorious_suggestion: add method for matching keys

* linter: fix tests

* account: add protected method for credentials minimizing access, display full account details to rpc.

* linter: spelling kweeeeeeen

* accounts/portfolio: clean/check portfolio code and quickly check balances from change. Add protected method for future matching.

* accounts: theres no point in pointerising everything

* linter: ok pointerise this then...

* exchanges: fix regression add in little notes.

* glorious: nits

* Update exchanges/account/credentials.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/account/credentials_test.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/account/credentials_test.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* gloriously: fix glorious glorious test gloriously

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2022-07-21 15:05:31 +10:00
Ryan O'Hara-Reid
a12cd3d733 orders: Add methods to derive SubmitResponse and Detail types (#955)
* orders: deprecate SubmitResponse return and change to *order.Detail construct detail from order.Submit struct

* orders: add coverage, fix tests

* coinut: rm test for checking

* orders: revert change for return and change field ID to a more explicit name OrderID

* orders: Add method to see if the order was placed

* order: change field name in Cancel type to be more explicit

* orders: standardize field -> OrderID

* backtester: populate change

* orders: add test

* gctscript: fix field name

* linter: fix issues

* linter: more fixes

* linter: forever

* exchanges_tests: populate order.Submit field exchange name

* Update exchanges/order/order_types.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/order/orders.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* glorious: nits

* thrasher: nits

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2022-06-06 11:52:15 +10:00
Ryan O'Hara-Reid
a63aa6b616 orders: Add derive modify struct method from order.Detail (#948)
* orders: Add derive modify struct method to order.Detail and then subsequent method to derive and standardize response details

* exchanges: call modify method in wrappers

* linter: fixes

* engine/wsroutineman: remove print summary

* glorious: nits, removed modifyOrder functionality for Bithumb. There are not docs to support this.

* Update exchanges/order/orders.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2022-05-30 14:38:21 +10:00
Ryan O'Hara-Reid
16a93b49a4 btcm: add modify order functionality, change return to pointer (#940)
* btcm: add modify order functionality, change return to pointer

* glorious: nits

* glorious: nits

* btcm: Adjust function name

* thrasher: nits

* thrasher: nits cont...
2022-05-16 10:55:23 +10:00
Ryan O'Hara-Reid
cdcc9630de order: slight optimizations (#917)
* order: slight optimizations

* orders: add benchmarks, small optimize and change order side to uin8 for comparitive optimizations.

* orders: continue to convert string type -> uint

* orders/backtester: interim move type to orders package, later can expand or deprecate.

* orders: handle errors

* orders: optimize filters and remove error returns when its clearly not needed

* orders: remove log call

* backtester: zero value check

* orders/futures: zero value -> flag

* linter: fix

* linter: more fixes

* linters: rides again

* glorious: nits

* common: Add zero value unix check for time values; also addresses glorious nits

* glorious scott: nits

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
2022-05-06 12:27:21 +10:00
Ryan O'Hara-Reid
6a02097431 account: storage, processing and method on balances update (#916)
* account: update account storage, retrieval and implement alert functionality when a currency change occurs.

* account: Add cancel channel

* account: remove old code

* account: don't embed mutex

* Update exchanges/account/account.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/account/account.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/account/account.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* account: addr nits

* account: Pull out test into indiv.

* account: Add test for update method

* account: add no change to test

* Update exchanges/account/account.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* account/portfolio: differentiate between asset type segregation and default to spot holdings.

* glorious: nits

* thrasher: nit

* ticker: fix spelling

* Update engine/portfolio_manager.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* thrasher: nits

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
2022-05-02 14:01:03 +10:00
Adrian Gallagher
9a4eb9de84 CI: Fix golangci-lint linter issues, add prealloc linter and bump version depends for Go 1.18 (#915)
* Bump CI versions

* Specifically set go version as 1.17.x bumps it to 1.18

* Another

* Adjust AppVeyor

* Part 1 of linter issues

* Part 2

* Fix various linters and improvements

* Part 3

* Finishing touches

* Tests and EqualFold

* Fix nitterinos plus bonus requester jobs bump for exchanges with large number of tests

* Fix nitterinos and bump golangci-lint timeout for AppVeyor

* Address nits, ensure all books are returned on err due to syncer regression

* Fix the wiggins

* Fix duplication

* Fix nitterinos
2022-04-20 13:45:15 +10:00
Ryan O'Hara-Reid
09fa2f236a context: Add authenticated HTTP credentials (#892)
* gRPC: context overide

* exchanges: continue update

* exchange: Update context handling
*Add setter methods for API credentials
*Shift credentials functionality to its own file in exchanges package
*Add tests
*Refactor function DeployCredentialsToContext for library usage
*Add function to process credential metadata from API boundary to internal use context value.
*Add OTP rpc handling

* exchanges: reverts to old style in GetFeeByType, reverts some code I accidently deleted. Plus things and other. XD

* template: update

* exchanges: fix linter issues

* REMOVE THAT AWESOME NEW LINE!

* gct: fix some tests

* I cant spell :(

* exchanges/gctscript: fix more tests

* coinnut: fix tests

* Update exchanges/credentials.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/credentials.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/credentials.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/credentials.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/credentials.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* exchanges/gctcli: stop applying empty credentials

* fix linters

* exchanges: add test

* rpceserver: actually check error for errors

* rpcserver: fix up tests

* Update exchanges/credentials.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* exchanges/creds: move tests to corresponding files, add protection and segration for Credentials struct & ptr values

* exchanges/creds: allow subaccount to override default credentials via gRPC

* exchanges/credentials: don't return nil in GetCredentials

* creds: spelling

* exchanges: fix glorious NITS!

* credentials: Add in test and refactor IsEmpty method.

* credentials: change type positioning (glorious)

* exchange_template: Fix template changes

* DOCS: Refresh

* docs: fix spelling

* DOCS: fix alignment and add package

* DOCS: ALIGN!

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2022-03-21 13:58:08 +11:00
Scott
86d3724507 Futures order position tracking & FTX scaled collateral calculation (#868)
* implements futures functions and GRPC functions on new branch

* lint and test fixes

* Fix uneven split pnl. Adds collateral weight test. docs. New clear func

* Test protection if someone has zero collateral

* Uses string instead of double for accuracy

* Fixes old code panic

* context, match, docs

* Addresses Shazniterinos, var names, expanded tests

* Returns subaccount name, provides USD values when offlinecalc

* Fixes oopsie

* Fixes cool bug which allowed made up subaccount results

* Subaccount override on FTX, subaccount results for collateral

* Strenghten collateral account info checks. Improve FTX test

* English is my first language

* Fixes oopsies

* Fixes for unrealised PNL & collateral rendering

* Fixes lint and tests

* Shaznit fixes

* Secret Shaznit

* Updates account information across wrappers to include more fields

* Updates online collateral calculations. Updates RPC data

* Accurately calculates collateral offline and online minus testing

* Tests and lint chocolate

* Simplifies accountinfo results

* Fixes shaznits

* Adds new func

* Increases collateral accuracy again again again x 200

* Increases accuracy of collateral rendering

* Fixes minor merge/test issues

* Linterino

* Fixes ws test. Improves collateral calculations and rendering

* Make it prettier

* Removes the lock I put on 👀

* Adds `additional_collateral_used` field, renders orig currency

* Fixes unrelated test

* Fix test

* Correctly calculate spot margin borrow collateral

* Address fun lint surprise

See https://github.com/golangci/golangci-lint/issues/741#issuecomment-1017014331

* Strange lint fixing x2

* Continued lint journey

* Nolint the nolint to not lint the lint

* Adds two new fields to response

* More linting issues arising

* fIX3s_c4s|NG

* Fixes command flags' incorrect numbering

* FairMarket = Won
2022-02-28 16:39:36 +11:00
Ryan O'Hara-Reid
6127e2ab73 Request: Fix http.Client race issue when setting transport layer proxy and timeouts (#885)
* backtester/request: trying to fix panic (WIP)

* request: fix race for transport layer

* request: linter issue fix

* request: more linter issues

* requester: Add function to remove the tracking of underlying http client and add to engine unload exchange.

* request: add more context to error return

* request: Fix after cherry pick issues

* request: fix niterinos

* exchanges: change return to package variable

* request: changed named

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
2022-02-18 09:22:10 +11:00
Ryan O'Hara-Reid
da3402476e tool/exchange_wrapper_coverage: fix regression and implement reflection (#837)
* cmd/tools/exchange: fix regression and implement reflection so as this can dynamically scale to our interface

* exchanges: add comment and fix whoopsie

* exchanges: fix linter issues

* wrapper_cov_tool: add actual full interface method count to get a better perceived deployment

* exchanges/tool: addr glorious nits

* kraken: remove string in test

* exchange_template_tool: fix tmpl issue
2021-11-17 15:41:21 +11:00
Scott
7c7aebe22f Binance: Fixes expiring futures contract pair formatting (#836)
* Fixes expiring contract formatting

* minor simplification

* shazniterinos

* Removes secret scientific test

* continue to address nits

* Shazniterinos

* Adds `bypassConfigFormatUpgrades` feature. Fixes Binance pair parsing.

* lint fix

* Adds test for binance func

* Consistent messaging and oopsie fix
2021-11-16 15:02:31 +11:00
Ryan O'Hara-Reid
099ffa1a60 stream/websocket: Consolidate fields by using exchange config pointer (#809)
* stream: add exchange config pointer to setup WebsocketSetup struct to reduce and consolidate setting of variables.

* config: reduce stutter

* config: reduce minor stutter

* glorious: nits addr.

* Update exchanges/stream/websocket.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* websocket: implement fix

* engine/helpers: fix test

* exchanges: fix after merge issues

* exchange_template: fix output

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2021-10-20 15:45:06 +11:00
Luis Rascão
a70224d123 exchanges/websocket: Allow configuration of orderbook publish period (#805)
* Allow configuration of orderbook publish period

For some applications that import GCT it's more interesting to be
immediately notified of an exchange orderbook update instead of
only getting notified every 10 seconds. This option allows that
to happen while keeping the previous default.

* exchanges: allow configuration of orderbook update period
2021-10-20 11:44:24 +11:00
Adrian Gallagher
0c00b7e1df exchanges/engine: Add multichain deposit/withdrawal support (#794)
* Add exchange multichain support

* Start tidying up

* Add multichain transfer support for Bitfinex and fix poloniex bug

* Add Coinbene multichain support

* Start adjusting the deposit address manager

* Fix deposit tests and further enhancements

* Cleanup

* Add bypass flag, expand tests plus error coverage for Huobi

Adjust helpers

* Address nitterinos

* BFX wd changes

* Address nitterinos

* Minor fixes rebasing on master

* Fix BFX acceptableMethods test

* Add some TO-DOs for 2 tests WRT races

* Fix acceptableMethods test round 2

* Address nitterinos
2021-10-15 15:55:38 +11:00
Ryan O'Hara-Reid
d636049fb2 exchanges: Initial context propagation (#744)
* gct: phase one context awareness pass

* exchanges: context propagation pass

* common/requester: force context requirement

* gctcli/exchanges: linter fix

* rpcserver: fix test using dummy rpc server

* backtester: fix comments

* grpc: add correct cancel and timeout for commands

* rpcserver_test: add comment on dummy server

* common: deprecated SendHTTPGetRequest

* linter: fix

* linter: turn on no context check

* apichecker: fix context linter issue

* binance: use param context

* common: remove checks as this gets executed before main

* common: change mutex to RW as clients can be used by multiple go routines.

* common: remove init and JIT default client. Unexport global variables and add protection.

* common: Add comments

* bithumb: after dinner mints fix
2021-09-11 13:52:07 +10:00
Ryan O'Hara-Reid
66fbd43cf0 websocket: fix deadlock when enabling/disabling via gctrpc (#754)
* websocket: select case error if no receiver, add in functionality to reset to initial sync for books on a new websocket connection

* websocket: fix tests

* websocket: log error instead of losing it

* websocket: fix whoopsie

* exchanges: fix test

* websocket: force requirement of specific functionality

* exchanges: fix tests

* exchanges/websocket: move waitgroup add before scheduling across exchanges

* gateio: add feature subscribe

* bithumb/bittrex: include connection state reset, fix reconnection bug for Bithumb

* huobi: Add listen to shutdown to routine so it actually returns and stops being a naughty boy.

* huobi: add missing waitgroup add.

* exchanges: bleed comms channels

* binance: fix reconnection bug with buffer

* bithumb: fix reconnection bug with ws orderbook when websocket is diabled/enabled

* bithumb/bittrex: add bleeders for ws orderbook jobs

* linter: fix

* kraken: reduce code block from double assertion

* This bug ruined my day.

* glorious: error checking

* zb: add correct path for websocket connection

* exchange: Add verbosity when config conflicts and overwrites default values

* zb: add https to path

* exchanges: glorious nits

* stream: Add checkAndSetMonitoring to reduce potential routine bundling, increase timeout and check state in tests

* stream: remove check that is not needed.

* glorious: nits addr.

* lint: test
2021-09-03 17:21:23 +10:00
Ryan O'Hara-Reid
8020e1ec6a linter: Enable error checking linter (#766)
* golangci: Enable err checking linter to expose unchecked errors.

* gct: handle errors across the board

* gct: handle errors NOTE: Found bug in FTX (WIP)

* linter: fix issues

* ftx/exchanges: fix bug where error was being returned when setting pair management variables to an already enabled state

* bitmex: fix bug where a dangly supported asset in config danglied up the place.

* linter: fix more linter issues

* linter: fix my terrible spelling.

* currency: fix test

* exchanges: fix tests

* logger: fix test

* exchanges: fix tests

* glorious: nits

* vm: revert rm variable and instigate test
2021-08-30 14:06:40 +10:00