Commit Graph

13 Commits

Author SHA1 Message Date
Ryan O'Hara-Reid
d2561402c4 common: update Errors type (#1129)
* common: adjust common error slice to allow multi errors.Is matching and conform to interface better

* zb: forgot to save?

* linties: fixies

* linties: word change as well.

* nitters: glorious

* buts

* nitters: fix glorious bug

* Update common/common.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* nitters: shifty

---------

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2023-02-20 10:48:24 +11:00
Ryan O'Hara-Reid
a12cd3d733 orders: Add methods to derive SubmitResponse and Detail types (#955)
* orders: deprecate SubmitResponse return and change to *order.Detail construct detail from order.Submit struct

* orders: add coverage, fix tests

* coinut: rm test for checking

* orders: revert change for return and change field ID to a more explicit name OrderID

* orders: Add method to see if the order was placed

* order: change field name in Cancel type to be more explicit

* orders: standardize field -> OrderID

* backtester: populate change

* orders: add test

* gctscript: fix field name

* linter: fix issues

* linter: more fixes

* linter: forever

* exchanges_tests: populate order.Submit field exchange name

* Update exchanges/order/order_types.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/order/orders.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* glorious: nits

* thrasher: nits

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2022-06-06 11:52:15 +10:00
Ryan O'Hara-Reid
a63aa6b616 orders: Add derive modify struct method from order.Detail (#948)
* orders: Add derive modify struct method to order.Detail and then subsequent method to derive and standardize response details

* exchanges: call modify method in wrappers

* linter: fixes

* engine/wsroutineman: remove print summary

* glorious: nits, removed modifyOrder functionality for Bithumb. There are not docs to support this.

* Update exchanges/order/orders.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2022-05-30 14:38:21 +10:00
Ryan O'Hara-Reid
14cde7b786 exchange: expose GetCredentials() and split GetAuthenticatedAPISupport() (#954)
* exchange/wrapper: expose GetCredentials func to IBotInterface

* exchanges: split up GetAuthenticatedAPISupport into specific function calls, organize IBotExchange functionality getter functions

* interface: change name - RPCSercer: rm GetBase func call.

* glorious: nits (fix panic)

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
2022-05-18 10:52:16 +10:00
Adrian Gallagher
9a4eb9de84 CI: Fix golangci-lint linter issues, add prealloc linter and bump version depends for Go 1.18 (#915)
* Bump CI versions

* Specifically set go version as 1.17.x bumps it to 1.18

* Another

* Adjust AppVeyor

* Part 1 of linter issues

* Part 2

* Fix various linters and improvements

* Part 3

* Finishing touches

* Tests and EqualFold

* Fix nitterinos plus bonus requester jobs bump for exchanges with large number of tests

* Fix nitterinos and bump golangci-lint timeout for AppVeyor

* Address nits, ensure all books are returned on err due to syncer regression

* Fix the wiggins

* Fix duplication

* Fix nitterinos
2022-04-20 13:45:15 +10:00
Ryan O'Hara-Reid
09fa2f236a context: Add authenticated HTTP credentials (#892)
* gRPC: context overide

* exchanges: continue update

* exchange: Update context handling
*Add setter methods for API credentials
*Shift credentials functionality to its own file in exchanges package
*Add tests
*Refactor function DeployCredentialsToContext for library usage
*Add function to process credential metadata from API boundary to internal use context value.
*Add OTP rpc handling

* exchanges: reverts to old style in GetFeeByType, reverts some code I accidently deleted. Plus things and other. XD

* template: update

* exchanges: fix linter issues

* REMOVE THAT AWESOME NEW LINE!

* gct: fix some tests

* I cant spell :(

* exchanges/gctscript: fix more tests

* coinnut: fix tests

* Update exchanges/credentials.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/credentials.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/credentials.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/credentials.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* Update exchanges/credentials.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* exchanges/gctcli: stop applying empty credentials

* fix linters

* exchanges: add test

* rpceserver: actually check error for errors

* rpcserver: fix up tests

* Update exchanges/credentials.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* exchanges/creds: move tests to corresponding files, add protection and segration for Credentials struct & ptr values

* exchanges/creds: allow subaccount to override default credentials via gRPC

* exchanges/credentials: don't return nil in GetCredentials

* creds: spelling

* exchanges: fix glorious NITS!

* credentials: Add in test and refactor IsEmpty method.

* credentials: change type positioning (glorious)

* exchange_template: Fix template changes

* DOCS: Refresh

* docs: fix spelling

* DOCS: fix alignment and add package

* DOCS: ALIGN!

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2022-03-21 13:58:08 +11:00
Adrian Gallagher
f0d45aa1d2 golangci-lint/CI: Bump versions and introduce new linters (#798)
* golangci-lint/CI: Bump versions

Fix remaining linter issues

* Specifically set AppVeyor version

* Fix the infamous typos 👀

* Add go env cmd to AppVeyor

* Add go version cmd to AppVeyor

* Specify AppVeyor image, adjust linters

* Update go get to go install due to deprecation

* Bump golangci-lint timeout time for AppVeyor

* Change NW contract to NQ

* Address nitters

* GetRandomPair -> Pair{}

* Address nits

* Address time nitterinos plus additional tweaks

* More time inception upgrades!

* Bending time and space
2021-10-14 16:38:53 +11:00
Ryan O'Hara-Reid
d636049fb2 exchanges: Initial context propagation (#744)
* gct: phase one context awareness pass

* exchanges: context propagation pass

* common/requester: force context requirement

* gctcli/exchanges: linter fix

* rpcserver: fix test using dummy rpc server

* backtester: fix comments

* grpc: add correct cancel and timeout for commands

* rpcserver_test: add comment on dummy server

* common: deprecated SendHTTPGetRequest

* linter: fix

* linter: turn on no context check

* apichecker: fix context linter issue

* binance: use param context

* common: remove checks as this gets executed before main

* common: change mutex to RW as clients can be used by multiple go routines.

* common: remove init and JIT default client. Unexport global variables and add protection.

* common: Add comments

* bithumb: after dinner mints fix
2021-09-11 13:52:07 +10:00
Ryan O'Hara-Reid
66fbd43cf0 websocket: fix deadlock when enabling/disabling via gctrpc (#754)
* websocket: select case error if no receiver, add in functionality to reset to initial sync for books on a new websocket connection

* websocket: fix tests

* websocket: log error instead of losing it

* websocket: fix whoopsie

* exchanges: fix test

* websocket: force requirement of specific functionality

* exchanges: fix tests

* exchanges/websocket: move waitgroup add before scheduling across exchanges

* gateio: add feature subscribe

* bithumb/bittrex: include connection state reset, fix reconnection bug for Bithumb

* huobi: Add listen to shutdown to routine so it actually returns and stops being a naughty boy.

* huobi: add missing waitgroup add.

* exchanges: bleed comms channels

* binance: fix reconnection bug with buffer

* bithumb: fix reconnection bug with ws orderbook when websocket is diabled/enabled

* bithumb/bittrex: add bleeders for ws orderbook jobs

* linter: fix

* kraken: reduce code block from double assertion

* This bug ruined my day.

* glorious: error checking

* zb: add correct path for websocket connection

* exchange: Add verbosity when config conflicts and overwrites default values

* zb: add https to path

* exchanges: glorious nits

* stream: Add checkAndSetMonitoring to reduce potential routine bundling, increase timeout and check state in tests

* stream: remove check that is not needed.

* glorious: nits addr.

* lint: test
2021-09-03 17:21:23 +10:00
Ryan O'Hara-Reid
8020e1ec6a linter: Enable error checking linter (#766)
* golangci: Enable err checking linter to expose unchecked errors.

* gct: handle errors across the board

* gct: handle errors NOTE: Found bug in FTX (WIP)

* linter: fix issues

* ftx/exchanges: fix bug where error was being returned when setting pair management variables to an already enabled state

* bitmex: fix bug where a dangly supported asset in config danglied up the place.

* linter: fix more linter issues

* linter: fix my terrible spelling.

* currency: fix test

* exchanges: fix tests

* logger: fix test

* exchanges: fix tests

* glorious: nits

* vm: revert rm variable and instigate test
2021-08-30 14:06:40 +10:00
Ryan O'Hara-Reid
279b53827f requester: defer execution of request.Item generation in closure (#723)
* requester: defer execution of request.Item generation in closure.

* bithumb: fix issue

* coinut/itbit: fix linter issues

* binance: fix bug on recvWindow setting

* requester: standardize sendpayload + add readme update

* nonce: remove inc() function

* request: defer unlockiflocked

* binance: revert changes for open orders

* btcmarkets: defer auth generation functionality, rm context deadline as this will be created just before sending HTTP request.

* binance: move const to top

* exmo: remove debug output as its generated in the requester function

* ftx: defer auth functionality

* requester: move error to top

* bittrex: defer auth functionality

* bitmex: defer auth functionality and remove deadline as generation occurs after rate limiting.

* btse: defer auth functionality

* coinbasepro: defer auth functionality and removed context deadline as this is generated after rate limiting

* coinbene: defer auth functionality and remove context deadline as this is generated after rate limiting

* huobi: defer auth functionality and remove context deadline as this is generated after rate limiting

* huobi-futures: defer auth functionality and remove context deadline as this is generated after rate limiting

* kraken: defer auth functionality and remove context deadline as this is generated after rate limiting

* kraken: remove deadline protection for timestamp generation

* okgroup: defer auth functionality and remove context deadline as this is generated after rate limiting

* poloniex: defer auth functionality

* zb: defer auth functionality and remove context deadline as this is generated after rate limiting

* exchanges: clean up log output which are done and inspected in the requester package

* binance: fix path bug on every retry, rm timeout context as this is not needed

* coinbene: fix path bug on retry

* binance: consolidate functionality

* coinbene: fix linter issues

* poloniex: linter fix

* kraken: change add -> set

* bitstamp: fix path bug for retry

* BTSE: fix retry path bug

* coinbene: fix path bug whoopsie by me

* gateio: fix bug where on retry it does not reset reader

* localbitcoins: fix path bug on retry

* zb: change domain to land

* exchanges: make sure io.Reader is generated every request

* exchanges: move reader generation into function scope

* wrapper_issues: setup exchange manager

* engine: expand withdraw manager test

* engine: dont look for environment

* bitstamp: fix pathing bug (@thrasher-)

* engine/withdraw_manager: purge tests as this is covered in repository withdraw
2021-08-06 17:24:38 +10:00
Scott
197ef2df21 Feature: Data history manager engine subsystem (#693)
* Adds lovely initial concept for historical data doer

* Adds ability to save tasks. Adds config. Adds startStop to engine

* Has a database microservice without use of globals! Further infrastructure design. Adds readme

* Commentary to help design

* Adds migrations for database

* readme and adds database models

* Some modelling that doesn't work end of day

* Completes datahistoryjob sql.Begins datahistoryjobresult

* Adds datahistoryjob functions to retreive job results. Adapts subsystem

* Adds process for upserting jobs and job results to the database

* Broken end of day weird sqlboiler crap

* Fixes issue with SQL generation.

* RPC generation and addition of basic upsert command

* Renames types

* Adds rpc functions

* quick commit before context swithc. Exchanges aren't being populated

* Begin the tests!

* complete sql tests. stop failed jobs. CLI command creation

* Defines rpc commands

* Fleshes out RPC implementation

* Expands testing

* Expands testing, removes double remove

* Adds coverage of data history subsystem, expands errors and nil checks

* Minor logic improvement

* streamlines datahistory test setup

* End of day minor linting

* Lint, convert simplify, rpc expansion, type expansion, readme expansion

* Documentation update

* Renames for consistency

* Completes RPC server commands

* Fixes tests

* Speeds up testing by reducing unnecessary actions. Adds maxjobspercycle config

* Comments for everything

* Adds missing result string. checks interval supported. default start end cli

* Fixes ID problem. Improves binance trade fetch. job ranges are processed

* adds dbservice coverage. adds rpcserver coverage

* docs regen, uses dbcon interface, reverts binance, fixes races, toggle manager

* Speed up tests, remove bad global usage, fix uuid check

* Adds verbose. Updates docs. Fixes postgres

* Minor changes to logging and start stop

* Fixes postgres db tests, fixes postgres column typo

* Fixes old string typo,removes constraint,error parsing for nonreaders

* prevents dhm running when table doesn't exist. Adds prereq documentation

* Adds parallel, rmlines, err fix, comment fix, minor param fixes

* doc regen, common time range check and test updating

* Fixes job validation issues. Updates candle range checker.

* Ensures test cannot fail due to time.Now() shenanigans

* Fixes oopsie, adds documentation and a warn

* Fixes another time test, adjusts copy

* Drastically speeds up data history manager tests via function overrides

* Fixes summary bug and better logs

* Fixes local time test, fixes websocket tests

* removes defaults and comment,updates error messages,sets cli command args

* Fixes FTX trade processing

* Fixes issue where jobs got stuck if data wasn't returned but retrieval was successful

* Improves test speed. Simplifies trade verification SQL. Adds command help

* Fixes the oopsies

* Fixes use of query within transaction. Fixes trade err

* oopsie, not needed

* Adds missing data status. Properly ends job even when data is missing

* errors are more verbose and so have more words to describe them

* Doc regen for new status

* tiny test tinkering

* str := string("Removes .String()").String()

* Merge fixups

* Fixes a data race discovered during github actions

* Allows websocket test to pass consistently

* Fixes merge issue preventing datahistorymanager from starting via config

* Niterinos cmd defaults and explanations

* fixes default oopsie

* Fixes lack of nil protection

* Additional oopsie

* More detailed error for validating job exchange
2021-07-01 16:21:48 +10:00
TaltaM
7e9266e438 Bittrex: Update API v1.1 to v3 and add websocket support (#646)
* Update Bittrex API from v1.1 to v3

V1.1 has been retired as of 9/30/2020

- Update REST to V3
- Add initial websocket support

* Bittrex update - enable websockets in testdata config file

* Update Bittrex - add Websocket capability to docs

* Update Bittrex connector

- AppVeyor warnings
- Update tests
- Generate documentation
- Fix nits

* Update Bittrex - add websocket order processing

* Update Bittrex connector

* Bittrex connector - fix ineffectual err assignment

* Fix nits

* Orderbook synchronization

* Remove redundant nil

* Log WS fetch orderbook message as debug message instead of as warning

* Update after rebase

* Add tests

* Add allowed candle interval values

* Replace literals with declared constants

* Replace variable name 'request' with 'req'

* Add check and update for deprecated REST URL

* Nits and some cleaning up

* Change ParseInt bit size to 64

* [FIX] Remove several shadow declarations

* Do not export constructTicker

* Remove parseTime()

* Update GetHistoricCandles()

* [FIX] Address gocritic nits

* [FIX] Address gocritic nits

* Use SendMessageReturnResponse() instead of local map

* Rate limit subscribing and unsubscribing

* [FIX] use go routine for subscribing and unsubscribing

* [FIX] Set correct index for map

* [FIX] Address unused vars, literals, time format

* Adjusted timing when subscribing to many order books

* Cache partial updates to tickers instead of calling REST function

* [FIX] Update sequence nr when multiple updates are queued

* Address golint issues

* Fix nits
2021-05-18 11:33:53 +10:00