Commit Graph

1001 Commits

Author SHA1 Message Date
Adrian Gallagher
9ff4471a45 Merge branch 'master' into engine 2019-06-21 18:10:55 +10:00
Ryan O'Hara-Reid
897bcfd9a4 Documentation Update (#318)
* Initial update

* update

* Fix linter issues

* Add new documentation template and fix
2019-06-21 12:38:35 +10:00
Scott
3a66e99899 Authenticated Websocket support (#315)
* Improves subscribing by not allowing duplicates. Adds bitmex auth support

* Adds coinbase pro support. Partial BTCC support. Adds WebsocketAuthenticatedEndpointsSupported websocket feature. Adds GateIO support

* Adds Coinut support

* Moves Coinut WS types to file. Implements Gemini's secure WS endpoint

* Adds HitBTC ws authenticated support. Fixes var names

* Adds huobi and hadax authenticated websocket support

* Adds auth to okgroup (okex, okcoin). Fixes some linting

* Adds Poloniex support

* Adds ZB support

* Adds proper bitmex support

* Improves bitfinex support, improves websocket functionality definitions

* Fixes coinbasepro auth

* Tests all endpoints

* go formatting, importing, linting run

* Adds wrapper supports

* General clean up. Data race destruction

* Improves testing on all exchanges except ZB

* Fixes ZB hashing, parsing and tests

* minor nits before someone else sees them <_<

* Fixes some nits pertaining to variable usage, comments, typos and rate limiting

* Addresses nits regarding types and test responses where applicable

* fmt import

* Fixes linting issues

* No longer returns an error on failure to authenticate, just logs. Adds new AuthenticatedWebsocketAPISupport config value to allow a user to seperate auth from REST and WS. Prevents WS auth if AuthenticatedWebsocketAPISupport is false, adds additional login check 'CanUseAuthenticatedEndpoints' for when login only occurs once (not per request). Removes unnecessary time.Sleeps from code. Moves WS auth error logic to auth function so that wrappers can get involved in all the auth fun. New-fandangled shared test package, used exclusively in testing, will be the store of all the constant boilerplate things like timeout values. Moves WS test setup function to only run once when there are multiple WS endpoint tests. Cleans up some struct types

* Increases test coverage with tests for config.areAuthenticatedCredentialsValid config.CheckExchangeConfigValues, exchange.SetAPIKeys, exchange.GetAuthenticatedAPISupport, exchange_websocket.CanUseAuthenticatedEndpoitns and exchange_websocket.SetCanUseAuthenticatedEndpoints. Adds b.Websocket.SetCanUseAuthenticatedEndpoints(false) when bitfinex fails to authenticate
Fixes a typo. gofmt and goimport

* Trim Test Typos

* Reformats various websocket types. Adds more specific error messaging to config.areAuthenticatedCredentialsValid
2019-06-19 13:19:01 +10:00
Adrian Gallagher
4a41c5cff3 Merge branch 'master' into engine 2019-06-17 14:36:06 +10:00
Adrian Gallagher
67a58a10bd Fix ticker empty err and improve test/consistency for orderbook/ticke… (#317)
* Fix ticker empty err and improve test/consistency for orderbook/ticker pkgs
2019-06-17 13:43:13 +10:00
Ryan O'Hara-Reid
20c24601fb Asset update to fix minor stutter (#316) 2019-06-17 09:02:07 +10:00
Adrian Gallagher
b901c4b670 Engine improvements
Add back events tests
Fill out SMTP comms handler
Add getcommunicationrelayers gRPC command
2019-06-14 18:00:42 +10:00
Adrian Gallagher
6b2cfe7905 Daily engine improvements
Link up various subsystems to be managed atomically with the ability to start/stop them
New subsystem APIs
Comms changes
2019-06-13 17:30:50 +10:00
Adrian Gallagher
33085318c4 Switch connchecker to service 2019-06-12 17:52:40 +10:00
Adrian Gallagher
2ad808e70c Daily engine improvements:
New GetExchangeOTPs API
CLI validation
Standardised pairs for GCTCLI
Expand test coverage
Trim SMS global from name is len > 11
Linter fixes
2019-06-11 17:02:00 +10:00
Adrian Gallagher
f777e68716 Engine improvements 2019-06-10 20:02:09 +10:00
Adrian Gallagher
04c7c4895f Split common package more and QA 2019-06-07 20:52:44 +10:00
Adrian Gallagher
cbd3e7bacd Order manager changes 2019-06-07 17:52:53 +10:00
Adrian Gallagher
bd8dc47c38 daily progress build 2019-06-06 17:20:40 +10:00
Adrian Gallagher
26795508ff Merge branch 'master' into engine 2019-06-06 14:57:32 +10:00
Adrian Gallagher
92d798ef2a BTCMarkets: Fix order JSON unmarshal response (#314) 2019-06-06 11:37:54 +10:00
Adrian Gallagher
3010b62ac1 Use string.EqualFold where necessary 2019-06-04 17:27:00 +10:00
Adrian Gallagher
e965e54e09 Split up common.go, file path fixes and much more 2019-06-04 17:04:32 +10:00
Adrian Gallagher
8c62316e82 websocket QA 2019-06-04 14:34:00 +10:00
Adrian Gallagher
6ebafb0f33 Merge branch 'master' into engine 2019-06-04 12:25:40 +10:00
Adrian Gallagher
d639f6e4a6 Ignore gocryptotrader binary on macOS/Linux builds
Also fix go fmt issues
2019-06-03 00:48:33 -07:00
Adrian Gallagher
8048962b60 Remove BTCC exchange 2019-06-03 17:17:17 +10:00
Adrian Gallagher
94ac7c917d Huobi/Hadax: Update test symbol to fix tests 2019-06-03 17:00:34 +10:00
Ryan O'Hara-Reid
a80acb16de General fixes for CancelAllOrders exchange wrapper function (#313)
* General fixes for CancelAllOrders

* Fix error shadowing issue

* Initialise order status map and rm reduntant initialisations
2019-05-31 16:06:10 +10:00
Shogin Michael
416fbbd5ae Unit tests for communication/base package (#312)
* Added tests

* Added tests communications base

* Removed unnecessary field

* Review corrections: linter

* Review corrections: typo
2019-05-30 13:35:22 +10:00
Adrian Gallagher
39718b7f01 Merge branch 'master' into engine 2019-05-29 11:15:50 +10:00
Ryan O'Hara-Reid
e8bc3b5e76 Bump golangci-lint version to 1.16.0 (#311) 2019-05-27 14:10:54 +10:00
Adrian Gallagher
993b73f5cc AppVeyor: Add cache (#309)
* AppVeyor: Add cache

* Add go-build to cache
2019-05-24 14:13:43 +10:00
Adrian Gallagher
f5914e8c10 Engine changes 2019-05-22 17:06:38 +10:00
Adrian Gallagher
53964aadf5 COINUT: Fix SendHTTPRequest authenticated requests (#308) 2019-05-22 17:03:35 +10:00
Andrew
506d6011c2 Bitstamp: Websocket API upgrade to v2 (#307)
* Subscribe/Unsubscribe methods added

* migration to v3

* removed orderbook from rest

* WsUpdateOrderbook updated to reflect changes to v2

* Added comment for exported func

* removed logging

* unexported structs that are not used globally moved seed to own function

* unexported functions not used outside package

* Support reconnection message from bitstamp

* moved from range key/val

* using ticket.Spot instead of string

* Seperated out WsReadData & WsHandleData to allow for better testing of websocket messages

* ah should continue to next iteration and not break execution on json decode

* code formatting clean up

* reworded connection message

* return out of method instead of just breaking loop

* formatting changes and replaced SPOT with ticket.Spot type
2019-05-22 15:46:55 +10:00
Scott
7a4ffefc93 Config: Fix AuthenticatedAPISupport default values check (#306)
* Fixes issue where first if statement always gets hit

* fmtimport

* Slightly more granular checks for keys, secret keys and clientids

* Moves log warn and removes extra checks
2019-05-22 12:36:17 +10:00
Vadim Zhuk
c08222699c Huobi/HuobiHadax/Binance: Fix GetActiveOrders orders date (#304)
* fix balance responce

* Fixed OrderHistory unmarshalling on the GateIo

* fix int convertation in the GetOrderHistory function

* fix issue on order submit : Binance exchange raw response: {"code":-1102,"msg":"Mandatory parameter 'timeInForce' was not sent, was empty/null, or malformed."}

* Okgroup: fix issue on empty notional GetSpotOrderResponse

* Okgroup: fix issue with empty order status. order start is required for orderHistory request

* HUOBU: fix order currency mapping in GetActiveOrders method

* OKGroup: fix currency name on GetAccountInfo

* ZB: fixed wrong error code on order send method

* OKGroup: add missing fields to GetActiveOrders mapping

* huobi: add missing fields to GetActiveOrders/GetOrderHistory response mapping

* gateio: add order status field to GetActiveOrders response

* gateio: fix orderId on SubmitOrder response

* - poloniex: fix 422 on private API call
- poloniex: fix empty order history

* - poloniex: fix 422 on private API call (revert)

* GateIo: Fix unmarshalling issue

* Huobi: removing redundant breaks

* Huobi: fix rangeValCopy in GetActiveOrders/GetOrderHistory methods

* - poloniex: fix 422 on private API call

* Revert "- poloniex: fix 422 on private API call"

This reverts commit 4748227a

Took 39 minutes

* Replace nonce increment with mutex to atomic incrementation

Took 20 minutes

* Fix ZB order unmarshaling

Took 6 minutes

* Fix CancelSpotOrderResponse unmarshaling on OKGroup

Took 2 minutes

* fix OrderDate on huobi

Took 3 minutes

* fix OrderDate on binance

Took 1 minute

* fix OrderDate on huobihadax

Took 24 minutes
2019-05-22 08:23:51 +10:00
Adrian Gallagher
0c6cd7fedf Go mod: Perioidc update of depends 2019-05-20 15:40:35 +10:00
Scott
08542b6ef7 WS test improvements (#303)
* Improves okgroup and kraken websocket tests so there is less overhead required

* Fixes declaration shadowing

* Fixes duplication

* Switches error to fatal for wsconnection problems in test.
2019-05-20 10:32:39 +10:00
Andrew
05a7ae70ba RPC Server Host valdiation fix for #296 (#302)
* listenAddress updated to correctly set if RPC listens on any interface

* Update restful_router.go
2019-05-17 11:45:22 +10:00
leilaes
46267e5975 Add missing close file and missing stop ticker (#300)
* Add missing close file and missing stop ticker

* Defer close file after checking error

* Update exchange_template.go
2019-05-17 11:17:35 +10:00
Adrian Gallagher
17a6acf58a Travis: Cache $GOPATH/pkg/mod (#301)
Speeds up CI testing
2019-05-16 17:01:23 +10:00
Scott
6c850e73e2 Websocket connection handling and subscription management (#297)
* Step one: Sets up  connection handler for websockets to always be connected until a shutdown event is received.
Sets up a vague subscription handler to ensure subscriptions are subscribed

* Adds support for resubscriptions for bitfinex, bitstamp, bitmex and btcc. Adds subscription params for special websocket subscription requirements. Removes subscription monitor from wait group so that it can exist despite a shutdown and continuously check

* Adds channel subscription support to bitmex, btse, coibasepro, coinut, gateio, gemini, hitbtc, huobi, hadax, kraken, okgroup, poloniex and zb

* Implements unsubscribe for bitfinex, btcc, btse, coinbasepro, gateio, gitbtc, huobi, hadax

* ManageSubscriptions now called from WSConnect and made private instead of inside individual exchanges. ManageSubscriptions can now unsubscribe. exchange_websocket_types.go now contains all exchange_websocket.go types to avoid clutter

* Adds it to websocket functionality so managesubscriptions will close when not supported

* Separates functions into testable functions to ensure logic works. Adds tests. Updates websocket setup to include verbosity (inherited from exchange). Adds no connection tolerance to fatal on failed reconnects

* More exchange_websocket tests. Updating to use pointers. Creation of equals func to make comparison easier

* Fixes okex, okcoin tests. Fixes race conditions. Removes pointer usage again.

* Adds subscribe and unsubscribe to wrappers

* Fixes deadlock. Fixes ws verbosity.

* Updates all exchanges to properly support subscription/connection feature. Also reintroduces race conditions....

* Moves connection varialbes to struct from package to allow each websocket to have their own reconnection checks. Neatens up logs

* Fixes lint/critic issues. Fixes tests. Removes unused function.

* Moves websocket ratelimiter to their own const variables. Fixes more race conditions with connecting variable

* Removes redundant subscribe functions. Ensuring only the exchange_websocket.go can manage subscriptions. Fixes debug logs to be verbose wrapped

* Fixes issue with slice copying. Re-adds okgroup default channels

* Adds nolint to append

* Adds comments and adds support for gateio auth request subscriptions

* Adds new test to ensure slices dont point to the same vars

* removes fatals. gofmt goimports

* more gofmts

* Addresses PR comments, removing empty and redundant lines

* Addresses PR comments. Ensures that writing to the websocket is single-threaded by adding a mutex to exchanges. Minimises wrapper code and moves subscription loops to exchange_websocket. Privatises ChannelsToSubscribe, Connecting properties and removeChannelToSubscribe func to prevent unnecessary tampering.

* Removes unused mutex. FMTS and IMPORTS

* Fixes request lock time change

* More specific logs

* Renames ws mutex. Fixes bitmex subscriptions. Increased gateio ratelimiter to 120ms. Removes ratelimiter from bitfinex, bitmex, bitstamp, btcc, btse, coibasepro, hitbtc, huobi, hadax, poloniex and zb

* changes recieved typo due to not being well received

* Fixes parsing issue with Huobi and hadax

* Fixes data race with more locks

* removes defer locks. fixes huobi/hadax verbose output

* Fixes double JSONEncode for coinut. Fixes verbose output for coinut

* gofmt,goimport for coinut

* Fixes issue where multiple connection monitors can spawn

* Removes defer exchange.WebsocketConn.Close() in defer handledata exit as connectionmonitor handles connections instead

* gofmt and go import

* More fmts
2019-05-16 16:39:16 +10:00
Adrian Gallagher
0b27096376 Periodic available pairs config update 2019-05-16 14:28:10 +10:00
Andrew
5bce701d43 Add httputil.DumpRequestOut & httputil.DumpResponse to SendPayload (#298)
* First pass adding DumpResponse

* WIP

* Added httpDebug flag to SendPayload

* anx -> coinbase converted

* HTTPDebug flag added to all exchanges

* Updated Alphapoint:

* reworded error message

* Corrected itbit and huobihadax sendpayload

* Removed setting of default value

* don't return body for response as we already get this in verbose mode

* body is now included based on verbose setting

* WIP

* Due to issues with DumpResponse and gzip decoding use raw response from exchange for body output Also adds gzip decoding back as bug fix for OK group exchange

* removed additional new line

* Added URL path to dumpbody display as well
2019-05-16 13:46:28 +10:00
Adrian Gallagher
166761454f ZB: Change API URL to .cn host as the .com one is down 2019-05-16 10:49:48 +10:00
leilaes
cdab89a58a Replace a zero-length, non-nil slice with a nil slice (#299)
* Replace a zero-length, non-nil slice with a nil slice

* Update codelingo.yaml
2019-05-16 10:19:08 +10:00
Andrew
07216a4422 Gateio expand wrappers, websocket and bug fix (#291)
* Added fix for balances as GateIO returns object or array depending on if you have any balances

* Adds GetOrderInfo function for REST API

* Correctly assign WebsocketURL

* Adds websocket auth support to GateIO

* reverts Host changes

* unexported getbalance and signin

* Add WsGetOrderInfo to retreive information on a set order

* unexport wsGetOrderInfo

* renamed freeze to locked to match rest interface

* Removed old logging

* Added detailed error messages and testing disabling auth api support if WS auth fails

* Removed old code for nonce

* reworked GetOrder to return an error if no order is found

* gofmt tests

* removed unneeded event from websocketresponse

* fixed casing on websocket

* Fixed condition
2019-05-13 13:21:39 +10:00
Ryan O'Hara-Reid
8279a036c2 Connection checker update (#294)
* Adds logic to ensure nothing proceeds a connectivity check and adds tests

* Adds waitgroup to verify monitor routine startup before shutdown is called thus eliminating race condition

* fix func desc and test func naming convention

* Add invalid address check

* Fix race condition for okgroup on wait group when instant shutdown called

* Fix fmt issue with gofmt
2019-05-13 12:02:39 +10:00
Andrew
de8916a0df Fix typo in config.go (#295) 2019-05-10 15:18:58 +10:00
leilaes
ee28d18f18 Omit unnecessary else statements when if does not flow into the next statement (#293)
* Removing unnecessary else

* Fix go fmt issues
2019-05-09 14:13:48 +10:00
Andrew
8be0682bca RPC server host/port fix (#292)
* Fix to conform with standards on Host header needing a port if its not on port 80

* Fix test to use correct address
2019-05-08 13:22:13 +10:00
Ryan O'Hara-Reid
35b94268e0 Update request.go to fix concurrency nonce issues (#285)
* Updates nonce generation to adhere to fifo channel buffer before request executes by routine

* removed unused variables, lns etc

* Fix requested changes and added in timer that disengages lock if out of scope error occurs

* Fixed woopsy daisy issue

* Add benchmark, reduce time in force to unlock before stack insertion, add nil check for edge case

* Remove unusued waitgroup field

* use return nonce.Value and method, rm redundant nonce code, fix tests.

* Fix linter issue: unnecessary conversion
2019-05-06 13:46:34 +10:00
Vadim Zhuk
1967507d40 Fix bugs with unmarshaling on ZB and OKGroup (#290)
* Replace nonce increment with mutex to atomic incrementation
* Fix ZB order unmarshaling
* Fix CancelSpotOrderResponse unmarshaling on OKGroup
2019-05-03 20:38:14 +10:00