Commit Graph

973 Commits

Author SHA1 Message Date
Adrian Gallagher
f5914e8c10 Engine changes 2019-05-22 17:06:38 +10:00
Adrian Gallagher
53964aadf5 COINUT: Fix SendHTTPRequest authenticated requests (#308) 2019-05-22 17:03:35 +10:00
Andrew
506d6011c2 Bitstamp: Websocket API upgrade to v2 (#307)
* Subscribe/Unsubscribe methods added

* migration to v3

* removed orderbook from rest

* WsUpdateOrderbook updated to reflect changes to v2

* Added comment for exported func

* removed logging

* unexported structs that are not used globally moved seed to own function

* unexported functions not used outside package

* Support reconnection message from bitstamp

* moved from range key/val

* using ticket.Spot instead of string

* Seperated out WsReadData & WsHandleData to allow for better testing of websocket messages

* ah should continue to next iteration and not break execution on json decode

* code formatting clean up

* reworded connection message

* return out of method instead of just breaking loop

* formatting changes and replaced SPOT with ticket.Spot type
2019-05-22 15:46:55 +10:00
Scott
7a4ffefc93 Config: Fix AuthenticatedAPISupport default values check (#306)
* Fixes issue where first if statement always gets hit

* fmtimport

* Slightly more granular checks for keys, secret keys and clientids

* Moves log warn and removes extra checks
2019-05-22 12:36:17 +10:00
Vadim Zhuk
c08222699c Huobi/HuobiHadax/Binance: Fix GetActiveOrders orders date (#304)
* fix balance responce

* Fixed OrderHistory unmarshalling on the GateIo

* fix int convertation in the GetOrderHistory function

* fix issue on order submit : Binance exchange raw response: {"code":-1102,"msg":"Mandatory parameter 'timeInForce' was not sent, was empty/null, or malformed."}

* Okgroup: fix issue on empty notional GetSpotOrderResponse

* Okgroup: fix issue with empty order status. order start is required for orderHistory request

* HUOBU: fix order currency mapping in GetActiveOrders method

* OKGroup: fix currency name on GetAccountInfo

* ZB: fixed wrong error code on order send method

* OKGroup: add missing fields to GetActiveOrders mapping

* huobi: add missing fields to GetActiveOrders/GetOrderHistory response mapping

* gateio: add order status field to GetActiveOrders response

* gateio: fix orderId on SubmitOrder response

* - poloniex: fix 422 on private API call
- poloniex: fix empty order history

* - poloniex: fix 422 on private API call (revert)

* GateIo: Fix unmarshalling issue

* Huobi: removing redundant breaks

* Huobi: fix rangeValCopy in GetActiveOrders/GetOrderHistory methods

* - poloniex: fix 422 on private API call

* Revert "- poloniex: fix 422 on private API call"

This reverts commit 4748227a

Took 39 minutes

* Replace nonce increment with mutex to atomic incrementation

Took 20 minutes

* Fix ZB order unmarshaling

Took 6 minutes

* Fix CancelSpotOrderResponse unmarshaling on OKGroup

Took 2 minutes

* fix OrderDate on huobi

Took 3 minutes

* fix OrderDate on binance

Took 1 minute

* fix OrderDate on huobihadax

Took 24 minutes
2019-05-22 08:23:51 +10:00
Adrian Gallagher
0c6cd7fedf Go mod: Perioidc update of depends 2019-05-20 15:40:35 +10:00
Scott
08542b6ef7 WS test improvements (#303)
* Improves okgroup and kraken websocket tests so there is less overhead required

* Fixes declaration shadowing

* Fixes duplication

* Switches error to fatal for wsconnection problems in test.
2019-05-20 10:32:39 +10:00
Andrew
05a7ae70ba RPC Server Host valdiation fix for #296 (#302)
* listenAddress updated to correctly set if RPC listens on any interface

* Update restful_router.go
2019-05-17 11:45:22 +10:00
leilaes
46267e5975 Add missing close file and missing stop ticker (#300)
* Add missing close file and missing stop ticker

* Defer close file after checking error

* Update exchange_template.go
2019-05-17 11:17:35 +10:00
Adrian Gallagher
17a6acf58a Travis: Cache $GOPATH/pkg/mod (#301)
Speeds up CI testing
2019-05-16 17:01:23 +10:00
Scott
6c850e73e2 Websocket connection handling and subscription management (#297)
* Step one: Sets up  connection handler for websockets to always be connected until a shutdown event is received.
Sets up a vague subscription handler to ensure subscriptions are subscribed

* Adds support for resubscriptions for bitfinex, bitstamp, bitmex and btcc. Adds subscription params for special websocket subscription requirements. Removes subscription monitor from wait group so that it can exist despite a shutdown and continuously check

* Adds channel subscription support to bitmex, btse, coibasepro, coinut, gateio, gemini, hitbtc, huobi, hadax, kraken, okgroup, poloniex and zb

* Implements unsubscribe for bitfinex, btcc, btse, coinbasepro, gateio, gitbtc, huobi, hadax

* ManageSubscriptions now called from WSConnect and made private instead of inside individual exchanges. ManageSubscriptions can now unsubscribe. exchange_websocket_types.go now contains all exchange_websocket.go types to avoid clutter

* Adds it to websocket functionality so managesubscriptions will close when not supported

* Separates functions into testable functions to ensure logic works. Adds tests. Updates websocket setup to include verbosity (inherited from exchange). Adds no connection tolerance to fatal on failed reconnects

* More exchange_websocket tests. Updating to use pointers. Creation of equals func to make comparison easier

* Fixes okex, okcoin tests. Fixes race conditions. Removes pointer usage again.

* Adds subscribe and unsubscribe to wrappers

* Fixes deadlock. Fixes ws verbosity.

* Updates all exchanges to properly support subscription/connection feature. Also reintroduces race conditions....

* Moves connection varialbes to struct from package to allow each websocket to have their own reconnection checks. Neatens up logs

* Fixes lint/critic issues. Fixes tests. Removes unused function.

* Moves websocket ratelimiter to their own const variables. Fixes more race conditions with connecting variable

* Removes redundant subscribe functions. Ensuring only the exchange_websocket.go can manage subscriptions. Fixes debug logs to be verbose wrapped

* Fixes issue with slice copying. Re-adds okgroup default channels

* Adds nolint to append

* Adds comments and adds support for gateio auth request subscriptions

* Adds new test to ensure slices dont point to the same vars

* removes fatals. gofmt goimports

* more gofmts

* Addresses PR comments, removing empty and redundant lines

* Addresses PR comments. Ensures that writing to the websocket is single-threaded by adding a mutex to exchanges. Minimises wrapper code and moves subscription loops to exchange_websocket. Privatises ChannelsToSubscribe, Connecting properties and removeChannelToSubscribe func to prevent unnecessary tampering.

* Removes unused mutex. FMTS and IMPORTS

* Fixes request lock time change

* More specific logs

* Renames ws mutex. Fixes bitmex subscriptions. Increased gateio ratelimiter to 120ms. Removes ratelimiter from bitfinex, bitmex, bitstamp, btcc, btse, coibasepro, hitbtc, huobi, hadax, poloniex and zb

* changes recieved typo due to not being well received

* Fixes parsing issue with Huobi and hadax

* Fixes data race with more locks

* removes defer locks. fixes huobi/hadax verbose output

* Fixes double JSONEncode for coinut. Fixes verbose output for coinut

* gofmt,goimport for coinut

* Fixes issue where multiple connection monitors can spawn

* Removes defer exchange.WebsocketConn.Close() in defer handledata exit as connectionmonitor handles connections instead

* gofmt and go import

* More fmts
2019-05-16 16:39:16 +10:00
Adrian Gallagher
0b27096376 Periodic available pairs config update 2019-05-16 14:28:10 +10:00
Andrew
5bce701d43 Add httputil.DumpRequestOut & httputil.DumpResponse to SendPayload (#298)
* First pass adding DumpResponse

* WIP

* Added httpDebug flag to SendPayload

* anx -> coinbase converted

* HTTPDebug flag added to all exchanges

* Updated Alphapoint:

* reworded error message

* Corrected itbit and huobihadax sendpayload

* Removed setting of default value

* don't return body for response as we already get this in verbose mode

* body is now included based on verbose setting

* WIP

* Due to issues with DumpResponse and gzip decoding use raw response from exchange for body output Also adds gzip decoding back as bug fix for OK group exchange

* removed additional new line

* Added URL path to dumpbody display as well
2019-05-16 13:46:28 +10:00
Adrian Gallagher
166761454f ZB: Change API URL to .cn host as the .com one is down 2019-05-16 10:49:48 +10:00
leilaes
cdab89a58a Replace a zero-length, non-nil slice with a nil slice (#299)
* Replace a zero-length, non-nil slice with a nil slice

* Update codelingo.yaml
2019-05-16 10:19:08 +10:00
Andrew
07216a4422 Gateio expand wrappers, websocket and bug fix (#291)
* Added fix for balances as GateIO returns object or array depending on if you have any balances

* Adds GetOrderInfo function for REST API

* Correctly assign WebsocketURL

* Adds websocket auth support to GateIO

* reverts Host changes

* unexported getbalance and signin

* Add WsGetOrderInfo to retreive information on a set order

* unexport wsGetOrderInfo

* renamed freeze to locked to match rest interface

* Removed old logging

* Added detailed error messages and testing disabling auth api support if WS auth fails

* Removed old code for nonce

* reworked GetOrder to return an error if no order is found

* gofmt tests

* removed unneeded event from websocketresponse

* fixed casing on websocket

* Fixed condition
2019-05-13 13:21:39 +10:00
Ryan O'Hara-Reid
8279a036c2 Connection checker update (#294)
* Adds logic to ensure nothing proceeds a connectivity check and adds tests

* Adds waitgroup to verify monitor routine startup before shutdown is called thus eliminating race condition

* fix func desc and test func naming convention

* Add invalid address check

* Fix race condition for okgroup on wait group when instant shutdown called

* Fix fmt issue with gofmt
2019-05-13 12:02:39 +10:00
Andrew
de8916a0df Fix typo in config.go (#295) 2019-05-10 15:18:58 +10:00
leilaes
ee28d18f18 Omit unnecessary else statements when if does not flow into the next statement (#293)
* Removing unnecessary else

* Fix go fmt issues
2019-05-09 14:13:48 +10:00
Andrew
8be0682bca RPC server host/port fix (#292)
* Fix to conform with standards on Host header needing a port if its not on port 80

* Fix test to use correct address
2019-05-08 13:22:13 +10:00
Ryan O'Hara-Reid
35b94268e0 Update request.go to fix concurrency nonce issues (#285)
* Updates nonce generation to adhere to fifo channel buffer before request executes by routine

* removed unused variables, lns etc

* Fix requested changes and added in timer that disengages lock if out of scope error occurs

* Fixed woopsy daisy issue

* Add benchmark, reduce time in force to unlock before stack insertion, add nil check for edge case

* Remove unusued waitgroup field

* use return nonce.Value and method, rm redundant nonce code, fix tests.

* Fix linter issue: unnecessary conversion
2019-05-06 13:46:34 +10:00
Vadim Zhuk
1967507d40 Fix bugs with unmarshaling on ZB and OKGroup (#290)
* Replace nonce increment with mutex to atomic incrementation
* Fix ZB order unmarshaling
* Fix CancelSpotOrderResponse unmarshaling on OKGroup
2019-05-03 20:38:14 +10:00
Adrian Gallagher
f19cf37752 Add AppVeyor support (#289)
Add AppVeyor support
2019-05-03 13:09:01 +10:00
Adrian Gallagher
5ea82f163a OKGroup: Fix unmarshalling issues after API update (#288) 2019-05-02 12:10:29 +10:00
Ryan O'Hara-Reid
6e2cba566f Add in routine that checks internet connectivity for bot services (#287)
* Add in routine that checks internet connectivity for bot services

* Packaged connection checker
2019-05-02 10:14:42 +10:00
Ryan O'Hara-Reid
2f1405ead4 Add OS X CI Matrix support to Travis (#284)
Add OS X CI Matrix support to Travis
2019-04-30 15:27:16 +10:00
Ryan O'Hara-Reid
88bc17a17f ZB: GetOrderbook out of range fix (#286)
ZB: GetOrderbook out of range fix
2019-04-29 17:27:03 +10:00
Adrian Gallagher
47152c87e3 Update CONTRIBUTORS and doc templates 2019-04-26 17:36:41 +10:00
Adrian Gallagher
9bdcc22ae1 Relax case sensitive string comparisons in various parts of GCT 2019-04-23 14:22:00 +10:00
Adrian Gallagher
32e4dcb63d Bitmex: Fix UpdateTicker/UpdateOrderbook wrappers
UpdateTicker: drop need for start timestamp as the latest entry will always be returned by default
UpdateOrderbook: fix string comparison so that orderbook entries will actually be appended

NOTE: As of this time, Bitmex doesn't support a ticker API endpoint so the only two options are a) last trade for an asset (which GCT uses) or b) orderbook data query
Fixes issue: https://github.com/thrasher-/gocryptotrader/issues/282
2019-04-23 11:02:30 +10:00
Vadim Zhuk
b02d03da8b Fix order issues on GateIo, Huobi, OkGroup and Poloniex (#280)
* fix balance responce

* Fixed OrderHistory unmarshalling on the GateIo

* fix int convertation in the GetOrderHistory function

* fix issue on order submit : Binance exchange raw response: {"code":-1102,"msg":"Mandatory parameter 'timeInForce' was not sent, was empty/null, or malformed."}

* Okgroup: fix issue on empty notional GetSpotOrderResponse

* Okgroup: fix issue with empty order status. order start is required for orderHistory request

* HUOBU: fix order currency mapping in GetActiveOrders method

* OKGroup: fix currency name on GetAccountInfo

* ZB: fixed wrong error code on order send method

* OKGroup: add missing fields to GetActiveOrders mapping

* huobi: add missing fields to GetActiveOrders/GetOrderHistory response mapping

* gateio: add order status field to GetActiveOrders response

* gateio: fix orderId on SubmitOrder response

* - poloniex: fix 422 on private API call
- poloniex: fix empty order history

* - poloniex: fix 422 on private API call (revert)

* GateIo: Fix unmarshalling issue

* Huobi: removing redundant breaks

* Huobi: fix rangeValCopy in GetActiveOrders/GetOrderHistory methods
2019-04-18 17:38:47 +10:00
Andrew Jackson
c449568f6d Fix incorrect variable usage (err -> errNTP) 2019-04-18 10:23:19 +10:00
Andrew
071f4f68a8 New NTP Client (#277)
* WIP

* Added check for time out of sync

* merged upstream/master

* added tests

* Increased configuration options for NTPclient and test coverage

* removed unneeded config save at end of ntp update

* Added test for empty response to confirm it will loop

* formatting correction

* converted to pointer to allow for default allowance settings to be checked

* added readme for NTP server

* corrected some formatting

* updated configtest negativedifference value

* gofmt config_test.go for correct import order

* corrected typo value in test

* bugfix for windows newline and changes based on PR feedback

* added minus sign to output

* fixed negative number input

* Fixed spelling mistakes and removed redundant test

* reverted back to a positive number in the config instead of negative for allowednegativedifference

* restructured code for cleaner output
2019-04-18 10:08:19 +10:00
Adam
400c1cc84d Improved test coverage and functions in common.go (#266)
Improved directory creation, including permissions and expanded test coverage
2019-04-17 17:33:40 +10:00
Vadim Zhuk
ca344bad51 ZB: Fix SendAuthenticatedHTTPRequest error code handling (#278)
* fix balance responce

* Fixed OrderHistory unmarshalling on the GateIo

* fix int convertation in the GetOrderHistory function

* fix issue on order submit : Binance exchange raw response: {"code":-1102,"msg":"Mandatory parameter 'timeInForce' was not sent, was empty/null, or malformed."}

* Okgroup: fix issue on empty notional GetSpotOrderResponse

* Okgroup: fix issue with empty order status. order start is required for orderHistory request

* HUOBU: fix order currency mapping in GetActiveOrders method

* OKGroup: fix currency name on GetAccountInfo

* ZB: fixed wrong error code on order send method
2019-04-17 09:25:06 +10:00
Harry Ma
6ebebf1b16 Fix the unsuitable definition of a function (#279) 2019-04-17 09:20:46 +10:00
Adrian Gallagher
388d160f2b BTSE: Fix JSONDecode error upon successful websocket connection
BTSE updated their websocket API to send a "connect success" message upon a successful connection
2019-04-15 14:53:38 +10:00
Adrian Gallagher
5a42a4162b Improve currency pair handling code
Fixes these happy little accidents:
1) Config: Actually use enabled pairs for config.GetEnabledPairs
2) Pair format: Handle edge case for pairs DASHKRW vs DSHKRW and improve testing
2019-04-15 14:49:46 +10:00
Scott
32b43387fd Kraken websocket orderbook buffer (#276)
* Adds a orderbook buffer to prevent out of order issues with WS orderbooks. Internalises latest orderbook for comparisons sake as getorderbookex is unreliable due to REST updates

* Adds a basic rate limiter for WS requests. Updates buffer to support multiple channels

* Uses earliest buffer 'lastupdate' to compate to existing orderbook's 'lastupdate' instead of last

* Adds WS test for buffer use

* Adds out of order test

* Fixes blocking bid updates, fixes issue where orderbook processing occured in wrong area, adds detailed verbose debugging, uses pointer for channelData use

* Updates test to adapt to buffer limits, reduces kraken buffer to 3

* Change websocket connection check in tests. Change error handling. Implement requestID in data responses for WS. Change test to prevent default subscriptions from preventing test execution with extra data. Updates orderbook tests to call correct functions to prevent resubscriptions. Removes resubscribe on subscription status failure

* Fixes linting issues

* Fixes error handling
2019-04-12 09:21:37 +10:00
Adrian Gallagher
7ac8854c43 Specifically set dist to xenial to avoid go linker build issue 2019-04-11 08:11:05 +10:00
Adam
d69713f59d Fix rangeValCopy linter issues (#275)
* first pass of linter fixes for range val

* fixed the remaining issues for pointers

* changed as requested
2019-04-10 17:04:32 +10:00
Scott
e56fc26d93 Offline worst case trade fees (#274)
* Really basic getSimulated fee function everywhere

* Worst case fees for all exchanges

* Adds tests, fixes comment spacing. Adds wrapper logic. Makes test api key var name consistent. Removes some okcoin ETT tests

* Removes redundant functions

* linting issues. Fixes introduces huobi issues

* More linting

* Stops trying to hide ETT problems, uses iota

* Skips ETT tests for now
2019-04-09 19:38:31 +10:00
Andrew
eeda97bbaf Binance websocket fixes for #271 & #270 (#272)
* Fixes bug reported in #271 switch statement

* Time conversion fix for timestamp

* remove debug statement

* reverted back to CurrDayClose
2019-04-07 05:41:47 +10:00
Adrian Gallagher
8251fdd6ae CoinbasePro: Fix websocket ticker inconsistency
Addresses issue: https://github.com/thrasher-/gocryptotrader/issues/270
2019-04-06 15:08:06 +11:00
Vadim Zhuk
3956613831 OKGroup: fix currency name on GetAccountInfo (#273)
OKGroup: fix currency name on GetAccountInfo
2019-04-06 11:42:07 +11:00
Andrew
6c8f8ff66b Fixes DNS rebinding vulnerability in router by explicitly passing Host parameter based off config listenaddress (#209) 2019-04-04 17:41:13 +11:00
Andrew
ca55f2f965 Optimisation: Large struct pointer conversion (final part) (#265)
Completes large struct pointer optomisations over the entire codebase and enables hugeParams linter by default
2019-04-04 15:31:49 +11:00
Scott
107cf76373 Kraken websocket support (#264)
* Initial commit. Adds ticker, candle and trade, subscription support

* Adds support for spread and orderbooks

* Adds new currency pair delimiter ("/"), Adds dedicated websocket Connected channel handler, Updates Kraken websocket capability definition, Refines websocket tests to connect and disconnect without freezing, separates WebsocketUnsubscribeEventRequest ChannelID into its own struct WebsocketUnsubscribeByChannelIDEventRequest to prevent bad json WS requests, Adds asset type to orderbook, Kraken WS handles connection better

* Removes duplicate type, reverts config value

* Addresses error returns and changes writeToWebsocket to use byte array. Removes deferred funcs in tests. Increases test listening limit for rare cases

* Fixes verbose log. Rearranges WS Connect async ordering. Fixes DATA RACE. Fixes random okex tests. Ensures Kraken WS tests only connect once
2019-04-04 10:21:44 +11:00
Andrew
a0e291097e Fix CoinbasePro websocket bug introduced in PR #262 (#269) 2019-04-04 10:19:48 +11:00
Vadim Zhuk
866efd069d Fix issues (#263)
OKGroup/Huobi: Fix order history/active orders issues
2019-03-28 13:06:42 +11:00