mirror of
https://github.com/d0zingcat/gocryptotrader.git
synced 2026-05-14 07:26:47 +00:00
34ef09dad655a5db059eb44941a7eb9467f8f447
41 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Scott
|
34ef09dad6 |
bugfixes: Backtester credentials, Binance ExecutionLimits, Mock recording (#1539)
* fix mock recording, binance, backtester * minor credential streamlining |
||
Ryan O'Hara-Reid
|
9657a570dd |
exchanges: shift GetDefaultConfig wrapper function to exchange.go (#1472)
* Shift wrapper function GetDefaultConfig to exchange.Base method definition, to ensure set defaults doesn't get called twice and to reduce code * rm alphapoint bootstrap method as is defined as exchange.Base method * add tests * glorious: make it a function and make it IBOTEXCHANGE --------- Co-authored-by: shazbert <ryan.oharareid@thrasher.io> |
||
Gareth Kirwan
|
52c6b3bf0b |
Websocket: Various refactors and test improvements (#1466)
* Websocket: Remove IsInit and simplify SetProxyAddress IsInit was basically the same as IsConnected. Any time Connect was called both would be set to true. Any time we had a disconnect they'd both be set to false Shutdown() incorrectly didn't setInit(false) SetProxyAddress simplified to only reconnect a connected Websocket. Any other state means it hasn't been Connected, or it's about to reconnect anyway. There's no handling for IsConnecting previously, either, so I've wrapped that behind the main mutex. * Websocket: Expand and Assertify tests * Websocket: Simplify state transistions * Websocket: Simplify Connecting/Connected state * Websocket: Tests and errors for websocket * Websocket: Make WebsocketNotEnabled a real error This allows for testing and avoids the repetition. If each returned error is a error.New() you can never use errors.Is() * Websocket: Add more testable errors * Websocket: Improve GenerateMessageID test Testing just the last id doesn't feel very robust * Websocket: Protect Setup() from races * Websocket: Use atomics instead of mutex This was spurred by looking at the setState call in trafficMonitor and the effect on blocking and efficiency. With the new atomic types in Go 1.19, and the small types in use here, atomics should be safe for our usage. bools should be truly atomic, and uint32 is atomic when the accepted value range is less than one byte/uint8 since that can be written atomicly by concurrent processors. Maybe that's not even a factor any more, however we don't even have to worry enough to check. * Websocket: Fix and simplify traffic monitor trafficMonitor had a check throttle at the end of the for loop to stop it just gobbling the (blocking) trafficAlert channel non-stop. That makes sense, except that nothing is sent to the trafficAlert channel if there's no listener. So that means that it's out by one second on the trafficAlert, because any traffic received during the pause is doesn't try to send a traffic alert. The unstopped timer is deliberately leaked for later GC when shutdown. It won't delay/block anything, and it's a trivial memory leak during an infrequent event. Deliberately Choosing to recreate the timer each time instead of using Stop, drain and reset * Websocket: Split traficMonitor test on behaviours * Websocket: Remove trafficMonitor connected status trafficMonitor does not need to set the connection to be connected. Connect() does that. Anything after that should result in a full shutdown and restart. It can't and shouldn't become connected unexpectedly, and this is most likely a race anyway. Also dropped trafficCheckInterval to 100ms to mitigate races of traffic alerts being buffered for too long. * Websocket: Set disconnected earlier in Shutdown This caused a possible race where state is still connected, but we start to trigger interested actors via ShutdownC and Wait. They may check state and then call Shutdown again, such as trafficMonitor * Websocket: Wait 5s for slow tests to pass traffic draining Keep getting failures upstream on test rigs. Think they can be very contended, so this pushes the boundary right out to 5s |
||
|
Gareth Kirwan
|
d7818ea956 |
Exchanges: Remove bespoke pair upgrade handling and abstract Start/Run (#1424)
* Exchanges: Remove Pair upgrade handling Now redundant behind #1401. These paths should never be met. Several legacy coin upgrade paths being deprecated as well: ZUSD and CNY Expecting any users with bad config from 3+ years ago would have to reset anyway. Also: At the time the intention of this was to upgrade the config format. However now, instead, it'd mostly serve to reset enabled pairs if there's a config mistake, which doesn't feel right. * Kraken: Fix typo in Kraken type struct * Exchanges: Abstract exchange Start() and Run() * Exchanges: Add test for abstracted Start * Exchanges: Move Start to Bootstrap * Simplify waitgroup usage * Add call to exchange.Bootstrap to allow overide or supplementation * Exchanges: Concurrent common bootstap actions * Gateio: Remove incorrect Run in test * GateIO: Fix pair dependencies in tests This ensures that the pairs are initialised no more than needed and kind-of just-in-time. Better pattern might be to use a function to get these pairs when we need them. * Exchanges: Complete UpdatePairs before ExecLims If we're going to update pairs, it needs to complete before we check for limits to avoid errors on old pairs * Exchanges: Remove Start and Run from tmpl Since they're replaced by bootstrap now and shouldn't need customisation normally * Alphapoint: Move Start to Bootstrap * GateIO: Fix linter shadow var |
||
|
Gareth Kirwan
|
e007f69f7c |
exchanges/websocket: Implement subscription configuration (#1394)
* Websockets: Move Subscription to its own package This allows the small type to be imported from both `config` and from `stream` without an import cycle, so we don't have to repeat ourselves * Subs: Renamed Currency to Pair This was being mis-used through much of the code, and since we're already touching everything, we might as well fix it * Websockets: Add Subscription configuration * Binance: Add subscription configuration * Kucoin: Subscription configuration * Simplify GenerateDefaultSubs * Improve TestGenSubs coverage * Test Candle Sub generation * Support Candle intervals * Full responsibility for formatting Channel name on GenerateDefaultSubs OR consumer of Subscribe * Simplify generatePayloads as a result * Fix test coverage of asset types in processMarketSnapshot * Exchanges: Abstract ParallelChanOp * Tests: Generic ws mock instances * Kucoin: Fix intermittent conflict in test currs Use isolated test instance for `TestGetOpenInterest`. `TestGetOpenInterest` would occassionally change pairs before GenerateDefault Subs. |
||
|
Scott
|
b71bf1f3d1 |
exchanges/futures: Implement open interest (#1417)
* adds open interest to exchanges * ADDS TESTING YEAH * New endpoints, BTSE, RPCS, cached * slight design change, begin gateio You will need to get cached for each exchange that supports it * gateio, huobi, rpc * fix up kraken, cache retrieval * okx, gateio * finalising all implementations and tests * definitely my final ever commit on this * Well, well, well * final v2 * quick fix of bug * test coverage, assert notempty, test helper Added a new testhelper for currency management because its very annoying in a parallel test setting which wastes so much space otherwise * minimises REST requests for Open Interest * types.Number merge misses * Minimises Kraken REST calls * len change, value -> pointer receiver * further fixup * fixes gateio, batch calculates open interest * single gateio, lint const fixes * rejig and more thorough oi for huobi * formatting expansion * minor fix for handling expiring contracts * rm unused Binance strings * add bybit support, fix bybit issues * oopsie doopsie, dont look at my whoopsie * Fix issue, remove feature * move an irrelevant function for the pr * mini bybit upgrades * fixes cli request bug |
||
|
Gareth Kirwan
|
32f08f82f4 |
Kucoin: Fix unmarshalling of FuturesTicker resp (#1414)
* Kucoin: Fix unmarshalling of FuturesTicker resp * Fix several string float64 fields not unmarshalling * Fix time field misnamed; Make a time.Time * Use order.Side for sides * Add tests This change opts to expose float64 instead of StringToFloat64 because there's no wrapper to convert the type over, and it's not a websocket. Essentially choosing to not expose internal convert types for methods called directly. Possible we'll eventually harmonise this with a types.Number or something. * Kucoin: FuturesRealTimeTicker => FuturesTicker Kucoin only call it RealTime in the first sentence of their docs. None of the API names, endpoints or any thing mention RealTime. This is just a ticker * Tests: Improve clarity of failed push data * Kucoin: Move PushData to a fixture file This avoids both the linter errors, and the bloat. Loses the name of the "test", but I think that's okay |
||
|
Scott
|
70690d9a04 |
futures: Implement GetLatestFundingRates across exchanges (#1339)
* adds funding rate implementations and improvements * merge fixes x1 * lint * kucoin funding rates func make * migrate sync-manager to keys * some kucoin work * adds some kucoin wrapper funcs * ehhh, todo * kucoin position * start of orders * adds the kucoin tests yay * multiplier * nits, EWS includes order limits * NotYetImplemented, IsPerp improvements, cleaning * lint, test fix, huobi time * fixes issues, improves testing * fixes linters I WRECKED * local lint but remote lint, lint, lint, lint * fixes err * skip CI * lint * Supported rates, binance endpoints * fixes weird mocktest problems * no, CZ is invalid * fixes some new EWS test errors |
||
|
Gareth Kirwan
|
8430509807 |
Okx: Websocket order channel fixes (#1346)
* Okx: Fix WS order fields * Fixes float64 with string annotation erroring on empty strings: Okx Order Push Data error json: invalid use of ,string struct tag, trying to unmarshal "" into float64 Specifically this came from px field from a market order * Switch to convert.StringToFloat64 instead of okxNumericalValue * Fix typo in Notional* field names; Ironically prevented them from erroring * Okx: Add tests for first order fields * Okx: CID and maybe set WS order Filled time * Tests: Set TestFixtureToDataHandler to t.Helper * Orders: Add UnmarshalJSON to order.Side * Okx: Fix FillTime not parsed for PendingOrder * Okx: Switch to order.Side Unmarshal throughout * Okx: Add Fee and FeeAsset to order processing * Okx: Fix WS order.Detail amounts and Test This fixes Amount vs QuoteAmount for market sells where tgtCcy is quote_ccy * Add comment to order.Side.UnmarshalJSON * Okx: Replace PendingOrderItem Unmarshal with local types * Okx: string type for WS order reduceOnly Note: Not yet in unit tests, since it's not part of the spot tests I was originally fixing. I'll circle back to adding full test support for Reduce only and deleveraging positions. * Okx: Fix TestOrderPushData Amount We were expecting 0 when we're given a quoteAmount In reality, we'll calculate the size from the price * Okx: Fix order and remAmount in wsOrders Improved handling for Float64 issues and boundaries when the order is fully executed but not yet marked as Filled * Fix ErrSideIsInvalid in tests |
||
|
Gareth Kirwan
|
d86da76b1f |
Bitstamp: Add auth channel subscription handling (#1333)
* Bitstamp: Add auth channel subscription handling * Bitstamp: Avoid searching for asset type We've hardcoded asset.Spot in order to find the pair. Looking the asset up from the pair makes no sense. * Bitstamp: Add type for wsOrders * Bitstamp: Working test of Generic DataHandler * Bitstamp: WS Order chan tests and remove type orderType could be derived from status == New and Buy & price == 9+e9 or Sell & price == 0 But it would only be true for the first update and it really doesn't feel worth the risk Consumers are going to have to merge to original request anyway * Bitstamp: Linter fixes * Kraken: Switch to shared fixture test * Bitstamp: Fix lint on TestFixtureToDataHandler * Engine: Add Clone for PairsManager go-vet highlighted that the mutex here is a value when we copied the PairsManager in a test. Options to fix: * Add a deep clone method to PairsManager * Add a shallow clone method with a disclaimer * Make the mutex a pointer * Make the PairsManager itself a pointer Options 3 and 4 are too invasive to justify changing at this point. There's an inherent risk of PM being passed by value, but govet should catch the copylock. There's more risk in changing everything to use a pointer at this stage. * Engine: Fix linter again, ironically * Bitstamp: Rename OHLC const * Bitstamp: Minor fixes to syntax * Bitstamp: Simplify chanSymb=>pair * Bitstamp: Still process order updates without ID If there's a ClientOrderID we'll still process the order. It doesn't seem likely we'd have this happen, but if it does we still want consumers to get something. * Bitstamp: Replace Clone with Lock methods * Engine: Expose PairsManager's Mutex Makes more sense than wrapping functions * Bitstamp: Fix linter copylock (again) * fixup! Engine: Expose PairsManager's Mutex Omit Mutex from Json * fixup! Bitstamp: Add auth channel subscription handling Remove unused wsAuthToken * Bitstamp: Simplify OrderData Unmarshal * Bitstamp: Remove unused contexts I added these following best practices, but the reality is that when/if we get context awareness in GCT, there will be a lot more to fix and this will be a drop in the ocean anyway. * Bitstamp: Only call handleWSOrder for MyOrders * Bitstamp: Avoid allocating again in handleWSOrder * CurrencyPairs: Remove public mutex Simplified to a Load method to avoid making mutex public * Tests: Improve test readability and clarity * Bitstamp: Wrap errWSPairParsingError Co-authored-by: Scott <gloriousCode@users.noreply.github.com> * Bitstamp: FetchWSAuth mock and live test --------- Co-authored-by: Scott <gloriousCode@users.noreply.github.com> |
||
|
Scott
|
7f0faf7850 |
futures: add GetFuturesContractDetails wrapper function (#1274)
* all in a days work * cleanup * cleanup for real, also stop it binance.json * minor coverage * adds gateio to the slurry * cleanup of types * verbose verbose verbose verbose verbose verbose * fixes huobi parsing issue * fix bybit contract identification * cleanup * merge fixes * addresses many big problems raised by SHAZ * tracking errors and fixes * funding rate if avail, fixes currency formatting * Addresses nits and sneaks in extra fixes * lint * minor fixes after rebase * better contract splitter for currencies like T-USDT * forgot to add the exchange name like a fool * merge fixes x1 * kucoin, direction, contract size * rn direction, fix kucoin time * WHOOPS * Update exchanges/kucoin/kucoin_wrapper.go Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io> * misdirection --------- Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io> |
||
|
Scott
|
fcc5ad4551 |
exchanges/qa: Add exchange wrapper testing suite (#1159)
* initial concept of a nice validation tester for exchanges * adds some datahandler design * expand testing * more tests and fixes * minor end of day fix for bithumb * fixes implementation issues * more test coverage and improvements, but not sure if i should continue * fix more wrapper implementations * adds error type, more fixes * changes signature, fixes implementations * fixes more wrapper implementations * one more bit * more cleanup * WOW things work? * lintle 1/1337 * mini bump * fixes all linting * neaten * GetOrderInfo+ asset pair fixes+improvements * adds new websocket test * expand ws testing * fix bug, expand tests, improve implementation * code coverage of a lot of new codes * fixes everything * reverts accidental changes * minor fixes from reviewing code * removes Bitfinex cancelBatchOrder implementation * fixes dumb baby typo for babies * mini nit fixes * so many nits to address * addresses all the nits * Titlecase * switcheroo * removes websocket testing for now * fix appveyor, minor test fix * fixes typo, re-kindles killed kode * skip binance wrapper tests when running CI * expired context, huobi okx fixes * kodespull * fix ordering * time fix because why not * fix exmo, others * hopefully this fixes all of my life's problems * last thing today * huobi, more like hypotrophy * golangci-lint, more like mypooroldknee-splint * fix huobi times by removing them * should fix okx currency issues * blocks the application * adds last little contingency for pairs * addresses most nits and new problems * lovely fixed before seeing why okx sucks * fixes issues with okx websocket * the classic receieieivaier * lintle * adds test and fixes existing tests * expands error handling messages during setup * fixes dumb okx bugs introduced * quick fix for lint and exmo * fixes nixes * fix exmo deposit issue * lint * fixes issue with extra asset runs missing * fix surprise race * all the lint and merge fixes * fixes surprise bugs in OKx * fixes issues with times and chains * fixing all the merge stuff * merge fix * rm logs and a panic potential * lovely lint lament * an easy demonstration of scenario, but not of initial purpose * put it in the bin * Revert "put it in the bin" This reverts commit 15c6490f713233d43f10957367fcbf18e3818bdd. * re-add after immediate error popup * fix mini poor test design * okx okay * merge fixes * fixes issues discovered in lovely test * I FORGOT TO COMMIT THIS * nit fixaroonaboo * forgoetten test fix * revert old okx asset intrument work * fixes * revert problems I didnt understand. update bybit * fix merge bugs * test cleanup * further improvements * reshuffle and lint * rm redundant CI_TEST by rm the CI_TEST field that is redundant * path fix * move to its own section, dont run on 32 bit + appveyor * lint * fix lbank * address nits * let it rip * fix failing test time range * niteroo boogaloo * mod tidy, use common.SimpleTimeFormat |
||
|
Ryan O'Hara-Reid
|
981a08af83 |
bybit: assortment of updates (#1192)
* bybit: cherry-pickable * bybit: implement fee fetching v5 * bybit: update to use nullable type * bybit: fix some tests * bybit: spell check fix * remove redunant asset dec, and rm output * rm comment code * linter: fixerinos woooo * bybit: constrict rate limit on public spot to v5 * exchanges/bybit/limits: update (CHERRY PICK ME) * glorious: nIIIIIIIIIIIIIIITS * glorious: nits continued * updated comment * update even more * RM LINE! * glorious: nits * Update exchanges/sharedtestvalues/sharedtestvalues.go Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io> * fix --------- Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io> Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io> |
||
|
Ryan O'Hara-Reid
|
b20cf75d13 |
sharedtestvalues: Add helper functions and implement throughout exchange tests (#1163)
* exchanges/sharedtestvalues: implement new functions to handle test skipping and announcements for standardising. * exchanges: fin test impl. * linter: fixes * exchange_template: fix test * allocate so it doesn't make a panic at the disco * glorious: nits * glorious: nits * Update exchanges/sharedtestvalues/sharedtestvalues.go Co-authored-by: Scott <gloriousCode@users.noreply.github.com> * Update exchanges/sharedtestvalues/sharedtestvalues.go Co-authored-by: Scott <gloriousCode@users.noreply.github.com> * glorious: nits * linter: fix * linter: shhhh --------- Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io> Co-authored-by: Scott <gloriousCode@users.noreply.github.com> |
||
Samuael A
|
969669cb0d |
linters: Bump golangci-lint version to v1.52.2 and fix issues (#1169)
* fix linter issues * linter issue suppressing and fixing * change linter version and linter issues fix * Bump version |
||
|
Ryan O'Hara-Reid
|
7cadc5cffb |
cmd/wrapper_coverage: prevent non-needed REST requests via context timeout (#1154)
* wrapper_coverage/exchanges: cancel context to not send rest requests/ populate context through functions that do rest requests * linter: fix * exchange_template: fix test --------- Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io> |
||
|
Ryan O'Hara-Reid
|
83cfefa45c |
kline/exchanges: automatic creation of unsupported candle intervals (#1091)
* kline: Add builder and testing * Ideas * kline: deploy builder functionality across GCT * exchanges: implement across gct * exchanges: Add tests and fix implementations before kline package testing and veri. * kline: Add tests and start to fix ConvertToNewInterval * kline: fix ConvertToNewInterval add tests * kline: complete overarching tests now on to exchanges * kline: finish exchange tests and implement limits * exchanges: more fixes * linter: fix * engine: fix tests * kraken: fix recent trades and other fixes * zb: fix tests * bithumb: fix empty insertion * kline: refactor/optimize CreateKline function * kline: remove the mooos! * kline: prealloc CalculateCandleDateRanges * linter: fix * exchanges: prealloc extended * fix whoopsie * reverse fix because this is a whoopsie * okx: fix risidual issues * linter: fix * kline: initial nits from @gloriouscode * kline: rename builder -> request and cascade change * linter: fix + test * kline: update forced alignment on start and end times when CreateKlineRequest is called. * nits: more more more * NITS: Addressed * tests: fix race issue * Update exchanges/kline/request.go Co-authored-by: Scott <gloriousCode@users.noreply.github.com> * kline: add method AddPadding() to automatically fill in holes in kline.Request functionality and reject if missing data when converting * kline: Add params start and end to addPadding() to insert blanks in between block * kline: remove test comment code as it's not needed anymore * kline: fix lint and test * kline: sort slice without extra bool check every iteration * okx: fix issues with timeing and candles and such from niterinos & address typo * Update exchanges/kline/kline.go Co-authored-by: Scott <gloriousCode@users.noreply.github.com> * glorious: niterinos * Update exchanges/poloniex/poloniex_wrapper.go Co-authored-by: Scott <gloriousCode@users.noreply.github.com> * glorious: nits now onto conflicts YAYA!!! * Update exchanges/exchange_test.go Co-authored-by: Scott <gloriousCode@users.noreply.github.com> * glorious: nits again * thrasher: nitters * thrasher: niterinos - adds partial flag for incomplete recent candles and fetching. * kline: rm fmtizzle packageizzle * glorious: nitters * glorious: more niterinos * fix last niterinos Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io> Co-authored-by: Scott <gloriousCode@users.noreply.github.com> |
||
|
Ryan O'Hara-Reid
|
e4d487e586 |
exchanges: Change wrapper function FetchTradablePairs to return currency.Pairs (#1093)
* exchanges: FetchTradablePairs currency pair slice return (1) * exchanges: finish conversion * thrasher: nits + linter * exchanges: shift var dec into for loop * exchanges: Apply changes thanks @thrasher- * Update exchanges/alphapoint/alphapoint_wrapper.go Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io> * thrasher: nitters * Update exchanges/bitflyer/bitflyer_wrapper.go Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io> * thrasher: fix more nitters Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io> Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io> |
||
|
Samuael A
|
2ac165a477 |
exchanges: Add OKX support (#1005)
* public endpoints methods added * Completing mapping REST endpoints * Binanceus Wrapper methods -Partially * Binanceus Wrapper methods -Partially * BinaWra functions with test funs; Not Completed * Finalizing wrapper methods & test * Fix & Complete wrapper functions * Finalizing wrapper methods & test * Adding Stream Datas * WS Test functions * CI: Fix golangci-lint linter issues * CI: Fix reverting unnessesary changes and type conversion issues * CI: Fix reverting unnessesary changes and type conversion issues * Adding Public endpoints and tests * Adding Market and Public Endpoints * Adding Public endoints * Public Trading Endpoints & Authenticated Trade order methods * Adding Authenticated Methods and Tests * Adding algo and Funding Authenticated endpoints * Adding funding trading endpoints and correspondint tests * adding authenticated endpoints * Completing Block Trading endpoints and added subaccount endpoints * Completing sub account and grid Trading endpoints * Adding Rate Limit and missing endpoints * Wrapper and Websocket handlers * Fixing Websocket Test and Push Data Handler Issues * Fixing Websocket Test and Push Data Handler Issues * Fixing linter issues, package dependency, and other slight tempos * Fixing linter and slight tempos * Update on test functions, and Rest and Websocket Endpoint handlers * Remove okex, adding comments, and slight fixes on endpoints. * Fixing linter issues and adding comments * Slight code changes, updating documentation, and n and linter issues * Fix context and configuration endpoint issues * slight fixes on config and test files * adding some missing test and fix linter issues * fix linter issue * Slight fixes on code structure, shorthand exp,and ot and other * Fix slight linter issue * Slight code fixes and fixing linter issues * fixing linter iissues * fixing linter iissues * slight linter issue fix * slight linter issue fix * Fix on models, type convert funcs and endpoints * Adding Error messages map and update of models * Fix on error message string and linter issues * Fix slight linter issue * Fix slight linter issue * Fixing type converts, models, and linter issues * Adding Ws fixes * Slight fix on websocket and other issues * Adding slight websocket fixes * Remove 'books5' channel default subscription * Small changes on default subscription and checksum * Fix slight websocket tempos * Fix Wrapper function tempost and linter issues * Resolving slight naming and other issues * Resolve slight pointer issues * resolve slight linter issues * Resolve config files issue * Update websocket and wrapper funcs with test and docs * fixs on websocket multiplexer, types, and other slight issues * fix slight linter issues * slight update on web-socket orderbook and tickers * fix slight issues and websocket runtime errors * Slight unit test fix and assing simple semaphore * FIx race issue * Update on authenticated endpoints * Fix wsSetupRun check in websocket 'setupWsAuth' func * Update wsSetupRun check in websocket 'setupWsAuth' func * Slight update on websocket handling * Fix some race conditions * fix slight tempos * fix authenticated test issues * Update on conditional statements * slight update on unit test * fix unit test tempos * Fix slight tempos * Change check map from struct valued to bool valued * slight fix trial * Slight unit test update * Fix websocket timeout error * Updating websocket subscription endpoints, and unit tests * update unit tests * Slight issue on wrapper method 'GetActiveOrders' * Overall code update * Addressing missing review comments * Fix unit test tempo and linter issue * Monor fix * Slight update * Slight unit test fix * Slight fixes * Slight fixes * Fixing on missing review comments * Adding WS Fixes * slight fix * Monor fix on unit test * Minor convert issue * Minor change on WS * Monor logic fix * Fix code structure and logic issues * Fixing small typos * fix slight data format issue * Update on trade and order wrapper methods * Adding slight update * fix on order detail * Slight update on FetchTradablePairs wrapper method * Slight update on wrapper * Update on deserialization and other slight issues * Final update * Resolve missing review comments * Slight update on config and unit test * minor fix on GetDepositAddress param * Minor fix |
||
|
Ryan O'Hara-Reid
|
558a70e87f |
orders: add Filter() method to GetOrdersRequest and force FilteredOrders return type on wrapper functions (#1055)
* orders: filter options return on validate * exchanges: force filter usage by wrapper return type and shift method functionality * exchanges: update tests and fix err shadow * exchanges: shadowland * exchanges: more shadow land spookyness * glorious: nits and ftx upgrade * linter: fix * orders: preserve unpopulated fields through filtering operation * glorious: nits Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io> |
||
Adrian Gallagher
|
68588560e3 |
CI: Bump go version, linters and fix minor issues (#1010)
* Bump golang, golangci-lint versions and fix issues * Add -fno-stack-protector * Fix AppVeyor golangci-lint ver * Nitters * Nitters round 2 |
||
Jaydeep Rajpurohit
|
247da918a8 |
exchanges: Add ByBit support (#887)
* few fixes and add ratelimiter * adds test * revert configtest.json changes * configtest updated * WIP: adds public endpoint support * WIP: adds public endpoint support * adds public endpoint support * WIP: adds auth. endpoint support * adds test for auth. endpoint * fixes * adds auth. endpoint support * WIP: ws support * WIP * WIP * WIP * WIP * WIP * WIP * WIP * Testing * Complete WS spot testing * adds support for ws events * minor change * WIP: adds REST support for CoinMarginedFutures * Fixes * WIP: adds REST support for CoinMarginedFutures * Fixes * improvement in SPOT REST * Typo fix * WIP: add REST support for CMF Account API * minor fixes * WIP: add support for CMF conditional orders and few minor fixes * complete support for CMF conditional orders * adds support for public CMF endpoint * adds support for CMF position API * Complete REST CMF support * WIP * Testing REST CMF support * Testing REST CMF support * Testing REST CMF support completed * WIP: add support for UMF * completed non-auth UMF * WIP: add support for REST Auth. UMF * WIP: add support for REST Auth. UMF and some improvements * WIP * WIP * WIP * completed REST UMF * renaming * adds REST support for futures * add testcases for UMF and some optimizations * add testcases for futures * Testing UMF, futures and its changes * Fixes * Fixes after testing * WIP * WIP * WIP * completed ws USDT futures support * WIP: ws support for futures * fixes in WS futures * fixes in WS support * roll back changes made for WS CMF, USDT and Futures * fixes * WIP * WIP * fixes * Steps for new PR * WIP * WIP * WIP * WIP * complete PR setup * fixes for successfully running tests * update in symbol for futures pair in test file * WIP * Fixes in test file and other minor fix * fix testdata/configtest.json * reset CONTRIBUTORS file * review changes * remove unwanted file * remove redundant code * improvisation * adds comment for exported functions * remove unwanted TODO and commented code * fix * improvisation * fix * defined errors * improvisation * improvisation * improvisation * updates test * adds comment for exported types * review changes * review changes * fix * fixes * Changes for making BYBIT compatible with existing code base * Test file changes * Changes for making BYBIT compatible with existing code base * Changes for making BYBIT compatible with existing code base * fix lint issues * fix * review changes * review changes * review changes * review changes * review changes * review changes * review changes * review changes * review changes * review changes * WIP * add test cases for new API's * minor improvements * add missing API and their tests * minor fixes * add bybitTime * add bybitTimeSec, bybitTimeMilliSec, bybitTimeNanoSec and necessary support * fix GetTradeHistory function * error handling * test fixes * add GetServerTime API * adds GetHistoricCandlesExtended and review changes * test fixes * minor fix * integrating CMF Bybit recent change log * minor fixes * adds extractCurrencyPair * minor fixes * minor fix * review changes * adds variable declaration of error * review commit * adds embeddable type in API response for all API and integrate it * fixes * adds authentication WS connection * review changes * review changes * compatible changes * adds asset to GetWithdrawalsHistory * adds asset_type in rpc.proto * adds asset argument in gctcli withdrawal request command * improve error handling in exchange API error * web socket fix * review changes * improvements * improvements * minor fix * review changes * fixing wrapper issues * fixes * fixes * review changes * add test cases * fix for GetActiveOrders * lint fixes * fixes in websocket * adds wrapper testcases * adds wrapper testcases * adds wrapper testcases * fixes * fix issue with GetHistoricCandlesExtended * fix merge issues * improving error reporting * adds wrapper testcases and a minor fix * gctrpc changes * adds test cases fixes in websocket * review changes for ws * review changes in WS * fix gctrpc * merge fixes * review changes * WIP * updates pair in configs * adds new asset USDCMarginedFutures * adds URL const for USDCMarginedFutures * adds API support * minor fixes * adds kline API * minor fix * adds API * adds API * adds API * WIP * WIP * WIP * adds support for USDC auth requests to SendAuthHTTPRequest * adds SendUSDCAuthHTTPRequest * run test and fix them * rollback support added for Auth. USDC request inside SendAuthHTTPRequest * adds API and test cases * adds API and test cases * adds APIs and test cases * adds APIs * adds rate limit for USDC * adds USDCMarginedFutures to wrapper * adds USDC testcases in wrapper and fix few issues * minor test fixes * minor test fixes * fix lint issues * WIP * Merge changes * minor fixes * remove "else" and optimize * review changes * review changes * review changes * fix lint issue * merge fix * fix test * fix templates and run them * changes after merge * review changes and improvements * code improvement * fixes with respect to changes in API response in documentation * fixed review change in test * adds check in CancelExistingOrder * update exchange template * review changes * adds GetDepositAddress API * WIP: adds GetOrderHistory * complete GetOrderHistory * fixes * adds test case * fixes and add WithdrawFund API * WIP * WIP * updating all SendAuthHTTPRequest call * adds WithdrawCryptocurrencyFunds * update test cases * fix lint issues * fixes after merge * adds GetAvailableTransferChains and few fixes * minor fix in GetDepositAddress * minor fix with WS ping/pong handling * add ping handler for WS Auth. * fix typo mistake * update doc |
||
|
Ryan O'Hara-Reid
|
a12cd3d733 |
orders: Add methods to derive SubmitResponse and Detail types (#955)
* orders: deprecate SubmitResponse return and change to *order.Detail construct detail from order.Submit struct * orders: add coverage, fix tests * coinut: rm test for checking * orders: revert change for return and change field ID to a more explicit name OrderID * orders: Add method to see if the order was placed * order: change field name in Cancel type to be more explicit * orders: standardize field -> OrderID * backtester: populate change * orders: add test * gctscript: fix field name * linter: fix issues * linter: more fixes * linter: forever * exchanges_tests: populate order.Submit field exchange name * Update exchanges/order/order_types.go Co-authored-by: Scott <gloriousCode@users.noreply.github.com> * Update exchanges/order/orders.go Co-authored-by: Scott <gloriousCode@users.noreply.github.com> * glorious: nits * glorious: nits * thrasher: nits Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io> Co-authored-by: Scott <gloriousCode@users.noreply.github.com> |
||
|
Ryan O'Hara-Reid
|
a63aa6b616 |
orders: Add derive modify struct method from order.Detail (#948)
* orders: Add derive modify struct method to order.Detail and then subsequent method to derive and standardize response details * exchanges: call modify method in wrappers * linter: fixes * engine/wsroutineman: remove print summary * glorious: nits, removed modifyOrder functionality for Bithumb. There are not docs to support this. * Update exchanges/order/orders.go Co-authored-by: Scott <gloriousCode@users.noreply.github.com> * glorious: nits Co-authored-by: Scott <gloriousCode@users.noreply.github.com> |
||
|
Ryan O'Hara-Reid
|
293d6104ed |
orders: adds method to retrieve snapshot of order execution limit values (#946)
* orders: add method to Limit to retrieve order execution limit snapshots * currency/btcmarkets: add error and update field name to standard * linter: fix * limts: don't return pointer * limit: Add notes * glorious: nits * linter: fix * limit: reinstate nil check * exchanges: change field names to be more consistent (@thrasher-) suggestion Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io> |
||
|
Ryan O'Hara-Reid
|
14cde7b786 |
exchange: expose GetCredentials() and split GetAuthenticatedAPISupport() (#954)
* exchange/wrapper: expose GetCredentials func to IBotInterface * exchanges: split up GetAuthenticatedAPISupport into specific function calls, organize IBotExchange functionality getter functions * interface: change name - RPCSercer: rm GetBase func call. * glorious: nits (fix panic) Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io> |
||
|
Ryan O'Hara-Reid
|
16a93b49a4 |
btcm: add modify order functionality, change return to pointer (#940)
* btcm: add modify order functionality, change return to pointer * glorious: nits * glorious: nits * btcm: Adjust function name * thrasher: nits * thrasher: nits cont... |
||
|
Ryan O'Hara-Reid
|
6127e2ab73 |
Request: Fix http.Client race issue when setting transport layer proxy and timeouts (#885)
* backtester/request: trying to fix panic (WIP) * request: fix race for transport layer * request: linter issue fix * request: more linter issues * requester: Add function to remove the tracking of underlying http client and add to engine unload exchange. * request: add more context to error return * request: Fix after cherry pick issues * request: fix niterinos * exchanges: change return to package variable * request: changed named Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io> |
||
|
Ryan O'Hara-Reid
|
da3402476e |
tool/exchange_wrapper_coverage: fix regression and implement reflection (#837)
* cmd/tools/exchange: fix regression and implement reflection so as this can dynamically scale to our interface * exchanges: add comment and fix whoopsie * exchanges: fix linter issues * wrapper_cov_tool: add actual full interface method count to get a better perceived deployment * exchanges/tool: addr glorious nits * kraken: remove string in test * exchange_template_tool: fix tmpl issue |
||
|
Ryan O'Hara-Reid
|
099ffa1a60 |
stream/websocket: Consolidate fields by using exchange config pointer (#809)
* stream: add exchange config pointer to setup WebsocketSetup struct to reduce and consolidate setting of variables. * config: reduce stutter * config: reduce minor stutter * glorious: nits addr. * Update exchanges/stream/websocket.go Co-authored-by: Scott <gloriousCode@users.noreply.github.com> * websocket: implement fix * engine/helpers: fix test * exchanges: fix after merge issues * exchange_template: fix output Co-authored-by: Scott <gloriousCode@users.noreply.github.com> |
||
|
Adrian Gallagher
|
0c00b7e1df |
exchanges/engine: Add multichain deposit/withdrawal support (#794)
* Add exchange multichain support * Start tidying up * Add multichain transfer support for Bitfinex and fix poloniex bug * Add Coinbene multichain support * Start adjusting the deposit address manager * Fix deposit tests and further enhancements * Cleanup * Add bypass flag, expand tests plus error coverage for Huobi Adjust helpers * Address nitterinos * BFX wd changes * Address nitterinos * Minor fixes rebasing on master * Fix BFX acceptableMethods test * Add some TO-DOs for 2 tests WRT races * Fix acceptableMethods test round 2 * Address nitterinos |
||
|
Ryan O'Hara-Reid
|
d636049fb2 |
exchanges: Initial context propagation (#744)
* gct: phase one context awareness pass * exchanges: context propagation pass * common/requester: force context requirement * gctcli/exchanges: linter fix * rpcserver: fix test using dummy rpc server * backtester: fix comments * grpc: add correct cancel and timeout for commands * rpcserver_test: add comment on dummy server * common: deprecated SendHTTPGetRequest * linter: fix * linter: turn on no context check * apichecker: fix context linter issue * binance: use param context * common: remove checks as this gets executed before main * common: change mutex to RW as clients can be used by multiple go routines. * common: remove init and JIT default client. Unexport global variables and add protection. * common: Add comments * bithumb: after dinner mints fix |
||
Luis Rascão
|
c9ab0b1164 |
exchanges: Add UpdateTickers method and reserve UpdateTicker for single ticker symbol update requests (if supported) (#764)
* exchanges: add an UpdateTickers method to the main exchange interface This method will fetch all currency pair tickers of a given asset type and update them internally, does nothing for now. * exchanges: refactor UpdateTicker on all exchanges Keep the exact previous behaviour but implement the UpdateTickers method and refactor UpdateTicker by using it where applicable. * sync_manager: update all tickers when batching is enabled * binance: UpdateTicker to fetch single ticker symbol * ftx: UpdateTicker to fetch single ticker symbol |
||
|
Luis Rascão
|
96669e29cf |
engine/exchange manager: Add support for custom exchanges (#749)
* engine: Setup exchange manager earlier So it's available for applications before starting. * engine: Add a custom exchange builder interface Allows applications that import GCT as a lib to build new custom exchanges without further modifications. |
||
Adam
|
504c2fad6d |
Feature: Implement funding rates, futures and coin margin (exchange API coverage) (#530)
* ALMOST THERE * more api wips * more api thingz * testing n more api wipz * more apiz * more wips * what is goin on * more wips * whip n testing * testing * testing no keys * remove log * kraken is broken ugh * still broken * fixing auth funcs + usdtm api docs * wip * api stuffs * whip * more wips * whip * more wip * api wip n testing * wip * wip * unsaved * wip n testing * wip * wip * wip * wip * wip * wip * wip * wip * wip * whip * wrapper authenticated functions * adding asset type and fixing dependencies * wip * binance auth wrapper start * wrapper functionality * wip * wip * wip * wrapper cancel functions * order submission for wrappers * wip * more error fixing and nits * websocket beginning n error fix * wip * WOW * glorious n shazzy nits * useless nits * wip * fixing things * merge stuffs * crapveyor * crapveyor rebuild * probably broke more things than he fixed * rm lns n other thangs * hope * please * stop it * done * ofcourse * rm vb * fix lbank * appveyor please * float lev * DONT ASK RYAN FOR HELP EVER * wip * wip * endpoint upgrades continued * path upgrade * NeeeNeeeNeeeNeeeNING * fix stuffs * fixing time issue * fixing broken funcs * glorious nits * shaz changes * fixing errors for fundmon * more error fixing for fundmon * test running past 30s * basic changes * THX AGAIN SHAZBERT * path system upgrade * config upgrade * unsaved stuffs * broken wip config upgrade * path system upgrade contd. * path system upgrade contd * path upgrade ready for review * testing verbose removed * linter stuffs * appveyor stuffs * appveyor stuff * fixed? * bugfix * wip * broken stuff * fix test * wierd hack fix * appveyor pls stop * error found * more useless nits * bitmex err * broken wip * broken wip path upgrade change to uint32 * changed url lookups to uint * WOW * ready4review * config fixed HOPEFULLY * config fix and glorious changes * efficient way of getting orders and open orders * binance wrapper logic fixing * testing, adding tests and fixing lot of errrrrs * merge master * appveyor stuffs * appveyor stuffs * fmt * test * octalLiteral issue fix? * octalLiteral fix? * rm vb * prnt ln to restart * adding testz * test fixzzz * READY FOR REVIEW * Actually ready now * FORMATTING * addressing shazzy n glorious nits * crapveyor * rm vb * small change * fixing err * shazbert nits * review changes * requested changes * more requested changes * noo * last nit fixes * restart appveyor * improving test cov * Update .golangci.yml * shazbert changes * moving pair formatting * format pair update wip * path upgrade complete * error fix * appveyor linters * more linters * remove testexch * more formatting changes * changes * shazbert changes * checking older requested changes to ensure completion * wip * fixing broken code * error fix * all fixed * additional changes * more changes * remove commented code * ftx margin api * appveyor fixes * more appveyor issues + test addition * more appveyor issues + test addition * remove unnecessary * testing * testing, fixing okex api, error fix * git merge fix * go sum * glorious changes and error fix * rm vb * more glorious changes and go mod tidy * fixed now * okex testing upgrade * old config migration and batch fetching fix * added test * glorious requested changes WIP * tested and fixed * go fmted * go fmt and test fix * additional funcs and tests for fundingRates * OKEX tested and fixed * appveyor fixes * ineff assign * 1 glorious change * error fix * typo * shazbert changes * glorious code changes and path fixing huobi WIP * adding assetType to accountinfo functions * fixing panic * panic fix and updating account info wrappers WIP * updateaccountinfo updated * testing WIP binance USDT n Coin Margined and Kraken Futures * auth functions tested and fixed * added test * config reverted * shazbert and glorious changes * shazbert and glorious changes * latest changes and portfolio update * go fmt change: * remove commented codes * improved error checking * index out of range fix * rm ln * critical nit * glorious latest changes * appveyor changes * shazbert change * easier readability * latest glorious changes * shadow dec * assetstore updated * last change * another last change * merge changes * go mod tidy * thrasher requested changes wip * improving struct layouts * appveyor go fmt * remove unnecessary code * shazbert changes * small change * oopsie * tidy * configtest reverted * error fix * oopsie * for what * test patch fix * insecurities * fixing tests * fix config |
||
|
Ryan O'Hara-Reid
|
14c72c9c6b |
Currency System Update (#448)
* initial update of currency system * WIP progress * Finish initial currency string error returns * fix whoopsie testing for non https insecureinos * Current WIP for getEnabledPairs check and error return * WIP continued * When getting enabled pairs throw error when item is not contained in available pairs list * More updates -WIP * Wip continued including potential interface * Current WIP * pairs manager pass * drop asset string and just use the map key, plus return some errors and create more work for myself. * clean and fixed a bug in currency.json, will not populate correctly without coinmarketcap api keys set. * purge logger references after merge * go mod tidy after merge * Pointer change WIP * fix some issues and added error returns to a few items (WIP) * WIP * Clean * Fix some linter issues * Fix more linter issues * even more linters * xtda nits * revert pointer change and rm field * Addr madcozbadd nits * fix linter issues: shadow declarations * Fix linter issues: gocritic huge things * linter issue fix * Addr nits * flush go mod files * after merge woops * fix shadow dec * Addr thrasher nits * addr nits * fix some issues * more fixes * RM println * Addr glorious nits * Add helper method for setting assets * add missing format directive * Addr nits * Actually process new futures contracts -_- derp * WIP for GRPC upgrade for pair management * update config pairs * finished disabling and enabling asset * linked update of tradable pairs to cli * fix oopsies * defer writing to file on program termination for currency storage system * update template * don't add disabled asset items to initial sync * Fix enable disabling a list of pairs and added in a slice error type so we can add whats allowable without throwing an error and return a report, also addressed some other nits * WIP on getting a channel to unsub * Wip track down unsubscribe bug and start creating streaming interface * purge websocket orderbook object and centralised updating routine for orderbook * general clean before interface implementation * stage one connection interface WIP * WIP * repackage wshandler WIP * find difference of subs and change signature of subscriber functions so we can batch subscriptions and unsubscription in exchanges that support it * design change on mange subscription routine WIP * integrated ZB with the new webosocket updates * WIP - okex conversion * integrate websocket upgrades for lakebtc, kraken, huobi, hitbtc, gateio, and WIP for coinbene * integrate another range of exchanges for websocket update * Added subscriber and unsubscriber methods to websocket functionality * fixed tests WIP * amalgamate cache setup with main websocket setup * reinstate exported fields traffic alert and shutdownC to accommodate gemini and lakebtc implementations * added in colon * Updated websocket auth handling as they werent getting passed through. Added a setter method for websocket URL due to the Binance generated auth key/listenKey. Fixed bug which stopped reconnection. * Fix subscribe candle bug Fix time conversion in candle Fix inititial candle history to datahandler Include funding to orderbook handling Include funding to trades Reduce code duplication in sub and unsub functions Added the ability to include funding currency websocket subscriptions validated all channels and added more items todo list (Auth items) * RM line * bitstamp pass * btcmarkets pass - still needs to implement unsubscriber functionality and pairs change test. * Batch outgoing subscriptions and fix unsubscribe bug * BTSE - bumped time to minute to reduce pinger calling by 75 calls per day. Fix authentication bug and add authentication pass into to-do. Batch outgoing subscription calls * fix type field and batch outgoing subs and unsubs for coinbasepro * Batch outgoing subs and unsubs * Fixes bug when matching return from authentication * Fix bug where params where being sent out of order due to map ,where depth items werent being subscribed too, where trying to subscribe to too many kline items caused error, where trying to get a nano secocond ID conflicted due to speed of generation. * Add websocket capability for currency pair change by utilizing full channel subscription list in subscribe function. * Add error handling * Fix public: time conversions, subscription list, stopped pushing heartbeat to data handler, aggregated list of connections. * hitBTC pass * returned nil instead of error due to period null bids and asks updates coming through. * Fix auth ping capture and reply. Added in interval handling for kline data. Added correct full trade data handling. Fix subscribe and unsubscribe. * Fix when websocket auth conn and token generation fail we don't try and auth sub. Fix bug between auth and normal connection id generation and matching. Batch outgoing payloads to increase efficiency. Updated matching functions to utilise channels instead of waitgroups and go routines. * RM debug output * rm func to get shutdown channel * Add unsubscriber functionality, added wsTicker type, removed return as this will impede data flow and cause reconnection when handling and processing data * okgroup WIP * *Added missing fields for websocket trades *Fix bug processing kline interval *Added fields for websocket ticker struct *Fix auth bug -Updated request and response matching param to interface so we can custom signature match. Stops auth subscribing before a reply is issued. -Updated channel inclusion of pair fo auth subs as this was missing. *Assortment of perfomance improvements * poloniex pass * send all trades to data handler, validated enabled and disable pairs * initial clean * centralised request matching mechanism * websocket main improvements WIP * WIP * Websocket management via gctcli WIP * GRPC expansion * Updated GCTCLI with websocket url and proxy setting functionality which flushes connection * Fix continuous spawning of routines bug on error with reconnection * Addr linter issues * fix subscription bug that I caused when I changed to a switch case * fix linter issue * fix woopsie * End of day WIP * Fix order submission REST, time conversion, order type conversion, orderID bugs * fix gateio test and unsubscribe bug * revert comment out code * websocketAPI changed to to true in configtest.json * fix race in gateio test * End of day WIP for websocket tests. * BugFix for binance when book isn't seeded. Updated websocket tests. Deprecated subscription manager. RM wrapper funcs. * Added string title to exchange name as they are saved as lower case in type, reinstated verbose check in websocket.go * Added verbosity check for setting websocket URL * fix bug where the asset had a mind of its own * purge dodgy coding * Fix tests, drop blocking chan in websocket Dial function * few more changes * race condition fix for websocket tests. * fix intermittent test failure due to underlying hash table storage * Address madcuzbad nit * RM superfluous printlines * Add quick top example with paramater fields * First pass Glorious nits * As per madcozbad suggestion return error when enabled pair not found in full return map. Add test. * addr madcozbadd nits * as per glorious suggestion rm'd loadedJSON field * adjusted ticker, added test and RM'd code that can never be executed * Addr nits and add in crypto rand genration for ID's * remove global channel declaration and rescoped as this was causing a lock * as per glorious suggestion restructured return error for websocket * addr glorious suggestions * fix linter issues * purge non-existent pair from testdata * add side field to struct and parse * addr glorious nits * Add verbosity to error returns and logs and fix string parsing in GCTRPC * fix speeling mistwake * Adds websocket functionality check before flushing websocket connection * Addr kraken panic and setting/flush websocket url stage one. * added websocket url check before setting with tests * Added in edge case test if by the time we call contains on available pairs it has been changed * remove error return for func * Continuation of tests * continuation of tests * Stop potential panic within pair creation function * Implement changes to upstream * rm sup comment * fix bug when subscribing and unsubscribing. Also add in boolean to determine there are currencies that need to be flushed via set pairs via gctcli * fix test * Fix linter issues * Fix tests * turn websocket off in config example * Fix issue where you cannot enable websocket when config is set to false, also added config websocket enable state saving * Introduced err var for same error returns * Add err var exchange base not found * restructure function * drop gctscript from generic response name * drop managesub delay const as its not being used * correctly implement websocket rate limiting for coinut * remove quotations * drop pair management check * fix spelling * return error in function to not update currency with unset role * amalagamted enable/disable into set function and added in pairstore fetch function * update error description * rm function * moved test function to sharedtestvals and move type to types.go * append delimiter onto currency delimiter strings * add test coverage * rm functions as they are set as methods in base * remove superfluous methods * Fix issue that would occur when a subscription errored and not appending successful subs * fix after rebase woopsie * fix linter issues * fix bug streamline code * fix linter issues * fix linter issues * fix case where it should not change ID if set but append new * fix whoopsie * fix websocket tests * fix readme, fix wrapper issues reporting template, go mod tidy * add test coverage * add test coverage and verified futures pariing * add in futures bypass as its not currently supported on BTSE until API update and implementation * removed downside/upside profit contract type as its no longer supported. Added in check in set config pairs to warn user of potential conflict and to manually remove or update. * If asset enabled add pair and increase code coverage * remove strings.title, set and fetch with strings.Lower but keep struct field exchangename unchanged. Streamline ticker and orderbook code. * Add code coverage * log error if setting default currency fails, add code coverage * address glorious nits * Addr xtda nits * fix linter issues * addr glorious nits * xtda nits * Addr glorious nits * add subscription protection and removed a superfluous wait call * fix test * fix whoopsie * addr xtda nits * addr glorious nits * Added asset types to subscriptions structs, also added in error handling for resubscription errors * consolidated rpc returned type and added in sucessful strings * dropped stream timing down to 100ms * DOC changes * proxy and url usage string additions * WIP * go mod tidy rides again * Addr nits * Addr nits, fix tests * fix wording * add in test case for currency matching * Add byte length check on outbound websocket payload subscriptions * addr thrasher nits * Addr madcozbadd nits * addr linter issues * Addr glorious nits by amalgamating function into one mega amazing function. * fix futures account subscription bug * addr glorious nits and reinstated wg.Wait() checks * changed string to currency delimiter string and setconnected by function |
||
|
Scott
|
b686cf2e0e |
Feature: Websocket order handling (#446)
* Initial changes, removing exchange name as an arg and puts it in the pointer struct. Adds case to ws routines * Adds CancelAllOrders func, adds GetByExchangeAndID. Adds modify handler in routines.go * initial poor attempts to have bitmex work with new datahandler handlers. fixes ordersides * bitmex Completes new order * Better bitmex handling, but not complete. Begins a gargantuan task of unifying order data structs. Sometimes an order update will contain lot's of information, so its best to be able to update all fields of our orders, rather than just an arbitrary subset. As a result, everything will be broken for the foreseeable future :glitch_crab: * Removes old order handler which did nothing. Updates order properties for everything everywhere - now consistent. Changes order status. Adds asset type and wallet address to all order types * Adds order updater to update only relevant fields since the object is generic, we don't know what fields are passed from what exchanges. Adds "lastupdated" field to order.Detail. Expands order cancellation for engine orders. * Ensures that new orders are added to the ordermanager's order store. Saaa many comments. Internalises orderStore get func. Adds internalOrderID to orderdetail and adds websocket support for it * Fixes a cancelAllOrders oopsie doopsie * Adds potential func to update orderdetails from an orderdetail struct. Unsure if will keep. * Begins btcmarkets implementation. Expands order "stringToOrder" funcs to allow for some more flexible string coversions. Removes order.Submit via websocket as it would cause unlimited order place issues :D * Finishes btc markets without testing * Adds untested ws auth func to btse * Finises btse, fixes btcmarkets bug * Adds coinbasepro support * Fixes a few more fields in coinbase pro and readds the extra subs * Begins work on coinbene. Plus theyve added a new ws connection yeee * Wasted a bunch of time adding support to an additional websocket that isn't needed ;_; Fixed a bug in coinbasepro. Fully kitted out coinbene support. Updates order types with all fields * Removes extra websocket connection ;_; * Finishes gemini. Fixes order side unknown * Adds okgroup support. Moves byte reading to another function to allow for unit testing. Updates routines to use pointers. Updates date update handling for order details * Finishes order data for okgroup websocket, but starts the STRANGE process of converting all other websocket endpoints to be a little less silly * Cleans up okroup websocket implementation. Fixes bug in Gemini * Adds poloniex support. Updates ws order handling * new bitmex support. Adds some tests now that its all in its own func. Fixes poloniex bug * Begins work on authenticated binance websocket * Attempts to track user data via binance websocket * Maybe finishes Binance websocket support * Begins adding test coverage to orders.go. Updates names of script properties to match updated * Begins an experiment with code coverage. Fixes more rebase issues * Completes orders coverage. Botches a few other things though. Fixes more scripting stuff * All tests in engine package pass * Adds some loevely routine tests * Moves ordermanager to test Bot ordermanager Adds lovely routine tests to ensure things that get sent to be handled the data handler are handled by the data handler by handling them * Replaces "wsHandleData" with "wsReadData" as that's what its going to do now. * Splits all wsHandleData into wsReadData and wsHandleData to allow for easy testing via sending []byte json examples to test proper functionality. Breaks so many tests * Fixes majority of test issues. But data races which are tough on the engine package * "Fixes" test by removing shutdown test. It interferes with too many things. Requires some thought * Tests all the binance websocket points * Adds better bitfinex websocket support. * Adds testing for bitfinex, bitstamp and btcmarkets. Fixes websocket bugs encountered * Adds BTSE ws tests. Fixes bugs in ws * Adds coinbase pro tests. Fixes any issues * Coinbene tests * Starts to handle coinut. Runs into a problem conceptually regarding websocket roundtrip and orders. Both events need to happen without impacting eachother/racing * Addresses a data race issue regarding websocket and bot order management submission - order submission locks at an earlier point to prevent routines.go from creating an order before order submission creates it. Updates rpcserver to use order management bot to submit orders. * Finishes the hectic coinut testing * Adds tests for gateio * Fixes rebase issues. Updates tests to work without being overloaded * Begins testing of gemini. fixes up minor issues * ginishes gemini tests and fixes * Adds hitbtc tests. Fixes all the many issues with hitbtc websocket * Adds remaining tests. Increases default test channel limit again * Begins work towards huobi tests * Finishes huobi tests * Fixed all mythical rebase adventures * Begins kraken transformation * Finishes kraken. Fixes coinbene leverage now that its changed * Begins okgroup testing * Adds okgroup ws tests * Does some poloniex * Fixes basic curreny issue by extracting to func * Begins redesign of poloniex websocket datahandling. Completes authenticated handling, now onto unauth * Finishes poloniex revision * Finishes ZB additions * Fixes data races * Fixes rebase issues. Fixes bad kraken logic * Fixes after reviewing code * lint everywhere * Fixes lingering lints * lint * Adds test coverage to order detail and modify updating * Fixes linting * Fixes huge int, fixes date tests * Adds GetByExchange, adds test for it. Protects fakepass echange. Renames DisplayQty to DisplayQuantity. Removes verbose. Adds some websocket properties. Updates bitmex asset type in test * Addresses timestamps, type abbreviations, verbosity. Expands binance kline switch cases. Updates some websocket capabilities. * Adds coverage to the stringToOrderType/Status functions introduced in PR * Minor fixes addressing some time, error text and use of StringDataCompareInsensitive * Introduces shiny new system which checks if there is an awaiting ID, if found, processes via wrapper method, else, goes through wsHandleData method. Removes weird locking system from wrapper/websocket data race. Updates bitfinex to properly handle websocket order requests and notifications * Moves fakePassingExchange to test_helper. Fixes some order side implementations for trades. Botches a new error type * Adds new error type to track and handle order classification errors separately * Fully fleshes out ClassificationError for all instances of status conversion. Even in order trades and some wrapper functions * Introduces common.SimpleTimeFormat for "2006-01-02 15:04:05". Fixes binance and bitfinex issues with auth endpoint use, map casting. Expands more order.ClassificationError usage. Fixes some more generic websocket response errors * Future proofs order updating by utilising asset types. Expands testing to accomodate. Adds shiny new time type. Expands wrapper websocket functionality definitions * minty linty * Broken end of day code addressing basic nits on comments, returns and currency conversion * Adds testing to btcmarkets websocket. Also updates websocket orderbook to use update instead * Fixes fun rebase fun fun so fun * Addresses minor nits regarding changed interface and comments * Creates new function `GetRequestFormattedPairAndAssetType` to retrieve a currency pair and asset type based on a string. It will iterate over enabled pairs and compare them to formatted pairs and then return that pair if found. * Fixes test * Adds a single line to the end of the file, because that would be really bad if it wasn't there * Updates fakepassexchange to not use params, updates test params, uses fatal in some tests where its important, updates order manager to have a rwmutex, removes some returns, improves ws key test for binance, updates properties to reflect their actual values, adds some more websocket properties * Addresses binance switch linting * Updates leverage property to int64 * Fixes what was broken |
||
|
Ryan O'Hara-Reid
|
6d8ba0a96a |
Initial implementation of HTTP mock testing framework (#310)
* Initial implementation of HTTP mock testing framework Convert to VCR testing server. Segregate live testing via build tags. Converted Binance to VCR server Convert Bitstamp to VCR mocking tests Added VCR mock testing for localbitcoins * Add server generation for concurrent testing * Fix linter issues * Fix linter issue * fix race - potentially * revert auto assigning of host vals * Fix requested changes * Adds mock testing for ANX Switch to using TestMain functionality Added cron job usage for travis-ci to live testing Added appveyor scheduled build check for live testing * WOOPS * silly correction * Fixes fantastic linter issues * fixed another whoopsie * WOOO! * Adds gemini mock testing with additional fixes * Add docs and sharedvalue * Added tls using httptest package * Fixed issues * added explicit mock recording reference to error * Fix requested changes * strip port from mock files as they are not needed on tls server * Change incorrect names * fix requested changes * lbank update * Fix another issue * Updated readme |
||
|
Scott
|
0fbf8b172a |
Websocket orderbook buffering (#333)
* Initial commit setting up a map orderbook system with a buffer. It will write to the buffer, sort apply to main orderbook and then process. * Moves namespaces again * Updates orderbook to use a sweet new WebsocketOrderbookUpdate type to handle all updates whether its using ID or not. So good. Adds many tests * Starting to implement orderbook update handling per exchange. Updates namespaces again. Hopefuylly will find a way to update via ID not timestamp, too many endpoints dont provide update timestamps * Changes orderbookbuffer to use BufferUpdate type instead of orderbook.Base to achieve more functionality and no need for type conversion functions. Updates tests * Updates all instances of ws.orderbook.Update. Simplifies some orderbook logic * Introduces toggleable buffer. Renames orderbooks. Completes implementation for everywhere but OKGroup due to hash calculation * Implements orderbook update for okgroup, but forgets about the orderbook hash checking * Fixes okgroup checksum calculation. Fixes linting issue. Removes redundant Kraken tests. * Introduces sorting toggle and separates from buffer toggle. Uses benchmarks to highlight performance gains * Fixes Gemini rate limit and parsing. Removes comments and fixes typos * Fixes bitfinex orderbook processing * Inbuilt sorting, minor fixes for websocket implementations. Improves test coverage * Adds surprise LakeBTC websocket support * Fixes data race * Fixes rebasing issues due to namespace movements * Addresses PR nits: moves folder namespace from ws to websocket. Removes line spaces in imports. Fixes lakebtc websocket returns and defer fucntions. Fixes comments * Adds poloniex orderook sorting support * Enables bitstamp and hitbtc orderbook sorting. Fixes poloniex's sorting * Renames namespaces and combines monitor and connection into wshandler. Removes unused SPOT const. Changes how orderbook stuff is loaded. It is done in startup with a setup. Removes exchange name from loadsnapshot as well * Removes the connection.go from rebasing issues. Removes error response from functions used in goroutines * Fixes test with exchange name output change * Fixes issues where copy and paste and replace all were used poorly |
||
|
Scott
|
e209d85d0d |
Websocket request-response correlation (#328)
* Establishes new websocket functionality. Begins the creation of the websocket request * Adding a wrapper over gorilla websocket connect,send,receive to handle ID messages. Doesn't work * Successfully moved exchange_websocket into its own wshandler namespace. oof * Sets up ZB to use a round trip WS request system * Adds Kraken ID support to subscriptions. Renames duplicate func name UnsubscribeToChannels to RemoveSubscribedChannels. Adds some helper methods in the WebsocketConn to reduce duplicate code. Cleans up ZB implementation * Fixes double locking which caused no websocket data to be read. Fixes requestid for kraken subscriptions * Completes Huobi and Hadax implementation. Extends ZB error handling. Adds GZip support for reading messages * Adds HitBTC support. Adds GetCurrencies, GetSymbols, GetTrades WS funcs. Adds super fun new parameter to GenerateMessageID for Unix and UnixNano * Adds GateIO id support * Adds Coinut support. Prevents nil reference error in constatus when there isnt one * Standardises all Exchange websockets to use the wshandler websocket. Removes the wsRequestMtx as wshandler handles that now. Makes the Dialer a dialer, its not externally referenced that I can see. * Fixes issue with coinut implementation. Updates bitmex currencies. Removes redundant log messages which are used to log messages * Starts testing. Renames files * Adds tests for websocket connection * Reverts request.go change * Linting everything * Fixes rebase issue * Final changes. Fixes variable names, removes log.Debug, removes lines, rearranges test types, removes order correlation websocket type * Final final commit, fixing ZB issues. * Adds traffic alerts where missed. Changes empty struct pointer addresses to nil instead. Removes empty lines * Fixed string conversion * Fixes issue with ZB not sending success codes * Fixes issue with coinut processing due to nonce handling with subscriptions * Fixes issue where ZB test failure was not caught. Removes unnecessary error handling from other ZB tests * Removes unused interface * Renames wshandler.Init() to wshandler.Run() * Updates template file * Capitalises cryptocurrencies in struct. Moves websocketResponseCheckTimeout and websocketResponseMaxLimit into config options. Moves connection configuration to main exchange Setup (where appropriate). Reverts currencylastupdated ticks. Improves reader close error checking * Fixes two inconsistent websocket delay times * Creates a default variable for websocket ResponseMaxLimit and ResponseCheckTimeout, then applies it to setdefaults and all tests * Updates exchange template to set and use default websocket response limits |
||
|
Scott
|
3a66e99899 |
Authenticated Websocket support (#315)
* Improves subscribing by not allowing duplicates. Adds bitmex auth support * Adds coinbase pro support. Partial BTCC support. Adds WebsocketAuthenticatedEndpointsSupported websocket feature. Adds GateIO support * Adds Coinut support * Moves Coinut WS types to file. Implements Gemini's secure WS endpoint * Adds HitBTC ws authenticated support. Fixes var names * Adds huobi and hadax authenticated websocket support * Adds auth to okgroup (okex, okcoin). Fixes some linting * Adds Poloniex support * Adds ZB support * Adds proper bitmex support * Improves bitfinex support, improves websocket functionality definitions * Fixes coinbasepro auth * Tests all endpoints * go formatting, importing, linting run * Adds wrapper supports * General clean up. Data race destruction * Improves testing on all exchanges except ZB * Fixes ZB hashing, parsing and tests * minor nits before someone else sees them <_< * Fixes some nits pertaining to variable usage, comments, typos and rate limiting * Addresses nits regarding types and test responses where applicable * fmt import * Fixes linting issues * No longer returns an error on failure to authenticate, just logs. Adds new AuthenticatedWebsocketAPISupport config value to allow a user to seperate auth from REST and WS. Prevents WS auth if AuthenticatedWebsocketAPISupport is false, adds additional login check 'CanUseAuthenticatedEndpoints' for when login only occurs once (not per request). Removes unnecessary time.Sleeps from code. Moves WS auth error logic to auth function so that wrappers can get involved in all the auth fun. New-fandangled shared test package, used exclusively in testing, will be the store of all the constant boilerplate things like timeout values. Moves WS test setup function to only run once when there are multiple WS endpoint tests. Cleans up some struct types * Increases test coverage with tests for config.areAuthenticatedCredentialsValid config.CheckExchangeConfigValues, exchange.SetAPIKeys, exchange.GetAuthenticatedAPISupport, exchange_websocket.CanUseAuthenticatedEndpoitns and exchange_websocket.SetCanUseAuthenticatedEndpoints. Adds b.Websocket.SetCanUseAuthenticatedEndpoints(false) when bitfinex fails to authenticate Fixes a typo. gofmt and goimport * Trim Test Typos * Reformats various websocket types. Adds more specific error messaging to config.areAuthenticatedCredentialsValid |