mirror of
https://github.com/d0zingcat/gocryptotrader.git
synced 2026-05-14 15:09:51 +00:00
977fecab19951be388690c22bbd5f8e2f4872cce
25 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Ryan O'Hara-Reid
|
cad7586e98 |
exchange/websocket, gateio: Rename/export package again, add websocket request functions for futures trading (#1603)
* gateio: Add multi asset websocket support WIP. * meow * Add tests and shenanigans * integrate flushing and for enabling/disabling pairs from rpc shenanigans * some changes * linter: fixes strikes again. * Change name ConnectionAssociation -> ConnectionCandidate for better clarity on purpose. Change connections map to point to candidate to track subscriptions for future dynamic connections holder and drop struct ConnectionDetails. * Add subscription tests (state functional) * glorious:nits + proxy handling * Spelling * linter: fixerino * instead of nil, dont do nil. * clean up nils * cya nils * don't need to set URL or check if its running * stream match update * update tests * linter: fix * glorious: nits + handle context cancellations * stop ping handler routine leak * * Fix bug where reader routine on error that is not a disconnection error but websocket frame error or anything really makes the reader routine return and then connection never cycles and the buffer gets filled. * Handle reconnection via an errors.Is check which is simpler and in that scope allow for quick disconnect reconnect without waiting for connection cycle. * Dial now uses code from DialContext but just calls context.Background() * Don't allow reader to return on parse binary response error. Just output error and return a non nil response * Allow rollback on connect on any error across all connections * fix shadow jutsu * glorious/gk: nitters - adds in ws mock server * linter: fix * fix deadlock on connection as the previous channel had no reader and would hang connection reader for eternity. * glorious: whooops * gk: nits * Leak issue and edge case * Websocket: Add SendMessageReturnResponses * whooooooopsie * gk: nitssssss * Update exchanges/stream/stream_match.go Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com> * Update exchanges/stream/stream_match_test.go Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com> * linter: appease the linter gods * gk: nits * gk: drain brain * started * more changes before merge match pr * gateio: still building out * gateio: finish spot * fix up tests in gateio * Add tests for stream package * rm unused field * glorious: nits * rn files, specifically set function names to asset and offload routing to websocket type. * linter: fix * Add futures websocket request support * gateio: integrate with IBOTExchange (cherry pick my nose) * linter: fix * glorious: nits * add counter and update gateio * fix collision issue * Update exchanges/stream/websocket.go Co-authored-by: Scott <gloriousCode@users.noreply.github.com> * glorious: nits * add tests * linter: fix * After merge * Add error connection info * upgrade to upstream merge * Fix edge case where it does not reconnect made by an already closed connection * stream coverage * glorious: nits * glorious: nits removed asset error handling in stream package * linter: fix * rm block * Add basic readme * fix asset enabled flush cycle for multi connection * spella: fix * linter: fix * Add glorious suggestions, fix some race thing * reinstate name before any routine gets spawned * stop on error in mock tests * glorious: nits * Set correct price * glorious: nits found in CI build * Add test for drain, bumped wait times as there seems to be something happening on macos CI builds, used context.WithTimeout because its instant. * mutex across shutdown and connect for protection * lint: fix * test time withoffset, reinstate stop * fix whoops * const trafficCheckInterval; rm testmain * y * fix lint * bump time check window * stream: fix intermittant test failures while testing routines and remove code that is not needed. * spells * cant do what I did * protect race due to routine. * update testURL * use mock websocket connection instead of test URL's * linter: fix * remove url because its throwing errors on CI builds * connections drop all the time, don't need to worry about not being able to echo back ws data as it can be easily reviewed _test file side. * remove another superfluous url thats not really set up for this * spawn overwatch routine when there is no errors, inline checker instead of waiting for a time period, add sleep inline with echo handler as this is really quick and wanted to ensure that latency is handing correctly * linter: fixerino uperino * fix ID bug, why I do this, I don't know. * glorious: panix * linter: things * whoops * dont need to make consecutive Unix() calls * websocket: fix potential panic on error and no responses and adding waitForResponses * rm json parser and handle in json package instead * in favour of json package unmarshalling * linter: fix * linter: fix again * * change field name OutboundRequestSignature to WrapperDefinedConnectionSignature for agnostic inbound and outbound connections. * change method name GetOutboundConnection to GetConnection for agnostic inbound and outbound connections. * drop outbound field map for improved performance just using a range and field check (less complex as well) * change field name connections to connectionToWrapper for better clarity * spells and magic and wands * glorious: nits * comparable check for signature * mv err var * glorious: nits and stuff * attempt to fix race * glorious: nits * gk: nits; engine log cleanup * gk: nits; OCD * gk: nits; move function change file names * gk: nits; 🚀 * gk: nits; convert variadic function and message inspection to interface and include a specific function for that handling so as to not need nil on every call * gk: nits; continued * gk: engine nits; rm loaded exchange * gk: nits; drop WebsocketLoginResponse * stream: Add match method EnsureMatchWithData * gk: nits; rn Inspect to IsFinal * gk: nits; rn to MessageFilter * linter: fix * gateio: update rate limit definitions (cherry-pick) * Add test and missing * Shared REST rate limit definitions with Websocket service, set lookup item to nil for systems that do not require rate limiting; add glorious nit * integrate rate limits for websocket trading spot * conform to match upstream changes * standardise names to upstream style * fix wrapper standards test when sending a auth request through a websocket connection * whoops * Update exchanges/gateio/gateio_types.go Co-authored-by: Scott <gloriousCode@users.noreply.github.com> * glorious: nits * linter: fix * linter: overload * whoops * spelling fixes on recent merge * glorious: nits * linter: fix? * glorious: nits * gk: assert errors touched * gk: unexport derive functions * gk: nitssssssss * fix test * gk: nitters v1 * gk: http status * gk/nits: Add getAssetFromFuturesPair * gk: nits single response when submitting * gk: new pair with delimiter in tests * gk: param update slice to slice of pointers * gk: add asset type in params, includes t.Context() for tests * linter: fix * linter: fix * fix merge whoopsie * glorious: nits * gk: nit * shift over to websocket package error * internal/exchange/websocket -> exchange/websocket * PEAK OCD! * appease the OCD gods * thrasher: nits --------- Co-authored-by: shazbert <ryan.oharareid@thrasher.io> Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com> Co-authored-by: Scott <gloriousCode@users.noreply.github.com> |
||
Gareth Kirwan
|
b4e45e9a1b |
Websocket: Restructure files and types (#1859)
* Websocket: Rename stream package * Websocket: Rename Websocket to Manager * Websocket: Replace explicit errs with common.NilGuard * Websocket: Move websocket_types.go to types.go * Websocket: Minor field comment and alignment in types * Webosocket: Rename WebsocketConnection to Connection * Alphapoint: Make gorilla ws import explicit Just to avoid confusion with our own packages. * Websocket: Move stream_match to match * Websocket: Move websocket_connection to connection * Websocket: Move websocket.go to manager.go * Websocket: Break out all subscription methods into subscriptions.go * Websocket: Move connection type into its file * Websocket: Remove PositionUpdated Type is not used anywhere * Kraken: Use local constant for pong Was the only use of websocket.Pong and doesn't really feel right to represent kraken's api resp in one of our packages * Websocket: Move connection sub-types to connection package * Websocket: Move manager types into manager * Websocket: Move ConnectionWrapper into manager * Websocket: Move websocket_test to manager_test * Websocket: Privatise connectionWrapper * Websocket: Remaining types into types.go These really belong somewhere else mostly, but this will do for now * Websocket: Tidy up connection method vars * Gofumpt: Moving package imports around * Websocket: Rename errDuplicateConnectionSetup * Websocket: Fix duplicate import of gws * Websocket: Fix gofumpt -extra * Websocket: Standardise import of gws across other pkgs * Kraken: Remove unused sub conf consts These were replaced by the generic Levels and Depth fields on all subs * Websocket: Privitise ConnectioWrapper fields * Websocket: inline single use var WebsocketNotAuthenticatedUsingRest * Websocket: Move documentation to template * Bithumb: Assertify TestWsHandleData |
||
Adrian Gallagher
|
4651af5767 |
modernise: Run new gopls modernise tool against the codebase and fix minor issues (#1826)
* modernise: Run new gopls modernise tool against codebase
* Address shazbert's nits
* apichecker, gctcli: Simplify HTML scraping functions and improve depth limit handling
* refactor: Create minSyncInterval const and update order book limit handling for binance and binanceUS
* refactor: Various slice usage improvements and rename TODO
* tranches: Revert deleteByID changes due to performance decrease
Shazbert was a F1 driver in a past lifetime 🏎️
* tranches: Simply retrieve copy
Thanks to shazbert
* documentation: Sort contributors list by contributions
* tranches: Remove deadcode in deleteByID
|
||
|
Adrian Gallagher
|
f5faca2eb2 |
linter: Enable gofumpt and run against codebase (#1848)
* linter: Enable gofumpt and run against codebase * Address shazbert's nits * gofumpt: Fix issues after rebase |
||
|
Gareth Kirwan
|
16d2d9f35a |
Config: AssetEnabled upgrade (#1735)
* Config: Move assetEnabled upgrade to Version management * Assets: Do not error on asset not enabled, or disabled This became more messy with Disabling something that's defaulted to disabled. Taking an idealogical stance against erroring that what you want to have done is already done. * CurrencyManager: Set AssetEnabled when StorePairs(enabled) * RPCServer: Fix tests expecting StoreAssetPairFormat to enable the asset Also assertifies * Bitfinex: Fix tests for MarginFunding subs * GCTWrapper: Improve TestMain clarity * BTSE: Add futures to testconfig * Exchanges: Rename StoreAssetPairStore Previously we were calling it "Format", but accepting everything from the PairStore. We were also defaulting to turning the Asset on. Now callers need to get their AssetEnabled set as they want it, so there's no magic This change also moves responsibility for error wrapping outside to the caller. * Config: AssetEnabled upgrade should respect assetTypes Previously we ignored the field and just turned on everything. I think that was because we couldn't get at the old value. In either case, we have the option to do better, and respect the assetEnabled value * Config: Improve exchange config version upgrade error messages |
||
cranktakular
|
90fee94c76 |
currency: Make pairs.Add variadic (#1566)
* Making Pairs(Add) Variatic * Slight improvements * Implementation overhaul * Improvements * Changing code which can use the variadic functionality * Fixing silliness * thing left over |
||
|
Gareth Kirwan
|
1199f38546 |
subscriptions: Encapsulate, replace Pair with Pairs and refactor; improve exchange support
* Websocket: Use ErrSubscribedAlready instead of errChannelAlreadySubscribed * Subscriptions: Replace Pair with Pairs Given that some subscriptions have multiple pairs, support that as the standard. * Docs: Update subscriptions in add new exch * RPC: Update Subscription Pairs * Linter: Disable testifylint.Len We deliberately use Equal over Len to avoid spamming the contents of large Slices * Websocket: Add suffix to state consts * Binance: Subscription Pairs support * Bitfinex: Subscription Pairs support * Bithumb: Subscription Pairs support * Bitmex: Subscription Pairs support * Bitstamp: Subscription Pairs support * BTCMarkets: Subscription Pairs support * BTSE: Subscription Pairs support * Coinbase: Subscription Pairs support * Coinut: Subscription Pairs support * GateIO: Subscription Pairs support * Gemini: Subscription Pairs support and improvement * Hitbtc: Subscription Pairs support * Huboi: Subscription Pairs support * Kucoin: Subscription Pairs support * Okcoin: Subscription Pairs support * Poloniex: Subscription Pairs support * Kraken: Add subscription Pairs support Note: This is a naieve implementation because we want to rebase the kraken websocket rewrite on top of this * Bybit: Subscription Pairs support * Okx: Subscription Pairs support * Bitmex: Subsription configuration * Fixes unauthenticated websocket left as CanUseAuth * Fixes auth subs happening privately * CoinbasePro: Subscription Configuration * Consolidate ProductIDs when all subscriptions are for the same list * Websocket: Log actual sent message when Verbose * Subscriptions: Improve clarity of which key is which in Match * Subscriptions: Lint fix for HugeParam * Subscriptions: Add AddPairs and move keys from test * Subscriptions: Simplify subscription keys and add key types * Subscriptions: Add List.GroupPairs Rename sub.AddPairs * Subscription: Fix ExactKey not matching 0 pairs * Subscriptions: Remove unused IdentityKey and HasPairKey * Subscriptions: Fix GetKey test * Subscriptions: Test coverage improvements * Websocket: Change State on Add/Remove * Subscriptions: Improve error context * Subscriptions: Fix Enable: false subs not ignored * Bitfinex: Fix WsAuth test failing on DataHandler DataHandler is eaten by dataMonitor now, so we need to use ToRoutine * Deribit: Subscription Pairs support * Websocket: Accept nil lists for checkSubscriptions If the user passes in a nil (implicitly empty) list, we would not panic. Therefore the burden of correctness about that data lies with them. The list of subscriptions is empty, and that's okay, and possibly convenient * Websocket: Add context to NilPointer errors * Subscriptions: Add context to nil errors * Exchange: Fix error expectations in UnsubToWSChans |
||
|
Gareth Kirwan
|
7d1eecfa7e |
Tests: Various race fixes and move TestFixtureToDataHandler (#1534)
* Tests: Move and simplify TestFixtureToDataHandler * Currency: Fix PairsManager.Load breaking matcher * Tests: Add multi-instance cache to UpdatePairsOnce * Kraken: Fix TestUpdateTickers race error Calling StorePairs on global instance can lead to race * Bitfinex: Fix TestUpdateTickers racing intermittently * Currency: Fix concurrent access to PM formats * Currency: Fix SupportsAsset implementation This should delegate entirely to PairManager's IsAssetSupported * Okx: Fix PM intrusion, rm GetPairFromInstrumentID * Exchange: Fix SetGlobalPairsManager to set asset enabled * Bitflyer: Fix race on set TestGetCurrURL TestGetCurrencyTradeURL would fail sometimes due to sequencing of enabling futures but not having pairs for it. * Tests: Simplify usage pattern for FixtureToDH |
||
Scott
|
34ef09dad6 |
bugfixes: Backtester credentials, Binance ExecutionLimits, Mock recording (#1539)
* fix mock recording, binance, backtester * minor credential streamlining |
||
|
Gareth Kirwan
|
52c6b3bf0b |
Websocket: Various refactors and test improvements (#1466)
* Websocket: Remove IsInit and simplify SetProxyAddress IsInit was basically the same as IsConnected. Any time Connect was called both would be set to true. Any time we had a disconnect they'd both be set to false Shutdown() incorrectly didn't setInit(false) SetProxyAddress simplified to only reconnect a connected Websocket. Any other state means it hasn't been Connected, or it's about to reconnect anyway. There's no handling for IsConnecting previously, either, so I've wrapped that behind the main mutex. * Websocket: Expand and Assertify tests * Websocket: Simplify state transistions * Websocket: Simplify Connecting/Connected state * Websocket: Tests and errors for websocket * Websocket: Make WebsocketNotEnabled a real error This allows for testing and avoids the repetition. If each returned error is a error.New() you can never use errors.Is() * Websocket: Add more testable errors * Websocket: Improve GenerateMessageID test Testing just the last id doesn't feel very robust * Websocket: Protect Setup() from races * Websocket: Use atomics instead of mutex This was spurred by looking at the setState call in trafficMonitor and the effect on blocking and efficiency. With the new atomic types in Go 1.19, and the small types in use here, atomics should be safe for our usage. bools should be truly atomic, and uint32 is atomic when the accepted value range is less than one byte/uint8 since that can be written atomicly by concurrent processors. Maybe that's not even a factor any more, however we don't even have to worry enough to check. * Websocket: Fix and simplify traffic monitor trafficMonitor had a check throttle at the end of the for loop to stop it just gobbling the (blocking) trafficAlert channel non-stop. That makes sense, except that nothing is sent to the trafficAlert channel if there's no listener. So that means that it's out by one second on the trafficAlert, because any traffic received during the pause is doesn't try to send a traffic alert. The unstopped timer is deliberately leaked for later GC when shutdown. It won't delay/block anything, and it's a trivial memory leak during an infrequent event. Deliberately Choosing to recreate the timer each time instead of using Stop, drain and reset * Websocket: Split traficMonitor test on behaviours * Websocket: Remove trafficMonitor connected status trafficMonitor does not need to set the connection to be connected. Connect() does that. Anything after that should result in a full shutdown and restart. It can't and shouldn't become connected unexpectedly, and this is most likely a race anyway. Also dropped trafficCheckInterval to 100ms to mitigate races of traffic alerts being buffered for too long. * Websocket: Set disconnected earlier in Shutdown This caused a possible race where state is still connected, but we start to trigger interested actors via ShutdownC and Wait. They may check state and then call Shutdown again, such as trafficMonitor * Websocket: Wait 5s for slow tests to pass traffic draining Keep getting failures upstream on test rigs. Think they can be very contended, so this pushes the boundary right out to 5s |
||
|
Gareth Kirwan
|
e007f69f7c |
exchanges/websocket: Implement subscription configuration (#1394)
* Websockets: Move Subscription to its own package This allows the small type to be imported from both `config` and from `stream` without an import cycle, so we don't have to repeat ourselves * Subs: Renamed Currency to Pair This was being mis-used through much of the code, and since we're already touching everything, we might as well fix it * Websockets: Add Subscription configuration * Binance: Add subscription configuration * Kucoin: Subscription configuration * Simplify GenerateDefaultSubs * Improve TestGenSubs coverage * Test Candle Sub generation * Support Candle intervals * Full responsibility for formatting Channel name on GenerateDefaultSubs OR consumer of Subscribe * Simplify generatePayloads as a result * Fix test coverage of asset types in processMarketSnapshot * Exchanges: Abstract ParallelChanOp * Tests: Generic ws mock instances * Kucoin: Fix intermittent conflict in test currs Use isolated test instance for `TestGetOpenInterest`. `TestGetOpenInterest` would occassionally change pairs before GenerateDefault Subs. |
||
|
Scott
|
b71bf1f3d1 |
exchanges/futures: Implement open interest (#1417)
* adds open interest to exchanges * ADDS TESTING YEAH * New endpoints, BTSE, RPCS, cached * slight design change, begin gateio You will need to get cached for each exchange that supports it * gateio, huobi, rpc * fix up kraken, cache retrieval * okx, gateio * finalising all implementations and tests * definitely my final ever commit on this * Well, well, well * final v2 * quick fix of bug * test coverage, assert notempty, test helper Added a new testhelper for currency management because its very annoying in a parallel test setting which wastes so much space otherwise * minimises REST requests for Open Interest * types.Number merge misses * Minimises Kraken REST calls * len change, value -> pointer receiver * further fixup * fixes gateio, batch calculates open interest * single gateio, lint const fixes * rejig and more thorough oi for huobi * formatting expansion * minor fix for handling expiring contracts * rm unused Binance strings * add bybit support, fix bybit issues * oopsie doopsie, dont look at my whoopsie * Fix issue, remove feature * move an irrelevant function for the pr * mini bybit upgrades * fixes cli request bug |
||
|
Gareth Kirwan
|
32f08f82f4 |
Kucoin: Fix unmarshalling of FuturesTicker resp (#1414)
* Kucoin: Fix unmarshalling of FuturesTicker resp * Fix several string float64 fields not unmarshalling * Fix time field misnamed; Make a time.Time * Use order.Side for sides * Add tests This change opts to expose float64 instead of StringToFloat64 because there's no wrapper to convert the type over, and it's not a websocket. Essentially choosing to not expose internal convert types for methods called directly. Possible we'll eventually harmonise this with a types.Number or something. * Kucoin: FuturesRealTimeTicker => FuturesTicker Kucoin only call it RealTime in the first sentence of their docs. None of the API names, endpoints or any thing mention RealTime. This is just a ticker * Tests: Improve clarity of failed push data * Kucoin: Move PushData to a fixture file This avoids both the linter errors, and the bloat. Loses the name of the "test", but I think that's okay |
||
|
Gareth Kirwan
|
8430509807 |
Okx: Websocket order channel fixes (#1346)
* Okx: Fix WS order fields * Fixes float64 with string annotation erroring on empty strings: Okx Order Push Data error json: invalid use of ,string struct tag, trying to unmarshal "" into float64 Specifically this came from px field from a market order * Switch to convert.StringToFloat64 instead of okxNumericalValue * Fix typo in Notional* field names; Ironically prevented them from erroring * Okx: Add tests for first order fields * Okx: CID and maybe set WS order Filled time * Tests: Set TestFixtureToDataHandler to t.Helper * Orders: Add UnmarshalJSON to order.Side * Okx: Fix FillTime not parsed for PendingOrder * Okx: Switch to order.Side Unmarshal throughout * Okx: Add Fee and FeeAsset to order processing * Okx: Fix WS order.Detail amounts and Test This fixes Amount vs QuoteAmount for market sells where tgtCcy is quote_ccy * Add comment to order.Side.UnmarshalJSON * Okx: Replace PendingOrderItem Unmarshal with local types * Okx: string type for WS order reduceOnly Note: Not yet in unit tests, since it's not part of the spot tests I was originally fixing. I'll circle back to adding full test support for Reduce only and deleveraging positions. * Okx: Fix TestOrderPushData Amount We were expecting 0 when we're given a quoteAmount In reality, we'll calculate the size from the price * Okx: Fix order and remAmount in wsOrders Improved handling for Float64 issues and boundaries when the order is fully executed but not yet marked as Filled * Fix ErrSideIsInvalid in tests |
||
|
Gareth Kirwan
|
d86da76b1f |
Bitstamp: Add auth channel subscription handling (#1333)
* Bitstamp: Add auth channel subscription handling * Bitstamp: Avoid searching for asset type We've hardcoded asset.Spot in order to find the pair. Looking the asset up from the pair makes no sense. * Bitstamp: Add type for wsOrders * Bitstamp: Working test of Generic DataHandler * Bitstamp: WS Order chan tests and remove type orderType could be derived from status == New and Buy & price == 9+e9 or Sell & price == 0 But it would only be true for the first update and it really doesn't feel worth the risk Consumers are going to have to merge to original request anyway * Bitstamp: Linter fixes * Kraken: Switch to shared fixture test * Bitstamp: Fix lint on TestFixtureToDataHandler * Engine: Add Clone for PairsManager go-vet highlighted that the mutex here is a value when we copied the PairsManager in a test. Options to fix: * Add a deep clone method to PairsManager * Add a shallow clone method with a disclaimer * Make the mutex a pointer * Make the PairsManager itself a pointer Options 3 and 4 are too invasive to justify changing at this point. There's an inherent risk of PM being passed by value, but govet should catch the copylock. There's more risk in changing everything to use a pointer at this stage. * Engine: Fix linter again, ironically * Bitstamp: Rename OHLC const * Bitstamp: Minor fixes to syntax * Bitstamp: Simplify chanSymb=>pair * Bitstamp: Still process order updates without ID If there's a ClientOrderID we'll still process the order. It doesn't seem likely we'd have this happen, but if it does we still want consumers to get something. * Bitstamp: Replace Clone with Lock methods * Engine: Expose PairsManager's Mutex Makes more sense than wrapping functions * Bitstamp: Fix linter copylock (again) * fixup! Engine: Expose PairsManager's Mutex Omit Mutex from Json * fixup! Bitstamp: Add auth channel subscription handling Remove unused wsAuthToken * Bitstamp: Simplify OrderData Unmarshal * Bitstamp: Remove unused contexts I added these following best practices, but the reality is that when/if we get context awareness in GCT, there will be a lot more to fix and this will be a drop in the ocean anyway. * Bitstamp: Only call handleWSOrder for MyOrders * Bitstamp: Avoid allocating again in handleWSOrder * CurrencyPairs: Remove public mutex Simplified to a Load method to avoid making mutex public * Tests: Improve test readability and clarity * Bitstamp: Wrap errWSPairParsingError Co-authored-by: Scott <gloriousCode@users.noreply.github.com> * Bitstamp: FetchWSAuth mock and live test --------- Co-authored-by: Scott <gloriousCode@users.noreply.github.com> |
||
|
Ryan O'Hara-Reid
|
981a08af83 |
bybit: assortment of updates (#1192)
* bybit: cherry-pickable * bybit: implement fee fetching v5 * bybit: update to use nullable type * bybit: fix some tests * bybit: spell check fix * remove redunant asset dec, and rm output * rm comment code * linter: fixerinos woooo * bybit: constrict rate limit on public spot to v5 * exchanges/bybit/limits: update (CHERRY PICK ME) * glorious: nIIIIIIIIIIIIIIITS * glorious: nits continued * updated comment * update even more * RM LINE! * glorious: nits * Update exchanges/sharedtestvalues/sharedtestvalues.go Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io> * fix --------- Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io> Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io> |
||
|
Ryan O'Hara-Reid
|
b20cf75d13 |
sharedtestvalues: Add helper functions and implement throughout exchange tests (#1163)
* exchanges/sharedtestvalues: implement new functions to handle test skipping and announcements for standardising. * exchanges: fin test impl. * linter: fixes * exchange_template: fix test * allocate so it doesn't make a panic at the disco * glorious: nits * glorious: nits * Update exchanges/sharedtestvalues/sharedtestvalues.go Co-authored-by: Scott <gloriousCode@users.noreply.github.com> * Update exchanges/sharedtestvalues/sharedtestvalues.go Co-authored-by: Scott <gloriousCode@users.noreply.github.com> * glorious: nits * linter: fix * linter: shhhh --------- Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io> Co-authored-by: Scott <gloriousCode@users.noreply.github.com> |
||
Samuael A
|
2ac165a477 |
exchanges: Add OKX support (#1005)
* public endpoints methods added * Completing mapping REST endpoints * Binanceus Wrapper methods -Partially * Binanceus Wrapper methods -Partially * BinaWra functions with test funs; Not Completed * Finalizing wrapper methods & test * Fix & Complete wrapper functions * Finalizing wrapper methods & test * Adding Stream Datas * WS Test functions * CI: Fix golangci-lint linter issues * CI: Fix reverting unnessesary changes and type conversion issues * CI: Fix reverting unnessesary changes and type conversion issues * Adding Public endpoints and tests * Adding Market and Public Endpoints * Adding Public endoints * Public Trading Endpoints & Authenticated Trade order methods * Adding Authenticated Methods and Tests * Adding algo and Funding Authenticated endpoints * Adding funding trading endpoints and correspondint tests * adding authenticated endpoints * Completing Block Trading endpoints and added subaccount endpoints * Completing sub account and grid Trading endpoints * Adding Rate Limit and missing endpoints * Wrapper and Websocket handlers * Fixing Websocket Test and Push Data Handler Issues * Fixing Websocket Test and Push Data Handler Issues * Fixing linter issues, package dependency, and other slight tempos * Fixing linter and slight tempos * Update on test functions, and Rest and Websocket Endpoint handlers * Remove okex, adding comments, and slight fixes on endpoints. * Fixing linter issues and adding comments * Slight code changes, updating documentation, and n and linter issues * Fix context and configuration endpoint issues * slight fixes on config and test files * adding some missing test and fix linter issues * fix linter issue * Slight fixes on code structure, shorthand exp,and ot and other * Fix slight linter issue * Slight code fixes and fixing linter issues * fixing linter iissues * fixing linter iissues * slight linter issue fix * slight linter issue fix * Fix on models, type convert funcs and endpoints * Adding Error messages map and update of models * Fix on error message string and linter issues * Fix slight linter issue * Fix slight linter issue * Fixing type converts, models, and linter issues * Adding Ws fixes * Slight fix on websocket and other issues * Adding slight websocket fixes * Remove 'books5' channel default subscription * Small changes on default subscription and checksum * Fix slight websocket tempos * Fix Wrapper function tempost and linter issues * Resolving slight naming and other issues * Resolve slight pointer issues * resolve slight linter issues * Resolve config files issue * Update websocket and wrapper funcs with test and docs * fixs on websocket multiplexer, types, and other slight issues * fix slight linter issues * slight update on web-socket orderbook and tickers * fix slight issues and websocket runtime errors * Slight unit test fix and assing simple semaphore * FIx race issue * Update on authenticated endpoints * Fix wsSetupRun check in websocket 'setupWsAuth' func * Update wsSetupRun check in websocket 'setupWsAuth' func * Slight update on websocket handling * Fix some race conditions * fix slight tempos * fix authenticated test issues * Update on conditional statements * slight update on unit test * fix unit test tempos * Fix slight tempos * Change check map from struct valued to bool valued * slight fix trial * Slight unit test update * Fix websocket timeout error * Updating websocket subscription endpoints, and unit tests * update unit tests * Slight issue on wrapper method 'GetActiveOrders' * Overall code update * Addressing missing review comments * Fix unit test tempo and linter issue * Monor fix * Slight update * Slight unit test fix * Slight fixes * Slight fixes * Fixing on missing review comments * Adding WS Fixes * slight fix * Monor fix on unit test * Minor convert issue * Minor change on WS * Monor logic fix * Fix code structure and logic issues * Fixing small typos * fix slight data format issue * Update on trade and order wrapper methods * Adding slight update * fix on order detail * Slight update on FetchTradablePairs wrapper method * Slight update on wrapper * Update on deserialization and other slight issues * Final update * Resolve missing review comments * Slight update on config and unit test * minor fix on GetDepositAddress param * Minor fix |
||
Adam
|
504c2fad6d |
Feature: Implement funding rates, futures and coin margin (exchange API coverage) (#530)
* ALMOST THERE * more api wips * more api thingz * testing n more api wipz * more apiz * more wips * what is goin on * more wips * whip n testing * testing * testing no keys * remove log * kraken is broken ugh * still broken * fixing auth funcs + usdtm api docs * wip * api stuffs * whip * more wips * whip * more wip * api wip n testing * wip * wip * unsaved * wip n testing * wip * wip * wip * wip * wip * wip * wip * wip * wip * whip * wrapper authenticated functions * adding asset type and fixing dependencies * wip * binance auth wrapper start * wrapper functionality * wip * wip * wip * wrapper cancel functions * order submission for wrappers * wip * more error fixing and nits * websocket beginning n error fix * wip * WOW * glorious n shazzy nits * useless nits * wip * fixing things * merge stuffs * crapveyor * crapveyor rebuild * probably broke more things than he fixed * rm lns n other thangs * hope * please * stop it * done * ofcourse * rm vb * fix lbank * appveyor please * float lev * DONT ASK RYAN FOR HELP EVER * wip * wip * endpoint upgrades continued * path upgrade * NeeeNeeeNeeeNeeeNING * fix stuffs * fixing time issue * fixing broken funcs * glorious nits * shaz changes * fixing errors for fundmon * more error fixing for fundmon * test running past 30s * basic changes * THX AGAIN SHAZBERT * path system upgrade * config upgrade * unsaved stuffs * broken wip config upgrade * path system upgrade contd. * path system upgrade contd * path upgrade ready for review * testing verbose removed * linter stuffs * appveyor stuffs * appveyor stuff * fixed? * bugfix * wip * broken stuff * fix test * wierd hack fix * appveyor pls stop * error found * more useless nits * bitmex err * broken wip * broken wip path upgrade change to uint32 * changed url lookups to uint * WOW * ready4review * config fixed HOPEFULLY * config fix and glorious changes * efficient way of getting orders and open orders * binance wrapper logic fixing * testing, adding tests and fixing lot of errrrrs * merge master * appveyor stuffs * appveyor stuffs * fmt * test * octalLiteral issue fix? * octalLiteral fix? * rm vb * prnt ln to restart * adding testz * test fixzzz * READY FOR REVIEW * Actually ready now * FORMATTING * addressing shazzy n glorious nits * crapveyor * rm vb * small change * fixing err * shazbert nits * review changes * requested changes * more requested changes * noo * last nit fixes * restart appveyor * improving test cov * Update .golangci.yml * shazbert changes * moving pair formatting * format pair update wip * path upgrade complete * error fix * appveyor linters * more linters * remove testexch * more formatting changes * changes * shazbert changes * checking older requested changes to ensure completion * wip * fixing broken code * error fix * all fixed * additional changes * more changes * remove commented code * ftx margin api * appveyor fixes * more appveyor issues + test addition * more appveyor issues + test addition * remove unnecessary * testing * testing, fixing okex api, error fix * git merge fix * go sum * glorious changes and error fix * rm vb * more glorious changes and go mod tidy * fixed now * okex testing upgrade * old config migration and batch fetching fix * added test * glorious requested changes WIP * tested and fixed * go fmted * go fmt and test fix * additional funcs and tests for fundingRates * OKEX tested and fixed * appveyor fixes * ineff assign * 1 glorious change * error fix * typo * shazbert changes * glorious code changes and path fixing huobi WIP * adding assetType to accountinfo functions * fixing panic * panic fix and updating account info wrappers WIP * updateaccountinfo updated * testing WIP binance USDT n Coin Margined and Kraken Futures * auth functions tested and fixed * added test * config reverted * shazbert and glorious changes * shazbert and glorious changes * latest changes and portfolio update * go fmt change: * remove commented codes * improved error checking * index out of range fix * rm ln * critical nit * glorious latest changes * appveyor changes * shazbert change * easier readability * latest glorious changes * shadow dec * assetstore updated * last change * another last change * merge changes * go mod tidy * thrasher requested changes wip * improving struct layouts * appveyor go fmt * remove unnecessary code * shazbert changes * small change * oopsie * tidy * configtest reverted * error fix * oopsie * for what * test patch fix * insecurities * fixing tests * fix config |
||
|
Ryan O'Hara-Reid
|
14c72c9c6b |
Currency System Update (#448)
* initial update of currency system * WIP progress * Finish initial currency string error returns * fix whoopsie testing for non https insecureinos * Current WIP for getEnabledPairs check and error return * WIP continued * When getting enabled pairs throw error when item is not contained in available pairs list * More updates -WIP * Wip continued including potential interface * Current WIP * pairs manager pass * drop asset string and just use the map key, plus return some errors and create more work for myself. * clean and fixed a bug in currency.json, will not populate correctly without coinmarketcap api keys set. * purge logger references after merge * go mod tidy after merge * Pointer change WIP * fix some issues and added error returns to a few items (WIP) * WIP * Clean * Fix some linter issues * Fix more linter issues * even more linters * xtda nits * revert pointer change and rm field * Addr madcozbadd nits * fix linter issues: shadow declarations * Fix linter issues: gocritic huge things * linter issue fix * Addr nits * flush go mod files * after merge woops * fix shadow dec * Addr thrasher nits * addr nits * fix some issues * more fixes * RM println * Addr glorious nits * Add helper method for setting assets * add missing format directive * Addr nits * Actually process new futures contracts -_- derp * WIP for GRPC upgrade for pair management * update config pairs * finished disabling and enabling asset * linked update of tradable pairs to cli * fix oopsies * defer writing to file on program termination for currency storage system * update template * don't add disabled asset items to initial sync * Fix enable disabling a list of pairs and added in a slice error type so we can add whats allowable without throwing an error and return a report, also addressed some other nits * WIP on getting a channel to unsub * Wip track down unsubscribe bug and start creating streaming interface * purge websocket orderbook object and centralised updating routine for orderbook * general clean before interface implementation * stage one connection interface WIP * WIP * repackage wshandler WIP * find difference of subs and change signature of subscriber functions so we can batch subscriptions and unsubscription in exchanges that support it * design change on mange subscription routine WIP * integrated ZB with the new webosocket updates * WIP - okex conversion * integrate websocket upgrades for lakebtc, kraken, huobi, hitbtc, gateio, and WIP for coinbene * integrate another range of exchanges for websocket update * Added subscriber and unsubscriber methods to websocket functionality * fixed tests WIP * amalgamate cache setup with main websocket setup * reinstate exported fields traffic alert and shutdownC to accommodate gemini and lakebtc implementations * added in colon * Updated websocket auth handling as they werent getting passed through. Added a setter method for websocket URL due to the Binance generated auth key/listenKey. Fixed bug which stopped reconnection. * Fix subscribe candle bug Fix time conversion in candle Fix inititial candle history to datahandler Include funding to orderbook handling Include funding to trades Reduce code duplication in sub and unsub functions Added the ability to include funding currency websocket subscriptions validated all channels and added more items todo list (Auth items) * RM line * bitstamp pass * btcmarkets pass - still needs to implement unsubscriber functionality and pairs change test. * Batch outgoing subscriptions and fix unsubscribe bug * BTSE - bumped time to minute to reduce pinger calling by 75 calls per day. Fix authentication bug and add authentication pass into to-do. Batch outgoing subscription calls * fix type field and batch outgoing subs and unsubs for coinbasepro * Batch outgoing subs and unsubs * Fixes bug when matching return from authentication * Fix bug where params where being sent out of order due to map ,where depth items werent being subscribed too, where trying to subscribe to too many kline items caused error, where trying to get a nano secocond ID conflicted due to speed of generation. * Add websocket capability for currency pair change by utilizing full channel subscription list in subscribe function. * Add error handling * Fix public: time conversions, subscription list, stopped pushing heartbeat to data handler, aggregated list of connections. * hitBTC pass * returned nil instead of error due to period null bids and asks updates coming through. * Fix auth ping capture and reply. Added in interval handling for kline data. Added correct full trade data handling. Fix subscribe and unsubscribe. * Fix when websocket auth conn and token generation fail we don't try and auth sub. Fix bug between auth and normal connection id generation and matching. Batch outgoing payloads to increase efficiency. Updated matching functions to utilise channels instead of waitgroups and go routines. * RM debug output * rm func to get shutdown channel * Add unsubscriber functionality, added wsTicker type, removed return as this will impede data flow and cause reconnection when handling and processing data * okgroup WIP * *Added missing fields for websocket trades *Fix bug processing kline interval *Added fields for websocket ticker struct *Fix auth bug -Updated request and response matching param to interface so we can custom signature match. Stops auth subscribing before a reply is issued. -Updated channel inclusion of pair fo auth subs as this was missing. *Assortment of perfomance improvements * poloniex pass * send all trades to data handler, validated enabled and disable pairs * initial clean * centralised request matching mechanism * websocket main improvements WIP * WIP * Websocket management via gctcli WIP * GRPC expansion * Updated GCTCLI with websocket url and proxy setting functionality which flushes connection * Fix continuous spawning of routines bug on error with reconnection * Addr linter issues * fix subscription bug that I caused when I changed to a switch case * fix linter issue * fix woopsie * End of day WIP * Fix order submission REST, time conversion, order type conversion, orderID bugs * fix gateio test and unsubscribe bug * revert comment out code * websocketAPI changed to to true in configtest.json * fix race in gateio test * End of day WIP for websocket tests. * BugFix for binance when book isn't seeded. Updated websocket tests. Deprecated subscription manager. RM wrapper funcs. * Added string title to exchange name as they are saved as lower case in type, reinstated verbose check in websocket.go * Added verbosity check for setting websocket URL * fix bug where the asset had a mind of its own * purge dodgy coding * Fix tests, drop blocking chan in websocket Dial function * few more changes * race condition fix for websocket tests. * fix intermittent test failure due to underlying hash table storage * Address madcuzbad nit * RM superfluous printlines * Add quick top example with paramater fields * First pass Glorious nits * As per madcozbad suggestion return error when enabled pair not found in full return map. Add test. * addr madcozbadd nits * as per glorious suggestion rm'd loadedJSON field * adjusted ticker, added test and RM'd code that can never be executed * Addr nits and add in crypto rand genration for ID's * remove global channel declaration and rescoped as this was causing a lock * as per glorious suggestion restructured return error for websocket * addr glorious suggestions * fix linter issues * purge non-existent pair from testdata * add side field to struct and parse * addr glorious nits * Add verbosity to error returns and logs and fix string parsing in GCTRPC * fix speeling mistwake * Adds websocket functionality check before flushing websocket connection * Addr kraken panic and setting/flush websocket url stage one. * added websocket url check before setting with tests * Added in edge case test if by the time we call contains on available pairs it has been changed * remove error return for func * Continuation of tests * continuation of tests * Stop potential panic within pair creation function * Implement changes to upstream * rm sup comment * fix bug when subscribing and unsubscribing. Also add in boolean to determine there are currencies that need to be flushed via set pairs via gctcli * fix test * Fix linter issues * Fix tests * turn websocket off in config example * Fix issue where you cannot enable websocket when config is set to false, also added config websocket enable state saving * Introduced err var for same error returns * Add err var exchange base not found * restructure function * drop gctscript from generic response name * drop managesub delay const as its not being used * correctly implement websocket rate limiting for coinut * remove quotations * drop pair management check * fix spelling * return error in function to not update currency with unset role * amalagamted enable/disable into set function and added in pairstore fetch function * update error description * rm function * moved test function to sharedtestvals and move type to types.go * append delimiter onto currency delimiter strings * add test coverage * rm functions as they are set as methods in base * remove superfluous methods * Fix issue that would occur when a subscription errored and not appending successful subs * fix after rebase woopsie * fix linter issues * fix bug streamline code * fix linter issues * fix linter issues * fix case where it should not change ID if set but append new * fix whoopsie * fix websocket tests * fix readme, fix wrapper issues reporting template, go mod tidy * add test coverage * add test coverage and verified futures pariing * add in futures bypass as its not currently supported on BTSE until API update and implementation * removed downside/upside profit contract type as its no longer supported. Added in check in set config pairs to warn user of potential conflict and to manually remove or update. * If asset enabled add pair and increase code coverage * remove strings.title, set and fetch with strings.Lower but keep struct field exchangename unchanged. Streamline ticker and orderbook code. * Add code coverage * log error if setting default currency fails, add code coverage * address glorious nits * Addr xtda nits * fix linter issues * addr glorious nits * xtda nits * Addr glorious nits * add subscription protection and removed a superfluous wait call * fix test * fix whoopsie * addr xtda nits * addr glorious nits * Added asset types to subscriptions structs, also added in error handling for resubscription errors * consolidated rpc returned type and added in sucessful strings * dropped stream timing down to 100ms * DOC changes * proxy and url usage string additions * WIP * go mod tidy rides again * Addr nits * Addr nits, fix tests * fix wording * add in test case for currency matching * Add byte length check on outbound websocket payload subscriptions * addr thrasher nits * Addr madcozbadd nits * addr linter issues * Addr glorious nits by amalgamating function into one mega amazing function. * fix futures account subscription bug * addr glorious nits and reinstated wg.Wait() checks * changed string to currency delimiter string and setconnected by function |
||
|
Scott
|
b686cf2e0e |
Feature: Websocket order handling (#446)
* Initial changes, removing exchange name as an arg and puts it in the pointer struct. Adds case to ws routines * Adds CancelAllOrders func, adds GetByExchangeAndID. Adds modify handler in routines.go * initial poor attempts to have bitmex work with new datahandler handlers. fixes ordersides * bitmex Completes new order * Better bitmex handling, but not complete. Begins a gargantuan task of unifying order data structs. Sometimes an order update will contain lot's of information, so its best to be able to update all fields of our orders, rather than just an arbitrary subset. As a result, everything will be broken for the foreseeable future :glitch_crab: * Removes old order handler which did nothing. Updates order properties for everything everywhere - now consistent. Changes order status. Adds asset type and wallet address to all order types * Adds order updater to update only relevant fields since the object is generic, we don't know what fields are passed from what exchanges. Adds "lastupdated" field to order.Detail. Expands order cancellation for engine orders. * Ensures that new orders are added to the ordermanager's order store. Saaa many comments. Internalises orderStore get func. Adds internalOrderID to orderdetail and adds websocket support for it * Fixes a cancelAllOrders oopsie doopsie * Adds potential func to update orderdetails from an orderdetail struct. Unsure if will keep. * Begins btcmarkets implementation. Expands order "stringToOrder" funcs to allow for some more flexible string coversions. Removes order.Submit via websocket as it would cause unlimited order place issues :D * Finishes btc markets without testing * Adds untested ws auth func to btse * Finises btse, fixes btcmarkets bug * Adds coinbasepro support * Fixes a few more fields in coinbase pro and readds the extra subs * Begins work on coinbene. Plus theyve added a new ws connection yeee * Wasted a bunch of time adding support to an additional websocket that isn't needed ;_; Fixed a bug in coinbasepro. Fully kitted out coinbene support. Updates order types with all fields * Removes extra websocket connection ;_; * Finishes gemini. Fixes order side unknown * Adds okgroup support. Moves byte reading to another function to allow for unit testing. Updates routines to use pointers. Updates date update handling for order details * Finishes order data for okgroup websocket, but starts the STRANGE process of converting all other websocket endpoints to be a little less silly * Cleans up okroup websocket implementation. Fixes bug in Gemini * Adds poloniex support. Updates ws order handling * new bitmex support. Adds some tests now that its all in its own func. Fixes poloniex bug * Begins work on authenticated binance websocket * Attempts to track user data via binance websocket * Maybe finishes Binance websocket support * Begins adding test coverage to orders.go. Updates names of script properties to match updated * Begins an experiment with code coverage. Fixes more rebase issues * Completes orders coverage. Botches a few other things though. Fixes more scripting stuff * All tests in engine package pass * Adds some loevely routine tests * Moves ordermanager to test Bot ordermanager Adds lovely routine tests to ensure things that get sent to be handled the data handler are handled by the data handler by handling them * Replaces "wsHandleData" with "wsReadData" as that's what its going to do now. * Splits all wsHandleData into wsReadData and wsHandleData to allow for easy testing via sending []byte json examples to test proper functionality. Breaks so many tests * Fixes majority of test issues. But data races which are tough on the engine package * "Fixes" test by removing shutdown test. It interferes with too many things. Requires some thought * Tests all the binance websocket points * Adds better bitfinex websocket support. * Adds testing for bitfinex, bitstamp and btcmarkets. Fixes websocket bugs encountered * Adds BTSE ws tests. Fixes bugs in ws * Adds coinbase pro tests. Fixes any issues * Coinbene tests * Starts to handle coinut. Runs into a problem conceptually regarding websocket roundtrip and orders. Both events need to happen without impacting eachother/racing * Addresses a data race issue regarding websocket and bot order management submission - order submission locks at an earlier point to prevent routines.go from creating an order before order submission creates it. Updates rpcserver to use order management bot to submit orders. * Finishes the hectic coinut testing * Adds tests for gateio * Fixes rebase issues. Updates tests to work without being overloaded * Begins testing of gemini. fixes up minor issues * ginishes gemini tests and fixes * Adds hitbtc tests. Fixes all the many issues with hitbtc websocket * Adds remaining tests. Increases default test channel limit again * Begins work towards huobi tests * Finishes huobi tests * Fixed all mythical rebase adventures * Begins kraken transformation * Finishes kraken. Fixes coinbene leverage now that its changed * Begins okgroup testing * Adds okgroup ws tests * Does some poloniex * Fixes basic curreny issue by extracting to func * Begins redesign of poloniex websocket datahandling. Completes authenticated handling, now onto unauth * Finishes poloniex revision * Finishes ZB additions * Fixes data races * Fixes rebase issues. Fixes bad kraken logic * Fixes after reviewing code * lint everywhere * Fixes lingering lints * lint * Adds test coverage to order detail and modify updating * Fixes linting * Fixes huge int, fixes date tests * Adds GetByExchange, adds test for it. Protects fakepass echange. Renames DisplayQty to DisplayQuantity. Removes verbose. Adds some websocket properties. Updates bitmex asset type in test * Addresses timestamps, type abbreviations, verbosity. Expands binance kline switch cases. Updates some websocket capabilities. * Adds coverage to the stringToOrderType/Status functions introduced in PR * Minor fixes addressing some time, error text and use of StringDataCompareInsensitive * Introduces shiny new system which checks if there is an awaiting ID, if found, processes via wrapper method, else, goes through wsHandleData method. Removes weird locking system from wrapper/websocket data race. Updates bitfinex to properly handle websocket order requests and notifications * Moves fakePassingExchange to test_helper. Fixes some order side implementations for trades. Botches a new error type * Adds new error type to track and handle order classification errors separately * Fully fleshes out ClassificationError for all instances of status conversion. Even in order trades and some wrapper functions * Introduces common.SimpleTimeFormat for "2006-01-02 15:04:05". Fixes binance and bitfinex issues with auth endpoint use, map casting. Expands more order.ClassificationError usage. Fixes some more generic websocket response errors * Future proofs order updating by utilising asset types. Expands testing to accomodate. Adds shiny new time type. Expands wrapper websocket functionality definitions * minty linty * Broken end of day code addressing basic nits on comments, returns and currency conversion * Adds testing to btcmarkets websocket. Also updates websocket orderbook to use update instead * Fixes fun rebase fun fun so fun * Addresses minor nits regarding changed interface and comments * Creates new function `GetRequestFormattedPairAndAssetType` to retrieve a currency pair and asset type based on a string. It will iterate over enabled pairs and compare them to formatted pairs and then return that pair if found. * Fixes test * Adds a single line to the end of the file, because that would be really bad if it wasn't there * Updates fakepassexchange to not use params, updates test params, uses fatal in some tests where its important, updates order manager to have a rwmutex, removes some returns, improves ws key test for binance, updates properties to reflect their actual values, adds some more websocket properties * Addresses binance switch linting * Updates leverage property to int64 * Fixes what was broken |
||
|
Ryan O'Hara-Reid
|
6d8ba0a96a |
Initial implementation of HTTP mock testing framework (#310)
* Initial implementation of HTTP mock testing framework Convert to VCR testing server. Segregate live testing via build tags. Converted Binance to VCR server Convert Bitstamp to VCR mocking tests Added VCR mock testing for localbitcoins * Add server generation for concurrent testing * Fix linter issues * Fix linter issue * fix race - potentially * revert auto assigning of host vals * Fix requested changes * Adds mock testing for ANX Switch to using TestMain functionality Added cron job usage for travis-ci to live testing Added appveyor scheduled build check for live testing * WOOPS * silly correction * Fixes fantastic linter issues * fixed another whoopsie * WOOO! * Adds gemini mock testing with additional fixes * Add docs and sharedvalue * Added tls using httptest package * Fixed issues * added explicit mock recording reference to error * Fix requested changes * strip port from mock files as they are not needed on tls server * Change incorrect names * fix requested changes * lbank update * Fix another issue * Updated readme |
||
|
Scott
|
0fbf8b172a |
Websocket orderbook buffering (#333)
* Initial commit setting up a map orderbook system with a buffer. It will write to the buffer, sort apply to main orderbook and then process. * Moves namespaces again * Updates orderbook to use a sweet new WebsocketOrderbookUpdate type to handle all updates whether its using ID or not. So good. Adds many tests * Starting to implement orderbook update handling per exchange. Updates namespaces again. Hopefuylly will find a way to update via ID not timestamp, too many endpoints dont provide update timestamps * Changes orderbookbuffer to use BufferUpdate type instead of orderbook.Base to achieve more functionality and no need for type conversion functions. Updates tests * Updates all instances of ws.orderbook.Update. Simplifies some orderbook logic * Introduces toggleable buffer. Renames orderbooks. Completes implementation for everywhere but OKGroup due to hash calculation * Implements orderbook update for okgroup, but forgets about the orderbook hash checking * Fixes okgroup checksum calculation. Fixes linting issue. Removes redundant Kraken tests. * Introduces sorting toggle and separates from buffer toggle. Uses benchmarks to highlight performance gains * Fixes Gemini rate limit and parsing. Removes comments and fixes typos * Fixes bitfinex orderbook processing * Inbuilt sorting, minor fixes for websocket implementations. Improves test coverage * Adds surprise LakeBTC websocket support * Fixes data race * Fixes rebasing issues due to namespace movements * Addresses PR nits: moves folder namespace from ws to websocket. Removes line spaces in imports. Fixes lakebtc websocket returns and defer fucntions. Fixes comments * Adds poloniex orderook sorting support * Enables bitstamp and hitbtc orderbook sorting. Fixes poloniex's sorting * Renames namespaces and combines monitor and connection into wshandler. Removes unused SPOT const. Changes how orderbook stuff is loaded. It is done in startup with a setup. Removes exchange name from loadsnapshot as well * Removes the connection.go from rebasing issues. Removes error response from functions used in goroutines * Fixes test with exchange name output change * Fixes issues where copy and paste and replace all were used poorly |
||
|
Scott
|
e209d85d0d |
Websocket request-response correlation (#328)
* Establishes new websocket functionality. Begins the creation of the websocket request * Adding a wrapper over gorilla websocket connect,send,receive to handle ID messages. Doesn't work * Successfully moved exchange_websocket into its own wshandler namespace. oof * Sets up ZB to use a round trip WS request system * Adds Kraken ID support to subscriptions. Renames duplicate func name UnsubscribeToChannels to RemoveSubscribedChannels. Adds some helper methods in the WebsocketConn to reduce duplicate code. Cleans up ZB implementation * Fixes double locking which caused no websocket data to be read. Fixes requestid for kraken subscriptions * Completes Huobi and Hadax implementation. Extends ZB error handling. Adds GZip support for reading messages * Adds HitBTC support. Adds GetCurrencies, GetSymbols, GetTrades WS funcs. Adds super fun new parameter to GenerateMessageID for Unix and UnixNano * Adds GateIO id support * Adds Coinut support. Prevents nil reference error in constatus when there isnt one * Standardises all Exchange websockets to use the wshandler websocket. Removes the wsRequestMtx as wshandler handles that now. Makes the Dialer a dialer, its not externally referenced that I can see. * Fixes issue with coinut implementation. Updates bitmex currencies. Removes redundant log messages which are used to log messages * Starts testing. Renames files * Adds tests for websocket connection * Reverts request.go change * Linting everything * Fixes rebase issue * Final changes. Fixes variable names, removes log.Debug, removes lines, rearranges test types, removes order correlation websocket type * Final final commit, fixing ZB issues. * Adds traffic alerts where missed. Changes empty struct pointer addresses to nil instead. Removes empty lines * Fixed string conversion * Fixes issue with ZB not sending success codes * Fixes issue with coinut processing due to nonce handling with subscriptions * Fixes issue where ZB test failure was not caught. Removes unnecessary error handling from other ZB tests * Removes unused interface * Renames wshandler.Init() to wshandler.Run() * Updates template file * Capitalises cryptocurrencies in struct. Moves websocketResponseCheckTimeout and websocketResponseMaxLimit into config options. Moves connection configuration to main exchange Setup (where appropriate). Reverts currencylastupdated ticks. Improves reader close error checking * Fixes two inconsistent websocket delay times * Creates a default variable for websocket ResponseMaxLimit and ResponseCheckTimeout, then applies it to setdefaults and all tests * Updates exchange template to set and use default websocket response limits |
||
|
Scott
|
3a66e99899 |
Authenticated Websocket support (#315)
* Improves subscribing by not allowing duplicates. Adds bitmex auth support * Adds coinbase pro support. Partial BTCC support. Adds WebsocketAuthenticatedEndpointsSupported websocket feature. Adds GateIO support * Adds Coinut support * Moves Coinut WS types to file. Implements Gemini's secure WS endpoint * Adds HitBTC ws authenticated support. Fixes var names * Adds huobi and hadax authenticated websocket support * Adds auth to okgroup (okex, okcoin). Fixes some linting * Adds Poloniex support * Adds ZB support * Adds proper bitmex support * Improves bitfinex support, improves websocket functionality definitions * Fixes coinbasepro auth * Tests all endpoints * go formatting, importing, linting run * Adds wrapper supports * General clean up. Data race destruction * Improves testing on all exchanges except ZB * Fixes ZB hashing, parsing and tests * minor nits before someone else sees them <_< * Fixes some nits pertaining to variable usage, comments, typos and rate limiting * Addresses nits regarding types and test responses where applicable * fmt import * Fixes linting issues * No longer returns an error on failure to authenticate, just logs. Adds new AuthenticatedWebsocketAPISupport config value to allow a user to seperate auth from REST and WS. Prevents WS auth if AuthenticatedWebsocketAPISupport is false, adds additional login check 'CanUseAuthenticatedEndpoints' for when login only occurs once (not per request). Removes unnecessary time.Sleeps from code. Moves WS auth error logic to auth function so that wrappers can get involved in all the auth fun. New-fandangled shared test package, used exclusively in testing, will be the store of all the constant boilerplate things like timeout values. Moves WS test setup function to only run once when there are multiple WS endpoint tests. Cleans up some struct types * Increases test coverage with tests for config.areAuthenticatedCredentialsValid config.CheckExchangeConfigValues, exchange.SetAPIKeys, exchange.GetAuthenticatedAPISupport, exchange_websocket.CanUseAuthenticatedEndpoitns and exchange_websocket.SetCanUseAuthenticatedEndpoints. Adds b.Websocket.SetCanUseAuthenticatedEndpoints(false) when bitfinex fails to authenticate Fixes a typo. gofmt and goimport * Trim Test Typos * Reformats various websocket types. Adds more specific error messaging to config.areAuthenticatedCredentialsValid |