* OrderManager: Fix race condition in submit with ws
If the ws sees the order before processSubmittedOrder then it will have
assigned it an internal order id already and added it to the store.
Don't treat that as an error. Instead just use the newer ws details
* OrderManager: Fix error comparisson
Should always use errors.Is when possible
* Tests: Simplify btcusd test pair declaration
* OrderManager: Improve test readability
* OrderManager: Add orderstore.getByDetail test
* Return a fresh pointer from orderstore.getByDetail
This protects the order.Details in the store from direct access.
The use-case was to allow the returned objects to be references so that
future changes to them would be reflected.
However we're not ready yet to allow people to touch the orders
directly, because they're not protected directly by a mutex, and nothing
would stop consumers contaminating the integrity of the data.
We can revisit this topic later atomicly, but it's definitely tangental
to the cause of action for PR #1336.
* Fix GetByDetail tests to assert a new pointer
* OrderManager: Avoid possible lock races
This fix internalises the getByDetail because the implication of moving
lock ownership out of exists/getByDetail to consumers breaks the order
store struct encapsulation in a way we really don't want to.
It's also more efficient
* Fix spelling mistake
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
* OrderManager: Fix TestSubmitOrder... description
* OrderManager: Improve clarity of comment
* OrderManager: Capitalise error message
On failure to add to orderstore, capitalise the error message
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
---------
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
* Bitstamp: Add auth channel subscription handling
* Bitstamp: Avoid searching for asset type
We've hardcoded asset.Spot in order to find the pair.
Looking the asset up from the pair makes no sense.
* Bitstamp: Add type for wsOrders
* Bitstamp: Working test of Generic DataHandler
* Bitstamp: WS Order chan tests and remove type
orderType could be derived from status == New and Buy & price == 9+e9 or Sell & price == 0
But it would only be true for the first update and it really doesn't feel worth the risk
Consumers are going to have to merge to original request anyway
* Bitstamp: Linter fixes
* Kraken: Switch to shared fixture test
* Bitstamp: Fix lint on TestFixtureToDataHandler
* Engine: Add Clone for PairsManager
go-vet highlighted that the mutex here is a value when we copied the
PairsManager in a test.
Options to fix:
* Add a deep clone method to PairsManager
* Add a shallow clone method with a disclaimer
* Make the mutex a pointer
* Make the PairsManager itself a pointer
Options 3 and 4 are too invasive to justify changing at this point.
There's an inherent risk of PM being passed by value, but govet should
catch the copylock.
There's more risk in changing everything to use a pointer at this stage.
* Engine: Fix linter again, ironically
* Bitstamp: Rename OHLC const
* Bitstamp: Minor fixes to syntax
* Bitstamp: Simplify chanSymb=>pair
* Bitstamp: Still process order updates without ID
If there's a ClientOrderID we'll still process the order.
It doesn't seem likely we'd have this happen, but if it does we still
want consumers to get something.
* Bitstamp: Replace Clone with Lock methods
* Engine: Expose PairsManager's Mutex
Makes more sense than wrapping functions
* Bitstamp: Fix linter copylock (again)
* fixup! Engine: Expose PairsManager's Mutex
Omit Mutex from Json
* fixup! Bitstamp: Add auth channel subscription handling
Remove unused wsAuthToken
* Bitstamp: Simplify OrderData Unmarshal
* Bitstamp: Remove unused contexts
I added these following best practices, but the reality is that when/if
we get context awareness in GCT, there will be a lot more to fix and
this will be a drop in the ocean anyway.
* Bitstamp: Only call handleWSOrder for MyOrders
* Bitstamp: Avoid allocating again in handleWSOrder
* CurrencyPairs: Remove public mutex
Simplified to a Load method to avoid making mutex public
* Tests: Improve test readability and clarity
* Bitstamp: Wrap errWSPairParsingError
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
* Bitstamp: FetchWSAuth mock and live test
---------
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
* engine: changes (cherry_pick_me_pls_ser)
* engine/helpers: refactor
* glorious nits: purge code
* engine/helpers: update tests so that we can ensure a default exchange and setup is good to go.
* -_-
* glorious: nits
---------
Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
* moves everything to use single map keys, also breaks
* full rollout
* tests
* fix a little bug
* minor test fixups
* Fix Key use
* rm 🔑 from 🔑 struct name
* all in a days work
* cleanup
* cleanup for real, also stop it binance.json
* minor coverage
* adds gateio to the slurry
* cleanup of types
* verbose verbose verbose verbose verbose verbose
* fixes huobi parsing issue
* fix bybit contract identification
* cleanup
* merge fixes
* addresses many big problems raised by SHAZ
* tracking errors and fixes
* funding rate if avail, fixes currency formatting
* Addresses nits and sneaks in extra fixes
* lint
* minor fixes after rebase
* better contract splitter for currencies like T-USDT
* forgot to add the exchange name like a fool
* merge fixes x1
* kucoin, direction, contract size
* rn direction, fix kucoin time
* WHOOPS
* Update exchanges/kucoin/kucoin_wrapper.go
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
* misdirection
---------
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
* Okx: Remove redundant asset types from conversion
* Okx: Fix list of private channels for subscription
* Okx: Fix ws order update Acc and Rem amounts
The amounts aren't treated as deltas by anything receiving them
upstream, e.g. orderManager or wsRoutineManager.
So we should treat them the same as the REST implementation
* Okx: Add context to stream unmarshall errors
I encountered a series of errors with unmarshalling from okx and have no
idea what the messages which weren't valid json actually were.
This fix adds the raw message for context to the error
* Okx: Reduce ping interval
Seeing occassional disconnects and suspect dropping the ping interval
down might help
* Fix ClientSupplier and ClientSupplied inconsist
This simply removes the Supplie. part in favour of the simpler name used
in binance. Seems safe since its already inconsistent and not populated
in at least ws order handlin
* Okx: Fix ClientOrderID ignored in ws orders
* Okx: Standardise Rfq abbrev; Fix RfqIDs
* Okx: Fix RfqIds outside of ClientRfqIds
* Okx: Simplify TestUpdateOrderExecutionLimits
* Okx: Fix panic on no avgPrice and QuoteAmount
* init
* updates config
* wrapper configuration
* updates exchange readme
* adds SendAuthHTTPRequest and SendHTTPRequest
* adds ratelimit file
* adds test case and minor fixes
* improve error handling
* update testcases and improve GetSymbols API
* adds SPOT API's
* minor fix
* WIP
* WIP
* adds test case
* adds check in test case
* fixes in Auth. HTTP
* improvements
* adds trade, kline support and testcases
* adds SPOT API and testcases for same
* adds SPOT API and testcases
* adds SPOT API and testcase
* WIP
* adds API's
* adds API's
* adds test cases
* adds comment to exported data types
* adds API and test cases
* adds API
* adds API
* rearrange functions
* WIP: adds API
* adds API for Post Order SPOT
* adds API and few fixes
* fixes
* WIP
* WIP
* add PostBulkOrder API and its test case
* fix issues
* adds cancel order APIs and test cases for same
* add minor test fixes
* add API
* adds API
* fixes
* add API
* adds API and test cases
* fix test
* adds API
* adds test
* fix test
* adds API and test
* adds deposit API and test cases
* WIP
* adds API and test cases
* WIP
* WIP
* add public future API and test cases
* WIP
* remove v2 API and replace them with v1
* update test cases
* adds future order API and test cases
* adds futures order API
* adds API
* add API and test cases
* adds API and test cases
* adds API and test cases
* adds API and test cases
* Adding wrapper functions
* Fix on wrapper function
* Adding websocket support
* Complete addressing WS push datas
* Adding spot push data unit tests
* adding futures websocket push data handlers
* Adding futures websocket push data handlers
* Added unit tests
* Updating unit tests
* Updating wrapper and unit test functions
* Adding missing wrapper functions and code cleaning up
* Resolved linter issues
* Fixing websocket issues
* Fixing websocket issues
* Slight fix on config_example file
* Minor update
* Basic nits updates
* Fix minor linter issues
* Minor update
* Minor unit test update
* Minor unit test update
* Code update and linter issues fix
* Removed unnecessary type conversion codes
* Monor update based on review comment
* Fix based on review comments
* Adding rate-limiter
* Websocket update and overall minor fixes
* Removed IsAssetTypeEnabled method implementation
* Fix connection and formatting issues
* Updating orderbook issues
* Very minor label fix
* Minor error returning fix
* code cleaning up and minor spelling fix
* Updates on unit test
* Update on unit tests and slight code structure
* unit test update
* orderbook update and minor fix
* fix on race
* Mini linter fix
* fix minor parameter and unit test issues
* handler funcs and models update
* Fixing websocket and unit test issues
* order side string for active orders
* Fix on websocket and unit tests
* Minor type changes
* Minor Orderbook fix and unit test update
* Small fix on orderbook
* Updating orderbook functionality
* FIx on websocket orderbook handlers
* Small update on kucoin websocket
* fix missed review comments
* fix based on review comments
* Updating websocket orderbook and fixing unit tests
* Minor fixes
* unit test update
* Updating unit test according to enabled asset type
* toggle canManipulateRealOrders const
* Unit test update
* Fix minor issues
* minor fix
* documentation fix
* wrapper coverage and unused params fix
* testing and minor changes
* documentation, websocket and unit test update
* minor linter fix
* Websocket spot/margin subscription update
* minor ticker update fix
* minor fixes on endpoints
* timestamp and number convert method and unit tests
* timestamp convert minor update
* minor type and conversion fix
* create a common timestamp convert and fix minor issues
* linter and ticker fix
* Updating unit tests and order placing endpoint methods
* Added a pairs check
* Fix config test error
* rm unused error variable
* Fix source of linter issue
* code update: convert, wrapper and websocket fix
* minor code update
* Websocket code and unit tests update
* Websocket ticker ask/bid type change and small error msg fix
* docs update
* fix: websocket orderbook handling
* change orderbook channel to marketOrderbookLevel2Channels and fix websocket orderbook update
* Minor func rename and reciever change
* Minor orderbook unit test issue fix
* comment: about why we used a random delimiter '-' for futures
* update config files and FetchTradablePair func for futures pairs
* futures config pairs update
* remove ConnextionMonitorDelay from websocket setup
* fix on types and futures pair conversion
* updating config pairs
* change NewPairFromString to DeriveFrom
* unit tests update
* unit tests update
* Added TickerBatching
* added GetStandardConfig to GetDefaultConfig
---------
Co-authored-by: Jaydeep Rajpurohit <jaydeeppurohit1996@gmail.com>
* init
* surprise train commit
* basic distinctions
* the terms of binance are confusing
* renames and introduction of allocatedMargin
* add new margin funcs
* pulling out wires
* implement proper getposition stuff
* bad coding day
* investigate order manager next
* a broken mess, but a progressing one
* finally completes some usdtmargined stuff
* coinMfutures eludes me
* expand to okx
* imports fix
* completes okx wrapper implementations
* cleans and polishes before rpc implementations
* rpc setup, order manager features, exch features
* more rpc, collateral and margin things
* mini test
* looking at rpc response, expansion of features
* reorganising before the storm
* changing how futures requests work
* cleanup and tests of cli usage
* remove silly client side logic
* cleanup
* collateral package, typo fix, margin err, rpc derive
* uses convert.StringToFloat ONLY ON STRUCTS FROM THIS PR
* fix binance order history bug
* niteroos
* adds new funcs to exchange standards testing
* more post merge fixes
* fix binance
* replace simepletimeformat
* fix for merge
* merge fixes
* micro fixes
* order side now required for leverage
* fix up the rest
* global -> portfolio collateral
* Update exchanges/collateral/collateral_test.go
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
* adds fields and todos
* rm field redundancy
* lint fix oopsie daisy
* fixes panic, expands error and cli explanations (sorry shaz)
* ensures casing is appropriate for underlying
* Adds a shiny TODO
---------
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
* exchanges: Add function to get standard config
* exchanges: add tests (cherry-pick here and above)
* after pick stuff
* cleanup
---------
Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
* WS: Relay disconnect errors to subscribers
Subscribers probably care when the WS got disconncted.
Tell them and expose a method to test the error for matching
* Fix linter error
* Engine: Fix false test passes for nil OrderManager
TestWebsocketRoutineManagerSetup tests that passing a nil value returns errNilOrderManager;
However that's not actually what would happen when order manager is configured off.
The arguments to setupWebsocketRoutineManager are interface types.
When a nil pointer of interface type is passed, it does NOT equal nil.
nil is a primitive type.
A nil pointer of interface type has the value (nil;type).
See [the Go lang spec](https://golang.org/ref/spec#Comparison_operators):
```
Interface values are comparable. Two interface values are equal if they have identical dynamic types and equal dynamic values or if both have value nil.
```
So that means that whilst this test was passing, because it was sending
in a real nil value and comparing it to a real nil, that's not what
would happen at runtime. At runtime the bot.OrderManager would be a nil
pointer to a concrete type *OrderManager, and so not comparible to nil.
This commit just fixes that oversight, and explains the often
misunderstood mechanics of comparing interface types to nil.
In practical terms this means that the tests assert that the WSRM would
not run without a OM, but in fact it actually would. And panic later.
This commit SHOULD introduce a FAILing test. Sorry if you're bisecting.
* WSM: Fix error on OrderManager not enabled
It's okay for OrderManager to not be enabled; It's configurable.
Remove the WSM setup protection and move it to runtime using IsRunning.
Left the WSM deps so they can be fixed as apporpriate to each.
* orderbook: Check assignment of time values and reject if not set.
* linter: fix
* buffer: additional linter winter fixter
* Implement through pending exchanges
* finished push
* linty: minty
* gomod: tidy
* thrasher: nits
* glorious: nits
* orderbook: purge type now in favour of external call allocation
* orderbook: push last param
* orderbook: only 1 unlock call is needed
---------
Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
When Kraken has a disconnect or failure that sets SetCanUseAuthenticatedEndpoints(false),
it's never re-enabled when the websocket is reconnected
That means all subsequent requests would fall back to rest
* Kraken: Fix error on WS update without status
* Kraken: Fix parsing of WS updates without Desc
* Kraken: Fix WS market order handling
* Kraken: Fix tests in parallel using DataHandler
* WebsocketManager: Fix order summary printing
* Kraken: Fix parallel tests race with DataHandler
* Kraken: Issue a classification err on asset 404
* Kraken: Switch to testify and close fixture
* Bitfinex: Fix cancel/update order WS ack not seen
Fixes#1288
* Bitfinex: Fix ws Unsubscribe and Resubscribe
Unsubscribe needed to use the channel id.
Resubscribe needs to have the original subscription params.
* Bitfinex: Fix ws Trades Fees on te
The ws channel for authenticated Trades sends two types of update:
* te, Trade Executed
* tu, Trade Execution Update
Only the second one contains fee information.
[See the docs](https://docs.bitfinex.com/reference/ws-auth-trades)
This commit fixes:
`exchange Bitfinex websocket error - unable to type assert trade fee`
after an executed market trade on the te update
* Bitfinex: Fix error on ws auth ok
This fixes:
`Bitfinex Could not find an existing channel subscription: account Pair:
ChannelID: 0`
It's not clear from history why we'd want to store a reference to the
ubiquitous 0 channel like this, but it's definitely wrong, and anything
that attempts to get channel information about 0 chan needs to be fixed
anyway.
* Bitfinex: Refactor wsUpdate handling
This commit doesn't break out all the sub-updater, but attempts to do
something about the unmanagable size of ws update handling
* Binfinex: Fix linter issue on chanId casing
* Bitfinex: Fix linter outdent complaint
* Bitfinex: Fix linter issues on test
* Bitfinex: Fix TestWsTradingPairSnapshot chan lookup
* Bitfinex: Remove unnecessary WsAddSubs in test
* Bitfinex: Fix TestWsSubscribedResponse chan
* Bitfinex: Throw a specific error for bad event
* Bitfinex: WS Type assertions for positionSnapshots
* Bitfinex: tradeUpdate type assertion
* Bitfinex: Reinstate default subscriptions
* Bitfinex: Assert chan assetType is the same
* Bitfinex: Lowercase error string
* Bitfinex: Refactor WS eventType/chanId handling
* Bitfinex: Fix linter issues
* Bitfinex: Fix delimiter for pairs with more than 6 chars
* Bitfinex: Fix WS handling of subscribed symbols
This simplifies the handling of subscription symbols.
Now that we know the channel up front from handling the subscribed
response we can limit the parsing forms needed
* Bitfinex: Placate the linter
* Bitfinex: Disable margin assets for WS
Margin WS Currently not fully implemented and causes subscription collisions with spot
* Bitfinex: Fix parsing of 4 part funding keys
This improves overall handling and errors on a few current assumptions
about key structure
* Bitfinex: Linter fixes
* Bitfinex: Remove key parsing from assetPairFromSymbol
* Bitfinex: Use native error wrapping
* Bitfinex: Skip disabled assets in default ws subs
* Orderbook: Add GetDepth to Base
Base.GetDepth returns the concrete book of which Base is a copy
This is probably useful for immutably monitoring orderbook health and state
whereas FetchOrderbook would trigger a refresh.
* Orderbook: Reword GetDepth comment
* Orderbook: Add test for Base.GetDepth
* allows sync manager customisation for values and logs
* config-example add
* who doesnt like more coverage?
* ensures you can actually disable it via config el oh el
* less ifs, better control
* fix verbose
* sync trades default false
* fix summary being printed when not enabled
* fixes config checker and output
* nits
* I can put this behind me now
* Fixed logCaSiNg
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
* combines if statements
---------
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
This prevents the frequent (5x / hour) disconn/reconns we're seeing with a
10s or even 20s traffic timeout.
I'd like to base the interval on the traffic timeout / 2, but that's
non-trivial right now, and 8s isn't an excessive default
* starting public endpoints
* Adding public endpoints
* added public spot market endpoints
* websocket subscriptions updates
* websocket push data handlers completing
* linter fix
* Added funding private endpoints
* Adding authenticated account endpoints
* Added fiat and OTC-RFQ authenticated endpoints
* trading authenticated endpoints
* completing trade endpoints and add public wrapper endpoints
* Authenticated wrapper functions and corresponding unit test
* Adding authenticated websocket endpoint and fixing wrapper functions
* Documentation and exchange websocket update
* update websocket orderbook checksum handling
* linter issues fix and unit test update
* remove invalid orderbook endpoint and unit test
* Documentation, handlers, and model types update
* minot fix
* Minor fixes
* Updating unit tests and added missing endpoints
* Add missing credential check
* Minor unit test fixes
* fix minor linter issue
* add snaphot test unit test
* Fix on update checksum and documentation update
* update exchange, add UpdateOrderExecutionLimits, and update documentation
* Minor fix on tickers fetching
* Minor websocket fix and smaill unit tests
* Minor websocket and naming fixes
* uncomment default channels
* Fix type and unit test issues
* websocket channels and data handling update
* Update Advanced-Algo websocket handling and minor fixes
* documentation and minor code fixes
* Fix name changes
* documentation contribution update
* intervalToString method update
* fix exchange_wrapper_standard tests
* Fix minor issues based on exchange_wrapper_standards_test
* Fix wrapper extended candlestick check
* websocket orders fetching error check method update
* Exchange name check and change
* docs: Add missing contributors
---------
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
* stream/buffer: Adds key map optimisation (cherry-pick)
* stream/buffer: Add buffer to DataHandler intermediary. Add field InitialUpdate bool to toggle when first update is seen for initial sync.
* whoops
* buffer: Add difference benchmarks for reference
* glorious: nits (reverting out of context changes)
* RM unused error that will be used later
* purge: benchmark
---------
Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
* fixes tests
* pair formats and extra fixes
* quick change before shazbert sees
* sneaky lint
* adds bybit mock testing and fixes test
* whoops
* error response instead
* classic forgetting to lint
* bybit live test no longer auto-records results
* ty thrasher- Update exchanges/bybit/bybit_wrapper.go
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
---------
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
* Bitstamp: Fix orderbook snapshot blanking LastUpdate
Drive-By: Remove:
`Bitstamp BTC/USDT spot orderbook has zero bid price, filtering.`
It's unactionable and happens every snapshot right now
* Bitstamp: Use ob timestamps for LastUpdate
Obviously.
* SubmitOrder- order.Buy replaced with IsLong
Progresses #1270
* Btcmarkets: order.Buy and order.Sell changed to order.Bid and order.Ask
* Bithumb: SubmitOrder- order.Buy replaced with IsLong
Progresses #1270
* Binanceus: SubmitOrder- order.Buy replaced with IsLong
Progresses #1720
* Bistamp: SubmitOrder- order.Buy replaced with IsLong
Progresses #1270
* Bitfinex: Changed order Side to IsLong
* Binance:switch added to SubmitOrder
* Binance: Replaced Buy and Sell in SubmitOrder with IsLong and IsShort
* Binance:SubmitOrder if statement for order side simplified
