d0zingcat/gocryptotrader
1
0
Fork 0
mirror of https://github.com/d0zingcat/gocryptotrader.git synced 2026-05-30 23:16:52 +00:00
Code Issues Packages Projects Releases Wiki Activity

Feature: Websocket order handling (#446)

Browse Source 
* Initial changes, removing exchange name as an arg and puts it in the pointer struct. Adds case to ws routines

* Adds CancelAllOrders func, adds GetByExchangeAndID. Adds modify handler in routines.go

* initial poor attempts to have bitmex work with new datahandler handlers. fixes ordersides

* bitmex Completes new order

* Better bitmex handling, but not complete. Begins a gargantuan task of unifying order data structs. Sometimes an order update will contain lot's of information, so its best to be able to update all fields of our orders, rather than just an arbitrary subset. As a result, everything will be broken for the foreseeable future :glitch_crab:

* Removes old order handler which did nothing. Updates order properties for everything everywhere - now consistent. Changes order status. Adds asset type and wallet address to all order types

* Adds order updater to update only relevant fields since the object is generic, we don't know what fields are passed from what exchanges. Adds "lastupdated" field to order.Detail. Expands order cancellation for engine orders.

* Ensures that new orders are added to the ordermanager's order store. Saaa many comments. Internalises orderStore get func. Adds internalOrderID to orderdetail and adds websocket support for it

* Fixes a cancelAllOrders oopsie doopsie

* Adds potential func to update orderdetails from an orderdetail struct. Unsure if will keep.

* Begins btcmarkets implementation. Expands order "stringToOrder" funcs to allow for some more flexible string coversions. Removes order.Submit via websocket as it would cause unlimited order place issues :D

* Finishes btc markets without testing

* Adds untested ws auth func to btse

* Finises btse, fixes btcmarkets bug

* Adds coinbasepro support

* Fixes a few more fields in coinbase pro and readds the extra subs

* Begins work on coinbene. Plus theyve added a new ws connection yeee

* Wasted a bunch of time adding support to an additional websocket that isn't needed ;_; Fixed a bug in coinbasepro. Fully kitted out coinbene support. Updates order types with all fields

* Removes extra websocket connection ;_;

* Finishes gemini. Fixes order side unknown

* Adds okgroup support. Moves byte reading to another function to allow for unit testing. Updates routines to use pointers. Updates date update handling for order details

* Finishes order data for okgroup websocket, but starts the STRANGE process of converting all other websocket endpoints to be a little less silly

* Cleans up okroup websocket implementation. Fixes bug in Gemini

* Adds poloniex support. Updates ws order handling

* new bitmex support. Adds some tests now that its all in its own func. Fixes poloniex bug

* Begins work on authenticated binance websocket

* Attempts to track user data via binance websocket

* Maybe finishes Binance websocket support

* Begins adding test coverage to orders.go. Updates names of script properties to match updated

* Begins an experiment with code coverage. Fixes more rebase issues

* Completes orders coverage. Botches a few other things though. Fixes more scripting stuff

* All tests in engine package pass

* Adds some loevely routine tests

* Moves ordermanager to test Bot ordermanager
Adds lovely routine tests to ensure things that get sent to be handled the data handler are handled by the data handler by handling them

* Replaces "wsHandleData" with "wsReadData" as that's what its going to do now.

* Splits all wsHandleData into wsReadData and wsHandleData to allow for easy testing via sending []byte json examples to test proper functionality. Breaks so many tests

* Fixes majority of test issues. But data races which are tough on the engine package

* "Fixes" test by removing shutdown test. It interferes with too many things. Requires some thought

* Tests all the binance websocket points

* Adds better bitfinex websocket support.

* Adds testing for bitfinex, bitstamp and btcmarkets. Fixes websocket bugs encountered

* Adds BTSE ws tests. Fixes bugs in ws

* Adds coinbase pro tests. Fixes any issues

* Coinbene tests

* Starts to handle coinut. Runs into a problem conceptually regarding websocket roundtrip and orders. Both events need to happen without impacting eachother/racing

* Addresses a data race issue regarding websocket and bot order management submission - order submission locks at an earlier point to prevent routines.go from creating an order before order submission creates it. Updates rpcserver to use order management bot to submit orders.

* Finishes the hectic coinut testing

* Adds tests for gateio

* Fixes rebase issues. Updates tests to work without being overloaded

* Begins testing of gemini. fixes up minor issues

* ginishes gemini tests and fixes

* Adds hitbtc tests. Fixes all the many issues with hitbtc websocket

* Adds remaining tests. Increases default test channel limit again

* Begins work towards huobi tests

* Finishes huobi tests

* Fixed all mythical rebase adventures

* Begins kraken transformation

* Finishes kraken. Fixes coinbene leverage now that its changed

* Begins okgroup testing

* Adds okgroup ws tests

* Does some poloniex

* Fixes basic curreny issue by extracting to func

* Begins redesign of poloniex websocket datahandling. Completes authenticated handling, now onto unauth

* Finishes poloniex revision

* Finishes ZB additions

* Fixes data races

* Fixes rebase issues. Fixes bad kraken logic

* Fixes after reviewing code

* lint everywhere

* Fixes lingering lints

* lint

* Adds test coverage to order detail and modify updating

* Fixes linting

* Fixes huge int, fixes date tests

* Adds GetByExchange, adds test for it. Protects fakepass echange. Renames DisplayQty to DisplayQuantity. Removes verbose. Adds some websocket properties.  Updates bitmex asset type in test

* Addresses timestamps, type abbreviations, verbosity. Expands binance kline switch cases. Updates some websocket capabilities.

* Adds coverage to the stringToOrderType/Status functions introduced in PR

* Minor fixes addressing some time, error text and use of StringDataCompareInsensitive

* Introduces shiny new system which checks if there is an awaiting ID, if found, processes via wrapper method, else, goes through wsHandleData method. Removes weird locking system from wrapper/websocket data race. Updates bitfinex to properly handle websocket order requests and notifications

* Moves fakePassingExchange to test_helper. Fixes some order side implementations for trades. Botches a new error type

* Adds new error type to track and handle order classification errors separately

* Fully fleshes out ClassificationError for all instances of status conversion. Even in order trades and some wrapper functions

* Introduces common.SimpleTimeFormat for "2006-01-02 15:04:05". Fixes binance and bitfinex issues with auth endpoint use, map casting. Expands more order.ClassificationError usage. Fixes some more generic websocket response errors

* Future proofs order updating by utilising asset types. Expands testing to accomodate. Adds shiny new time type. Expands wrapper websocket functionality definitions

* minty linty

* Broken end of day code addressing basic nits on comments, returns and currency conversion

* Adds testing to btcmarkets websocket. Also updates websocket orderbook to use update instead

* Fixes fun rebase fun fun so fun

* Addresses minor nits regarding changed interface and comments

* Creates new function `GetRequestFormattedPairAndAssetType` to retrieve a currency pair and asset type based on a string. It will iterate over enabled pairs and compare them to formatted pairs and then return that pair if found.

* Fixes test

* Adds a single line to the end of the file, because that would be really bad if it wasn't there

* Updates fakepassexchange to not use params, updates test params, uses fatal in some tests where its important, updates order manager to have a rwmutex, removes some returns, improves ws key test for binance, updates properties to reflect their actual values, adds some more websocket properties

* Addresses binance switch linting

* Updates leverage property to int64

* Fixes what was broken
This commit is contained in:
Scott
2020-03-03 13:32:14 +11:00
committed by GitHub
parent 9d49184bc6
commit b686cf2e0e
142 changed files with 13867 additions and 6043 deletions
Download Patch File Download Diff File Expand all files Collapse all files

28
engine/events_test.go
View File

@@ -13,10 +13,6 @@ const (
testExchange = "Bitstamp"
)
var (
configLoaded = false
)
func addValidEvent() (int64, error) {
return Add(testExchange,
ItemPrice,
@@ -27,10 +23,7 @@ func addValidEvent() (int64, error) {
}
func TestAdd(t *testing.T) {
if !configLoaded {
loadConfig(t)
}
SetupTestHelpers(t)
_, err := Add("", "", EventConditionParams{}, currency.Pair{}, "", "")
if err == nil {
t.Error("should err on invalid params")
@@ -52,10 +45,7 @@ func TestAdd(t *testing.T) {
}
func TestRemove(t *testing.T) {
if !configLoaded {
loadConfig(t)
}
SetupTestHelpers(t)
id, err := addValidEvent()
if err != nil {
t.Error("unexpected result", err)
@@ -71,10 +61,7 @@ func TestRemove(t *testing.T) {
}
func TestGetEventCounter(t *testing.T) {
if !configLoaded {
loadConfig(t)
}
SetupTestHelpers(t)
_, err := addValidEvent()
if err != nil {
t.Error("unexpected result", err)
@@ -254,10 +241,7 @@ func TestCheckEventCondition(t *testing.T) {
}
func TestIsValidEvent(t *testing.T) {
if !configLoaded {
loadConfig(t)
}
SetupTestHelpers(t)
// invalid exchange name
if err := IsValidEvent("meow", "", EventConditionParams{}, ""); err != errExchangeDisabled {
t.Error("unexpected result:", err)
@@ -308,9 +292,7 @@ func TestIsValidExchange(t *testing.T) {
if s := IsValidExchange("invalidexchangerino"); s {
t.Error("unexpected result")
}
if !configLoaded {
loadConfig(t)
}
SetupTestHelpers(t)
if s := IsValidExchange(testExchange); !s {
t.Error("unexpected result")
}

72
engine/exchange_test.go
View File

@@ -6,45 +6,29 @@ import (
"github.com/thrasher-corp/gocryptotrader/exchanges/bitfinex"
)
var testSetup = false
func SetupTest(t *testing.T) {
if !testSetup {
var err error
Bot, err = New()
if err != nil {
t.Fatal(err)
}
testSetup = true
}
if GetExchangeByName(testExchange) != nil {
return
}
err := LoadExchange(testExchange, false, nil)
if err != nil {
t.Errorf("SetupTest: Failed to load exchange: %s", err)
}
}
func CleanupTest(t *testing.T) {
if GetExchangeByName(testExchange) == nil {
return
if GetExchangeByName(testExchange) != nil {
err := UnloadExchange(testExchange)
if err != nil {
t.Fatalf("CleanupTest: Failed to unload exchange: %s",
err)
}
}
err := UnloadExchange(testExchange)
if err != nil {
t.Fatalf("CleanupTest: Failed to unload exchange: %s",
err)
if GetExchangeByName(fakePassExchange) != nil {
err := UnloadExchange(fakePassExchange)
if err != nil {
t.Fatalf("CleanupTest: Failed to unload exchange: %s",
err)
}
}
}
func TestExchangeManagerAdd(t *testing.T) {
t.Parallel()
var e exchangeManager
bitfinex := new(bitfinex.Bitfinex)
bitfinex.SetDefaults()
e.add(bitfinex)
b := new(bitfinex.Bitfinex)
b.SetDefaults()
e.add(b)
if exch := e.getExchanges(); exch[0].GetName() != "Bitfinex" {
t.Error("unexpected exchange name")
}
@@ -56,9 +40,9 @@ func TestExchangeManagerGetExchanges(t *testing.T) {
if exchanges := e.getExchanges(); exchanges != nil {
t.Error("unexpected value")
}
bitfinex := new(bitfinex.Bitfinex)
bitfinex.SetDefaults()
e.add(bitfinex)
b := new(bitfinex.Bitfinex)
b.SetDefaults()
e.add(b)
if exch := e.getExchanges(); exch[0].GetName() != "Bitfinex" {
t.Error("unexpected exchange name")
}
@@ -70,9 +54,9 @@ func TestExchangeManagerRemoveExchange(t *testing.T) {
if err := e.removeExchange("Bitfinex"); err != ErrNoExchangesLoaded {
t.Error("no exchanges should be loaded")
}
bitfinex := new(bitfinex.Bitfinex)
bitfinex.SetDefaults()
e.add(bitfinex)
b := new(bitfinex.Bitfinex)
b.SetDefaults()
e.add(b)
if err := e.removeExchange(testExchange); err != ErrExchangeNotFound {
t.Error("Bitstamp exchange should return an error")
}
@@ -85,7 +69,7 @@ func TestExchangeManagerRemoveExchange(t *testing.T) {
}
func TestCheckExchangeExists(t *testing.T) {
SetupTest(t)
SetupTestHelpers(t)
if GetExchangeByName(testExchange) == nil {
t.Errorf("TestGetExchangeExists: Unable to find exchange")
@@ -99,7 +83,7 @@ func TestCheckExchangeExists(t *testing.T) {
}
func TestGetExchangeByName(t *testing.T) {
SetupTest(t)
SetupTestHelpers(t)
exch := GetExchangeByName(testExchange)
if exch == nil {
@@ -129,7 +113,7 @@ func TestGetExchangeByName(t *testing.T) {
}
func TestUnloadExchange(t *testing.T) {
SetupTest(t)
SetupTestHelpers(t)
err := UnloadExchange("asdf")
if err.Error() != "exchange asdf not found" {
@@ -143,6 +127,12 @@ func TestUnloadExchange(t *testing.T) {
err)
}
err = UnloadExchange(fakePassExchange)
if err != nil {
t.Errorf("TestUnloadExchange: Failed to unload exchange. %s",
err)
}
err = UnloadExchange(testExchange)
if err != ErrNoExchangesLoaded {
t.Errorf("TestUnloadExchange: Incorrect result: %s",
@@ -153,7 +143,7 @@ func TestUnloadExchange(t *testing.T) {
}
func TestDryRunParamInteraction(t *testing.T) {
SetupTest(t)
SetupTestHelpers(t)
// Load bot as per normal, dry run and verbose for Bitfinex should be
// disabled

192
engine/fake_exchange_test.go Normal file
View File

@@ -0,0 +1,192 @@
package engine
import (
"sync"
"time"
"github.com/thrasher-corp/gocryptotrader/config"
"github.com/thrasher-corp/gocryptotrader/currency"
exchange "github.com/thrasher-corp/gocryptotrader/exchanges"
"github.com/thrasher-corp/gocryptotrader/exchanges/account"
"github.com/thrasher-corp/gocryptotrader/exchanges/asset"
"github.com/thrasher-corp/gocryptotrader/exchanges/order"
"github.com/thrasher-corp/gocryptotrader/exchanges/orderbook"
"github.com/thrasher-corp/gocryptotrader/exchanges/ticker"
"github.com/thrasher-corp/gocryptotrader/exchanges/websocket/wshandler"
"github.com/thrasher-corp/gocryptotrader/portfolio/withdraw"
)
const (
fakePassExchange = "FakePassExchange"
)
// FakePassingExchange is used to override IBotExchange responses in tests
// In this context, we don't care what FakePassingExchange does as we're testing
// the engine package
type FakePassingExchange struct {
exchange.Base
}
// addPassingFakeExchange adds an exchange to engine tests where all funcs return a positive result
func addPassingFakeExchange(baseExchangeName string) error {
testExch := GetExchangeByName(baseExchangeName)
if testExch == nil {
return ErrExchangeNotFound
}
base := testExch.GetBase()
Bot.Config.Exchanges = append(Bot.Config.Exchanges, config.ExchangeConfig{
Name: fakePassExchange,
Enabled: true,
Verbose: false,
})
Bot.exchangeManager.add(&FakePassingExchange{
Base: exchange.Base{
Name: fakePassExchange,
Enabled: true,
LoadedByConfig: true,
SkipAuthCheck: true,
API: base.API,
Features: base.Features,
HTTPTimeout: base.HTTPTimeout,
HTTPUserAgent: base.HTTPUserAgent,
HTTPRecording: base.HTTPRecording,
HTTPDebugging: base.HTTPDebugging,
WebsocketResponseCheckTimeout: base.WebsocketResponseCheckTimeout,
WebsocketResponseMaxLimit: base.WebsocketResponseMaxLimit,
WebsocketOrderbookBufferLimit: base.WebsocketOrderbookBufferLimit,
Websocket: base.Websocket,
Requester: base.Requester,
Config: base.Config,
},
})
return nil
}
func (h *FakePassingExchange) Setup(_ *config.ExchangeConfig) error { return nil }
func (h *FakePassingExchange) Start(_ *sync.WaitGroup) {}
func (h *FakePassingExchange) SetDefaults() {}
func (h *FakePassingExchange) GetName() string { return fakePassExchange }
func (h *FakePassingExchange) IsEnabled() bool { return true }
func (h *FakePassingExchange) SetEnabled(bool) {}
func (h *FakePassingExchange) ValidateCredentials() error { return nil }
func (h *FakePassingExchange) FetchTicker(_ currency.Pair, _ asset.Item) (*ticker.Price, error) {
return nil, nil
}
func (h *FakePassingExchange) UpdateTicker(_ currency.Pair, _ asset.Item) (*ticker.Price, error) {
return nil, nil
}
func (h *FakePassingExchange) FetchOrderbook(_ currency.Pair, _ asset.Item) (*orderbook.Base, error) {
return nil, nil
}
func (h *FakePassingExchange) UpdateOrderbook(_ currency.Pair, _ asset.Item) (*orderbook.Base, error) {
return nil, nil
}
func (h *FakePassingExchange) FetchTradablePairs(_ asset.Item) ([]string, error) {
return nil, nil
}
func (h *FakePassingExchange) UpdateTradablePairs(_ bool) error { return nil }
func (h *FakePassingExchange) GetEnabledPairs(_ asset.Item) currency.Pairs {
return currency.Pairs{}
}
func (h *FakePassingExchange) GetAvailablePairs(_ asset.Item) currency.Pairs {
return currency.Pairs{}
}
func (h *FakePassingExchange) FetchAccountInfo() (account.Holdings, error) {
return account.Holdings{}, nil
}
func (h *FakePassingExchange) UpdateAccountInfo() (account.Holdings, error) {
return account.Holdings{}, nil
}
func (h *FakePassingExchange) GetAuthenticatedAPISupport(_ uint8) bool { return true }
func (h *FakePassingExchange) SetPairs(_ currency.Pairs, _ asset.Item, _ bool) error {
return nil
}
func (h *FakePassingExchange) GetAssetTypes() asset.Items { return asset.Items{asset.Spot} }
func (h *FakePassingExchange) GetExchangeHistory(_ currency.Pair, _ asset.Item) ([]exchange.TradeHistory, error) {
return nil, nil
}
func (h *FakePassingExchange) SupportsAutoPairUpdates() bool { return true }
func (h *FakePassingExchange) SupportsRESTTickerBatchUpdates() bool { return true }
func (h *FakePassingExchange) GetFeeByType(_ *exchange.FeeBuilder) (float64, error) {
return 0, nil
}
func (h *FakePassingExchange) GetLastPairsUpdateTime() int64 { return 0 }
func (h *FakePassingExchange) GetWithdrawPermissions() uint32 { return 0 }
func (h *FakePassingExchange) FormatWithdrawPermissions() string { return "" }
func (h *FakePassingExchange) SupportsWithdrawPermissions(_ uint32) bool { return true }
func (h *FakePassingExchange) GetFundingHistory() ([]exchange.FundHistory, error) { return nil, nil }
func (h *FakePassingExchange) SubmitOrder(_ *order.Submit) (order.SubmitResponse, error) {
return order.SubmitResponse{
IsOrderPlaced: true,
FullyMatched: true,
OrderID: "FakePassingExchangeOrder",
}, nil
}
func (h *FakePassingExchange) ModifyOrder(_ *order.Modify) (string, error) { return "", nil }
func (h *FakePassingExchange) CancelOrder(_ *order.Cancel) error { return nil }
func (h *FakePassingExchange) CancelAllOrders(_ *order.Cancel) (order.CancelAllResponse, error) {
return order.CancelAllResponse{}, nil
}
func (h *FakePassingExchange) GetOrderInfo(_ string) (order.Detail, error) {
return order.Detail{}, nil
}
func (h *FakePassingExchange) GetDepositAddress(_ currency.Code, _ string) (string, error) {
return "", nil
}
func (h *FakePassingExchange) GetOrderHistory(_ *order.GetOrdersRequest) ([]order.Detail, error) {
return nil, nil
}
func (h *FakePassingExchange) GetActiveOrders(_ *order.GetOrdersRequest) ([]order.Detail, error) {
return []order.Detail{
{
Price: 1337,
Amount: 1337,
Exchange: fakePassExchange,
ID: "fakeOrder",
Type: order.Market,
Side: order.Buy,
Status: order.Active,
AssetType: asset.Spot,
Date: time.Now(),
Pair: currency.NewPairFromString("BTCUSD"),
},
}, nil
}
func (h *FakePassingExchange) SetHTTPClientUserAgent(_ string) {}
func (h *FakePassingExchange) GetHTTPClientUserAgent() string { return "" }
func (h *FakePassingExchange) SetClientProxyAddress(_ string) error { return nil }
func (h *FakePassingExchange) SupportsWebsocket() bool { return true }
func (h *FakePassingExchange) SupportsREST() bool { return true }
func (h *FakePassingExchange) IsWebsocketEnabled() bool { return true }
func (h *FakePassingExchange) GetWebsocket() (*wshandler.Websocket, error) { return nil, nil }
func (h *FakePassingExchange) SubscribeToWebsocketChannels(_ []wshandler.WebsocketChannelSubscription) error {
return nil
}
func (h *FakePassingExchange) UnsubscribeToWebsocketChannels(_ []wshandler.WebsocketChannelSubscription) error {
return nil
}
func (h *FakePassingExchange) AuthenticateWebsocket() error { return nil }
func (h *FakePassingExchange) GetSubscriptions() ([]wshandler.WebsocketChannelSubscription, error) {
return nil, nil
}
func (h *FakePassingExchange) GetDefaultConfig() (*config.ExchangeConfig, error) { return nil, nil }
func (h *FakePassingExchange) GetBase() *exchange.Base { return nil }
func (h *FakePassingExchange) SupportsAsset(_ asset.Item) bool { return true }
func (h *FakePassingExchange) GetHistoricCandles(_ currency.Pair, _, _ int64) ([]exchange.Candle, error) {
return []exchange.Candle{}, nil
}
func (h *FakePassingExchange) DisableRateLimiter() error { return nil }
func (h *FakePassingExchange) EnableRateLimiter() error { return nil }
func (h *FakePassingExchange) WithdrawCryptocurrencyFunds(_ *withdraw.Request) (*withdraw.ExchangeResponse, error) {
return nil, nil
}
func (h *FakePassingExchange) WithdrawFiatFunds(_ *withdraw.Request) (*withdraw.ExchangeResponse, error) {
return nil, nil
}
func (h *FakePassingExchange) WithdrawFiatFundsToInternationalBank(_ *withdraw.Request) (*withdraw.ExchangeResponse, error) {
return nil, nil
}

42
engine/helpers_test.go
View File

@@ -31,23 +31,33 @@ var (
func SetupTestHelpers(t *testing.T) {
if !helperTestLoaded {
if !testSetup {
if Bot == nil {
Bot = new(Engine)
}
Bot.Config = &config.Cfg
err := Bot.Config.LoadConfig(config.TestFile, true)
if err != nil {
t.Fatalf("SetupTest: Failed to load config: %s", err)
}
testSetup = true
if Bot == nil {
Bot = new(Engine)
}
err := Bot.Config.RetrieveConfigCurrencyPairs(true, asset.Spot)
Bot.Config = &config.Cfg
err := Bot.Config.LoadConfig(config.TestFile, true)
if err != nil {
t.Fatalf("SetupTest: Failed to load config: %s", err)
}
err = Bot.Config.RetrieveConfigCurrencyPairs(true, asset.Spot)
if err != nil {
t.Fatalf("Failed to retrieve config currency pairs. %s", err)
}
helperTestLoaded = true
}
if GetExchangeByName(testExchange) == nil {
err := LoadExchange(testExchange, false, nil)
if err != nil {
t.Fatalf("SetupTest: Failed to load exchange: %s", err)
}
}
if GetExchangeByName(fakePassExchange) == nil {
err := addPassingFakeExchange(testExchange)
if err != nil {
t.Fatalf("SetupTest: Failed to load exchange: %s", err)
}
}
}
func TestGetExchangeOTPs(t *testing.T) {
@@ -117,8 +127,8 @@ func TestGetExchangeoOTPByName(t *testing.T) {
func TestGetAuthAPISupportedExchanges(t *testing.T) {
SetupTestHelpers(t)
if result := GetAuthAPISupportedExchanges(); result != nil {
t.Fatal("Unexpected result")
if result := GetAuthAPISupportedExchanges(); len(result) != 1 {
t.Fatal("Unexpected result", result)
}
}
@@ -571,7 +581,7 @@ func TestGetCryptocurrenciesByExchange(t *testing.T) {
}
func TestGetExchangeNames(t *testing.T) {
SetupTest(t)
SetupTestHelpers(t)
if e := GetExchangeNames(true); len(e) == 0 {
t.Error("exchange names should be populated")
}
@@ -581,8 +591,8 @@ func TestGetExchangeNames(t *testing.T) {
if e := GetExchangeNames(true); common.StringDataCompare(e, testExchange) {
t.Error("Bitstamp should be missing")
}
if e := GetExchangeNames(false); len(e) != 27 {
t.Error("len should be all available exchanges")
if e := GetExchangeNames(false); len(e) != len(Bot.Config.Exchanges) {
t.Errorf("Expected %v Received %v", len(e), len(Bot.Config.Exchanges))
}
}

288
engine/orders.go
View File

@@ -17,15 +17,66 @@ import (
var (
OrderManagerDelay = time.Second * 10
ErrOrdersAlreadyExists = errors.New("order already exists")
ErrOrderNotFound = errors.New("order does not exist")
)
func (o *orderStore) Get() map[string][]order.Detail {
o.m.Lock()
defer o.m.Unlock()
// get returns all orders for all exchanges
// should not be exported as it can have large impact if used improperly
func (o *orderStore) get() map[string][]*order.Detail {
o.m.RLock()
defer o.m.RUnlock()
return o.Orders
}
// GetByExchangeAndID returns a specific order by exchange and id
func (o *orderStore) GetByExchangeAndID(exchange, id string) (*order.Detail, error) {
o.m.RLock()
defer o.m.RUnlock()
r, ok := o.Orders[exchange]
if !ok {
return nil, ErrExchangeNotFound
}
for x := range r {
if r[x].ID == id {
return r[x], nil
}
}
return nil, ErrOrderNotFound
}
// GetByExchange returns orders by exchange
func (o *orderStore) GetByExchange(exchange string) ([]*order.Detail, error) {
o.m.RLock()
defer o.m.RUnlock()
r, ok := o.Orders[exchange]
if !ok {
return nil, ErrExchangeNotFound
}
return r, nil
}
// GetByInternalOrderID will search all orders for our internal orderID
// and return the order
func (o *orderStore) GetByInternalOrderID(internalOrderID string) (*order.Detail, error) {
o.m.RLock()
defer o.m.RUnlock()
for _, v := range o.Orders {
for x := range v {
if v[x].InternalOrderID == internalOrderID {
return v[x], nil
}
}
}
return nil, ErrOrderNotFound
}
func (o *orderStore) exists(order *order.Detail) bool {
if order == nil {
return false
}
o.m.RLock()
defer o.m.RUnlock()
r, ok := o.Orders[order.Exchange]
if !ok {
return false
@@ -36,28 +87,47 @@ func (o *orderStore) exists(order *order.Detail) bool {
return true
}
}
return false
}
// Add Adds an order to the orderStore for tracking the lifecycle
func (o *orderStore) Add(order *order.Detail) error {
o.m.Lock()
defer o.m.Unlock()
if order == nil {
return errors.New("order store: Order is nil")
}
exch := GetExchangeByName(order.Exchange)
if exch == nil {
return ErrExchangeNotFound
}
if o.exists(order) {
return ErrOrdersAlreadyExists
}
// Untracked websocket orders will not have internalIDs yet
if order.InternalOrderID == "" {
id, err := uuid.NewV4()
if err != nil {
log.Warnf(log.OrderMgr,
"Order manager: Unable to generate UUID. Err: %s",
err)
} else {
order.InternalOrderID = id.String()
}
}
o.m.Lock()
defer o.m.Unlock()
orders := o.Orders[order.Exchange]
orders = append(orders, *order)
orders = append(orders, order)
o.Orders[order.Exchange] = orders
return nil
}
// Started returns the status of the orderManager
func (o *orderManager) Started() bool {
return atomic.LoadInt32(&o.started) == 1
}
// Start will boot up the orderManager
func (o *orderManager) Start() error {
if atomic.AddInt32(&o.started, 1) != 1 {
return errors.New("order manager already started")
@@ -66,11 +136,12 @@ func (o *orderManager) Start() error {
log.Debugln(log.OrderBook, "Order manager starting...")
o.shutdown = make(chan struct{})
o.orderStore.Orders = make(map[string][]order.Detail)
o.orderStore.Orders = make(map[string][]*order.Detail)
go o.run()
return nil
}
// Stop will attempt to shutdown the orderManager
func (o *orderManager) Stop() error {
if atomic.LoadInt32(&o.started) == 0 {
return errors.New("order manager not started")
@@ -92,39 +163,7 @@ func (o *orderManager) Stop() error {
func (o *orderManager) gracefulShutdown() {
if o.cfg.CancelOrdersOnShutdown {
log.Debugln(log.OrderMgr, "Order manager: Cancelling any open orders...")
orders := o.orderStore.Get()
if orders == nil {
return
}
for k, v := range orders {
log.Debugf(log.OrderMgr, "Order manager: Cancelling order(s) for exchange %s.\n", k)
for y := range v {
log.Debugf(log.OrderMgr, "order manager: Cancelling order ID %v [%v]",
v[y].ID, v[y])
err := o.Cancel(k, &order.Cancel{
OrderID: v[y].ID,
})
if err != nil {
msg := fmt.Sprintf("Order manager: Exchange %s unable to cancel order ID=%v. Err: %s",
k, v[y].ID, err)
log.Debugln(log.OrderBook, msg)
Bot.CommsManager.PushEvent(base.Event{
Type: "order",
Message: msg,
})
continue
}
msg := fmt.Sprintf("Order manager: Exchange %s order ID=%v cancelled.",
k, v[y].ID)
log.Debugln(log.OrderBook, msg)
Bot.CommsManager.PushEvent(base.Event{
Type: "order",
Message: msg,
})
}
}
o.CancelAllOrders(Bot.Config.GetEnabledExchanges())
}
}
@@ -149,35 +188,98 @@ func (o *orderManager) run() {
}
}
func (o *orderManager) CancelAllOrders() {}
func (o *orderManager) Cancel(exchName string, cancel *order.Cancel) error {
if exchName == "" {
return errors.New("order exchange name is empty")
// CancelAllOrders iterates and cancels all orders for each exchange provided
func (o *orderManager) CancelAllOrders(exchangeNames []string) {
orders := o.orderStore.get()
if orders == nil {
return
}
for k, v := range orders {
log.Debugf(log.OrderMgr, "Order manager: Cancelling order(s) for exchange %s.", k)
if !common.StringDataCompareInsensitive(exchangeNames, k) {
continue
}
for y := range v {
log.Debugf(log.OrderMgr, "Order manager: Cancelling order ID %v [%v]",
v[y].ID, v[y])
err := o.Cancel(&order.Cancel{
Exchange: k,
ID: v[y].ID,
AccountID: v[y].AccountID,
ClientID: v[y].ClientID,
WalletAddress: v[y].WalletAddress,
Type: v[y].Type,
Side: v[y].Side,
Pair: v[y].Pair,
})
if err != nil {
log.Error(log.OrderMgr, err)
Bot.CommsManager.PushEvent(base.Event{
Type: "order",
Message: err.Error(),
})
continue
}
msg := fmt.Sprintf("Order manager: Exchange %s order ID=%v cancelled.",
k, v[y].ID)
log.Debugln(log.OrderMgr, msg)
Bot.CommsManager.PushEvent(base.Event{
Type: "order",
Message: msg,
})
}
}
}
// Cancel will find the order in the orderManager, send a cancel request
// to the exchange and if successful, update the status of the order
func (o *orderManager) Cancel(cancel *order.Cancel) error {
if cancel == nil {
return errors.New("order cancel param is nil")
}
if cancel.OrderID == "" {
if cancel.Exchange == "" {
return errors.New("order exchange name is empty")
}
if cancel.ID == "" {
return errors.New("order id is empty")
}
exch := GetExchangeByName(exchName)
exch := GetExchangeByName(cancel.Exchange)
if exch == nil {
return errors.New("unable to get exchange by name")
return ErrExchangeNotFound
}
if cancel.AssetType.String() != "" && !exch.GetAssetTypes().Contains(cancel.AssetType) {
return errors.New("order asset type not supported by exchange")
}
return exch.CancelOrder(cancel)
err := exch.CancelOrder(cancel)
if err != nil {
return fmt.Errorf("%v - Failed to cancel order: %v", cancel.Exchange, err)
}
var od *order.Detail
od, err = o.orderStore.GetByExchangeAndID(cancel.Exchange, cancel.ID)
if err != nil {
return fmt.Errorf("%v - Failed to retrieve order %v to update cancelled status: %v", cancel.Exchange, cancel.ID, err)
}
od.Status = order.Cancelled
return nil
}
func (o *orderManager) Submit(exchName string, newOrder *order.Submit) (*orderSubmitResponse, error) {
if exchName == "" {
// Submit will take in an order struct, send it to the exchange and
// populate it in the orderManager if successful
func (o *orderManager) Submit(newOrder *order.Submit) (*orderSubmitResponse, error) {
if newOrder == nil {
return nil, errors.New("order cannot be nil")
}
if newOrder.Exchange == "" {
return nil, errors.New("order exchange name must be specified")
}
@@ -186,7 +288,7 @@ func (o *orderManager) Submit(exchName string, newOrder *order.Submit) (*orderSu
}
if o.cfg.EnforceLimitConfig {
if !o.cfg.AllowMarketOrders && newOrder.OrderType == order.Market {
if !o.cfg.AllowMarketOrders && newOrder.Type == order.Market {
return nil, errors.New("order market type is not allowed")
}
@@ -195,7 +297,7 @@ func (o *orderManager) Submit(exchName string, newOrder *order.Submit) (*orderSu
}
if len(o.cfg.AllowedExchanges) > 0 &&
!common.StringDataCompareInsensitive(o.cfg.AllowedExchanges, exchName) {
!common.StringDataCompareInsensitive(o.cfg.AllowedExchanges, newOrder.Exchange) {
return nil, errors.New("order exchange not found in allowed list")
}
@@ -204,18 +306,10 @@ func (o *orderManager) Submit(exchName string, newOrder *order.Submit) (*orderSu
}
}
exch := GetExchangeByName(exchName)
exch := GetExchangeByName(newOrder.Exchange)
if exch == nil {
return nil, errors.New("unable to get exchange by name")
return nil, ErrExchangeNotFound
}
id, err := uuid.NewV4()
if err != nil {
log.Warnf(log.OrderMgr,
"Order manager: Unable to generate UUID. Err: %s\n",
err)
}
result, err := exch.SubmitOrder(newOrder)
if err != nil {
return nil, err
@@ -225,42 +319,84 @@ func (o *orderManager) Submit(exchName string, newOrder *order.Submit) (*orderSu
return nil, errors.New("order unable to be placed")
}
var id uuid.UUID
id, err = uuid.NewV4()
if err != nil {
log.Warnf(log.OrderMgr,
"Order manager: Unable to generate UUID. Err: %s",
err)
}
msg := fmt.Sprintf("Order manager: Exchange %s submitted order ID=%v [Ours: %v] pair=%v price=%v amount=%v side=%v type=%v.",
exchName,
newOrder.Exchange,
result.OrderID,
id.String(),
newOrder.Pair,
newOrder.Price,
newOrder.Amount,
newOrder.OrderSide,
newOrder.OrderType)
newOrder.Side,
newOrder.Type)
log.Debugln(log.OrderMgr, msg)
Bot.CommsManager.PushEvent(base.Event{
Type: "order",
Message: msg,
})
status := order.New
if result.FullyMatched {
status = order.Filled
}
err = o.orderStore.Add(&order.Detail{
ImmediateOrCancel: newOrder.ImmediateOrCancel,
HiddenOrder: newOrder.HiddenOrder,
FillOrKill: newOrder.FillOrKill,
PostOnly: newOrder.PostOnly,
Price: newOrder.Price,
Amount: newOrder.Amount,
LimitPriceUpper: newOrder.LimitPriceUpper,
LimitPriceLower: newOrder.LimitPriceLower,
TriggerPrice: newOrder.TriggerPrice,
TargetAmount: newOrder.TargetAmount,
ExecutedAmount: newOrder.ExecutedAmount,
RemainingAmount: newOrder.RemainingAmount,
Fee: newOrder.Fee,
Exchange: newOrder.Exchange,
InternalOrderID: id.String(),
ID: result.OrderID,
AccountID: newOrder.AccountID,
ClientID: newOrder.ClientID,
WalletAddress: newOrder.WalletAddress,
Type: newOrder.Type,
Side: newOrder.Side,
Status: status,
AssetType: newOrder.AssetType,
Date: time.Now(),
LastUpdated: time.Now(),
Pair: newOrder.Pair,
})
if err != nil {
return nil, fmt.Errorf("unable to add %v order %v to orderStore: %s", newOrder.Exchange, result.OrderID, err)
}
return &orderSubmitResponse{
SubmitResponse: order.SubmitResponse{
OrderID: result.OrderID,
},
OurOrderID: id.String(),
InternalOrderID: id.String(),
}, nil
}
func (o *orderManager) processOrders() {
authExchanges := GetAuthAPISupportedExchanges()
for x := range authExchanges {
log.Debugf(log.OrderMgr, "Order manager: Procesing orders for exchange %v.\n", authExchanges[x])
log.Debugf(log.OrderMgr, "Order manager: Procesing orders for exchange %v.", authExchanges[x])
exch := GetExchangeByName(authExchanges[x])
req := order.GetOrdersRequest{
OrderSide: order.AnySide,
OrderType: order.AnyType,
Side: order.AnySide,
Type: order.AnyType,
}
result, err := exch.GetActiveOrders(&req)
if err != nil {
log.Warnf(log.OrderMgr, "Order manager: Unable to get active orders: %s\n", err)
log.Warnf(log.OrderMgr, "Order manager: Unable to get active orders: %s", err)
continue
}
@@ -269,8 +405,8 @@ func (o *orderManager) processOrders() {
result := o.orderStore.Add(ord)
if result != ErrOrdersAlreadyExists {
msg := fmt.Sprintf("Order manager: Exchange %s added order ID=%v pair=%v price=%v amount=%v side=%v type=%v.",
ord.Exchange, ord.ID, ord.CurrencyPair, ord.Price, ord.Amount, ord.OrderSide, ord.OrderType)
log.Debugf(log.OrderMgr, "%v\n", msg)
ord.Exchange, ord.ID, ord.Pair, ord.Price, ord.Amount, ord.Side, ord.Type)
log.Debugf(log.OrderMgr, "%v", msg)
Bot.CommsManager.PushEvent(base.Event{
Type: "order",
Message: msg,

379
engine/orders_test.go Normal file
View File

@@ -0,0 +1,379 @@
package engine
import (
"testing"
"time"
"github.com/thrasher-corp/gocryptotrader/currency"
"github.com/thrasher-corp/gocryptotrader/exchanges/asset"
"github.com/thrasher-corp/gocryptotrader/exchanges/order"
)
var ordersSetupRan bool
func OrdersSetup(t *testing.T) {
SetupTestHelpers(t)
if !ordersSetupRan {
err := Bot.OrderManager.Start()
if err != nil {
t.Fatal(err)
}
if !Bot.OrderManager.Started() {
t.Fatal("Order manager not started")
}
ordersSetupRan = true
}
}
func TestOrdersGet(t *testing.T) {
OrdersSetup(t)
if Bot.OrderManager.orderStore.get() == nil {
t.Error("orderStore not established")
}
}
func TestOrdersAdd(t *testing.T) {
OrdersSetup(t)
err := Bot.OrderManager.orderStore.Add(&order.Detail{
Exchange: testExchange,
ID: "TestOrdersAdd",
})
if err != nil {
t.Error(err)
}
err = Bot.OrderManager.orderStore.Add(&order.Detail{
Exchange: "testTest",
ID: "TestOrdersAdd",
})
if err == nil {
t.Error("Expected error from non existent exchange")
}
err = Bot.OrderManager.orderStore.Add(nil)
if err == nil {
t.Error("Expected error from nil order")
}
err = Bot.OrderManager.orderStore.Add(&order.Detail{
Exchange: testExchange,
ID: "TestOrdersAdd",
})
if err == nil {
t.Error("Expected error re-adding order")
}
}
func TestGetByInternalOrderID(t *testing.T) {
OrdersSetup(t)
err := Bot.OrderManager.orderStore.Add(&order.Detail{
Exchange: testExchange,
ID: "TestGetByInternalOrderID",
InternalOrderID: "internalTest",
})
if err != nil {
t.Error(err)
}
o, err := Bot.OrderManager.orderStore.GetByInternalOrderID("internalTest")
if err != nil {
t.Error(err)
}
if o == nil {
t.Fatal("Expected a matching order")
}
if o.ID != "TestGetByInternalOrderID" {
t.Error("Expected to retrieve order")
}
_, err = Bot.OrderManager.orderStore.GetByInternalOrderID("NoOrder")
if err != ErrOrderNotFound {
t.Error(err)
}
}
func TestGetByExchange(t *testing.T) {
OrdersSetup(t)
err := Bot.OrderManager.orderStore.Add(&order.Detail{
Exchange: testExchange,
ID: "TestGetByExchange",
InternalOrderID: "internalTestGetByExchange",
})
if err != nil {
t.Error(err)
}
err = Bot.OrderManager.orderStore.Add(&order.Detail{
Exchange: testExchange,
ID: "TestGetByExchange2",
InternalOrderID: "internalTestGetByExchange2",
})
if err != nil {
t.Error(err)
}
err = Bot.OrderManager.orderStore.Add(&order.Detail{
Exchange: fakePassExchange,
ID: "TestGetByExchange3",
InternalOrderID: "internalTest3",
})
if err != nil {
t.Error(err)
}
var o []*order.Detail
o, err = Bot.OrderManager.orderStore.GetByExchange(testExchange)
if err != nil {
t.Error(err)
}
if o == nil {
t.Error("Expected non nil response")
}
var o1Found, o2Found bool
for i := range o {
if o[i].ID == "TestGetByExchange" && o[i].Exchange == testExchange {
o1Found = true
}
if o[i].ID == "TestGetByExchange2" && o[i].Exchange == testExchange {
o2Found = true
}
}
if !o1Found || !o2Found {
t.Error("Expected orders 'TestGetByExchange' and 'TestGetByExchange2' to be returned")
}
_, err = Bot.OrderManager.orderStore.GetByInternalOrderID("NoOrder")
if err != ErrOrderNotFound {
t.Error(err)
}
err = Bot.OrderManager.orderStore.Add(&order.Detail{
Exchange: "thisWillFail",
})
if err == nil {
t.Error("Expected exchange not found error")
}
}
func TestGetByExchangeAndID(t *testing.T) {
OrdersSetup(t)
err := Bot.OrderManager.orderStore.Add(&order.Detail{
Exchange: testExchange,
ID: "TestGetByExchangeAndID",
})
if err != nil {
t.Error(err)
}
o, err := Bot.OrderManager.orderStore.GetByExchangeAndID(testExchange, "TestGetByExchangeAndID")
if err != nil {
t.Error(err)
}
if o.ID != "TestGetByExchangeAndID" {
t.Error("Expected to retrieve order")
}
_, err = Bot.OrderManager.orderStore.GetByExchangeAndID("", "TestGetByExchangeAndID")
if err != ErrExchangeNotFound {
t.Error(err)
}
_, err = Bot.OrderManager.orderStore.GetByExchangeAndID(testExchange, "")
if err != ErrOrderNotFound {
t.Error(err)
}
}
func TestExists(t *testing.T) {
OrdersSetup(t)
if Bot.OrderManager.orderStore.exists(nil) {
t.Error("Expected false")
}
o := &order.Detail{
Exchange: testExchange,
ID: "TestExists",
}
err := Bot.OrderManager.orderStore.Add(o)
if err != nil {
t.Error(err)
}
b := Bot.OrderManager.orderStore.exists(o)
if !b {
t.Error("Expected true")
}
}
func TestCancelOrder(t *testing.T) {
OrdersSetup(t)
err := Bot.OrderManager.Cancel(nil)
if err == nil {
t.Error("Expected error due to empty order")
}
err = Bot.OrderManager.Cancel(&order.Cancel{})
if err == nil {
t.Error("Expected error due to empty order")
}
err = Bot.OrderManager.Cancel(&order.Cancel{
Exchange: testExchange,
})
if err == nil {
t.Error("Expected error due to no order ID")
}
err = Bot.OrderManager.Cancel(&order.Cancel{
ID: "ID",
})
if err == nil {
t.Error("Expected error due to no Exchange")
}
err = Bot.OrderManager.Cancel(&order.Cancel{
ID: "ID",
Exchange: testExchange,
AssetType: asset.Binary,
})
if err == nil {
t.Error("Expected error due to bad asset type")
}
o := &order.Detail{
Exchange: fakePassExchange,
ID: "TestCancelOrder",
Status: order.New,
}
err = Bot.OrderManager.orderStore.Add(o)
if err != nil {
t.Error(err)
}
err = Bot.OrderManager.Cancel(&order.Cancel{
ID: "Unknown",
Exchange: fakePassExchange,
AssetType: asset.Spot,
})
if err == nil {
t.Error("Expected error due to no order found")
}
cancel := &order.Cancel{
Exchange: fakePassExchange,
ID: "TestCancelOrder",
Side: order.Sell,
Status: order.New,
AssetType: asset.Spot,
Date: time.Now(),
Pair: currency.NewPairFromString("BTCUSD"),
}
err = Bot.OrderManager.Cancel(cancel)
if err != nil {
t.Error(err)
}
if o.Status != order.Cancelled {
t.Error("Failed to cancel")
}
}
func TestCancelAllOrders(t *testing.T) {
OrdersSetup(t)
o := &order.Detail{
Exchange: fakePassExchange,
ID: "TestCancelAllOrders",
Status: order.New,
}
err := Bot.OrderManager.orderStore.Add(o)
if err != nil {
t.Error(err)
}
Bot.OrderManager.CancelAllOrders([]string{"NotFound"})
if o.Status == order.Cancelled {
t.Error("Order should not be cancelled")
}
Bot.OrderManager.CancelAllOrders([]string{fakePassExchange})
if o.Status != order.Cancelled {
t.Error("Order should be cancelled")
}
o.Status = order.New
Bot.OrderManager.CancelAllOrders(nil)
if o.Status != order.New {
t.Error("Order should not be cancelled")
}
}
func TestSubmit(t *testing.T) {
OrdersSetup(t)
_, err := Bot.OrderManager.Submit(nil)
if err == nil {
t.Error("Expected error from nil order")
}
o := &order.Submit{
Exchange: "",
ID: "FakePassingExchangeOrder",
Status: order.New,
Type: order.Market,
}
_, err = Bot.OrderManager.Submit(o)
if err == nil {
t.Error("Expected error from empty exchange")
}
o.Exchange = fakePassExchange
_, err = Bot.OrderManager.Submit(o)
if err == nil {
t.Error("Expected error from validation")
}
Bot.OrderManager.cfg.EnforceLimitConfig = true
Bot.OrderManager.cfg.AllowMarketOrders = false
o.Pair = currency.NewPairFromString("BTCUSD")
o.AssetType = asset.Spot
o.Side = order.Buy
o.Amount = 1
o.Price = 1
_, err = Bot.OrderManager.Submit(o)
if err == nil {
t.Error("Expected fail due to order market type is not allowed")
}
Bot.OrderManager.cfg.AllowMarketOrders = true
Bot.OrderManager.cfg.LimitAmount = 1
o.Amount = 2
_, err = Bot.OrderManager.Submit(o)
if err == nil {
t.Error("Expected fail due to order limit exceeds allowed limit")
}
Bot.OrderManager.cfg.LimitAmount = 0
Bot.OrderManager.cfg.AllowedExchanges = []string{"fake"}
_, err = Bot.OrderManager.Submit(o)
if err == nil {
t.Error("Expected fail due to order exchange not found in allowed list")
}
Bot.OrderManager.cfg.AllowedExchanges = nil
Bot.OrderManager.cfg.AllowedPairs = currency.Pairs{currency.NewPairFromString("BTCAUD")}
_, err = Bot.OrderManager.Submit(o)
if err == nil {
t.Error("Expected fail due to order pair not found in allowed list")
}
Bot.OrderManager.cfg.AllowedPairs = nil
_, err = Bot.OrderManager.Submit(o)
if err != nil {
t.Error(err)
}
o2, err := Bot.OrderManager.orderStore.GetByExchangeAndID(fakePassExchange, "FakePassingExchangeOrder")
if err != nil {
t.Error(err)
}
if o2.InternalOrderID == "" {
t.Error("Failed to assign internal order id")
}
}
func TestProcessOrders(t *testing.T) {
OrdersSetup(t)
Bot.OrderManager.processOrders()
}

6
engine/orders_types.go
View File

@@ -18,8 +18,8 @@ type orderManagerConfig struct {
}
type orderStore struct {
m sync.Mutex
Orders map[string][]order.Detail
m sync.RWMutex
Orders map[string][]*order.Detail
}
type orderManager struct {
@@ -32,5 +32,5 @@ type orderManager struct {
type orderSubmitResponse struct {
order.SubmitResponse
OurOrderID string
InternalOrderID string
}

31
engine/restful_server_test.go
View File

@@ -12,16 +12,6 @@ import (
"github.com/thrasher-corp/gocryptotrader/config"
)
func loadConfig(t *testing.T) *config.Config {
cfg := config.GetConfig()
err := cfg.LoadConfig("", true)
if err != nil {
t.Error("GetCurrencyConfig LoadConfig error", err)
}
configLoaded = true
return cfg
}
func makeHTTPGetRequest(t *testing.T, response interface{}) *http.Response {
w := httptest.NewRecorder()
@@ -34,8 +24,8 @@ func makeHTTPGetRequest(t *testing.T, response interface{}) *http.Response {
// TestConfigAllJsonResponse test if config/all restful json response is valid
func TestConfigAllJsonResponse(t *testing.T) {
cfg := loadConfig(t)
resp := makeHTTPGetRequest(t, cfg)
SetupTestHelpers(t)
resp := makeHTTPGetRequest(t, Bot.Config)
body, err := ioutil.ReadAll(resp.Body)
resp.Body.Close()
if err != nil {
@@ -48,16 +38,13 @@ func TestConfigAllJsonResponse(t *testing.T) {
t.Error("Response not parseable as json", err)
}
if reflect.DeepEqual(responseConfig, cfg) {
if reflect.DeepEqual(responseConfig, Bot.Config) {
t.Error("Json not equal to config")
}
}
func TestInvalidHostRequest(t *testing.T) {
Bot = &Engine{
Config: loadConfig(t),
}
SetupTestHelpers(t)
req, err := http.NewRequest(http.MethodGet, "/config/all", nil)
if err != nil {
t.Fatal(err)
@@ -73,10 +60,7 @@ func TestInvalidHostRequest(t *testing.T) {
}
func TestValidHostRequest(t *testing.T) {
Bot = &Engine{
Config: loadConfig(t),
}
SetupTestHelpers(t)
req, err := http.NewRequest(http.MethodGet, "/config/all", nil)
if err != nil {
t.Fatal(err)
@@ -92,10 +76,7 @@ func TestValidHostRequest(t *testing.T) {
}
func TestProfilerEnabledShouldEnableProfileEndPoint(t *testing.T) {
Bot = &Engine{
Config: loadConfig(t),
}
SetupTestHelpers(t)
req, err := http.NewRequest(http.MethodGet, "/debug/pprof/", nil)
if err != nil {
t.Fatal(err)

212
engine/routines.go
View File

@@ -5,11 +5,11 @@ import (
"fmt"
"strings"
"sync"
"time"
"github.com/thrasher-corp/gocryptotrader/common"
"github.com/thrasher-corp/gocryptotrader/currency"
"github.com/thrasher-corp/gocryptotrader/exchanges/asset"
"github.com/thrasher-corp/gocryptotrader/exchanges/order"
"github.com/thrasher-corp/gocryptotrader/exchanges/orderbook"
"github.com/thrasher-corp/gocryptotrader/exchanges/stats"
"github.com/thrasher-corp/gocryptotrader/exchanges/ticker"
@@ -232,7 +232,7 @@ func WebsocketRoutine() {
}
// Data handler routine
go WebsocketDataHandler(ws)
go WebsocketDataReceiver(ws)
err = ws.Connect()
if err != nil {
@@ -252,35 +252,9 @@ func WebsocketRoutine() {
var shutdowner = make(chan struct{}, 1)
var wg sync.WaitGroup
// Websocketshutdown shuts down the exchange routines and then shuts down
// governing routines
func Websocketshutdown(ws *wshandler.Websocket) error {
err := ws.Shutdown() // shutdown routines on the exchange
if err != nil {
log.Errorf(log.WebsocketMgr, "routines.go error - failed to shutdown %s\n", err)
}
timer := time.NewTimer(5 * time.Second)
c := make(chan struct{}, 1)
go func(c chan struct{}) {
close(shutdowner)
wg.Wait()
c <- struct{}{}
}(c)
select {
case <-timer.C:
return errors.New("routines.go error - failed to shutdown routines")
case <-c:
return nil
}
}
// WebsocketDataHandler handles websocket data coming from a websocket feed
// WebsocketDataReceiver handles websocket data coming from a websocket feed
// associated with an exchange
func WebsocketDataHandler(ws *wshandler.Websocket) {
func WebsocketDataReceiver(ws *wshandler.Websocket) {
wg.Add(1)
defer wg.Done()
@@ -288,85 +262,109 @@ func WebsocketDataHandler(ws *wshandler.Websocket) {
select {
case <-shutdowner:
return
case data := <-ws.DataHandler:
switch d := data.(type) {
case string:
switch d {
case wshandler.WebsocketNotEnabled:
if Bot.Settings.Verbose {
log.Warnf(log.WebsocketMgr, "routines.go warning - exchange %s websocket not enabled\n",
ws.GetName())
}
default:
log.Info(log.WebsocketMgr, d)
}
case error:
log.Errorf(log.WebsocketMgr, "routines.go exchange %s websocket error - %s", ws.GetName(), data)
case wshandler.TradeData:
// Websocket Trade Data
if Bot.Settings.Verbose {
log.Infof(log.WebsocketMgr, "%s websocket %s %s trade updated %+v\n",
ws.GetName(),
FormatCurrency(d.CurrencyPair),
d.AssetType,
d)
}
case wshandler.FundingData:
// Websocket Funding Data
if Bot.Settings.Verbose {
log.Infof(log.WebsocketMgr, "%s websocket %s %s funding updated %+v\n",
ws.GetName(),
FormatCurrency(d.CurrencyPair),
d.AssetType,
d)
}
case *ticker.Price:
// Websocket Ticker Data
if Bot.Settings.EnableExchangeSyncManager && Bot.ExchangeCurrencyPairManager != nil {
Bot.ExchangeCurrencyPairManager.update(ws.GetName(),
d.Pair,
d.AssetType,
SyncItemTicker,
nil)
}
err := ticker.ProcessTicker(ws.GetName(), d, d.AssetType)
printTickerSummary(d, d.Pair, d.AssetType, ws.GetName(), "websocket", err)
case wshandler.KlineData:
// Websocket Kline Data
if Bot.Settings.Verbose {
log.Infof(log.WebsocketMgr, "%s websocket %s %s kline updated %+v\n",
ws.GetName(),
FormatCurrency(d.Pair),
d.AssetType,
d)
}
case wshandler.WebsocketOrderbookUpdate:
// Websocket Orderbook Data
result := data.(wshandler.WebsocketOrderbookUpdate)
if Bot.Settings.EnableExchangeSyncManager && Bot.ExchangeCurrencyPairManager != nil {
Bot.ExchangeCurrencyPairManager.update(ws.GetName(),
result.Pair,
result.Asset,
SyncItemOrderbook,
nil)
}
if Bot.Settings.Verbose {
log.Infof(log.WebsocketMgr,
"%s websocket %s %s orderbook updated\n",
ws.GetName(),
FormatCurrency(result.Pair),
d.Asset)
}
default:
if Bot.Settings.Verbose {
log.Warnf(log.WebsocketMgr,
"%s websocket Unknown type: %+v\n",
ws.GetName(),
d)
}
err := WebsocketDataHandler(ws.GetName(), data)
if err != nil {
log.Error(log.WebsocketMgr, err)
}
}
}
}
// WebsocketDataHandler is a central point for exchange websocket implementations to send
// processed data. WebsocketDataHandler will then pass that to an appropriate handler
func WebsocketDataHandler(exchName string, data interface{}) error {
if data == nil {
return fmt.Errorf("routines.go - exchange %s nil data sent to websocket",
exchName)
}
switch d := data.(type) {
case string:
log.Info(log.WebsocketMgr, d)
case error:
return fmt.Errorf("routines.go exchange %s websocket error - %s", exchName, data)
case wshandler.TradeData:
if Bot.Settings.Verbose {
log.Infof(log.WebsocketMgr, "%s websocket %s %s trade updated %+v",
exchName,
FormatCurrency(d.CurrencyPair),
d.AssetType,
d)
}
case wshandler.FundingData:
if Bot.Settings.Verbose {
log.Infof(log.WebsocketMgr, "%s websocket %s %s funding updated %+v",
exchName,
FormatCurrency(d.CurrencyPair),
d.AssetType,
d)
}
case *ticker.Price:
if Bot.Settings.EnableExchangeSyncManager && Bot.ExchangeCurrencyPairManager != nil {
Bot.ExchangeCurrencyPairManager.update(exchName,
d.Pair,
d.AssetType,
SyncItemTicker,
nil)
}
err := ticker.ProcessTicker(exchName, d, d.AssetType)
printTickerSummary(d, d.Pair, d.AssetType, exchName, "websocket", err)
case wshandler.KlineData:
if Bot.Settings.Verbose {
log.Infof(log.WebsocketMgr, "%s websocket %s %s kline updated %+v",
exchName,
FormatCurrency(d.Pair),
d.AssetType,
d)
}
case wshandler.WebsocketOrderbookUpdate:
if Bot.Settings.EnableExchangeSyncManager && Bot.ExchangeCurrencyPairManager != nil {
Bot.ExchangeCurrencyPairManager.update(exchName,
d.Pair,
d.Asset,
SyncItemOrderbook,
nil)
}
if Bot.Settings.Verbose {
log.Infof(log.WebsocketMgr,
"%s websocket %s %s orderbook updated",
exchName,
FormatCurrency(d.Pair),
d.Asset)
}
case *order.Detail:
if !Bot.OrderManager.orderStore.exists(d) {
err := Bot.OrderManager.orderStore.Add(d)
if err != nil {
return err
}
} else {
od, err := Bot.OrderManager.orderStore.GetByExchangeAndID(d.Exchange, d.ID)
if err != nil {
return err
}
od.UpdateOrderFromDetail(d)
}
case *order.Cancel:
return Bot.OrderManager.Cancel(d)
case *order.Modify:
od, err := Bot.OrderManager.orderStore.GetByExchangeAndID(d.Exchange, d.ID)
if err != nil {
return err
}
od.UpdateOrderFromModify(d)
case order.ClassificationError:
return errors.New(d.Error())
case wshandler.UnhandledMessageWarning:
log.Warn(log.WebsocketMgr, d.Message)
default:
if Bot.Settings.Verbose {
log.Warnf(log.WebsocketMgr,
"%s websocket Unknown type: %+v",
exchName,
d)
}
}
return nil
}

127
engine/routines_test.go Normal file
View File

@@ -0,0 +1,127 @@
package engine
import (
"errors"
"testing"
"time"
"github.com/thrasher-corp/gocryptotrader/exchanges/order"
"github.com/thrasher-corp/gocryptotrader/exchanges/sharedtestvalues"
"github.com/thrasher-corp/gocryptotrader/exchanges/ticker"
"github.com/thrasher-corp/gocryptotrader/exchanges/websocket/wshandler"
)
func TestWebsocketDataHandlerProcess(t *testing.T) {
ws := wshandler.New()
err := ws.Setup(&wshandler.WebsocketSetup{Enabled: true})
if err != nil {
t.Error(err)
}
ws.DataHandler = sharedtestvalues.GetWebsocketInterfaceChannelOverride()
go WebsocketDataReceiver(ws)
ws.DataHandler <- "string"
time.Sleep(time.Second)
close(shutdowner)
}
func TestHandleData(t *testing.T) {
OrdersSetup(t)
var exchName = "exch"
var orderID = "testOrder.Detail"
err := WebsocketDataHandler(exchName, errors.New("error"))
if err == nil {
t.Error("Error not handled correctly")
}
err = WebsocketDataHandler(exchName, nil)
if err == nil {
t.Error("Expected nil data error")
}
err = WebsocketDataHandler(exchName, wshandler.TradeData{})
if err != nil {
t.Error(err)
}
err = WebsocketDataHandler(exchName, wshandler.FundingData{})
if err != nil {
t.Error(err)
}
err = WebsocketDataHandler(exchName, &ticker.Price{})
if err != nil {
t.Error(err)
}
err = WebsocketDataHandler(exchName, wshandler.KlineData{})
if err != nil {
t.Error(err)
}
err = WebsocketDataHandler(exchName, wshandler.WebsocketOrderbookUpdate{})
if err != nil {
t.Error(err)
}
origOrder := &order.Detail{
Exchange: fakePassExchange,
ID: orderID,
Amount: 1337,
Price: 1337,
}
err = WebsocketDataHandler(exchName, origOrder)
if err != nil {
t.Error(err)
}
// Send it again since it exists now
err = WebsocketDataHandler(exchName, &order.Detail{
Exchange: fakePassExchange,
ID: orderID,
Amount: 1338,
})
if err != nil {
t.Error(err)
}
if origOrder.Amount != 1338 {
t.Error("Bad pipeline")
}
err = WebsocketDataHandler(exchName, &order.Modify{
Exchange: fakePassExchange,
ID: orderID,
Status: order.Active,
})
if err != nil {
t.Error(err)
}
if origOrder.Status != order.Active {
t.Error("Expected order to be modified to Active")
}
err = WebsocketDataHandler(exchName, &order.Cancel{
Exchange: fakePassExchange,
ID: orderID,
})
if err != nil {
t.Error(err)
}
if origOrder.Status != order.Cancelled {
t.Error("Expected order status to be cancelled")
}
// Send some gibberish
err = WebsocketDataHandler(exchName, order.Stop)
if err != nil {
t.Error(err)
}
err = WebsocketDataHandler(exchName, wshandler.UnhandledMessageWarning{Message: "there's an issue here's a tissue"})
if err != nil {
t.Error(err)
}
classificationError := order.ClassificationError{
Exchange: "test",
OrderID: "one",
Err: errors.New("lol"),
}
err = WebsocketDataHandler(exchName, classificationError)
if err == nil {
t.Error("Expected error")
}
if err.Error() != classificationError.Error() {
t.Errorf("Problem formatting error. Expected %v Received %v", classificationError.Error(), err.Error())
}
}

45
engine/rpcserver.go
View File

@@ -718,7 +718,7 @@ func (s *RPCServer) GetOrders(ctx context.Context, r *gctrpc.GetOrdersRequest) (
}
resp, err := exch.GetActiveOrders(&order.GetOrdersRequest{
Currencies: []currency.Pair{
Pairs: []currency.Pair{
currency.NewPairWithDelimiter(r.Pair.Base,
r.Pair.Quote, r.Pair.Delimiter),
},
@@ -732,12 +732,12 @@ func (s *RPCServer) GetOrders(ctx context.Context, r *gctrpc.GetOrdersRequest) (
orders = append(orders, &gctrpc.OrderDetails{
Exchange: r.Exchange,
Id: resp[x].ID,
BaseCurrency: resp[x].CurrencyPair.Base.String(),
QuoteCurrency: resp[x].CurrencyPair.Quote.String(),
BaseCurrency: resp[x].Pair.Base.String(),
QuoteCurrency: resp[x].Pair.Quote.String(),
AssetType: asset.Spot.String(),
OrderType: resp[x].OrderType.String(),
OrderSide: resp[x].OrderSide.String(),
CreationTime: resp[x].OrderDate.Unix(),
OrderType: resp[x].Type.String(),
OrderSide: resp[x].Side.String(),
CreationTime: resp[x].Date.Unix(),
Status: resp[x].Status.String(),
Price: resp[x].Price,
Amount: resp[x].Amount,
@@ -761,18 +761,17 @@ func (s *RPCServer) SubmitOrder(ctx context.Context, r *gctrpc.SubmitOrderReques
}
p := currency.NewPairFromStrings(r.Pair.Base, r.Pair.Quote)
submission := &order.Submit{
Pair: p,
OrderSide: order.Side(r.Side),
OrderType: order.Type(r.OrderType),
Amount: r.Amount,
Price: r.Price,
ClientID: r.ClientId,
}
result, err := exch.SubmitOrder(submission)
resp, err := Bot.OrderManager.Submit(&order.Submit{
Pair: p,
Side: order.Side(r.Side),
Type: order.Type(r.OrderType),
Amount: r.Amount,
Price: r.Price,
ClientID: r.ClientId,
})
return &gctrpc.SubmitOrderResponse{
OrderId: result.OrderID,
OrderPlaced: result.IsOrderPlaced,
OrderId: resp.OrderID,
OrderPlaced: resp.IsOrderPlaced,
}, err
}
@@ -860,7 +859,7 @@ func (s *RPCServer) CancelOrder(ctx context.Context, r *gctrpc.CancelOrderReques
err := exch.CancelOrder(&order.Cancel{
AccountID: r.AccountId,
OrderID: r.OrderId,
ID: r.OrderId,
Side: order.Side(r.Side),
WalletAddress: r.WalletAddress,
})
@@ -1082,12 +1081,12 @@ func (s *RPCServer) WithdrawalEventsByExchange(ctx context.Context, r *gctrpc.Wi
// WithdrawalEventsByDate returns previous withdrawal request details by exchange
func (s *RPCServer) WithdrawalEventsByDate(ctx context.Context, r *gctrpc.WithdrawalEventsByDateRequest) (*gctrpc.WithdrawalEventsByExchangeResponse, error) {
UTCStartTime, err := time.Parse(audit.TableTimeFormat, r.Start)
UTCStartTime, err := time.Parse(common.SimpleTimeFormat, r.Start)
if err != nil {
return nil, err
}
UTCSEndTime, err := time.Parse(audit.TableTimeFormat, r.End)
UTCSEndTime, err := time.Parse(common.SimpleTimeFormat, r.End)
if err != nil {
return nil, err
}
@@ -1423,12 +1422,12 @@ func (s *RPCServer) GetExchangeTickerStream(r *gctrpc.GetExchangeTickerStreamReq
// GetAuditEvent returns matching audit events from database
func (s *RPCServer) GetAuditEvent(ctx context.Context, r *gctrpc.GetAuditEventRequest) (*gctrpc.GetAuditEventResponse, error) {
UTCStartTime, err := time.Parse(audit.TableTimeFormat, r.StartDate)
UTCStartTime, err := time.Parse(common.SimpleTimeFormat, r.StartDate)
if err != nil {
return nil, err
}
UTCSEndTime, err := time.Parse(audit.TableTimeFormat, r.EndDate)
UTCSEndTime, err := time.Parse(common.SimpleTimeFormat, r.EndDate)
if err != nil {
return nil, err
}
@@ -1449,7 +1448,7 @@ func (s *RPCServer) GetAuditEvent(ctx context.Context, r *gctrpc.GetAuditEventRe
Type: v[x].Type,
Identifier: v[x].Identifier,
Message: v[x].Message,
Timestamp: v[x].CreatedAt.In(loc).Format(audit.TableTimeFormat),
Timestamp: v[x].CreatedAt.In(loc).Format(common.SimpleTimeFormat),
}
resp.Events = append(resp.Events, tempEvent)

2
engine/syncer.go
View File

@@ -505,7 +505,7 @@ func (e *ExchangeCurrencyPairSyncer) Start() {
}
if !ws.IsConnected() && !ws.IsConnecting() {
go WebsocketDataHandler(ws)
go WebsocketDataReceiver(ws)
err = ws.Connect()
if err != nil {

31
engine/withdraw_test.go
View File

@@ -14,7 +14,6 @@ import (
)
const (
exchangeName = "BTC Markets"
bankAccountID = "test-bank-01"
)
@@ -30,14 +29,6 @@ var (
}
)
func setupEngine() (err error) {
Bot, err = NewFromSettings(&settings)
if err != nil {
return err
}
return Bot.Start()
}
func cleanup() {
err := os.RemoveAll(settings.DataDir)
if err != nil {
@@ -46,11 +37,7 @@ func cleanup() {
}
func TestSubmitWithdrawal(t *testing.T) {
err := setupEngine()
if err != nil {
t.Fatal(err)
}
SetupTestHelpers(t)
banking.Accounts = append(banking.Accounts,
banking.Account{
Enabled: true,
@@ -66,7 +53,7 @@ func TestSubmitWithdrawal(t *testing.T) {
SWIFTCode: "91272837",
IBAN: "98218738671897",
SupportedCurrencies: "AUD,USD",
SupportedExchanges: exchangeName,
SupportedExchanges: testExchange,
},
)
@@ -75,9 +62,9 @@ func TestSubmitWithdrawal(t *testing.T) {
t.Fatal(err)
}
req := &withdraw.Request{
Exchange: exchangeName,
Exchange: testExchange,
Currency: currency.AUD,
Description: exchangeName,
Description: testExchange,
Amount: 1.0,
Type: 1,
Fiat: &withdraw.FiatRequest{
@@ -85,12 +72,12 @@ func TestSubmitWithdrawal(t *testing.T) {
},
}
_, err = SubmitWithdrawal(exchangeName, req)
_, err = SubmitWithdrawal(testExchange, req)
if err != nil {
t.Fatal(err)
}
_, err = SubmitWithdrawal(exchangeName, nil)
_, err = SubmitWithdrawal(testExchange, nil)
if err != nil {
if err.Error() != withdraw.ErrRequestCannotBeNil.Error() {
t.Fatal(err)
@@ -122,21 +109,21 @@ func TestWithdrawEventByID(t *testing.T) {
}
func TestWithdrawalEventByExchange(t *testing.T) {
_, err := WithdrawalEventByExchange(exchangeName, 1)
_, err := WithdrawalEventByExchange(testExchange, 1)
if err == nil {
t.Fatal(err)
}
}
func TestWithdrawEventByDate(t *testing.T) {
_, err := WithdrawEventByDate(exchangeName, time.Now(), time.Now(), 1)
_, err := WithdrawEventByDate(testExchange, time.Now(), time.Now(), 1)
if err == nil {
t.Fatal(err)
}
}
func TestWithdrawalEventByExchangeID(t *testing.T) {
_, err := WithdrawalEventByExchangeID(exchangeName, exchangeName)
_, err := WithdrawalEventByExchangeID(testExchange, testExchange)
if err == nil {
t.Fatal(err)
}