diff --git a/CONTRIBUTORS b/CONTRIBUTORS index 3457ffb5..fb32de0c 100644 --- a/CONTRIBUTORS +++ b/CONTRIBUTORS @@ -5,11 +5,11 @@ shazbert | https://github.com/shazbert gloriousCode | https://github.com/gloriousCode dependabot-preview[bot] | https://github.com/apps/dependabot-preview xtda | https://github.com/xtda +dependabot[bot] | https://github.com/apps/dependabot Rots | https://github.com/Rots vazha | https://github.com/vazha ermalguni | https://github.com/ermalguni MadCozBadd | https://github.com/MadCozBadd -dependabot[bot] | https://github.com/apps/dependabot vadimzhukck | https://github.com/vadimzhukck 140am | https://github.com/140am marcofranssen | https://github.com/marcofranssen @@ -21,6 +21,7 @@ azhang | https://github.com/azhang andreygrehov | https://github.com/andreygrehov bretep | https://github.com/bretep Christian-Achilli | https://github.com/Christian-Achilli +MarkDzulko | https://github.com/MarkDzulko yangrq1018 | https://github.com/yangrq1018 gam-phon | https://github.com/gam-phon cornelk | https://github.com/cornelk @@ -29,7 +30,6 @@ lozdog245 | https://github.com/lozdog245 soxipy | https://github.com/soxipy mshogin | https://github.com/mshogin herenow | https://github.com/herenow -blombard | https://github.com/blombard tk42 | https://github.com/tk42 daniel-cohen | https://github.com/daniel-cohen DirectX | https://github.com/DirectX @@ -47,3 +47,4 @@ merkeld | https://github.com/merkeld CodeLingoTeam | https://github.com/CodeLingoTeam Daanikus | https://github.com/Daanikus CodeLingoBot | https://github.com/CodeLingoBot +blombard | https://github.com/blombard diff --git a/README.md b/README.md index 1be939d5..f65bb785 100644 --- a/README.md +++ b/README.md @@ -142,16 +142,16 @@ Binaries will be published once the codebase reaches a stable condition. |User|Contribution Amount| |--|--| -| [thrasher-](https://github.com/thrasher-) | 655 | -| [shazbert](https://github.com/shazbert) | 207 | -| [gloriousCode](https://github.com/gloriousCode) | 180 | +| [thrasher-](https://github.com/thrasher-) | 656 | +| [shazbert](https://github.com/shazbert) | 209 | +| [gloriousCode](https://github.com/gloriousCode) | 184 | | [dependabot-preview[bot]](https://github.com/apps/dependabot-preview) | 88 | | [xtda](https://github.com/xtda) | 47 | +| [dependabot[bot]](https://github.com/apps/dependabot) | 18 | | [Rots](https://github.com/Rots) | 15 | | [vazha](https://github.com/vazha) | 15 | | [ermalguni](https://github.com/ermalguni) | 14 | -| [MadCozBadd](https://github.com/MadCozBadd) | 12 | -| [dependabot[bot]](https://github.com/apps/dependabot) | 12 | +| [MadCozBadd](https://github.com/MadCozBadd) | 13 | | [vadimzhukck](https://github.com/vadimzhukck) | 10 | | [140am](https://github.com/140am) | 8 | | [marcofranssen](https://github.com/marcofranssen) | 8 | @@ -163,6 +163,7 @@ Binaries will be published once the codebase reaches a stable condition. | [andreygrehov](https://github.com/andreygrehov) | 2 | | [bretep](https://github.com/bretep) | 2 | | [Christian-Achilli](https://github.com/Christian-Achilli) | 2 | +| [MarkDzulko](https://github.com/MarkDzulko) | 2 | | [yangrq1018](https://github.com/yangrq1018) | 2 | | [gam-phon](https://github.com/gam-phon) | 2 | | [cornelk](https://github.com/cornelk) | 2 | @@ -171,7 +172,6 @@ Binaries will be published once the codebase reaches a stable condition. | [soxipy](https://github.com/soxipy) | 2 | | [mshogin](https://github.com/mshogin) | 2 | | [herenow](https://github.com/herenow) | 2 | -| [blombard](https://github.com/blombard) | 1 | | [tk42](https://github.com/tk42) | 2 | | [daniel-cohen](https://github.com/daniel-cohen) | 1 | | [DirectX](https://github.com/DirectX) | 1 | @@ -189,3 +189,4 @@ Binaries will be published once the codebase reaches a stable condition. | [CodeLingoTeam](https://github.com/CodeLingoTeam) | 1 | | [Daanikus](https://github.com/Daanikus) | 1 | | [CodeLingoBot](https://github.com/CodeLingoBot) | 1 | +| [blombard](https://github.com/blombard) | 1 | diff --git a/backtester/README.md b/backtester/README.md index a627326d..3297c069 100644 --- a/backtester/README.md +++ b/backtester/README.md @@ -39,6 +39,20 @@ An event-driven backtesting tool to test and iterate trading strategies using hi - Helpful statistics to help determine whether a strategy was effective - Compliance manager to keep snapshots of every transaction and their changes at every interval +## Planned Features +We welcome pull requests on any feature for the Backtester! We will be especially appreciative of any contribution towards the following planned features: + +| Feature | Description | +|---------|-------------| +| Add quote-based portfolio funding feature | Funds are currently currency-pair based which is helpful for running the same strategy against many pairs simultaneously. This feature would allow for shared funding pool for an overarching strategy | +| Add backtesting support for futures asset types | Spot trading is currently the only supported asset type. Futures trading greatly expands the Backtester's potential | +| Example futures pairs trading strategy | Providing a basic example will allow for esteemed traders to build and customise their own | +| Save Backtester results to database | This will allow for easier comparison of results over time | +| Backtester result comparison report | Providing an executive summary of Backtester database results | +| Currency correlation | Compare multiple exchange, asset, currencies for a candle interval against indicators to highlight correlated pairs for use in pairs trading | +| Improve live trading functionality | Live trading is currently only a proof Of concept. Adding live support for running multiple currencies and running off orderbook data will allow for esteemed traders to use their backtested strategies | + + ## How does it work? - The application will load a `.strat` config file as specified at runtime - The `.strat` config file will contain @@ -72,7 +86,7 @@ Creating strategies requires programming skills. [Here](/backtester/eventhandler - The readmes linked in the "How does it work" covers the main parts of the application. - If you are still unsure, please raise an issue, ask a question in our Slack or open a pull request - Here is an overview -![workflow](https://user-images.githubusercontent.com/9261323/104982257-61d97900-5a5e-11eb-930e-3b431d6e6bab.png) +![workflow](https://i.imgur.com/Kup6IA9.png) # Important notes diff --git a/backtester/backtest/README.md b/backtester/backtest/README.md index f1fa7502..754c4036 100644 --- a/backtester/backtest/README.md +++ b/backtester/backtest/README.md @@ -31,7 +31,7 @@ It is responsible for the following functionality A flow of the application is as follows: -![workflow](https://user-images.githubusercontent.com/9261323/104982257-61d97900-5a5e-11eb-930e-3b431d6e6bab.png) +![workflow](https://i.imgur.com/Kup6IA9.png) ### Please click GoDocs chevron above to view current GoDoc information for this package diff --git a/backtester/backtest/backtest.go b/backtester/backtest/backtest.go index bf81f9f6..37e73263 100644 --- a/backtester/backtest/backtest.go +++ b/backtester/backtest/backtest.go @@ -251,7 +251,7 @@ func (bt *BackTest) setupExchangeSettings(cfg *config.Config) (exchange.Exchange } if cfg.CurrencySettings[i].MaximumSlippagePercent < 0 { - log.Warnf(log.BackTester, "invalid maximum slippage percent '%v'. Slippage percent is defined as a number, eg '100.00', defaulting to '%v'", + log.Warnf(log.BackTester, "invalid maximum slippage percent '%f'. Slippage percent is defined as a number, eg '100.00', defaulting to '%f'", cfg.CurrencySettings[i].MaximumSlippagePercent, slippage.DefaultMaximumSlippagePercent) cfg.CurrencySettings[i].MaximumSlippagePercent = slippage.DefaultMaximumSlippagePercent @@ -260,7 +260,7 @@ func (bt *BackTest) setupExchangeSettings(cfg *config.Config) (exchange.Exchange cfg.CurrencySettings[i].MaximumSlippagePercent = slippage.DefaultMaximumSlippagePercent } if cfg.CurrencySettings[i].MinimumSlippagePercent < 0 { - log.Warnf(log.BackTester, "invalid minimum slippage percent '%v'. Slippage percent is defined as a number, eg '80.00', defaulting to '%v'", + log.Warnf(log.BackTester, "invalid minimum slippage percent '%f'. Slippage percent is defined as a number, eg '80.00', defaulting to '%f'", cfg.CurrencySettings[i].MinimumSlippagePercent, slippage.DefaultMinimumSlippagePercent) cfg.CurrencySettings[i].MinimumSlippagePercent = slippage.DefaultMinimumSlippagePercent diff --git a/backtester/config/config.go b/backtester/config/config.go index 961997d0..ec8d5be1 100644 --- a/backtester/config/config.go +++ b/backtester/config/config.go @@ -116,19 +116,19 @@ func (c *Config) PrintSetting() { func (m *MinMax) Validate() { if m.MaximumSize < 0 { m.MaximumSize *= -1 - log.Warnf(log.BackTester, "invalid maximum size set to %v", m.MaximumSize) + log.Warnf(log.BackTester, "invalid maximum size set to %f", m.MaximumSize) } if m.MinimumSize < 0 { m.MinimumSize *= -1 - log.Warnf(log.BackTester, "invalid minimum size set to %v", m.MinimumSize) + log.Warnf(log.BackTester, "invalid minimum size set to %f", m.MinimumSize) } if m.MaximumSize <= m.MinimumSize && m.MinimumSize != 0 && m.MaximumSize != 0 { m.MaximumSize = m.MinimumSize + 1 - log.Warnf(log.BackTester, "invalid maximum size set to %v", m.MaximumSize) + log.Warnf(log.BackTester, "invalid maximum size set to %f", m.MaximumSize) } if m.MaximumTotal < 0 { m.MaximumTotal *= -1 - log.Warnf(log.BackTester, "invalid maximum total set to %v", m.MaximumTotal) + log.Warnf(log.BackTester, "invalid maximum total set to %f", m.MaximumTotal) } } diff --git a/backtester/eventhandlers/README.md b/backtester/eventhandlers/README.md index 2e71271e..c8b10690 100644 --- a/backtester/eventhandlers/README.md +++ b/backtester/eventhandlers/README.md @@ -22,7 +22,7 @@ Join our slack to discuss all things related to GoCryptoTrader! [GoCryptoTrader Event handlers are responsible for taking in an event, analysing its contents and outputting another event to be handled. An individual candle is turned into a data event which handled via the strategy event handler. The strategy handler outputs a signal event, which the portfolio eventhandler will size and risk analyse before raising an order event. The event is then sent to the portfolio manager to determine whether there is appropriate funding, adequate risk and proper order sizing before raising an order event. The order event is taken to the exchange handler which will place the order and create a fill event. Below is an overview of how event handlers are used -![workflow](https://user-images.githubusercontent.com/9261323/104982257-61d97900-5a5e-11eb-930e-3b431d6e6bab.png) +![workflow](https://i.imgur.com/Kup6IA9.png) ### Please click GoDocs chevron above to view current GoDoc information for this package diff --git a/backtester/eventhandlers/exchange/exchange.go b/backtester/eventhandlers/exchange/exchange.go index 38288150..30c70ab8 100644 --- a/backtester/eventhandlers/exchange/exchange.go +++ b/backtester/eventhandlers/exchange/exchange.go @@ -5,6 +5,7 @@ import ( "github.com/gofrs/uuid" "github.com/thrasher-corp/gocryptotrader/backtester/common" + "github.com/thrasher-corp/gocryptotrader/backtester/config" "github.com/thrasher-corp/gocryptotrader/backtester/data" "github.com/thrasher-corp/gocryptotrader/backtester/eventhandlers/exchange/slippage" "github.com/thrasher-corp/gocryptotrader/backtester/eventtypes/event" @@ -35,9 +36,8 @@ func (e *Exchange) ExecuteOrder(o order.Event, data data.Handler, bot *engine.En Interval: o.GetInterval(), Reason: o.GetReason(), }, - Direction: o.GetDirection(), - Amount: o.GetAmount(), - + Direction: o.GetDirection(), + Amount: o.GetAmount(), ClosePrice: data.Latest().ClosePrice(), } @@ -85,41 +85,27 @@ func (e *Exchange) ExecuteOrder(o order.Event, data data.Handler, bot *engine.En return f, err } } - reducedAmount := reduceAmountToFitPortfolioLimit(adjustedPrice, amount, o.GetFunds()) - if reducedAmount != amount { - f.AppendReason(fmt.Sprintf("Order size shrunk from %v to %v to remain within portfolio limits", amount, reducedAmount)) + + portfolioLimitedAmount := reduceAmountToFitPortfolioLimit(adjustedPrice, amount, o.GetFunds(), f.GetDirection()) + if portfolioLimitedAmount != amount { + f.AppendReason(fmt.Sprintf("Order size shrunk from %f to %f to remain within portfolio limits", amount, portfolioLimitedAmount)) } - var limitReducedAmount float64 + limitReducedAmount := portfolioLimitedAmount if cs.CanUseExchangeLimits { // Conforms the amount to the exchange order defined step amount // reducing it when needed - limitReducedAmount = cs.Limits.ConformToAmount(reducedAmount) - if limitReducedAmount != reducedAmount { - f.AppendReason(fmt.Sprintf("Order size shrunk from %v to %v to remain within exchange step amount limits", - reducedAmount, + limitReducedAmount = cs.Limits.ConformToAmount(portfolioLimitedAmount) + if limitReducedAmount != portfolioLimitedAmount { + f.AppendReason(fmt.Sprintf("Order size shrunk from %f to %f to remain within exchange step amount limits", + portfolioLimitedAmount, limitReducedAmount)) } - } else { - limitReducedAmount = reducedAmount } - // Conforms the amount to fall into the minimum size and maximum size limit after reduced - switch f.GetDirection() { - case gctorder.Buy: - if ((limitReducedAmount < cs.BuySide.MinimumSize && cs.BuySide.MinimumSize > 0) || (limitReducedAmount > cs.BuySide.MaximumSize && cs.BuySide.MaximumSize > 0)) && (cs.BuySide.MaximumSize > 0 || cs.BuySide.MinimumSize > 0) { - f.SetDirection(common.CouldNotBuy) - e := fmt.Sprintf("Order size %.8f exceed minimum size %.8f or maximum size %.8f ", limitReducedAmount, cs.BuySide.MinimumSize, cs.BuySide.MaximumSize) - f.AppendReason(e) - return f, fmt.Errorf(e) - } - case gctorder.Sell: - if ((limitReducedAmount < cs.SellSide.MinimumSize && cs.SellSide.MinimumSize > 0) || (limitReducedAmount > cs.SellSide.MaximumSize && cs.SellSide.MaximumSize > 0)) && (cs.SellSide.MaximumSize > 0 || cs.SellSide.MinimumSize > 0) { - f.SetDirection(common.CouldNotSell) - e := fmt.Sprintf("Order size %.8f exceed minimum size %.8f or maximum size %.8f ", limitReducedAmount, cs.SellSide.MinimumSize, cs.SellSide.MaximumSize) - f.AppendReason(e) - return f, fmt.Errorf(e) - } + err = verifyOrderWithinLimits(f, limitReducedAmount, &cs) + if err != nil { + return f, err } orderID, err := e.placeOrder(adjustedPrice, limitReducedAmount, cs.UseRealOrders, cs.CanUseExchangeLimits, f, bot) @@ -152,11 +138,62 @@ func (e *Exchange) ExecuteOrder(o order.Event, data data.Handler, bot *engine.En return f, nil } -func reduceAmountToFitPortfolioLimit(adjustedPrice, amount, sizedPortfolioTotal float64) float64 { - if adjustedPrice*amount > sizedPortfolioTotal { - // adjusted amounts exceeds portfolio manager's allowed funds - // the amount has to be reduced to equal the sizedPortfolioTotal - amount = sizedPortfolioTotal / adjustedPrice +// verifyOrderWithinLimits conforms the amount to fall into the minimum size and maximum size limit after reduced +func verifyOrderWithinLimits(f *fill.Fill, limitReducedAmount float64, cs *Settings) error { + if f == nil { + return common.ErrNilEvent + } + if cs == nil { + return errNilCurrencySettings + } + exceeded := false + var minMax config.MinMax + var direction gctorder.Side + switch f.GetDirection() { + case gctorder.Buy: + minMax = cs.BuySide + direction = common.CouldNotBuy + case gctorder.Sell: + minMax = cs.SellSide + direction = common.CouldNotSell + default: + direction = f.GetDirection() + f.SetDirection(common.DoNothing) + return fmt.Errorf("%w: %v", errInvalidDirection, direction) + } + var exceededLimit string + var size float64 + if limitReducedAmount < minMax.MinimumSize && minMax.MinimumSize > 0 { + exceeded = true + exceededLimit = "minimum" + size = minMax.MinimumSize + } + if limitReducedAmount > minMax.MaximumSize && minMax.MaximumSize > 0 { + exceeded = true + exceededLimit = "maximum" + size = minMax.MaximumSize + } + if exceeded { + f.SetDirection(direction) + e := fmt.Sprintf("Order size %.8f exceeded %v size %.8f", limitReducedAmount, exceededLimit, size) + f.AppendReason(e) + return fmt.Errorf("%w %v", errExceededPortfolioLimit, e) + } + return nil +} + +func reduceAmountToFitPortfolioLimit(adjustedPrice, amount, sizedPortfolioTotal float64, side gctorder.Side) float64 { + switch side { + case gctorder.Buy: + if adjustedPrice*amount > sizedPortfolioTotal { + // adjusted amounts exceeds portfolio manager's allowed funds + // the amount has to be reduced to equal the sizedPortfolioTotal + amount = sizedPortfolioTotal / adjustedPrice + } + case gctorder.Sell: + if amount > sizedPortfolioTotal { + amount = sizedPortfolioTotal + } } return amount } @@ -220,7 +257,7 @@ func (e *Exchange) sizeOfflineOrder(high, low, volume float64, cs *Settings, f * slippageRate := slippage.EstimateSlippagePercentage(cs.MinimumSlippageRate, cs.MaximumSlippageRate) f.VolumeAdjustedPrice, adjustedAmount = ensureOrderFitsWithinHLV(f.ClosePrice, f.Amount, high, low, volume) if adjustedAmount != f.Amount { - f.AppendReason(fmt.Sprintf("Order size shrunk from %v to %v to fit candle", f.Amount, adjustedAmount)) + f.AppendReason(fmt.Sprintf("Order size shrunk from %f to %f to fit candle", f.Amount, adjustedAmount)) } if adjustedAmount <= 0 && f.Amount > 0 { diff --git a/backtester/eventhandlers/exchange/exchange_test.go b/backtester/eventhandlers/exchange/exchange_test.go index 316e05c0..152a7b0e 100644 --- a/backtester/eventhandlers/exchange/exchange_test.go +++ b/backtester/eventhandlers/exchange/exchange_test.go @@ -14,6 +14,7 @@ import ( "github.com/thrasher-corp/gocryptotrader/backtester/eventtypes/order" "github.com/thrasher-corp/gocryptotrader/currency" "github.com/thrasher-corp/gocryptotrader/engine" + exchange "github.com/thrasher-corp/gocryptotrader/exchanges" "github.com/thrasher-corp/gocryptotrader/exchanges/asset" gctkline "github.com/thrasher-corp/gocryptotrader/exchanges/kline" gctorder "github.com/thrasher-corp/gocryptotrader/exchanges/order" @@ -374,11 +375,8 @@ func TestExecuteOrderBuySellSizeLimit(t *testing.T) { } d.Next() _, err = e.ExecuteOrder(o, d, bot) - if err != nil && !strings.Contains(err.Error(), "exceed minimum size") { - t.Error(err) - } - if err == nil { - t.Error("Order size 0.99999999 should exceed minimum size 0.00000000 or maximum size 0.01000000") + if !errors.Is(err, errExceededPortfolioLimit) { + t.Errorf("received %v expected %v", err, errExceededPortfolioLimit) } o = &order.Order{ Base: ev, @@ -423,19 +421,15 @@ func TestExecuteOrderBuySellSizeLimit(t *testing.T) { cs.SellSide.MinimumSize = 1 e.CurrencySettings = []Settings{cs} _, err = e.ExecuteOrder(o, d, bot) - if err != nil && !strings.Contains(err.Error(), "exceed minimum size") { - t.Error(err) - } - - if err == nil { - t.Error(" Order size 0.50000000 should exceed minimum size 1.00000000") + if !errors.Is(err, errExceededPortfolioLimit) { + t.Errorf("received %v expected %v", err, errExceededPortfolioLimit) } o = &order.Order{ Base: ev, Direction: gctorder.Sell, Amount: 0.02, - Funds: 1337, + Funds: 0.01337, } cs.SellSide.MaximumSize = 0 cs.SellSide.MinimumSize = 0.01 @@ -445,8 +439,8 @@ func TestExecuteOrderBuySellSizeLimit(t *testing.T) { o.Direction = gctorder.Sell e.CurrencySettings = []Settings{cs} _, err = e.ExecuteOrder(o, d, bot) - if err != nil && !strings.Contains(err.Error(), "unset/default API keys") { - t.Error(err) + if !errors.Is(err, exchange.ErrAuthenticatedRequestWithoutCredentialsSet) { + t.Errorf("received %v expected %v", err, exchange.ErrAuthenticatedRequestWithoutCredentialsSet) } } @@ -469,8 +463,71 @@ func TestReduceAmountToFitPortfolioLimit(t *testing.T) { portfolioAdjustedTotal := initialAmount * initialPrice adjustedPrice := 1000.0 amount := 2.0 - finalAmount := reduceAmountToFitPortfolioLimit(adjustedPrice, amount, portfolioAdjustedTotal) + finalAmount := reduceAmountToFitPortfolioLimit(adjustedPrice, amount, portfolioAdjustedTotal, gctorder.Buy) if finalAmount*adjustedPrice != portfolioAdjustedTotal { t.Errorf("expected value %v to match portfolio total %v", finalAmount*adjustedPrice, portfolioAdjustedTotal) } + finalAmount = reduceAmountToFitPortfolioLimit(adjustedPrice, 133333333337, portfolioAdjustedTotal, gctorder.Sell) + if finalAmount != portfolioAdjustedTotal { + t.Errorf("expected value %v to match portfolio total %v", finalAmount, portfolioAdjustedTotal) + } + finalAmount = reduceAmountToFitPortfolioLimit(adjustedPrice, 1, portfolioAdjustedTotal, gctorder.Sell) + if finalAmount != 1 { + t.Errorf("expected value %v to match portfolio total %v", finalAmount, portfolioAdjustedTotal) + } +} + +func TestVerifyOrderWithinLimits(t *testing.T) { + t.Parallel() + err := verifyOrderWithinLimits(nil, 0, nil) + if !errors.Is(err, common.ErrNilEvent) { + t.Errorf("received %v expected %v", err, common.ErrNilEvent) + } + + err = verifyOrderWithinLimits(&fill.Fill{}, 0, nil) + if !errors.Is(err, errNilCurrencySettings) { + t.Errorf("received %v expected %v", err, errNilCurrencySettings) + } + + err = verifyOrderWithinLimits(&fill.Fill{}, 0, &Settings{}) + if !errors.Is(err, errInvalidDirection) { + t.Errorf("received %v expected %v", err, errInvalidDirection) + } + f := &fill.Fill{ + Direction: gctorder.Buy, + } + err = verifyOrderWithinLimits(f, 0, &Settings{}) + if !errors.Is(err, nil) { + t.Errorf("received %v expected %v", err, nil) + } + s := &Settings{ + BuySide: config.MinMax{ + MinimumSize: 1, + MaximumSize: 1, + }, + } + err = verifyOrderWithinLimits(f, 0.5, s) + if !errors.Is(err, errExceededPortfolioLimit) { + t.Errorf("received %v expected %v", err, errExceededPortfolioLimit) + } + f.Direction = gctorder.Buy + err = verifyOrderWithinLimits(f, 2, s) + if !errors.Is(err, errExceededPortfolioLimit) { + t.Errorf("received %v expected %v", err, errExceededPortfolioLimit) + } + + f.Direction = gctorder.Sell + s.SellSide = config.MinMax{ + MinimumSize: 1, + MaximumSize: 1, + } + err = verifyOrderWithinLimits(f, 0.5, s) + if !errors.Is(err, errExceededPortfolioLimit) { + t.Errorf("received %v expected %v", err, errExceededPortfolioLimit) + } + f.Direction = gctorder.Sell + err = verifyOrderWithinLimits(f, 2, s) + if !errors.Is(err, errExceededPortfolioLimit) { + t.Errorf("received %v expected %v", err, errExceededPortfolioLimit) + } } diff --git a/backtester/eventhandlers/exchange/exchange_types.go b/backtester/eventhandlers/exchange/exchange_types.go index 8c7c90a8..cb96b1b3 100644 --- a/backtester/eventhandlers/exchange/exchange_types.go +++ b/backtester/eventhandlers/exchange/exchange_types.go @@ -14,7 +14,10 @@ import ( ) var ( - errDataMayBeIncorrect = errors.New("data may be incorrect") + errDataMayBeIncorrect = errors.New("data may be incorrect") + errExceededPortfolioLimit = errors.New("exceeded portfolio limit") + errNilCurrencySettings = errors.New("received nil currency settings") + errInvalidDirection = errors.New("received invalid order direction") ) // ExecutionHandler interface dictates what functions are required to submit an order diff --git a/backtester/eventhandlers/exchange/slippage/slippage_types.go b/backtester/eventhandlers/exchange/slippage/slippage_types.go index 08b0a5ff..afd9fc7f 100644 --- a/backtester/eventhandlers/exchange/slippage/slippage_types.go +++ b/backtester/eventhandlers/exchange/slippage/slippage_types.go @@ -3,6 +3,6 @@ package slippage // Default slippage rates. It works on a percentage basis // 100 means unaffected, 95 would mean 95% const ( - DefaultMaximumSlippagePercent = 100 - DefaultMinimumSlippagePercent = 100 + DefaultMaximumSlippagePercent float64 = 100 + DefaultMinimumSlippagePercent float64 = 100 ) diff --git a/backtester/eventhandlers/portfolio/risk/risk.go b/backtester/eventhandlers/portfolio/risk/risk.go index bae4d8c6..07524947 100644 --- a/backtester/eventhandlers/portfolio/risk/risk.go +++ b/backtester/eventhandlers/portfolio/risk/risk.go @@ -31,16 +31,16 @@ func (r *Risk) EvaluateOrder(o order.Event, latestHoldings []holdings.Holding, s } ratio := existingLeverageRatio(s) if ratio > lookup.MaximumOrdersWithLeverageRatio && lookup.MaximumOrdersWithLeverageRatio > 0 { - return nil, fmt.Errorf("proceeding with the order would put maximum orders using leverage ratio beyond its limit of %v to %v and %w", lookup.MaximumOrdersWithLeverageRatio, ratio, errCannotPlaceLeverageOrder) + return nil, fmt.Errorf("proceeding with the order would put maximum orders using leverage ratio beyond its limit of %f to %f and %w", lookup.MaximumOrdersWithLeverageRatio, ratio, errCannotPlaceLeverageOrder) } if retOrder.GetLeverage() > lookup.MaxLeverageRate && lookup.MaxLeverageRate > 0 { - return nil, fmt.Errorf("proceeding with the order would put leverage rate beyond its limit of %v to %v and %w", lookup.MaxLeverageRate, retOrder.GetLeverage(), errCannotPlaceLeverageOrder) + return nil, fmt.Errorf("proceeding with the order would put leverage rate beyond its limit of %f to %f and %w", lookup.MaxLeverageRate, retOrder.GetLeverage(), errCannotPlaceLeverageOrder) } } if len(latestHoldings) > 1 { ratio := assessHoldingsRatio(o.Pair(), latestHoldings) if lookup.MaximumHoldingRatio > 0 && ratio != 1 && ratio > lookup.MaximumHoldingRatio { - return nil, fmt.Errorf("order would exceed maximum holding ratio of %v to %v for %v %v %v. %w", lookup.MaximumHoldingRatio, ratio, ex, a, p, errCannotPlaceLeverageOrder) + return nil, fmt.Errorf("order would exceed maximum holding ratio of %f to %f for %v %v %v. %w", lookup.MaximumHoldingRatio, ratio, ex, a, p, errCannotPlaceLeverageOrder) } } return retOrder, nil diff --git a/backtester/eventhandlers/portfolio/size/size.go b/backtester/eventhandlers/portfolio/size/size.go index 790ad850..da5827a3 100644 --- a/backtester/eventhandlers/portfolio/size/size.go +++ b/backtester/eventhandlers/portfolio/size/size.go @@ -85,7 +85,7 @@ func (s *Size) calculateBuySize(price, availableFunds, feeRate, buyLimit float64 amount = minMaxSettings.MaximumTotal * (1 - feeRate) / price } if amount < minMaxSettings.MinimumSize && minMaxSettings.MinimumSize > 0 { - return 0, fmt.Errorf("%w. Sized: '%.8f' Minimum: '%v'", errLessThanMinimum, amount, minMaxSettings.MinimumSize) + return 0, fmt.Errorf("%w. Sized: '%.8f' Minimum: '%f'", errLessThanMinimum, amount, minMaxSettings.MinimumSize) } return amount, nil } @@ -114,7 +114,7 @@ func (s *Size) calculateSellSize(price, baseAmount, feeRate, sellLimit float64, amount = minMaxSettings.MaximumTotal * (1 - feeRate) / price } if amount < minMaxSettings.MinimumSize && minMaxSettings.MinimumSize > 0 { - return 0, fmt.Errorf("%w. Sized: '%.8f' Minimum: '%v'", errLessThanMinimum, amount, minMaxSettings.MinimumSize) + return 0, fmt.Errorf("%w. Sized: '%.8f' Minimum: '%f'", errLessThanMinimum, amount, minMaxSettings.MinimumSize) } return amount, nil diff --git a/backtester/eventhandlers/statistics/statistics.go b/backtester/eventhandlers/statistics/statistics.go index 4e1ffad4..885d39b4 100644 --- a/backtester/eventhandlers/statistics/statistics.go +++ b/backtester/eventhandlers/statistics/statistics.go @@ -206,7 +206,7 @@ func (s *Statistic) PrintTotalResults() { log.Infof(log.BackTester, "Exchange: %v Asset: %v Currency: %v", s.BiggestDrawdown.Exchange, s.BiggestDrawdown.Asset, s.BiggestDrawdown.Pair) log.Infof(log.BackTester, "Highest Price: $%.2f", s.BiggestDrawdown.MaxDrawdown.Highest.Price) log.Infof(log.BackTester, "Highest Price Time: %v", s.BiggestDrawdown.MaxDrawdown.Highest.Time) - log.Infof(log.BackTester, "Lowest Price: $%v", s.BiggestDrawdown.MaxDrawdown.Lowest.Price) + log.Infof(log.BackTester, "Lowest Price: $%.2f", s.BiggestDrawdown.MaxDrawdown.Lowest.Price) log.Infof(log.BackTester, "Lowest Price Time: %v", s.BiggestDrawdown.MaxDrawdown.Lowest.Time) log.Infof(log.BackTester, "Calculated Drawdown: %.2f%%", s.BiggestDrawdown.MaxDrawdown.DrawdownPercent) log.Infof(log.BackTester, "Difference: $%.2f", s.BiggestDrawdown.MaxDrawdown.Highest.Price-s.BiggestDrawdown.MaxDrawdown.Lowest.Price) @@ -214,8 +214,8 @@ func (s *Statistic) PrintTotalResults() { } if s.BestMarketMovement != nil && s.BestStrategyResults != nil { log.Info(log.BackTester, "------------------Orders----------------------------------") - log.Infof(log.BackTester, "Best performing market movement: %v %v %v %v%%", s.BestMarketMovement.Exchange, s.BestMarketMovement.Asset, s.BestMarketMovement.Pair, s.BestMarketMovement.MarketMovement) - log.Infof(log.BackTester, "Best performing strategy movement: %v %v %v %v%%\n\n", s.BestStrategyResults.Exchange, s.BestStrategyResults.Asset, s.BestStrategyResults.Pair, s.BestStrategyResults.StrategyMovement) + log.Infof(log.BackTester, "Best performing market movement: %v %v %v %f%%", s.BestMarketMovement.Exchange, s.BestMarketMovement.Asset, s.BestMarketMovement.Pair, s.BestMarketMovement.MarketMovement) + log.Infof(log.BackTester, "Best performing strategy movement: %v %v %v %f%%\n\n", s.BestStrategyResults.Exchange, s.BestStrategyResults.Asset, s.BestStrategyResults.Pair, s.BestStrategyResults.StrategyMovement) } } @@ -272,13 +272,13 @@ func (s *Statistic) PrintAllEvents() { direction == common.DoNothing || direction == common.MissingData || direction == "" { - log.Infof(log.BackTester, "%v | Price: $%v - Direction: %v - Reason: %s", + log.Infof(log.BackTester, "%v | Price: $%f - Direction: %v - Reason: %s", c.Events[i].FillEvent.GetTime().Format(gctcommon.SimpleTimeFormat), c.Events[i].FillEvent.GetClosePrice(), c.Events[i].FillEvent.GetDirection(), c.Events[i].FillEvent.GetReason()) } else { - log.Infof(log.BackTester, "%v | Price: $%v - Amount: %v - Fee: $%v - Total: $%v - Direction %v - Reason: %s", + log.Infof(log.BackTester, "%v | Price: $%f - Amount: %f - Fee: $%f - Total: $%f - Direction %v - Reason: %s", c.Events[i].FillEvent.GetTime().Format(gctcommon.SimpleTimeFormat), c.Events[i].FillEvent.GetPurchasePrice(), c.Events[i].FillEvent.GetAmount(), @@ -289,12 +289,12 @@ func (s *Statistic) PrintAllEvents() { ) } case c.Events[i].SignalEvent != nil: - log.Infof(log.BackTester, "%v | Price: $%v - Reason: %v", + log.Infof(log.BackTester, "%v | Price: $%f - Reason: %v", c.Events[i].SignalEvent.GetTime().Format(gctcommon.SimpleTimeFormat), c.Events[i].SignalEvent.GetPrice(), c.Events[i].SignalEvent.GetReason()) case c.Events[i].DataEvent != nil: - log.Infof(log.BackTester, "%v | Price: $%v - Reason: %v", + log.Infof(log.BackTester, "%v | Price: $%f - Reason: %v", c.Events[i].DataEvent.GetTime().Format(gctcommon.SimpleTimeFormat), c.Events[i].DataEvent.ClosePrice(), c.Events[i].DataEvent.GetReason()) diff --git a/backtester/eventtypes/README.md b/backtester/eventtypes/README.md index 4c444f09..b8f4e5c1 100644 --- a/backtester/eventtypes/README.md +++ b/backtester/eventtypes/README.md @@ -22,7 +22,7 @@ Join our slack to discuss all things related to GoCryptoTrader! [GoCryptoTrader Event types are created after retrieving candle data. An individual candle is turned into a data event which is sent to the strategy for analysis. The event is then sent to the portfolio manager to determine whether there is appropriate funding, adequate risk and proper order sizing before raising an order event. The order event is taken to the exchange handler which will place the order and create a fill event. The fill event is used to update the portfolios individual holdings for analysis and decision making. Below is an overview of how events are used -![workflow](https://user-images.githubusercontent.com/9261323/104982257-61d97900-5a5e-11eb-930e-3b431d6e6bab.png) +![workflow](https://i.imgur.com/Kup6IA9.png) ### Please click GoDocs chevron above to view current GoDoc information for this package diff --git a/cmd/documentation/backtester_templates/backtester_backtest_readme.tmpl b/cmd/documentation/backtester_templates/backtester_backtest_readme.tmpl index 99ed0d48..88c6c1fa 100644 --- a/cmd/documentation/backtester_templates/backtester_backtest_readme.tmpl +++ b/cmd/documentation/backtester_templates/backtester_backtest_readme.tmpl @@ -13,7 +13,7 @@ It is responsible for the following functionality A flow of the application is as follows: -![workflow](https://user-images.githubusercontent.com/9261323/104982257-61d97900-5a5e-11eb-930e-3b431d6e6bab.png) +![workflow](https://i.imgur.com/Kup6IA9.png) ### Please click GoDocs chevron above to view current GoDoc information for this package diff --git a/cmd/documentation/backtester_templates/backtester_eventhandlers_readme.tmpl b/cmd/documentation/backtester_templates/backtester_eventhandlers_readme.tmpl index c919c4c1..cfcfb8c7 100644 --- a/cmd/documentation/backtester_templates/backtester_eventhandlers_readme.tmpl +++ b/cmd/documentation/backtester_templates/backtester_eventhandlers_readme.tmpl @@ -4,7 +4,7 @@ Event handlers are responsible for taking in an event, analysing its contents and outputting another event to be handled. An individual candle is turned into a data event which handled via the strategy event handler. The strategy handler outputs a signal event, which the portfolio eventhandler will size and risk analyse before raising an order event. The event is then sent to the portfolio manager to determine whether there is appropriate funding, adequate risk and proper order sizing before raising an order event. The order event is taken to the exchange handler which will place the order and create a fill event. Below is an overview of how event handlers are used -![workflow](https://user-images.githubusercontent.com/9261323/104982257-61d97900-5a5e-11eb-930e-3b431d6e6bab.png) +![workflow](https://i.imgur.com/Kup6IA9.png) ### Please click GoDocs chevron above to view current GoDoc information for this package diff --git a/cmd/documentation/backtester_templates/backtester_eventtypes_readme.tmpl b/cmd/documentation/backtester_templates/backtester_eventtypes_readme.tmpl index 49ccd690..9d1a94ff 100644 --- a/cmd/documentation/backtester_templates/backtester_eventtypes_readme.tmpl +++ b/cmd/documentation/backtester_templates/backtester_eventtypes_readme.tmpl @@ -4,7 +4,7 @@ Event types are created after retrieving candle data. An individual candle is turned into a data event which is sent to the strategy for analysis. The event is then sent to the portfolio manager to determine whether there is appropriate funding, adequate risk and proper order sizing before raising an order event. The order event is taken to the exchange handler which will place the order and create a fill event. The fill event is used to update the portfolios individual holdings for analysis and decision making. Below is an overview of how events are used -![workflow](https://user-images.githubusercontent.com/9261323/104982257-61d97900-5a5e-11eb-930e-3b431d6e6bab.png) +![workflow](https://i.imgur.com/Kup6IA9.png) ### Please click GoDocs chevron above to view current GoDoc information for this package diff --git a/cmd/documentation/backtester_templates/backtester_readme.tmpl b/cmd/documentation/backtester_templates/backtester_readme.tmpl index d6bb3ada..63054e6d 100644 --- a/cmd/documentation/backtester_templates/backtester_readme.tmpl +++ b/cmd/documentation/backtester_templates/backtester_readme.tmpl @@ -21,6 +21,20 @@ An event-driven backtesting tool to test and iterate trading strategies using hi - Helpful statistics to help determine whether a strategy was effective - Compliance manager to keep snapshots of every transaction and their changes at every interval +## Planned Features +We welcome pull requests on any feature for the Backtester! We will be especially appreciative of any contribution towards the following planned features: + +| Feature | Description | +|---------|-------------| +| Add quote-based portfolio funding feature | Funds are currently currency-pair based which is helpful for running the same strategy against many pairs simultaneously. This feature would allow for shared funding pool for an overarching strategy | +| Add backtesting support for futures asset types | Spot trading is currently the only supported asset type. Futures trading greatly expands the Backtester's potential | +| Example futures pairs trading strategy | Providing a basic example will allow for esteemed traders to build and customise their own | +| Save Backtester results to database | This will allow for easier comparison of results over time | +| Backtester result comparison report | Providing an executive summary of Backtester database results | +| Currency correlation | Compare multiple exchange, asset, currencies for a candle interval against indicators to highlight correlated pairs for use in pairs trading | +| Improve live trading functionality | Live trading is currently only a proof Of concept. Adding live support for running multiple currencies and running off orderbook data will allow for esteemed traders to use their backtested strategies | + + ## How does it work? - The application will load a `.strat` config file as specified at runtime - The `.strat` config file will contain @@ -54,7 +68,7 @@ Creating strategies requires programming skills. [Here](/backtester/eventhandler - The readmes linked in the "How does it work" covers the main parts of the application. - If you are still unsure, please raise an issue, ask a question in our Slack or open a pull request - Here is an overview -![workflow](https://user-images.githubusercontent.com/9261323/104982257-61d97900-5a5e-11eb-930e-3b431d6e6bab.png) +![workflow](https://i.imgur.com/Kup6IA9.png) # Important notes diff --git a/cmd/documentation/documentation.go b/cmd/documentation/documentation.go index 8dbb85ad..20235834 100644 --- a/cmd/documentation/documentation.go +++ b/cmd/documentation/documentation.go @@ -170,6 +170,11 @@ func main() { // Github API missing contributors contributors = append(contributors, []Contributor{ + { + Login: "tk42", + URL: "https://github.com/tk42", + Contributions: 2, + }, { Login: "daniel-cohen", URL: "https://github.com/daniel-cohen", @@ -251,6 +256,11 @@ func main() { URL: "https://github.com/CodeLingoBot", Contributions: 1, }, + { + Login: "blombard", + URL: "https://github.com/blombard", + Contributions: 1, + }, }...) if verbose { diff --git a/cmd/documentation/engine_templates/datahistory_manager.tmpl b/cmd/documentation/engine_templates/datahistory_manager.tmpl index 77a36c23..3942f45b 100644 --- a/cmd/documentation/engine_templates/datahistory_manager.tmpl +++ b/cmd/documentation/engine_templates/datahistory_manager.tmpl @@ -41,7 +41,7 @@ For a breakdown of what a job consists of and what each parameter does, please r + Modify the following example command to your needs: `.\gctcli.exe datahistory upsertjob --nickname=binance-spot-bnb-btc-1h-candles --exchange=binance --asset=spot --pair=BNB-BTC --interval=3600 --start_date="2020-06-02 12:00:00" --end_date="2020-12-02 12:00:00" --request_size_limit=10 --data_type=0 --max_retry_attempts=3 --batch_size=3` ### Candle intervals and trade fetching -+ A candle interval is required for a job, even when fetching trade data. This is to appropriately break down requests into time interval chunks. However, it is restricted to only a small range of times. This is to prevent fetching issues as fetching trades over a period of days or weeks will take a significant amount of time. When setting a job to fetch trades, the allowable range is less than 4 hours and greater than 10 minutes. So an interval of 1 hour will then fetch an hour's worth of trade data. ++ A candle interval is required for a job, even when fetching trade data. This is to appropriately break down requests into time interval chunks. However, it is restricted to only a small range of times. This is to prevent fetching issues as fetching trades over a period of days or weeks will take a significant amount of time. When setting a job to fetch trades, the allowable range is less than 4 hours and greater than 10 minutes. ### Application run time parameters