Files
gocryptotrader/backtester
Scott 48434dfd46 Feature: Candle conversion & Candle validation (#716)
* Remove old concept. Introduce new job types and candle scaling

* Adds extra processing, commands

* new concept for queued jobs. Jobs can pause. New commands to manage status

* =End of day commit designing tables and implementing prerequisites further.

* Adds postgres data history relations

* Fixes table design for sqlite. Fixes all issues from merge

* Fixes craziness of database design. Adds some functions to get related jobs

* Fixes errors

* Updates some documentation, manages prerequisite jobs a little better, adds rpc funcs

* Fixes database design and adjust repo functions

* Tests database relationship

* Test coverage of new job functions

* Finishes coverage of new functions

* Commands and RPC coverage

* New database modifications for new job types

* Adds db support of new columns. Adds conversion validation. lint

* command blurb changes

* Allows websocket test to pass consistently

* Fixes merge issue preventing datahistorymanager from starting via config

* Minor fixes for different job type processing

* Fixes rangeholder issue, fixes validation, does not address jobs not starting or wrong status

* Fixes database tests, but at what cost. Fixes dhm tests

* Fixes dhj completion issue. Adds prerequisite by nickname

* Fixes validation processing. Adds db tests and validation

* Fixes validation job processing range

* Fixes trade sql. Reduces defaults. Validation processing and errors

* Updates cli job commands. adds validation decimal. fix job validation

* Expands run job handling and tests

* Validation work

* Fixes validation processing

* candle relations. new job type. updating database design

* Adds secondary exchange support. Sets stage for candle override

* Re adds accidentally deleted relationship

* Updates loading and saving candles to have relationship data when relevant

* Now validates and replaces candle data appropriately

* Fixes getting and setting datahistory data. Neatens DHM

* Test coverage

* Updates proto for new db types. New test coverage. Secondary exchange work

* Investigation into never-ending validation jobs. Now that intervals are ruled out, now need to complete the job....

* Fixes issues with validation job completion. Fixes validation volume issue for secondary exchange

* Adds candle warning support to the backtester

* Fixes warnings

* lint and begin docs

* Documentation updates. Final testing changes

* Minor fixes

* docs, prerequisite checks, more testing

* Fixes binance trade test. Rename err

* Documentation fixes. Figure fixes

* documentation update

* Fixes remote PSQL tests

* Fix binance mock test

* Remove unnecessary JSON

* regen proto

* Some minor nit fixes

* Var usage, query sorting, log improving, sql mirroring

* Extra coverage

* Experimental removal of m.jobs and mutex. Fix messaging

* Fixes error

* Lint fixes, command description improvements. More isRunning gates

* description improvements

* Lint

* BUFF regenerate

* Rough concept to fix insertions taking up long periods of time

* New calculation for trade data. Adds batch saving

This also adds an experimental request feature to shut down lingering requests. However, its uncertain whether or not this is having any impact. Initially thought it was the trades that was taking time and not SQL. Will investigate further

* Removes experimental requester. Adds documentation. Fixes typo

* rm unused error

* re-adds more forgotten contributors

* Now with proper commit count
2021-08-05 10:27:27 +10:00
..

GoCryptoTrader Backtester: Backtester package

Build Status Software License GoDoc Coverage Status Go Report Card

This backtester package is part of the GoCryptoTrader codebase.

This is still in active development

You can track ideas, planned features and what's in progress on this Trello board: https://trello.com/b/ZAhMhpOy/gocryptotrader.

Join our slack to discuss all things related to GoCryptoTrader! GoCryptoTrader Slack

GoCryptoTrader Backtester

An event-driven backtesting tool to test and iterate trading strategies using historical or custom data.

Features

  • Works with all GoCryptoTrader exchanges that support trade/candle retrieval. See candle readme and trade readme for supported exchanges
  • CSV data import
  • Database data import
  • Proof of concept live data running
  • Can run strategies against multiple cryptocurrencies
  • Can run strategies that can assess multiple currencies simultaneously to make complex decisions
  • Dollar cost strategy implementation
  • RSI strategy implementation
  • Rules customisation via config .strat files
  • Strategy customisation without requiring recompilation. For example, customising RSI high, low and length values via config .strat files.
  • Report generation
  • Portfolio manager to help size orders based on config rules, risk and candle volume
  • Order manager to place orders with customisable slippage estimator
  • Helpful statistics to help determine whether a strategy was effective
  • Compliance manager to keep snapshots of every transaction and their changes at every interval

Planned Features

We welcome pull requests on any feature for the Backtester! We will be especially appreciative of any contribution towards the following planned features:

Feature Description
Add quote-based portfolio funding feature Funds are currently currency-pair based which is helpful for running the same strategy against many pairs simultaneously. This feature would allow for shared funding pool for an overarching strategy
Add backtesting support for futures asset types Spot trading is currently the only supported asset type. Futures trading greatly expands the Backtester's potential
Example futures pairs trading strategy Providing a basic example will allow for esteemed traders to build and customise their own
Save Backtester results to database This will allow for easier comparison of results over time
Backtester result comparison report Providing an executive summary of Backtester database results
Currency correlation Compare multiple exchange, asset, currencies for a candle interval against indicators to highlight correlated pairs for use in pairs trading
Improve live trading functionality Live trading is currently only a proof Of concept. Adding live support for running multiple currencies and running off orderbook data will allow for esteemed traders to use their backtested strategies

How does it work?

  • The application will load a .strat config file as specified at runtime
  • The .strat config file will contain
    • Start & end dates
    • The strategy to run
    • The candle interval
    • Where the data is to be sourced (API, CSV, database, live)
    • Whether to use trade or candle data (readme)
    • A nickname for the strategy (to help differentiate between runs/configs using the same strategy)
    • The currency/currencies to use
    • The exchange(s) to run against
    • See readme for a breakdown of all config features
  • The GoCryptoTrader Backtester will retrieve the data specified in the config (readme)
  • The data is converted into candles and each candle is streamed as a data event.
  • The data event is analysed by the strategy which will output a purchasing signal such as BUY, SELL or DONOTHING (readme)
  • The purchase signal is then processed by the portfolio manager (readme) which will size the order (readme) and assess risk (readme) before sending it to the exchange
  • The exchange order event handler will size to the candle data and run a slippage estimator (readme) and place the order (readme)
  • Upon an order being placed, the order is snapshot for analysis in both the statistics package (readme) and the report package (readme)

Cool story, how do I use it?

To run the application using the provided dollar cost average strategy, simply run go run . from gocryptotrader/backtester. An output of the results will be put in the results folder.

How do I create my own config?

There is a config generating helper application under /backtester/config/configbuilder to help you create a .strat file. Read more about it here. There are also a number of tests under /config/config_test.go which generate configs into the examples folder, which if you have code knowledge, can write your own configs programmatically.

How do I create my own strategy?

Creating strategies requires programming skills. Here is a readme on the subject. After reading the readmes, please review the strategies here to gain an understanding on how to write your own.

How does it work technically?

  • The readmes linked in the "How does it work" covers the main parts of the application.
    • If you are still unsure, please raise an issue, ask a question in our Slack or open a pull request
  • Here is an overview workflow

Important notes

  • This application is not considered production ready and you may experience issues
    • If you encounter any issues, you can raise them in our Slack channel or via Github issues
  • Past performance is no guarantee of future results
  • While an experimental feature, it is not recommended to ever use live trading and real orders
  • Past performance is no guarantee of future results

Please click GoDocs chevron above to view current GoDoc information for this package

Contribution

Please feel free to submit any pull requests or suggest any desired features to be added.

When submitting a PR, please abide by our coding guidelines:

  • Code must adhere to the official Go formatting guidelines (i.e. uses gofmt).
  • Code must be documented adhering to the official Go commentary guidelines.
  • Code must adhere to our coding style.
  • Pull requests need to be based on and opened against the master branch.

Donations

If this framework helped you in any way, or you would like to support the developers working on it, please donate Bitcoin to:

bc1qk0jareu4jytc0cfrhr5wgshsq8282awpavfahc