Files
gocryptotrader/backtester/config
Scott f929b4d51e backtester: Futures handling & FTX Cash and Carry example strategy (#930)
* implements futures functions and GRPC functions on new branch

* lint and test fixes

* Fix uneven split pnl. Adds collateral weight test. docs. New clear func

* Test protection if someone has zero collateral

* Uses string instead of double for accuracy

* Fixes old code panic

* context, match, docs

* Addresses Shazniterinos, var names, expanded tests

* Returns subaccount name, provides USD values when offlinecalc

* Fixes oopsie

* Fixes cool bug which allowed made up subaccount results

* Subaccount override on FTX, subaccount results for collateral

* Strenghten collateral account info checks. Improve FTX test

* English is my first language

* Fixes oopsies

* Adds some conceptual futures order details to track PNL

* Initial design of future order processing in the backtester

* Introduces futures concept for collateral and spot/futures config diffs

* Fixes most tests

* Simple designs for collateral funding pair concept

* Expands interface use so much it hurts

* Implements more collateral interfaces

* Adds liquidation, adds strategy, struggles with Binance

* Attempts at getting FTX to work

* Adds calculatePNL as a wrapper function and adds an `IsFutures` asset check

* Successfully loads backtester with collateral currency

* Fails to really get much going for supporting futures

* Merges master changes

* Fleshes out how FTX processes collateral

* Further FTX collateral workings

* hooks up more ftx collateral and pnl calculations

* more funcs to flesh out handling

* Adds more links, just can't fit the pieces together :(

* Greatly expands futures order processing

* Fleshes out position tracker to also handle asset and exchange +testing

* RM linkedOrderID. rn positioncontroller, unexport

* Successfully tracks futures order positions

* Fails to calculate PNL

* Calculates pnl from orders accurately with exception to flipping orders

* Calculates PNL from orders

* Adds another controller layer to make it ez from orderstore

* Backtester now compiles. Adds test coverage

* labels things add scaling collateral test

* Calculates pnl in line with fees

* Mostly accurate PNL, with exception to appending with diff prices

* Adds locks, adds rpc function

* grpc implementations

* Gracefully handles rpc function

* beautiful tests!

* rejiggles tests to polish

* Finishes FTX testing, adds comments

* Exposes collateral calculations to rpc

* Adds commands and testing for rpcserver.go functions

* Increase testing and fix up backtester code

* Returns cool changes to original branch

* end of day fixes

* Fixing some tests

* Fixing tests 🎉

* Fixes all the tests

* Splits the backtester setup and running into different files

* Merge, minor fixes

* Messing with some strategy updates

* Failed understanding at collateral usage

* Begins the creation of cash and carry strategy

* Adds underlying pair, adds filldependentevent for futures

* Completes fill prerequsite event implementation. Can't short though

* Some bug fixes

* investigating funds

* CAN NOW CREATE A SHORT ORDER

* Minor change in short size

* Fixes for unrealised PNL & collateral rendering

* Fixes lint and tests

* Adds some verbosity

* Updates to pnl calc

* Tracks pnl for short orders, minor update to strategy

* Close and open event based on conditions

* Adds pnl data for currency statistics

* Working through PNL calculation automatically. Now panics

* Adds tracking, is blocked from design

* Work to flesh out closing a position

* vain attempts at tracking zeroing out bugs

* woww, super fun new subloggers 🎉

* Begins attempt at automatically handling contracts and collateral based on direction

* Merge master + fixes

* Investigating issues with pnl and holdings

* Minor pnl fixes

* Fixes future position sizing, needs contract sizing

* Can render pnl results, focussing on funding statistics

* tracking candles for futures, but why not btc

* Improves funding statistics

* Colours and stats

* Fixes collateral and snapshot bugs

* Completes test

* Fixes totals bug

* Fix double buy, expand stats, fixes usd totals, introduce interface

* Begins report formatting and calculations

* Appends pnl to receiving curr. Fixes map[time]. accurate USD

* Improves report output rendering

* PNL stats in report. New tests for futures

* Fixes existing tests before adding new coverage

* Test coverage

* Completes portfolio coverage

* Increase coverage exchange, portfolio. fix size bug. NEW CHART

* WHAT IS GOING ON WITH PNL

* Fixes PNL calculation. Adds ability to skip om futures tracking

* minor commit before merge

* Adds basic liquidation to backtester

* Changes liquidation to order based

* Liquidationnnnnn

* Further fleshes out liquidations

* Completes liquidations in a honorable manner. Adds AppendReasonf

* Beginnings of spot futures gap chart. Needs to link currencies to render difference

* Removes fake liquidation. Adds cool new chart

* Fixes somet tests,allows for zero fee value v nil distinction,New tests

* Some annoying test fixes that took too long

* portfolio coverage

* holding coverage, privatisation funding

* Testwork

* boring tests

* engine coverage

* More backtesting coverage

* Funding, strategy, report test coverage

* Completes coverage of report package

* Documentation, fixes some assumptions on asset errors

* Changes before master merge

* Lint and Tests

* defaults to non-coloured rendering

* Chart rendering

* Fixes surprise non-local-lints

* Niterinos to the extremeos

* Fixes merge problems

* The linter splintered across the glinting plinths

* Many nits addressed. Now sells spot position on final candle

* Adds forgotten coverage

* Adds ability to size futures contracts to match spot positions.

* fixes order sell sizing

* Adds tests to sizing. Fixes charting issue

* clint splintered the linters with flint

* Improves stats, stat rendering

* minifix

* Fixes tests and fee bug

* Merge fixeroos

* Microfixes

* Updates orderPNL on first Correctly utilises fees. Adds committed funds

* New base funcs. New order summary

* Fun test updates

* Fix logo colouring

* Fixes niteroonies

* Fix report

* BAD COMMIT

* Fixes funding issues.Updates default fee rates.Combines cashcarry case

* doc regen

* Now returns err

* Fixes sizing bug issue introduced in PR

* Fixes fun fee/total US value bug

* Fix chart bug. Show log charts with disclaimer

* sellside fee

* fixes fee and slippage view

* Fixed slippage price issue

* Fixes calculation and removes rendering

* Fixes stats and some rendering

* Merge fix

* Fixes merge issues

* go mod tidy, lint updates

* New linter attempt

* Version bump in appveyor and makefile

* Regex filename, config fixes, template h2 fixes

* Removes bad stats.

* neatens config builder. Moves filename generator

* Fixes issue where linter wants to fix my spelling

* Fixes pointers and starts
2022-06-30 15:43:41 +10:00
..

GoCryptoTrader Backtester: Config package

Build Status Software License GoDoc Coverage Status Go Report Card

This config package is part of the GoCryptoTrader codebase.

This is still in active development

You can track ideas, planned features and what's in progress on this Trello board: https://trello.com/b/ZAhMhpOy/gocryptotrader.

Join our slack to discuss all things related to GoCryptoTrader! GoCryptoTrader Slack

Config package overview

What does the config package do?

The config package contains a set of structs which allow for the customisation of the GoCryptoTrader Backtester when running. The GoCryptoTrader Backtester runs from reading config files (.strat files by default under /examples).

What does Simultaneous Processing mean?

GoCryptoTrader Backtester config files may contain multiple ExchangeSettings which defined exchange, asset and currency pairs to iterate through a period of time.

If there are multiple entries to ExchangeSettings and SimultaneousProcessing is disabled, then each individual exchange, asset and currency pair candle event is evaluated individually and does not know about other exchange, asset and currency pair data events. It is a way to test a singular strategy against multiple assets simultaneously. But it isn't defined as Simultaneous Processing Simultaneous Signal Processing is a setting which allows multiple ExchangeSettings data events for a candle event to be considered simultaneously. This means that you can check if the price of BTC-USDT is 5% greater on Binance than it is on Kraken and choose to make signal a BUY event for Kraken and not Binance.

It allows for complex strategical decisions to be made when you consider the scope of the entire market at a given time, rather than in a vacuum when SimultaneousSignalProcessing is disabled.

How do I customise the GoCryptoTrader Backtester?

See below for a set of tables and fields, expected values and what they can do

Config

Key Description
Nickname A nickname for the specific config. When running multiple variants of the same strategy, use the nickname to help differentiate between runs
Goal A description of what you would hope the outcome to be. When verifying output, you can review and confirm whether the strategy met that goal
CurrencySettings Currency settings is an array of settings for each individual currency you wish to run the strategy against
StrategySettings Select which strategy to run, what custom settings to load and whether the strategy can assess multiple currencies at once to make more in-depth decisions
FundingSettings Defines whether individual funding settings can be used. Defines the funding exchange, asset, currencies at an individual level
PortfolioSettings Contains a list of global rules for the portfolio manager. CurrencySettings contain their own rules on things like how big a position is allowable, the portfolio manager rules are the same, but override any individual currency's settings
StatisticSettings Contains settings that impact statistics calculation. Such as the risk-free rate for the sharpe ratio

Strategy Settings

Key Description Example
Name The strategy to use rsi
UsesSimultaneousProcessing This denotes whether multiple currencies are processed simultaneously with the strategy function OnSimultaneousSignals. Eg If you have multiple CurrencySettings and only wish to purchase BTC-USDT when XRP-DOGE is 1337, this setting is useful as you can analyse both signal events to output a purchase call for BTC true
CustomSettings This is a map where you can enter custom settings for a strategy. The RSI strategy allows for customisation of the upper, lower and length variables to allow you to change them from 70, 30 and 14 respectively to 69, 36, 12 "custom-settings": { "rsi-high": 70, "rsi-low": 30, "rsi-period": 14 }
DisableUSDTracking If false, will track all currencies used in your strategy against USD equivalent candles. For example, if you are running a strategy for BTC/XRP, then the GoCryptoTrader Backtester will also retreive candles data for BTC/USD and XRP/USD to then track strategy performance against a single currency. This also tracks against USDT and other USD tracked stablecoins, so one exchange supporting USDT and another BUSD will still allow unified strategy performance analysis. If disabled, will not track against USD, this can be especially helpful when running strategies under live, database and CSV based data false

Funding Config Settings

Key Description Example
UseExchangeLevelFunding Allows shared funding at an exchange asset level. You can set funding for USDT and all pairs that feature USDT will have access to those funds when making orders. See this for more information false
ExchangeLevelFunding An array of exchange level funding settings. See below, or this for more information []
Funding Item Config Settings
Key Description Example
ExchangeName The exchange to set funds. See here for a list of supported exchanges Binance
Asset The asset type to set funds. Typically, this will be spot, however, see this package for the various asset types GoCryptoTrader supports spot
Currency The currency to set funds BTC
InitialFunds The initial funding for the currency 1337
TransferFee If your strategy utilises transferring of funds via the Funding Manager, this is deducted upon doing so 0.005

Currency Settings

Key Description Example
ExchangeName The exchange to load. See here for a list of supported exchanges Binance
Asset The asset type. Typically, this will be spot, however, see this package for the various asset types GoCryptoTrader supports spot
Base The base of a currency BTC
Quote The quote of a currency USDT
InitialFunds A legacy field, will be temporarily migrated to InitialQuoteFunds if present in your strat config ``
BuySide This struct defines the buying side rules this specific currency setting must abide by such as maximum purchase amount -
SellSide This struct defines the selling side rules this specific currency setting must abide by such as maximum selling amount -
MinimumSlippagePercent Is the lower bounds in a random number generated that make purchases more expensive, or sell events less valuable. If this value is 90, then the most a price can be affected is 10% 90
MaximumSlippagePercent Is the upper bounds in a random number generated that make purchases more expensive, or sell events less valuable. If this value is 99, then the least a price can be affected is 1%. Set both upper and lower to 100 to have no randomness applied to purchase events 100
MakerFee The fee to use when sizing and purchasing currency. If nil, will lookup an exchange's fee details 0.001
TakerFee Unused fee for when an order is placed in the orderbook, rather than taken from the orderbook. If nil, will lookup an exchange's fee details 0.002
MaximumHoldingsRatio When multiple currency settings are used, you may set a maximum holdings ratio to prevent having too large a stake in a single currency 0.5
CanUseExchangeLimits Will lookup exchange rules around purchase sizing eg minimum order increments of 0.0005. Note: Will retrieve up-to-date rules which may not have existed for the data you are using. Best to use this when considering to use this strategy live false
SkipCandleVolumeFitting When placing orders, by default the BackTester will shrink an order's size to fit the candle data's volume so as to not rewrite history. Set this to true to ignore this and to set order size at what the portfolio manager prescribes false
SpotSettings An optional field which contains initial funding data for SPOT currency pairs See SpotSettings table below
FuturesSettings An optional field which contains leverage data for FUTURES currency pairs See FuturesSettings table below
SpotSettings
Key Description Example
InitialBaseFunds The funds that the GoCryptoTraderBacktester has for the base currency. This is only required if the strategy setting UseExchangeLevelFunding is false 2
InitialQuoteFunds The funds that the GoCryptoTraderBacktester has for the quote currency. This is only required if the strategy setting UseExchangeLevelFunding is false 10000
FuturesSettings
Key Description Example
Leverage This struct defines the leverage rules that this specific currency setting must abide by 1

PortfolioSettings

Key Description
Leverage This struct defines the leverage rules that this specific currency setting must abide by
BuySide This struct defines the buying side rules this specific currency setting must abide by such as maximum purchase amount
SellSide This struct defines the selling side rules this specific currency setting must abide by such as maximum selling amount

StatisticsSettings

Key Description Example
RiskFreeRate The risk free rate used in the calculation of sharpe and sortino ratios 0.03

APIData

Key Description Example
DataType Choose whether candle or trade data is used. If trades are used, they will be converted to candles trade
Interval The candle interval in time.Duration format eg set as15000000000 for a value of time.Second * 15 15000000000
StartDate The start date to retrieve data 2021-01-23T11:00:00+11:00
EndDate The end date to retrieve data 2021-01-24T11:00:00+11:00
InclusiveEndDate When enabled, the end date's candle is included in the results. ie 2021-01-24T11:00:00+11:00 with a one hour candle, the final candle will be 2021-01-24T11:00:00+11:00 to 2021-01-24T12:00:00+11:00 false

CSVData

Key Description Example
DataType Choose whether candle or trade data is used. If trades are used, they will be converted to candles candle
Interval The candle interval in time.Duration format eg set as15000000000 for a value of time.Second * 15 15000000000
FullPath The file to load /data/exchangelist.csv

DatabaseData

Key Description Example
DataType Choose whether candle or trade data is used. If trades are used, they will be converted to candles trade
Interval The candle interval in time.Duration format eg set as15000000000 for a value of time.Second * 15 15000000000
StartDate The start date to retrieve data 2021-01-23T11:00:00+11:00
EndDate The end date to retrieve data 2021-01-24T11:00:00+11:00
Config This is the same struct used as your GoCryptoTrader database config. See below tables for breakdown see below
Path If using SQLite, the path to the directory, not the file. Leaving blank will use GoCryptoTrader's default database path ``
InclusiveEndDate When enabled, the end date's candle is included in the results. ie 2021-01-24T11:00:00+11:00 with a one hour candle, the final candle will be 2021-01-24T11:00:00+11:00 to 2021-01-24T12:00:00+11:00 false
database
Config Description Example
enabled Enabled or disables the database connection subsystem true
verbose Displays more information to the logger which can be helpful for debugging false
driver The SQL driver to use. Can be postgres or sqlite sqlite
connectionDetails See below
connectionDetails
Config Description Example
host The host address of the database localhost
port The port used to connect to the database 5432
username An optional username to connect to the database username
password An optional password to connect to the database password
database The name of the database database.db
sslmode The connection type of the database for Postgres databases only disable

LiveData

Key Description Example
DataType Choose whether candle or trade data is used. If trades are used, they will be converted to candles candle
Interval The candle interval in time.Duration format eg set as15000000000 for a value of time.Second * 15 15000000000
APIKeyOverride Will set the GoCryptoTrader exchange to use the following API Key 1234
APISecretOverride Will set the GoCryptoTrader exchange to use the following API Secret 5678
APIClientIDOverride Will set the GoCryptoTrader exchange to use the following API Client ID 9012
API2FAOverride Will set the GoCryptoTrader exchange to use the following 2FA seed hello-moto
APISubaccountOverride Will set the GoCryptoTrader exchange to use the following subaccount on supported exchanges subzero
RealOrders Whether to place real orders. You really should never consider using this. Ever ever true
Leverage Settings
Key Description Example
CanUseLeverage Allows the use of leverage false
MaximumOrdersWithLeverageRatio If the ratio of leveraged orders for a currency exceeds this, the order cannot be placed 0.5
MaximumLeverageRate Orders cannot be placed with leverage over this amount 100
Buy/Sell Settings
Key Description Example
MinimumSize If the order's quantity is below this, the order cannot be placed 0.1
MaximumSize If the order's quantity is over this amount, it cannot be placed and will be reduced to the maximum amount 10
MaximumTotal If the order's price * amount exceeds this number, the order cannot be placed and will be reduced to this figure 1337

Please click GoDocs chevron above to view current GoDoc information for this package

Contribution

Please feel free to submit any pull requests or suggest any desired features to be added.

When submitting a PR, please abide by our coding guidelines:

  • Code must adhere to the official Go formatting guidelines (i.e. uses gofmt).
  • Code must be documented adhering to the official Go commentary guidelines.
  • Code must adhere to our coding style.
  • Pull requests need to be based on and opened against the master branch.

Donations

If this framework helped you in any way, or you would like to support the developers working on it, please donate Bitcoin to:

bc1qk0jareu4jytc0cfrhr5wgshsq8282awpavfahc