Commit Graph

8 Commits

Author SHA1 Message Date
Scott
91d699be9d maps: expansion of Key concept (#1349)
* moves everything to use single map keys, also breaks

* full rollout

* tests

* fix a little bug

* minor test fixups

* Fix Key use

* rm 🔑 from 🔑 struct name
2023-10-04 10:19:41 +11:00
Scott
7f0faf7850 futures: add GetFuturesContractDetails wrapper function (#1274)
* all in a days work

* cleanup

* cleanup for real, also stop it binance.json

* minor coverage

* adds gateio to the slurry

* cleanup of types

* verbose verbose verbose verbose verbose verbose

* fixes huobi parsing issue

* fix bybit contract identification

* cleanup

* merge fixes

* addresses many big problems raised by SHAZ

* tracking errors and fixes

* funding rate if avail, fixes currency formatting

* Addresses nits and sneaks in extra fixes

* lint

* minor fixes after rebase

* better contract splitter for currencies like T-USDT

* forgot to add the exchange name like a fool

* merge fixes x1

* kucoin, direction, contract size

* rn direction, fix kucoin time

* WHOOPS

* Update exchanges/kucoin/kucoin_wrapper.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* misdirection

---------

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
2023-10-03 16:04:45 +11:00
Ryan O'Hara-Reid
db8735ec99 log: Add structured logging (#1171)
* basic implementation

* log: deprecate duplicate function, add tests and refine calls.

* linter: fixes

* linter: update struct

* linter and new type

* log tests: update to not lint issue

* linter: stop complaining please

* glorious: nits

* log: rm comment code

* glorious: nits

* glorious: nits

* glorious: nits

* glorious: nits missed

---------

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
2023-05-10 17:52:53 +10:00
Scott
017cdf1384 Backtester: Live trading upgrades (#1023)
* Modifications for a smoother live run

* Fixes data appending

* Successfully allows multi-currency live trading. Adds multiple currencies to live DCA strategy

* Attempting to get cash and carry working

* Poor attempts at sorting out data and appending it properly with USD in mind

* =designs new live data handler

* Updates cash and carry strat to work

* adds test coverage. begins closeallpositions function

* Updates cash and carry to work live

* New kline.Event type. Cancels orders on close. Rn types

* =Fixes USD funding issue

* =fixes tests

* fixes tests AGAIN

* adds coverage to close all orders

* crummy tests, should override

* more tests

* more tests

* more coverage

* removes scourge of currency.Pair maps. More tests

* missed currency stuff

* Fixes USD data issue & collateral issue. Needs to close ALL orders

* Now triggers updates on the very first data entry

* All my problems are solved now????

* fixes tests, extends coverage

* there is some really funky candle stuff going on

* my brain is melting

* better shutdown management, fixes freezing bug

* fixes data duplication issues, adds retries to requests

* reduces logging, adds verbose options

* expands coverage over all new functionality

* fixes fun bug from curr == curr to curr.Equal(curr)

* fixes setup issues and tests

* starts adding external wallet amounts for funding

* more setup for assets

* setup live fund calcs and placing orders

* successfully performs automated cash and carry

* merge fixes

* funding properly set at all times

* fixes some bugs, need to address currencystatistics still

* adds 'appeneded' trait, attempts to fix some stats

* fixes stat bugs, adds cool new fetchfees feature

* fixes terrible processing bugs

* tightens realorder stats, sadly loses some live stats

* this actually sets everything correctly for bothcd ..cd ..cd ..cd ..cd ..!

* fix tests

* coverage

* beautiful new test coverage

* docs

* adds new fee getter delayer

* commits from the correct directory

* Lint

* adds verbose to fund manager

* Fix bug in t2b2 strat. Update dca live config. Docs

* go mod tidy

* update buf

* buf + test improvement

* Post merge fixes

* fixes surprise offset bug

* fix sizing restrictions for cash and carry

* fix server lints

* merge fixes

* test fixesss

* lintle fixles

* slowloris

* rn run to task, bug fixes, close all on close

* rpc lint and fixes

* bugfix: order manager not processing orders properly

* somewhat addresses nits

* absolutely broken end of day commit

* absolutely massive knockon effects from nits

* massive knockon effects continue

* fixes things

* address remaining nits

* jk now fixes things

* addresses the easier nits

* more nit fixers

* more niterinos addressederinos

* refactors holdings and does some nits

* so buf

* addresses some nits, fixes holdings bugs

* cleanup

* attempts to fix alert chans to prevent many chans waiting?

* terrible code, will revert

* to be reviewed in detail tomorrow

* Fixes up channel system

* smashes those nits

* fixes extra candles, fixes collateral bug, tests

* fixes data races, introduces reflection

* more checks n tests

* Fixes cash and carry issues. Fixes more cool bugs

* fixes ~typer~ typo

* replace spot strats from ftx to binance

* fixes all the tests I just destroyed

* removes example path, rm verbose

* 1) what 2) removes FTX references from the Backtester

* renamed, non-working strategies

* Removes FTX references almost as fast as sbf removes funds

* regen docs, add contrib names,sort contrib names

* fixes merge renamings

* Addresses nits. Fixes setting API credentials. Fixes Binance limit retrieval

* Fixes live order bugs with real orders and without

* Apply suggestions from code review

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update backtester/engine/live.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update backtester/engine/live.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update backtester/config/strategyconfigbuilder/main.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* updates docs

* even better docs

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
2023-01-05 13:03:17 +11:00
Scott
2232478415 backtester: run manager (#1040)
* begins defining run management options

* fleshes out concept

* completes fund manager and RPC commands

* coverage and improvements

* adds coverage, and bad log concept

* simplifies output at expense of races

* removes run logging for now. tightens races. adds cov

* Lints thine splints

* Fixes stopping and clearing bugs

* some niteroos

* fix races
2022-09-30 14:15:10 +10:00
Scott
1461cba363 backtester: standalone application (#988)
* Ramshackle early leads to GRPC backtester

* Adds GRPC server, default config generation

* Partial support for GRPC backtester config

* Update to use Buf, merge fixes

* Full config for GRPC

* Adds new commands, causes big panic

* Fixes panics

* Setup for the future

* Docs update

* test

* grpc tests

* Fix merge issues. Lint and test

* minor fixes after rebase

* Docs, formatting and main fixes

* Change buf owner

* shazNits

* test-123

* rpc fixes

* string fixes

* Removes --singlerun flag and just relies on --singlerunstrategypath

* fixes test

* initial post merge compatability fixes

* this actually all seems to work? unexpected

* adds pluginpath to config

* rm unused func. add gitignore

* rm unused func. add gitignore

* lintle

* tITLE cASE lOG fIX,rm auth package, gitignore, tmpdir fix

* buf updates + gen. go mod tidy

* x2

* Update default port, update error text
2022-09-08 16:22:30 +10:00
Scott
46cadd6f15 FTX: Funding rates, payments & stats + order manager tracking (#976)
* Adds basic PoC for calculating/retrieving position data

* A very unfortunate day of miscalculations

* Adds position summary and funding rate details to RPC

* Offline funding rate calculations

* More helpers, more stats, refining data, automated retrieval

* Adds new rpc server commands and attempts some organisation

* lower string, lower stress

* Adds ordermanager config. Fleshes outcli. Tracks positions automatically

* Adds new separation for funding payments/rates

* Combines funding rates and payments

* Fun test coverage

* ALL THE TESTS... I hope

* Fixes

* polishes ftx tests. improves perp check. Loops rates

* Final touches before nit attax

* buff 💪

* Stops NotYetImplemented spam with one simple trick!

* Some lovely little niteroos

* linteroo

* Clarifies a couple of errors to help narrow likely end user problems

* Fixes asset type bug, fixes closed position order return, fixes unset status bug

* Fixes order manager handling when no rates are available yet

* Continues on no funding rates instead. Removes err

* Don't show predicted rate if the time is zero

* Addresses scenario with no funding rate payments

* Bug fixes and commentary before updating maps to use *currency.Item

* Adds a pair key type

* Polishes pKey, fixes map order bug

* key is not a property in the event someone changes the base/quote

* Adds improvements to order processing...Breaks it all

* Shakes up the design of things by removing a function

* Fixes issues with order manager positions. Limits update range

* Fixes build issues. Identification of bad tests.

* Merges and fixes features from master and this branch

* buff linter 💪

* re-gen

* proto regen

* Addresses some nits. But not all of them.

* Fixes issue where funding rates weren't returned 🎉

* completes transition futures tracking to map[*currency.Item]map[*currency.Item]

* who did that? not me

* removes redundant check on account of being redundant and unnecessary

* so buf

* addresses nits: duplications, startTime, loops, go tidy, typos

* fixes minor mistakes

* fixes 🍣 🐻 changes to int64
2022-08-23 12:16:50 +10:00
Scott
f929b4d51e backtester: Futures handling & FTX Cash and Carry example strategy (#930)
* implements futures functions and GRPC functions on new branch

* lint and test fixes

* Fix uneven split pnl. Adds collateral weight test. docs. New clear func

* Test protection if someone has zero collateral

* Uses string instead of double for accuracy

* Fixes old code panic

* context, match, docs

* Addresses Shazniterinos, var names, expanded tests

* Returns subaccount name, provides USD values when offlinecalc

* Fixes oopsie

* Fixes cool bug which allowed made up subaccount results

* Subaccount override on FTX, subaccount results for collateral

* Strenghten collateral account info checks. Improve FTX test

* English is my first language

* Fixes oopsies

* Adds some conceptual futures order details to track PNL

* Initial design of future order processing in the backtester

* Introduces futures concept for collateral and spot/futures config diffs

* Fixes most tests

* Simple designs for collateral funding pair concept

* Expands interface use so much it hurts

* Implements more collateral interfaces

* Adds liquidation, adds strategy, struggles with Binance

* Attempts at getting FTX to work

* Adds calculatePNL as a wrapper function and adds an `IsFutures` asset check

* Successfully loads backtester with collateral currency

* Fails to really get much going for supporting futures

* Merges master changes

* Fleshes out how FTX processes collateral

* Further FTX collateral workings

* hooks up more ftx collateral and pnl calculations

* more funcs to flesh out handling

* Adds more links, just can't fit the pieces together :(

* Greatly expands futures order processing

* Fleshes out position tracker to also handle asset and exchange +testing

* RM linkedOrderID. rn positioncontroller, unexport

* Successfully tracks futures order positions

* Fails to calculate PNL

* Calculates pnl from orders accurately with exception to flipping orders

* Calculates PNL from orders

* Adds another controller layer to make it ez from orderstore

* Backtester now compiles. Adds test coverage

* labels things add scaling collateral test

* Calculates pnl in line with fees

* Mostly accurate PNL, with exception to appending with diff prices

* Adds locks, adds rpc function

* grpc implementations

* Gracefully handles rpc function

* beautiful tests!

* rejiggles tests to polish

* Finishes FTX testing, adds comments

* Exposes collateral calculations to rpc

* Adds commands and testing for rpcserver.go functions

* Increase testing and fix up backtester code

* Returns cool changes to original branch

* end of day fixes

* Fixing some tests

* Fixing tests 🎉

* Fixes all the tests

* Splits the backtester setup and running into different files

* Merge, minor fixes

* Messing with some strategy updates

* Failed understanding at collateral usage

* Begins the creation of cash and carry strategy

* Adds underlying pair, adds filldependentevent for futures

* Completes fill prerequsite event implementation. Can't short though

* Some bug fixes

* investigating funds

* CAN NOW CREATE A SHORT ORDER

* Minor change in short size

* Fixes for unrealised PNL & collateral rendering

* Fixes lint and tests

* Adds some verbosity

* Updates to pnl calc

* Tracks pnl for short orders, minor update to strategy

* Close and open event based on conditions

* Adds pnl data for currency statistics

* Working through PNL calculation automatically. Now panics

* Adds tracking, is blocked from design

* Work to flesh out closing a position

* vain attempts at tracking zeroing out bugs

* woww, super fun new subloggers 🎉

* Begins attempt at automatically handling contracts and collateral based on direction

* Merge master + fixes

* Investigating issues with pnl and holdings

* Minor pnl fixes

* Fixes future position sizing, needs contract sizing

* Can render pnl results, focussing on funding statistics

* tracking candles for futures, but why not btc

* Improves funding statistics

* Colours and stats

* Fixes collateral and snapshot bugs

* Completes test

* Fixes totals bug

* Fix double buy, expand stats, fixes usd totals, introduce interface

* Begins report formatting and calculations

* Appends pnl to receiving curr. Fixes map[time]. accurate USD

* Improves report output rendering

* PNL stats in report. New tests for futures

* Fixes existing tests before adding new coverage

* Test coverage

* Completes portfolio coverage

* Increase coverage exchange, portfolio. fix size bug. NEW CHART

* WHAT IS GOING ON WITH PNL

* Fixes PNL calculation. Adds ability to skip om futures tracking

* minor commit before merge

* Adds basic liquidation to backtester

* Changes liquidation to order based

* Liquidationnnnnn

* Further fleshes out liquidations

* Completes liquidations in a honorable manner. Adds AppendReasonf

* Beginnings of spot futures gap chart. Needs to link currencies to render difference

* Removes fake liquidation. Adds cool new chart

* Fixes somet tests,allows for zero fee value v nil distinction,New tests

* Some annoying test fixes that took too long

* portfolio coverage

* holding coverage, privatisation funding

* Testwork

* boring tests

* engine coverage

* More backtesting coverage

* Funding, strategy, report test coverage

* Completes coverage of report package

* Documentation, fixes some assumptions on asset errors

* Changes before master merge

* Lint and Tests

* defaults to non-coloured rendering

* Chart rendering

* Fixes surprise non-local-lints

* Niterinos to the extremeos

* Fixes merge problems

* The linter splintered across the glinting plinths

* Many nits addressed. Now sells spot position on final candle

* Adds forgotten coverage

* Adds ability to size futures contracts to match spot positions.

* fixes order sell sizing

* Adds tests to sizing. Fixes charting issue

* clint splintered the linters with flint

* Improves stats, stat rendering

* minifix

* Fixes tests and fee bug

* Merge fixeroos

* Microfixes

* Updates orderPNL on first Correctly utilises fees. Adds committed funds

* New base funcs. New order summary

* Fun test updates

* Fix logo colouring

* Fixes niteroonies

* Fix report

* BAD COMMIT

* Fixes funding issues.Updates default fee rates.Combines cashcarry case

* doc regen

* Now returns err

* Fixes sizing bug issue introduced in PR

* Fixes fun fee/total US value bug

* Fix chart bug. Show log charts with disclaimer

* sellside fee

* fixes fee and slippage view

* Fixed slippage price issue

* Fixes calculation and removes rendering

* Fixes stats and some rendering

* Merge fix

* Fixes merge issues

* go mod tidy, lint updates

* New linter attempt

* Version bump in appveyor and makefile

* Regex filename, config fixes, template h2 fixes

* Removes bad stats.

* neatens config builder. Moves filename generator

* Fixes issue where linter wants to fix my spelling

* Fixes pointers and starts
2022-06-30 15:43:41 +10:00