* add backtester support
* Prevent live data custom candles, prevent nanosecond candles
* test coverage
* a more interesting rsi strategy result
* actual custom candle and proper strat date
* add test to old funk
* typos 🌞🌞
* this was definitely worth failing linting for
* Adds stricter processing and adapts to it
* now compat with partial and absent candles
* test fixes, zb fixes
* fix more introduced bugeroos
* fix more introduced bugeroosx2
* linting for one space is so annoying
* addresseroos niteroos
* Update backtester/engine/setup.go
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
* kline: Add builder and testing
* Ideas
* kline: deploy builder functionality across GCT
* exchanges: implement across gct
* exchanges: Add tests and fix implementations before kline package testing and veri.
* kline: Add tests and start to fix ConvertToNewInterval
* kline: fix ConvertToNewInterval add tests
* kline: complete overarching tests now on to exchanges
* kline: finish exchange tests and implement limits
* exchanges: more fixes
* linter: fix
* engine: fix tests
* kraken: fix recent trades and other fixes
* zb: fix tests
* bithumb: fix empty insertion
* kline: refactor/optimize CreateKline function
* kline: remove the mooos!
* kline: prealloc CalculateCandleDateRanges
* linter: fix
* exchanges: prealloc extended
* fix whoopsie
* reverse fix because this is a whoopsie
* okx: fix risidual issues
* linter: fix
* kline: initial nits from @gloriouscode
* kline: rename builder -> request and cascade change
* linter: fix + test
* kline: update forced alignment on start and end times when CreateKlineRequest is called.
* nits: more more more
* NITS: Addressed
* tests: fix race issue
* Update exchanges/kline/request.go
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
* kline: add method AddPadding() to automatically fill in holes in kline.Request functionality and reject if missing data when converting
* kline: Add params start and end to addPadding() to insert blanks in between block
* kline: remove test comment code as it's not needed anymore
* kline: fix lint and test
* kline: sort slice without extra bool check every iteration
* okx: fix issues with timeing and candles and such from niterinos & address typo
* Update exchanges/kline/kline.go
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
* glorious: niterinos
* Update exchanges/poloniex/poloniex_wrapper.go
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
* glorious: nits now onto conflicts YAYA!!!
* Update exchanges/exchange_test.go
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
* glorious: nits again
* thrasher: nitters
* thrasher: niterinos - adds partial flag for incomplete recent candles and fetching.
* kline: rm fmtizzle packageizzle
* glorious: nitters
* glorious: more niterinos
* fix last niterinos
Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
* Modifications for a smoother live run
* Fixes data appending
* Successfully allows multi-currency live trading. Adds multiple currencies to live DCA strategy
* Attempting to get cash and carry working
* Poor attempts at sorting out data and appending it properly with USD in mind
* =designs new live data handler
* Updates cash and carry strat to work
* adds test coverage. begins closeallpositions function
* Updates cash and carry to work live
* New kline.Event type. Cancels orders on close. Rn types
* =Fixes USD funding issue
* =fixes tests
* fixes tests AGAIN
* adds coverage to close all orders
* crummy tests, should override
* more tests
* more tests
* more coverage
* removes scourge of currency.Pair maps. More tests
* missed currency stuff
* Fixes USD data issue & collateral issue. Needs to close ALL orders
* Now triggers updates on the very first data entry
* All my problems are solved now????
* fixes tests, extends coverage
* there is some really funky candle stuff going on
* my brain is melting
* better shutdown management, fixes freezing bug
* fixes data duplication issues, adds retries to requests
* reduces logging, adds verbose options
* expands coverage over all new functionality
* fixes fun bug from curr == curr to curr.Equal(curr)
* fixes setup issues and tests
* starts adding external wallet amounts for funding
* more setup for assets
* setup live fund calcs and placing orders
* successfully performs automated cash and carry
* merge fixes
* funding properly set at all times
* fixes some bugs, need to address currencystatistics still
* adds 'appeneded' trait, attempts to fix some stats
* fixes stat bugs, adds cool new fetchfees feature
* fixes terrible processing bugs
* tightens realorder stats, sadly loses some live stats
* this actually sets everything correctly for bothcd ..cd ..cd ..cd ..cd ..!
* fix tests
* coverage
* beautiful new test coverage
* docs
* adds new fee getter delayer
* commits from the correct directory
* Lint
* adds verbose to fund manager
* Fix bug in t2b2 strat. Update dca live config. Docs
* go mod tidy
* update buf
* buf + test improvement
* Post merge fixes
* fixes surprise offset bug
* fix sizing restrictions for cash and carry
* fix server lints
* merge fixes
* test fixesss
* lintle fixles
* slowloris
* rn run to task, bug fixes, close all on close
* rpc lint and fixes
* bugfix: order manager not processing orders properly
* somewhat addresses nits
* absolutely broken end of day commit
* absolutely massive knockon effects from nits
* massive knockon effects continue
* fixes things
* address remaining nits
* jk now fixes things
* addresses the easier nits
* more nit fixers
* more niterinos addressederinos
* refactors holdings and does some nits
* so buf
* addresses some nits, fixes holdings bugs
* cleanup
* attempts to fix alert chans to prevent many chans waiting?
* terrible code, will revert
* to be reviewed in detail tomorrow
* Fixes up channel system
* smashes those nits
* fixes extra candles, fixes collateral bug, tests
* fixes data races, introduces reflection
* more checks n tests
* Fixes cash and carry issues. Fixes more cool bugs
* fixes ~typer~ typo
* replace spot strats from ftx to binance
* fixes all the tests I just destroyed
* removes example path, rm verbose
* 1) what 2) removes FTX references from the Backtester
* renamed, non-working strategies
* Removes FTX references almost as fast as sbf removes funds
* regen docs, add contrib names,sort contrib names
* fixes merge renamings
* Addresses nits. Fixes setting API credentials. Fixes Binance limit retrieval
* Fixes live order bugs with real orders and without
* Apply suggestions from code review
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
* Update backtester/engine/live.go
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
* Update backtester/engine/live.go
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
* Update backtester/config/strategyconfigbuilder/main.go
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
* updates docs
* even better docs
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
* logger: reduce go routine generation
* logger: shift most of processing and prep work to the worker pool, add pool for fields because each log we are pushing the struct to the heap, has better segregation now and includes a buffer in scope instead of relying on a pool
* logger: shift fmt package calls to worker pool
* logger: conform tests to new design
* linter: fix issues
* Update log/logger_test.go
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
* Update log/logger_test.go
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
* UN-GLORIOUS: nits
* logger: Handle config variable
* logger: NITERINOS BY GLORIOUS CODE
* logger: revert
* glorious: nits
* Panic at the disco: fix
* Panic at the disco: fix
* logger: make sure logger closed and job channel emptied on start up error
* fix tests
* logger: reduce globals
* logger: finished reduces globals, reduce workers to one too keep everything in line.
* logger: remove comments
* logger/exhchange: linter issues
* db/test: fix linter
* logger: add tests shift wait before unlock
* logger: consolidate worker code; fix linter issue and make sure we can sustain writing for external testing.
* logger: fix race and warn for conflict in config
* logger: fix name and add to tests
* logger: remove zero value field
* glorious: panic fix and removal of code
* logger: reinstate channels in close
* logger: shift reinstate processing to SetupGlobalLogger
* logger: segregate config.json from internal log.Config
* logger: fix silly mistake that is silly
* engine: Add protection for nil issues and implement new constructor in tests
* logger: Force singular mutex usage throughout package, throw away funcs that are not used outside of this package, unexport a bunch. Fix tests.
* logger: actually set advanced settings
* Update log/loggers.go
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
* Update log/loggers.go
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
* Update log/loggers.go
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
* Update log/loggers.go
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
* Update log/logger_multiwriter.go
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
* glorious: nits
* logger: test issue when not purging temp file and contents
* loggertest: add more protections for the panics
* linter: fix
* glorious: nits
* cleanup
* logger: linter fix
* linter: fix(?) :/
* linter: revert change
* linter: fix
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
* begins defining run management options
* fleshes out concept
* completes fund manager and RPC commands
* coverage and improvements
* adds coverage, and bad log concept
* simplifies output at expense of races
* removes run logging for now. tightens races. adds cov
* Lints thine splints
* Fixes stopping and clearing bugs
* some niteroos
* fix races
* Ramshackle early leads to GRPC backtester
* Adds GRPC server, default config generation
* Partial support for GRPC backtester config
* Update to use Buf, merge fixes
* Full config for GRPC
* Adds new commands, causes big panic
* Fixes panics
* Setup for the future
* Docs update
* test
* grpc tests
* Fix merge issues. Lint and test
* minor fixes after rebase
* Docs, formatting and main fixes
* Change buf owner
* shazNits
* test-123
* rpc fixes
* string fixes
* Removes --singlerun flag and just relies on --singlerunstrategypath
* fixes test
* initial post merge compatability fixes
* this actually all seems to work? unexpected
* adds pluginpath to config
* rm unused func. add gitignore
* rm unused func. add gitignore
* lintle
* tITLE cASE lOG fIX,rm auth package, gitignore, tmpdir fix
* buf updates + gen. go mod tidy
* x2
* Update default port, update error text
* Adds custom strategy
* docs and structure
* docs
* rn
* Documents plugins, adds custom strat config
* mini fixes. Fleshes strategy test
* docgen
* Updates plugins to allow for multiple strategies to be loaded
* docs
* docs regen
* fix doc accuracy
* why did I add the word custom?
* Adds basic PoC for calculating/retrieving position data
* A very unfortunate day of miscalculations
* Adds position summary and funding rate details to RPC
* Offline funding rate calculations
* More helpers, more stats, refining data, automated retrieval
* Adds new rpc server commands and attempts some organisation
* lower string, lower stress
* Adds ordermanager config. Fleshes outcli. Tracks positions automatically
* Adds new separation for funding payments/rates
* Combines funding rates and payments
* Fun test coverage
* ALL THE TESTS... I hope
* Fixes
* polishes ftx tests. improves perp check. Loops rates
* Final touches before nit attax
* buff 💪
* Stops NotYetImplemented spam with one simple trick!
* Some lovely little niteroos
* linteroo
* Clarifies a couple of errors to help narrow likely end user problems
* Fixes asset type bug, fixes closed position order return, fixes unset status bug
* Fixes order manager handling when no rates are available yet
* Continues on no funding rates instead. Removes err
* Don't show predicted rate if the time is zero
* Addresses scenario with no funding rate payments
* Bug fixes and commentary before updating maps to use *currency.Item
* Adds a pair key type
* Polishes pKey, fixes map order bug
* key is not a property in the event someone changes the base/quote
* Adds improvements to order processing...Breaks it all
* Shakes up the design of things by removing a function
* Fixes issues with order manager positions. Limits update range
* Fixes build issues. Identification of bad tests.
* Merges and fixes features from master and this branch
* buff linter 💪
* re-gen
* proto regen
* Addresses some nits. But not all of them.
* Fixes issue where funding rates weren't returned 🎉
* completes transition futures tracking to map[*currency.Item]map[*currency.Item]
* who did that? not me
* removes redundant check on account of being redundant and unnecessary
* so buf
* addresses nits: duplications, startTime, loops, go tidy, typos
* fixes minor mistakes
* fixes 🍣🐻 changes to int64
* implements futures functions and GRPC functions on new branch
* lint and test fixes
* Fix uneven split pnl. Adds collateral weight test. docs. New clear func
* Test protection if someone has zero collateral
* Uses string instead of double for accuracy
* Fixes old code panic
* context, match, docs
* Addresses Shazniterinos, var names, expanded tests
* Returns subaccount name, provides USD values when offlinecalc
* Fixes oopsie
* Fixes cool bug which allowed made up subaccount results
* Subaccount override on FTX, subaccount results for collateral
* Strenghten collateral account info checks. Improve FTX test
* English is my first language
* Fixes oopsies
* Adds some conceptual futures order details to track PNL
* Initial design of future order processing in the backtester
* Introduces futures concept for collateral and spot/futures config diffs
* Fixes most tests
* Simple designs for collateral funding pair concept
* Expands interface use so much it hurts
* Implements more collateral interfaces
* Adds liquidation, adds strategy, struggles with Binance
* Attempts at getting FTX to work
* Adds calculatePNL as a wrapper function and adds an `IsFutures` asset check
* Successfully loads backtester with collateral currency
* Fails to really get much going for supporting futures
* Merges master changes
* Fleshes out how FTX processes collateral
* Further FTX collateral workings
* hooks up more ftx collateral and pnl calculations
* more funcs to flesh out handling
* Adds more links, just can't fit the pieces together :(
* Greatly expands futures order processing
* Fleshes out position tracker to also handle asset and exchange +testing
* RM linkedOrderID. rn positioncontroller, unexport
* Successfully tracks futures order positions
* Fails to calculate PNL
* Calculates pnl from orders accurately with exception to flipping orders
* Calculates PNL from orders
* Adds another controller layer to make it ez from orderstore
* Backtester now compiles. Adds test coverage
* labels things add scaling collateral test
* Calculates pnl in line with fees
* Mostly accurate PNL, with exception to appending with diff prices
* Adds locks, adds rpc function
* grpc implementations
* Gracefully handles rpc function
* beautiful tests!
* rejiggles tests to polish
* Finishes FTX testing, adds comments
* Exposes collateral calculations to rpc
* Adds commands and testing for rpcserver.go functions
* Increase testing and fix up backtester code
* Returns cool changes to original branch
* end of day fixes
* Fixing some tests
* Fixing tests 🎉
* Fixes all the tests
* Splits the backtester setup and running into different files
* Merge, minor fixes
* Messing with some strategy updates
* Failed understanding at collateral usage
* Begins the creation of cash and carry strategy
* Adds underlying pair, adds filldependentevent for futures
* Completes fill prerequsite event implementation. Can't short though
* Some bug fixes
* investigating funds
* CAN NOW CREATE A SHORT ORDER
* Minor change in short size
* Fixes for unrealised PNL & collateral rendering
* Fixes lint and tests
* Adds some verbosity
* Updates to pnl calc
* Tracks pnl for short orders, minor update to strategy
* Close and open event based on conditions
* Adds pnl data for currency statistics
* Working through PNL calculation automatically. Now panics
* Adds tracking, is blocked from design
* Work to flesh out closing a position
* vain attempts at tracking zeroing out bugs
* woww, super fun new subloggers 🎉
* Begins attempt at automatically handling contracts and collateral based on direction
* Merge master + fixes
* Investigating issues with pnl and holdings
* Minor pnl fixes
* Fixes future position sizing, needs contract sizing
* Can render pnl results, focussing on funding statistics
* tracking candles for futures, but why not btc
* Improves funding statistics
* Colours and stats
* Fixes collateral and snapshot bugs
* Completes test
* Fixes totals bug
* Fix double buy, expand stats, fixes usd totals, introduce interface
* Begins report formatting and calculations
* Appends pnl to receiving curr. Fixes map[time]. accurate USD
* Improves report output rendering
* PNL stats in report. New tests for futures
* Fixes existing tests before adding new coverage
* Test coverage
* Completes portfolio coverage
* Increase coverage exchange, portfolio. fix size bug. NEW CHART
* WHAT IS GOING ON WITH PNL
* Fixes PNL calculation. Adds ability to skip om futures tracking
* minor commit before merge
* Adds basic liquidation to backtester
* Changes liquidation to order based
* Liquidationnnnnn
* Further fleshes out liquidations
* Completes liquidations in a honorable manner. Adds AppendReasonf
* Beginnings of spot futures gap chart. Needs to link currencies to render difference
* Removes fake liquidation. Adds cool new chart
* Fixes somet tests,allows for zero fee value v nil distinction,New tests
* Some annoying test fixes that took too long
* portfolio coverage
* holding coverage, privatisation funding
* Testwork
* boring tests
* engine coverage
* More backtesting coverage
* Funding, strategy, report test coverage
* Completes coverage of report package
* Documentation, fixes some assumptions on asset errors
* Changes before master merge
* Lint and Tests
* defaults to non-coloured rendering
* Chart rendering
* Fixes surprise non-local-lints
* Niterinos to the extremeos
* Fixes merge problems
* The linter splintered across the glinting plinths
* Many nits addressed. Now sells spot position on final candle
* Adds forgotten coverage
* Adds ability to size futures contracts to match spot positions.
* fixes order sell sizing
* Adds tests to sizing. Fixes charting issue
* clint splintered the linters with flint
* Improves stats, stat rendering
* minifix
* Fixes tests and fee bug
* Merge fixeroos
* Microfixes
* Updates orderPNL on first Correctly utilises fees. Adds committed funds
* New base funcs. New order summary
* Fun test updates
* Fix logo colouring
* Fixes niteroonies
* Fix report
* BAD COMMIT
* Fixes funding issues.Updates default fee rates.Combines cashcarry case
* doc regen
* Now returns err
* Fixes sizing bug issue introduced in PR
* Fixes fun fee/total US value bug
* Fix chart bug. Show log charts with disclaimer
* sellside fee
* fixes fee and slippage view
* Fixed slippage price issue
* Fixes calculation and removes rendering
* Fixes stats and some rendering
* Merge fix
* Fixes merge issues
* go mod tidy, lint updates
* New linter attempt
* Version bump in appveyor and makefile
* Regex filename, config fixes, template h2 fixes
* Removes bad stats.
* neatens config builder. Moves filename generator
* Fixes issue where linter wants to fix my spelling
* Fixes pointers and starts
* orders: deprecate SubmitResponse return and change to *order.Detail construct detail from order.Submit struct
* orders: add coverage, fix tests
* coinut: rm test for checking
* orders: revert change for return and change field ID to a more explicit name OrderID
* orders: Add method to see if the order was placed
* order: change field name in Cancel type to be more explicit
* orders: standardize field -> OrderID
* backtester: populate change
* orders: add test
* gctscript: fix field name
* linter: fix issues
* linter: more fixes
* linter: forever
* exchanges_tests: populate order.Submit field exchange name
* Update exchanges/order/order_types.go
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
* Update exchanges/order/orders.go
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
* glorious: nits
* glorious: nits
* thrasher: nits
Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
* orders: add method to Limit to retrieve order execution limit snapshots
* currency/btcmarkets: add error and update field name to standard
* linter: fix
* limts: don't return pointer
* limit: Add notes
* glorious: nits
* linter: fix
* limit: reinstate nil check
* exchanges: change field names to be more consistent (@thrasher-) suggestion
Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
* order: slight optimizations
* orders: add benchmarks, small optimize and change order side to uin8 for comparitive optimizations.
* orders: continue to convert string type -> uint
* orders/backtester: interim move type to orders package, later can expand or deprecate.
* orders: handle errors
* orders: optimize filters and remove error returns when its clearly not needed
* orders: remove log call
* backtester: zero value check
* orders/futures: zero value -> flag
* linter: fix
* linter: more fixes
* linters: rides again
* glorious: nits
* common: Add zero value unix check for time values; also addresses glorious nits
* glorious scott: nits
Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
* test: use `T.TempDir` to create temporary test directory
This commit replaces `ioutil.TempDir` with `t.TempDir` in tests. The
directory created by `t.TempDir` is automatically removed when the test
and all its subtests complete.
Prior to this commit, temporary directory created using `ioutil.TempDir`
needs to be removed manually by calling `os.RemoveAll`, which is omitted
in some tests. The error handling boilerplate e.g.
defer func() {
if err := os.RemoveAll(dir); err != nil {
t.Fatal(err)
}
}
is also tedious, but `t.TempDir` handles this for us nicely.
Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
* test: fix TestEncryptTwiceReusesSaltButNewCipher on Windows
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
* test: fix TestCheckConnection on Windows
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
* test: fix TestRPCServer_GetTicker_LastUpdatedNanos on Windows
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
* test: cleanup TestGenerateReport
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
* Bump CI versions
* Specifically set go version as 1.17.x bumps it to 1.18
* Another
* Adjust AppVeyor
* Part 1 of linter issues
* Part 2
* Fix various linters and improvements
* Part 3
* Finishing touches
* Tests and EqualFold
* Fix nitterinos plus bonus requester jobs bump for exchanges with large number of tests
* Fix nitterinos and bump golangci-lint timeout for AppVeyor
* Address nits, ensure all books are returned on err due to syncer regression
* Fix the wiggins
* Fix duplication
* Fix nitterinos
* implements futures functions and GRPC functions on new branch
* lint and test fixes
* Fix uneven split pnl. Adds collateral weight test. docs. New clear func
* Test protection if someone has zero collateral
* Uses string instead of double for accuracy
* Fixes old code panic
* context, match, docs
* Addresses Shazniterinos, var names, expanded tests
* Returns subaccount name, provides USD values when offlinecalc
* Fixes oopsie
* Fixes cool bug which allowed made up subaccount results
* Subaccount override on FTX, subaccount results for collateral
* Strenghten collateral account info checks. Improve FTX test
* English is my first language
* Fixes oopsies
* Fixes for unrealised PNL & collateral rendering
* Fixes lint and tests
* Shaznit fixes
* Secret Shaznit
* Updates account information across wrappers to include more fields
* Updates online collateral calculations. Updates RPC data
* Accurately calculates collateral offline and online minus testing
* Tests and lint chocolate
* Simplifies accountinfo results
* Fixes shaznits
* Adds new func
* Increases collateral accuracy again again again x 200
* Increases accuracy of collateral rendering
* Fixes minor merge/test issues
* Linterino
* Fixes ws test. Improves collateral calculations and rendering
* Make it prettier
* Removes the lock I put on 👀
* Adds `additional_collateral_used` field, renders orig currency
* Fixes unrelated test
* Fix test
* Correctly calculate spot margin borrow collateral
* Address fun lint surprise
See https://github.com/golangci/golangci-lint/issues/741#issuecomment-1017014331
* Strange lint fixing x2
* Continued lint journey
* Nolint the nolint to not lint the lint
* Adds two new fields to response
* More linting issues arising
* fIX3s_c4s|NG
* Fixes command flags' incorrect numbering
* FairMarket = Won
* currency: Add method to derive pair
* currency: Add method to lower entire charset but used the slice copy and returned that. This will change the original, just gotta see if this is an issue, but the slice usually goes out of scope anyway.
* currency/pairs: add filter method
* currency: add function to derive select currencies from currency pairs
* currency/engine: slight adjustments
* currency: fix linter issue also shift burden of proof to caller instead of repair, more performant.
* currency: more linter
* pairs: optimize; reduce allocs/op and B/op
* currency: Add in function 'NewPairsFromString' for testing purposes
* currency: don't suppress error
* currency: stop panic on empty currency code
* currency: Add helper method to match currencies between exchanges
* currency: fixed my bad spelling
* currency: Implement stable coin checks, refactored base code methods, optimized upper and lower case strings for currency code/pairs
* currency: add pairs method to derive stable coins from internal list.
* Currency: Cleanup, fix tests.
* engine/exchanges/currency: fix whoops
* Currency: force govet no copy on Item datatype
* Currency: fix naughty linter issues
* exchange: revert change
* currency/config: fix config upgrade mistake
* currency: re-implement currency sub-systems
* *RetrieveConfigCurrencyPairs removed
*CheckCurrencyConfigValues to only provide warnings, add additional support when, disable when support is lost or not available and set default values.
*Drop Cryptocurrencies from configuration as this is not needed.
*Drop REST Poll delay field as this was unused.
*Update default values for currencyFileUpdateDuration & foreignExchangeUpdateDuration.
*Allow Role to be marshalled for file type.
*Refactor RunUpdater to verify and check config values and set default running foreign exchange provider.
* currency: cleanup
* currency: change match -> equal for comparison which is more of a standard and little easier to find
* currency: address nits
* currency: fix whoops
* currency: Add some more pairs methods
* currency: linter issues
* currency: RM unused field
* currency: rm verbose
* currency: fix word
* currency: gocritic
* currency: fix another whoopsie
* example_config: default to show log system name
* Currency: Force all support packages to use Equal method for comparison as there is a small comparison bug when checking upper and lower casing, this has a more of a pronounced impact between exchanges and client instances of currency generation
* currency: fix log name
* ordermanager: fix potential panic
* currency: small optim.
* engine: display correct bool and force shutdown
* currency: add function and fix regression
* Change ConvertCurrency -> ConvertFiat to be more precise
* ADD GetForeignExchangeRate to get specific exchange rate for fiat pair
* Fix currency display and formatting regression and tied in with config.Currency fields
* engine: fix tests
* currency: return the amount when no conversion needs to take place
* currency: reduce method name
* currency: Address nits glorious nits
* currency: fix linter
* currency: addr nits
* currency: check underlying role in test
* gct: change to EMPTYCODE and EMPTYPAIR across codebase
* currency: fix nits
* currency: this fixes test race but this issue has not been resolved. Please see: https://trello.com/c/54eizOIo/143-currency-package-upgrades
* currency: Add temp dir for testing
* Update engine/engine.go
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
* documentation: update and regen
* currency: Address niterinos
* currency: Add test case for config upgrade when falling over to exchange rate host as default from exchangeRates provider
* currency: addr nits
* currency: fix whoops
Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
* log: fix bugs expand coverage and optimise
* log: fix linter issues
* log: fix linter issue and pack methods in same file
* log: drop defer
* logger: move global check inside getfields and remove unused test function
* logger: Increase note thanks @gloriouscode
* logger: wrap error with writer type
* logger: change variable name
* logger: change variable names and remove validsublogger func as it doesn't add functionality over a standard map call
* logs: error when unsupported output is applied on setup calls
* logs: add glorious suggestion
* logger: add protection to reduce olympic gold medal races
* logger: fix linter issues
* log: glorious niterinos
* Fixes zero data issues on backtester & fixes drawdown calcs for zeros
* Fixes valuation of holdings + when simultaneous processing disabled
* Fixes an issue for when there is a lot of missing data
* Initial concept for creating price tracking pairs
* Completes coverage, even with a slow test
* I dont know what point to hook this stuff up
* Bit of a broken way of handling tracking pairs
* Correctly calculates USD rates against all currencies
* Removes dependency on GCT config
* Failed currency statistics redesign
* initial Update chart to use highcharts
* Minor changes to stats
* Creats funding stats to handle the stat calculations. Needs more work
* tracks USD snapshots and BREAKS THINGS FURTHER
* Fixed!
* Adds ratio calculations and such, but its WRONG. do it at totals level dummy
* End of day basic lint
* Remaining lints
* USD totals statistics
* Minor panic fixes
* Printing of funding stats, but its bad
* Properly calculates overall benchmark, moves funding stat output
* Adds some template charge, removes duplicate fields
* New charts!
* Darkcharts. funding protection when disabled
* Now works with usd tracking/funding disabled!
* Attempting to only show working stats based on settings.
* Spruces up the goose/reporting
* Completes report HTML rendering
* lint and test fixes
* funding statistics testing
* slightly more test coverage
* Test coverage
* Initial documentation
* Fixes tests
* Database testing and rendering improvements and breakages
* report and cmd rendering, linting. fix comma output. rm gct cfg
* PR mode 🎉 Path field, config builder support,testing,linting,docs
* minor calculation improvement
* Secret lint that did not show up locally
* Disable USD tracking for example configs
* ShazNitNoScope
* Forgotten errors
* ""
* literally Logarithmically logically renders the date 👀
* Fixes typos, fixes parallel test, fixes chart gui and exporting
* golangci-lint/CI: Bump versions
Fix remaining linter issues
* Specifically set AppVeyor version
* Fix the infamous typos 👀
* Add go env cmd to AppVeyor
* Add go version cmd to AppVeyor
* Specify AppVeyor image, adjust linters
* Update go get to go install due to deprecation
* Bump golangci-lint timeout time for AppVeyor
* Change NW contract to NQ
* Address nitters
* GetRandomPair -> Pair{}
* Address nits
* Address time nitterinos plus additional tweaks
* More time inception upgrades!
* Bending time and space
* Better designed backtester funding concept
* Fleshes out funding concepts further to allow two funding types
* Adds types, finishes adding to portfolio and adds to exchange
* Fixes a bug to reveal another
* Fixes issues with purchasing
* A partial conversion to using decimal.decimal for the backtester
* Further decimal rollout. Can compile and output report
* More cleanup
* Fix rendering and initial funds issue.
* Adds new concept for trading using the exchange level funding to see what happens
* Fixes a bug in funding not being found
* New strat config to test RSI and discover issues
* Can run with pairs that contain 0 funding
* Finally fixes the arrangement to share funds
* Adds testing and funding transfer
* end of day
* More comments, more tests!
* Improves item comparisons and completes testing
* Initial attempt at new strategy which utilisies shared funding and transfers
* end of day broken
* Chronological output. Fixes output bug where multi currency.
* End of day commit
* Fixes bug where events were being overwritten in a simultaneous context
* Begins transitioning from portfolio holdings to funding holdings. Am I doing the right thing
* End of day run around
* Likely fix for holding calculations
* Improvement to template. Improvement to holdings
* DARK MODE. Report upgrades. Even handling with funds. Fix output
* Output funding to cmd
* Add new trasnferred funds "side"
* Fixing test run 1
* Test updates
* Test updating
* More test fixing
* Fixes portfolio tests
* More test fixes
* Fixes remaining tests and lints
* Fixes currencystatistics tests. Adds decimal math implementations
* Fixes hilarious bug where there could only be on holding
* Adds funding support for config. Minor fixes
* Adds documentation
* Finishes config builder support for funding
* Logs inexact conversions, updates tests. adds config validation
* The quest to understand a new funding bug begins. New strategy
* Fixes bug where wrong funding was retrieved. Expands t2b2 strat
* End of the day commit. Gotta revert the nulldecimal stuff
* Fixes tests, adds extra funding transfer feature
* Fixes initial total values, tries to add a grand total value
* Rebase fixes, documentation updates, tests for strategy
* Swaps the err statement for tests. Regenerates tests. Math warnings
* Attempts to solve Live data problems. Fixes volume
* Fixes live data missing
* can trade at any interval. skip volume sizing. volume colours.
* config regen. display fixes
* test fixes, lint fixes
* Anti-funky errors
* docs
* Rmbad
* docs
* docs update
* Simplifies err handling. Updates readmes. Data type checks
* docs. new field initial-base-funds. comment errs. config test coverage
* minMaxing
* testfix
* Fixes fee calculation, re-bans minMax being equal
* Crazy concepts to attempt to solve totals. Addresses nits
* Adds in totals calculation for exchange level funding.Uses external API
In future, this will be replaced by proper pricing supplied by the same
exchange that is requested. This is an unknown price
* rm dollar signs in cmd and report. rm bad error. fix chart decimal. padding
* re-run docs post merge
* Fixes oopsie for fee parsing
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
* gct: phase one context awareness pass
* exchanges: context propagation pass
* common/requester: force context requirement
* gctcli/exchanges: linter fix
* rpcserver: fix test using dummy rpc server
* backtester: fix comments
* grpc: add correct cancel and timeout for commands
* rpcserver_test: add comment on dummy server
* common: deprecated SendHTTPGetRequest
* linter: fix
* linter: turn on no context check
* apichecker: fix context linter issue
* binance: use param context
* common: remove checks as this gets executed before main
* common: change mutex to RW as clients can be used by multiple go routines.
* common: remove init and JIT default client. Unexport global variables and add protection.
* common: Add comments
* bithumb: after dinner mints fix
* Remove old concept. Introduce new job types and candle scaling
* Adds extra processing, commands
* new concept for queued jobs. Jobs can pause. New commands to manage status
* =End of day commit designing tables and implementing prerequisites further.
* Adds postgres data history relations
* Fixes table design for sqlite. Fixes all issues from merge
* Fixes craziness of database design. Adds some functions to get related jobs
* Fixes errors
* Updates some documentation, manages prerequisite jobs a little better, adds rpc funcs
* Fixes database design and adjust repo functions
* Tests database relationship
* Test coverage of new job functions
* Finishes coverage of new functions
* Commands and RPC coverage
* New database modifications for new job types
* Adds db support of new columns. Adds conversion validation. lint
* command blurb changes
* Allows websocket test to pass consistently
* Fixes merge issue preventing datahistorymanager from starting via config
* Minor fixes for different job type processing
* Fixes rangeholder issue, fixes validation, does not address jobs not starting or wrong status
* Fixes database tests, but at what cost. Fixes dhm tests
* Fixes dhj completion issue. Adds prerequisite by nickname
* Fixes validation processing. Adds db tests and validation
* Fixes validation job processing range
* Fixes trade sql. Reduces defaults. Validation processing and errors
* Updates cli job commands. adds validation decimal. fix job validation
* Expands run job handling and tests
* Validation work
* Fixes validation processing
* candle relations. new job type. updating database design
* Adds secondary exchange support. Sets stage for candle override
* Re adds accidentally deleted relationship
* Updates loading and saving candles to have relationship data when relevant
* Now validates and replaces candle data appropriately
* Fixes getting and setting datahistory data. Neatens DHM
* Test coverage
* Updates proto for new db types. New test coverage. Secondary exchange work
* Investigation into never-ending validation jobs. Now that intervals are ruled out, now need to complete the job....
* Fixes issues with validation job completion. Fixes validation volume issue for secondary exchange
* Adds candle warning support to the backtester
* Fixes warnings
* lint and begin docs
* Documentation updates. Final testing changes
* Minor fixes
* docs, prerequisite checks, more testing
* Fixes binance trade test. Rename err
* Documentation fixes. Figure fixes
* documentation update
* Fixes remote PSQL tests
* Fix binance mock test
* Remove unnecessary JSON
* regen proto
* Some minor nit fixes
* Var usage, query sorting, log improving, sql mirroring
* Extra coverage
* Experimental removal of m.jobs and mutex. Fix messaging
* Fixes error
* Lint fixes, command description improvements. More isRunning gates
* description improvements
* Lint
* BUFF regenerate
* Rough concept to fix insertions taking up long periods of time
* New calculation for trade data. Adds batch saving
This also adds an experimental request feature to shut down lingering requests. However, its uncertain whether or not this is having any impact. Initially thought it was the trades that was taking time and not SQL. Will investigate further
* Removes experimental requester. Adds documentation. Fixes typo
* rm unused error
* re-adds more forgotten contributors
* Now with proper commit count
* orderbook: export orderbook nodes for external strategy inspection
* orderbook: Add in methods for locking and unlocking multiple books at the same time e.g. book1.LockWith(book2); defer book1.UnlockWith(book2)
* include waiting functionality for depth change alert
* backtester: add word.
* log: include logger changes to impl with downstream integration
* engine: reduce params for loading exchange
* assort: rm verbose in tests, change wording in ob, expose sync.waitgroup for ext. sync options
* ticker: reduce map look ups and contention when using RW mutex when there are over 80% writes adds find last function to get the latest rate
* engine/syncmanager: add in waitgroup for step over for external package calls
* cleaup
* engine: linter fix
* currency/fx: include all references to fiat currencies to default
* orderbook: Add in fields to Unsafe type for strategies to detect potential out of sync book operations
* syncmanager: changed config variable to display correct time
* ordermanager: Add time when none provided
* currency/manager: update getasset param to get enabled assets for minor optimizations
* ftx: use get all wallet balances for a better accounts breakdown
* orderbook: unlock in reverse order
* bithumb: fixes bug on market buy and sell orders
* bithumb: fix bug for nonce is also time window sensitive
* bithumb: get orders add required parameter
* bithumb: Add asset type to account struct
* currency: improve log output when checking currency and it fails
* bithumb: Add error return on incomplete pair
* ticker:unexport all service related methods
* ticker/currency: fixes
* orderbook: fix comment
* engine: revert variable name in LoadExchange method
* sync_manager: fix panic when enabling disabling manager
* engine: fix naming convention of exported function and comments
* engine: update comment
* orderbook: fix comment for unsafe type
* Speeds up tests
* Reduces time.Sleeps, lowers CreateTestBot complexity. Breaks things
* Removal of unecessary config reads. Parallel tests. Lower times
* Speeds up recent trades results
* mini update
* zoooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooom
* Removes the dupes
* Lint
* post cherrypick
* Fix rare kraken data race
* Fixes banking global issues. Fixes postgres trades
* rmline for appveyor test
* Expands timeout in event that channel is closed before send
* Fix data race
* No rows, no bows and definitely no shows
* Removes parallel from createsnapshot tests
* Extends timedmutext test a smidge. Exchange fatality
* Shorter end timeframe and bigger candle
* Adds lovely initial concept for historical data doer
* Adds ability to save tasks. Adds config. Adds startStop to engine
* Has a database microservice without use of globals! Further infrastructure design. Adds readme
* Commentary to help design
* Adds migrations for database
* readme and adds database models
* Some modelling that doesn't work end of day
* Completes datahistoryjob sql.Begins datahistoryjobresult
* Adds datahistoryjob functions to retreive job results. Adapts subsystem
* Adds process for upserting jobs and job results to the database
* Broken end of day weird sqlboiler crap
* Fixes issue with SQL generation.
* RPC generation and addition of basic upsert command
* Renames types
* Adds rpc functions
* quick commit before context swithc. Exchanges aren't being populated
* Begin the tests!
* complete sql tests. stop failed jobs. CLI command creation
* Defines rpc commands
* Fleshes out RPC implementation
* Expands testing
* Expands testing, removes double remove
* Adds coverage of data history subsystem, expands errors and nil checks
* Minor logic improvement
* streamlines datahistory test setup
* End of day minor linting
* Lint, convert simplify, rpc expansion, type expansion, readme expansion
* Documentation update
* Renames for consistency
* Completes RPC server commands
* Fixes tests
* Speeds up testing by reducing unnecessary actions. Adds maxjobspercycle config
* Comments for everything
* Adds missing result string. checks interval supported. default start end cli
* Fixes ID problem. Improves binance trade fetch. job ranges are processed
* adds dbservice coverage. adds rpcserver coverage
* docs regen, uses dbcon interface, reverts binance, fixes races, toggle manager
* Speed up tests, remove bad global usage, fix uuid check
* Adds verbose. Updates docs. Fixes postgres
* Minor changes to logging and start stop
* Fixes postgres db tests, fixes postgres column typo
* Fixes old string typo,removes constraint,error parsing for nonreaders
* prevents dhm running when table doesn't exist. Adds prereq documentation
* Adds parallel, rmlines, err fix, comment fix, minor param fixes
* doc regen, common time range check and test updating
* Fixes job validation issues. Updates candle range checker.
* Ensures test cannot fail due to time.Now() shenanigans
* Fixes oopsie, adds documentation and a warn
* Fixes another time test, adjusts copy
* Drastically speeds up data history manager tests via function overrides
* Fixes summary bug and better logs
* Fixes local time test, fixes websocket tests
* removes defaults and comment,updates error messages,sets cli command args
* Fixes FTX trade processing
* Fixes issue where jobs got stuck if data wasn't returned but retrieval was successful
* Improves test speed. Simplifies trade verification SQL. Adds command help
* Fixes the oopsies
* Fixes use of query within transaction. Fixes trade err
* oopsie, not needed
* Adds missing data status. Properly ends job even when data is missing
* errors are more verbose and so have more words to describe them
* Doc regen for new status
* tiny test tinkering
* str := string("Removes .String()").String()
* Merge fixups
* Fixes a data race discovered during github actions
* Allows websocket test to pass consistently
* Fixes merge issue preventing datahistorymanager from starting via config
* Niterinos cmd defaults and explanations
* fixes default oopsie
* Fixes lack of nil protection
* Additional oopsie
* More detailed error for validating job exchange
* Initial codes for a trade tracker
* Moving everything in a broken fashion
* Removes tradetracker. Removes some errors for subsystems
* Cleans up some subsystems, renames stuttering types. Removes some global Bot usage
* More basic subsystem renaming and file moving
* Removes engine dependency from events,ntpserver,ordermanager,comms manager
* Exports eventManager, fixes rpcserver. puts rpcserver back for now
* Removes redundant error message, further removes engine dependencies
* experimental end of day interface usage
* adds ability to build the application
* Withdraw and event manager handling
* cleans up apiserver and communications manager
* Cleans up some start/setup processes. Though should separate
* More consistency with Setup Start Stop IsRunning funcs
* Final consistency pass before testing phase
* Fixes engine tests. Fixes stop nil issue
* api server tests
* Communications manager testing
* Connection manager tests and nilsubsystem error
* End of day currencypairsyncer tests
* Adds databaseconnection/databaseconnection_test.go
* Adds withdrawal manager tests
* Deposit address testing. Moved orderbook sync first as its more important
* Adds test for event manager
* More full eventmanager testing
* Adds testfile. Enables skipped test.
* ntp manager tests
* Adds ordermanager tests, Extracts a whole new subsystem from engine and fanangles import cycles
* Adds websocket routine manager tests
* Basic portfolio manager testing
* Fixes issue with currency pair sync startup
* Fixes issue with event manager startup
* Starts the order manager before backtester starts
* Fixes fee tests. Expands testing. Doesnt fix races
* Fixes most test races
* Resolves data races
* Fixes subsystem test issues
* currency pair syncer coverage tests
* Refactors portfolio. Fixes tests. Withdraw validation
Portfolio didn't need to exist with a portfolio manager. Now the porfolio manager
is in charge how the portfolio is handled and all portfolio functions are attached
to the base instead of just exported at the package level
Withdrawal validation occurred at the exchange level when it can just be run at the
withdrawal manager level. All withdrawal requests go through that endpoint
* lint -fix
* golang lint fixes
* lints and comments everything
* Updates GCT logo, adds documentation for some subsystems
* More documentation and more logo updates
* Fixes backtesting and apiserver errors encountered
* Fixes errors and typos from reviewing
* More minor fixes
* Changes %h verb to %w
* reverbs to %s
* Humbly begins reverting to more flat engine package
The main reasoning for this is that the subsystem split doesn't make sense
in a golang environment. The subsystems are only meant to be used with engine
and so by placing them in a non-engine area, it does not work and is
inconsistent with the rest of the application's package layout.
This will begin salvaging the changes made by reverting to a flat
engine package, but maintaining the consistent designs introduced.
Further, I will look to remove any TestMains and decrease the scope
of testing to be more local and decrease the issues that have been
caused from our style of testing.
* Manages to re-flatten things. Everything is within its own file
* mini fixes
* Fixes tests and data races and lints
* Updates docs tool for engine to create filename readmes
* os -> ioutil
* remove err
* Appveyor version increase test
* Removes tCleanup as its unsupported on appveyor
* Adds stuff that I thought was in previous merge master commit
* Removes cancel from test
* Fixes really fun test-exclusive data race
* minor nit fixes
* niterinos
* docs gen
* rm;rf test
* Remove typoline. expands startstop helper. Splits apiserver
* Removes accidental folder
* Uses update instead of replace for order upsert
* addresses nits. Renames files. Regenerates documentation.
* lint and removal of comments
* Add new test for default scenario
* Fixes typo
* regen docs
* Add initial workflows config
* Add PSQL service
* Add 32bit and frontend jobs
* Add gcc-multilib and fix typo
* Fix backtester test for linux/unix filesystem errors and npm commands
* 32-bit Add -race to purposefully throw an error to ensure envs are set correctly (will revert)
* Revert "32-bit Add -race to purposefully throw an error to ensure envs are set correctly (will revert)"
This reverts commit b97e66c49fca859f8738e01107f96f9ca4040b2e.
* Bump workers count for orderbook processing
* Remove comments to trigger 32-bit cache hit after job success
* Cancel addition plus version bump, spawn more overlords
* Update build badge to Github Actions
* Brach -> Branch
* add buy and sell limit to signal event
* add buy limit and sell limit
* add test case
* add verify limit before order
* fix sell max && min bugs
* add equal when sell & buy limit comparison && add received to buy & sell limit testcase
* fix bugs in description of SetSellLimit
* remote backtester\eventhandlers\exchange\exchange.go:115: unnecessary trailing newline (whitespace)
* add timeout=10m to golangci-lint
* add timeout=10m to .golangci.yml
* Revert "remote backtester\eventhandlers\exchange\exchange.go:115: unnecessary trailing newline (whitespace)"
This reverts commit 5f7f34903eb9d11a83d3643141a26388c8364a67.
* Revert "add timeout=10m to .golangci.yml"
This reverts commit c83fa972b58327b8de7af3c8fc1d7c19f537838f.
* Revert "add timeout=10m to golangci-lint"
This reverts commit a9da40e91af05d4bb3eee52a61106686c03f9ff4.
* trailing whitespace && revert timeout for linter ci
* add check when buy & sell limit is 0 && passed test cases in size_test
* fix bugs when buy & sell min & max limit is zero && pass testcase TestExecuteOrder
* check MaximumSize if zero or not && add test cases TestExecuteOrderBuySellSizeLimit
* clean logs
* add update buy sell limit in exchange && update testcase
* fix bugs when max is zero calculateBuySize && add testcase TestMaximumBuySizeEqualZero
* fix bugs when max is zero calculateSellSize && add testcase TestMaximumSellSizeEqualZero
Co-authored-by: Tony Wang <tonywang.data@gmail.com>
* deleting the unwanted file created during testing + adding more verbose errors for cli
* wip
* checking params throughout wip
* improving errors
* wip
* thrasher patch
* better err name
* whip
* testing and fixing errors WIP
* upgrades and better errors
* broken test
* wip
* adding some tests
* using tempDir
* mini improvement
* little changes
* better time check
* fixing error
* more glorious changes
* end of day wip
* shazzy changes
* checking error
* appveyor
* last changes:
* FTX: Add REST API subaccount support
* Add API key check to GetSubaccounts
* Fix missing comment and expand to rest of the codebase
* Address glorious nits
* Address various nits
* Fix ZB typo
https://www.zb.com/api#hsptccieyyqomlp
