6 Commits

Author SHA1 Message Date
Scott
85403fe801 exchange/order/limits: Migrate to new package and integrate with exchanges (#1860)
* move limits, transition to key gen

* rollout NewExchangePairAssetKey everywhere

* test improvements

* self-review fixes

* ok, lets go

* fix merge issue

* slower value func,assertify,drop IsValidPairString

* remove binance reference for backtesting test

* Redundant nil checks removed due to redundancy

* Update order_test.go

* Move limits back into /exchanges/

* puts limits in a different box again

* SHAZBERT SPECIAL SUGGESTIONS

* Update gateio_wrapper.go

* fixes all build issues

* Many niteroos!

* something has gone awry

* bugfix

* gk's everywhere nits

* lint

* extra lint

* re-remove IsValidPairString

* lint fix

* standardise test

* revert some bads

* dupe rm

* another revert 360 mcgee

* un-in-revertify

* Update exchange/order/limits/levels_test.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* fix

* Update exchanges/binance/binance_test.go

HERE'S HOPING GITHUB FORMATS THIS CORRECTLY!

Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>

* update text

* rn func, same line err gk4202000

---------

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
Co-authored-by: Gareth Kirwan <gbjkirwan@gmail.com>
2025-08-26 12:30:21 +10:00
Adrian Gallagher
3cc9a2b9e0 exchanges: Refactor time handling and other minor improvements (#1948)
* exchanges: Refactor time handling and other minor improvements

- Updated Kraken wrapper to utilise new time handling methods.
- Simplified Kucoin types by removing unnecessary structures and using direct JSON unmarshalling.
- Improved websocket handling in Kucoin to directly parse candlestick data.
- Modified Lbank types to use the new time representation.
- Adjusted Poloniex wrapper and types to utilise the new time handling.
- Updated Yobit types and wrapper to reflect changes in time representation.
- Introduced DateTime type for better handling of specific time formats.
- Added tests for DateTime unmarshalling to ensure correctness.
- Rid UTC().Unix and UTC().UnixMilli as it's not needed
- Correct Huobi timestamp usage for some endpoints.
- Rid RFC3339 time parsing since Go does that automatically.

* exchanges: Refactor JSON unmarshalling for various types and improve test coverage

* linter: Update error message in TestGetKlines

* refactor: Simplify JSON unmarshalling in MovementHistory and improve test assertions in GetKlines

* refactor: Improve JSON unmarshalling for channel name and clarify comment in wsProcessOpenOrders

* refactor: Update time handling in Huobi types to use types.Time for createdAt fields and relax GetLiquidationOrders test

* refactor: Move wsTicker, wsSpread, wsTrades, and wsCandle types to kraken_types.go for better organistion

* refactor: Add validation for underlying parameter in GetExpirationTime and update tests
2025-07-01 09:11:55 +10:00
Scott
91d699be9d maps: expansion of Key concept (#1349)
* moves everything to use single map keys, also breaks

* full rollout

* tests

* fix a little bug

* minor test fixups

* Fix Key use

* rm 🔑 from 🔑 struct name
2023-10-04 10:19:41 +11:00
Scott
017cdf1384 Backtester: Live trading upgrades (#1023)
* Modifications for a smoother live run

* Fixes data appending

* Successfully allows multi-currency live trading. Adds multiple currencies to live DCA strategy

* Attempting to get cash and carry working

* Poor attempts at sorting out data and appending it properly with USD in mind

* =designs new live data handler

* Updates cash and carry strat to work

* adds test coverage. begins closeallpositions function

* Updates cash and carry to work live

* New kline.Event type. Cancels orders on close. Rn types

* =Fixes USD funding issue

* =fixes tests

* fixes tests AGAIN

* adds coverage to close all orders

* crummy tests, should override

* more tests

* more tests

* more coverage

* removes scourge of currency.Pair maps. More tests

* missed currency stuff

* Fixes USD data issue & collateral issue. Needs to close ALL orders

* Now triggers updates on the very first data entry

* All my problems are solved now????

* fixes tests, extends coverage

* there is some really funky candle stuff going on

* my brain is melting

* better shutdown management, fixes freezing bug

* fixes data duplication issues, adds retries to requests

* reduces logging, adds verbose options

* expands coverage over all new functionality

* fixes fun bug from curr == curr to curr.Equal(curr)

* fixes setup issues and tests

* starts adding external wallet amounts for funding

* more setup for assets

* setup live fund calcs and placing orders

* successfully performs automated cash and carry

* merge fixes

* funding properly set at all times

* fixes some bugs, need to address currencystatistics still

* adds 'appeneded' trait, attempts to fix some stats

* fixes stat bugs, adds cool new fetchfees feature

* fixes terrible processing bugs

* tightens realorder stats, sadly loses some live stats

* this actually sets everything correctly for bothcd ..cd ..cd ..cd ..cd ..!

* fix tests

* coverage

* beautiful new test coverage

* docs

* adds new fee getter delayer

* commits from the correct directory

* Lint

* adds verbose to fund manager

* Fix bug in t2b2 strat. Update dca live config. Docs

* go mod tidy

* update buf

* buf + test improvement

* Post merge fixes

* fixes surprise offset bug

* fix sizing restrictions for cash and carry

* fix server lints

* merge fixes

* test fixesss

* lintle fixles

* slowloris

* rn run to task, bug fixes, close all on close

* rpc lint and fixes

* bugfix: order manager not processing orders properly

* somewhat addresses nits

* absolutely broken end of day commit

* absolutely massive knockon effects from nits

* massive knockon effects continue

* fixes things

* address remaining nits

* jk now fixes things

* addresses the easier nits

* more nit fixers

* more niterinos addressederinos

* refactors holdings and does some nits

* so buf

* addresses some nits, fixes holdings bugs

* cleanup

* attempts to fix alert chans to prevent many chans waiting?

* terrible code, will revert

* to be reviewed in detail tomorrow

* Fixes up channel system

* smashes those nits

* fixes extra candles, fixes collateral bug, tests

* fixes data races, introduces reflection

* more checks n tests

* Fixes cash and carry issues. Fixes more cool bugs

* fixes ~typer~ typo

* replace spot strats from ftx to binance

* fixes all the tests I just destroyed

* removes example path, rm verbose

* 1) what 2) removes FTX references from the Backtester

* renamed, non-working strategies

* Removes FTX references almost as fast as sbf removes funds

* regen docs, add contrib names,sort contrib names

* fixes merge renamings

* Addresses nits. Fixes setting API credentials. Fixes Binance limit retrieval

* Fixes live order bugs with real orders and without

* Apply suggestions from code review

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update backtester/engine/live.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update backtester/engine/live.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* Update backtester/config/strategyconfigbuilder/main.go

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>

* updates docs

* even better docs

Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
2023-01-05 13:03:17 +11:00
Ryan O'Hara-Reid
f843b7d277 exchange: upgrade UpdatePair method (#991)
* exchange: upgrade update pair

* exchanges: Add enabled string matching and format handling if discrepency is found.

* linter: fixes

* bithumb: fix tests

* BTSE: api change fix ordering

* huobi: fix tests

* gloriousnits: stage 1

* gloriousnits: stage 2

* currency: more nits

* bitmex: add spot and process pairs before currency package call.

* bitmex: finished correct orderbook matching and other implementations

* linter: fix issue

* currency: Fix linter

* currency: segregate and protect pair store, update tests

* currency/manager: clean code, rm log output

* currency: Add store method and make sure formatting stays nil if not stored.

* gct: check errors

* engine/websocketroutineman: fix tests

* bybit: fix duplication bug

* huobi: fix test

* btse: fix tests?

* ob/buffer: fix tests

* Update currency/manager.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* glorious: nits strikes again.

* exchange: add bypassConfigFormatUpgrades to stop formatting

* GLORIOUS LINTER

* Update exchanges/bithumb/bithumb_wrapper.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

* glorious: nits

* exchange: fix pair upgrade issue when duplications are in both avail and enabled pairs

* linter: fix shadow dec

* config: fix test

* Update currency/pair_test.go

Co-authored-by: Scott <gloriousCode@users.noreply.github.com>

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Scott <gloriousCode@users.noreply.github.com>
2022-09-16 08:59:27 +10:00
Scott
f929b4d51e backtester: Futures handling & FTX Cash and Carry example strategy (#930)
* implements futures functions and GRPC functions on new branch

* lint and test fixes

* Fix uneven split pnl. Adds collateral weight test. docs. New clear func

* Test protection if someone has zero collateral

* Uses string instead of double for accuracy

* Fixes old code panic

* context, match, docs

* Addresses Shazniterinos, var names, expanded tests

* Returns subaccount name, provides USD values when offlinecalc

* Fixes oopsie

* Fixes cool bug which allowed made up subaccount results

* Subaccount override on FTX, subaccount results for collateral

* Strenghten collateral account info checks. Improve FTX test

* English is my first language

* Fixes oopsies

* Adds some conceptual futures order details to track PNL

* Initial design of future order processing in the backtester

* Introduces futures concept for collateral and spot/futures config diffs

* Fixes most tests

* Simple designs for collateral funding pair concept

* Expands interface use so much it hurts

* Implements more collateral interfaces

* Adds liquidation, adds strategy, struggles with Binance

* Attempts at getting FTX to work

* Adds calculatePNL as a wrapper function and adds an `IsFutures` asset check

* Successfully loads backtester with collateral currency

* Fails to really get much going for supporting futures

* Merges master changes

* Fleshes out how FTX processes collateral

* Further FTX collateral workings

* hooks up more ftx collateral and pnl calculations

* more funcs to flesh out handling

* Adds more links, just can't fit the pieces together :(

* Greatly expands futures order processing

* Fleshes out position tracker to also handle asset and exchange +testing

* RM linkedOrderID. rn positioncontroller, unexport

* Successfully tracks futures order positions

* Fails to calculate PNL

* Calculates pnl from orders accurately with exception to flipping orders

* Calculates PNL from orders

* Adds another controller layer to make it ez from orderstore

* Backtester now compiles. Adds test coverage

* labels things add scaling collateral test

* Calculates pnl in line with fees

* Mostly accurate PNL, with exception to appending with diff prices

* Adds locks, adds rpc function

* grpc implementations

* Gracefully handles rpc function

* beautiful tests!

* rejiggles tests to polish

* Finishes FTX testing, adds comments

* Exposes collateral calculations to rpc

* Adds commands and testing for rpcserver.go functions

* Increase testing and fix up backtester code

* Returns cool changes to original branch

* end of day fixes

* Fixing some tests

* Fixing tests 🎉

* Fixes all the tests

* Splits the backtester setup and running into different files

* Merge, minor fixes

* Messing with some strategy updates

* Failed understanding at collateral usage

* Begins the creation of cash and carry strategy

* Adds underlying pair, adds filldependentevent for futures

* Completes fill prerequsite event implementation. Can't short though

* Some bug fixes

* investigating funds

* CAN NOW CREATE A SHORT ORDER

* Minor change in short size

* Fixes for unrealised PNL & collateral rendering

* Fixes lint and tests

* Adds some verbosity

* Updates to pnl calc

* Tracks pnl for short orders, minor update to strategy

* Close and open event based on conditions

* Adds pnl data for currency statistics

* Working through PNL calculation automatically. Now panics

* Adds tracking, is blocked from design

* Work to flesh out closing a position

* vain attempts at tracking zeroing out bugs

* woww, super fun new subloggers 🎉

* Begins attempt at automatically handling contracts and collateral based on direction

* Merge master + fixes

* Investigating issues with pnl and holdings

* Minor pnl fixes

* Fixes future position sizing, needs contract sizing

* Can render pnl results, focussing on funding statistics

* tracking candles for futures, but why not btc

* Improves funding statistics

* Colours and stats

* Fixes collateral and snapshot bugs

* Completes test

* Fixes totals bug

* Fix double buy, expand stats, fixes usd totals, introduce interface

* Begins report formatting and calculations

* Appends pnl to receiving curr. Fixes map[time]. accurate USD

* Improves report output rendering

* PNL stats in report. New tests for futures

* Fixes existing tests before adding new coverage

* Test coverage

* Completes portfolio coverage

* Increase coverage exchange, portfolio. fix size bug. NEW CHART

* WHAT IS GOING ON WITH PNL

* Fixes PNL calculation. Adds ability to skip om futures tracking

* minor commit before merge

* Adds basic liquidation to backtester

* Changes liquidation to order based

* Liquidationnnnnn

* Further fleshes out liquidations

* Completes liquidations in a honorable manner. Adds AppendReasonf

* Beginnings of spot futures gap chart. Needs to link currencies to render difference

* Removes fake liquidation. Adds cool new chart

* Fixes somet tests,allows for zero fee value v nil distinction,New tests

* Some annoying test fixes that took too long

* portfolio coverage

* holding coverage, privatisation funding

* Testwork

* boring tests

* engine coverage

* More backtesting coverage

* Funding, strategy, report test coverage

* Completes coverage of report package

* Documentation, fixes some assumptions on asset errors

* Changes before master merge

* Lint and Tests

* defaults to non-coloured rendering

* Chart rendering

* Fixes surprise non-local-lints

* Niterinos to the extremeos

* Fixes merge problems

* The linter splintered across the glinting plinths

* Many nits addressed. Now sells spot position on final candle

* Adds forgotten coverage

* Adds ability to size futures contracts to match spot positions.

* fixes order sell sizing

* Adds tests to sizing. Fixes charting issue

* clint splintered the linters with flint

* Improves stats, stat rendering

* minifix

* Fixes tests and fee bug

* Merge fixeroos

* Microfixes

* Updates orderPNL on first Correctly utilises fees. Adds committed funds

* New base funcs. New order summary

* Fun test updates

* Fix logo colouring

* Fixes niteroonies

* Fix report

* BAD COMMIT

* Fixes funding issues.Updates default fee rates.Combines cashcarry case

* doc regen

* Now returns err

* Fixes sizing bug issue introduced in PR

* Fixes fun fee/total US value bug

* Fix chart bug. Show log charts with disclaimer

* sellside fee

* fixes fee and slippage view

* Fixed slippage price issue

* Fixes calculation and removes rendering

* Fixes stats and some rendering

* Merge fix

* Fixes merge issues

* go mod tidy, lint updates

* New linter attempt

* Version bump in appveyor and makefile

* Regex filename, config fixes, template h2 fixes

* Removes bad stats.

* neatens config builder. Moves filename generator

* Fixes issue where linter wants to fix my spelling

* Fixes pointers and starts
2022-06-30 15:43:41 +10:00