Commit Graph

32 Commits

Author SHA1 Message Date
Ryan O'Hara-Reid
e99adca86f encoding/json: Add custom JSON package with build tag support for Sonic (#1623)
* tag optional sonic and allow full library conversion

* Add workflow and disallow arm and darwin usage

* Add basic hotswap benchmark

* linter: fix

* use bash

* linter: fix?

* Fix whoopsie, add to make file, also add mention in features list.

* test enforcement

* actually read documentation see if this works

* linter: fix

* linter: fix

* sonic: bump tagged version

* encoding/json: drop build tag arch and os filters

* encoding/json: consolidate tests

* encoding/json: log build tag usage

* rm superfluous builds

* glorious/nits: add template change and regen docs

* glorious/nits: update commentary on nolint directive

* glorious/nits: rm init func and log results in main.go

* Test to actually pull flag in

* linter: fix

* thrasher: nits

* gk: nits 4 goflags goooooooooo!

* gk: nits rn

* make sonic default json implementation

* screen 386

* linter: fix

* Add commentary

* glorious: nits Makefile not working

* gk: nits

* gk: nits whoops

* whoopsirino

* mention 32bit systems won't be sonic

* gk: super-duper nit of extremes

---------

Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
2025-02-20 16:05:55 +11:00
Adrian Gallagher
225429bda6 CI/build: Update Go version, linters and fix minor issues (#1612)
* CI/build: Update Go version, linters and fix minor issues

* linters: Add intrange, copyloopvar, additional go vet linters to match gopls and fix issues
2024-08-16 17:41:11 +10:00
Adrian Gallagher
1e95ae9b79 qa: Enforce JSON standard for GCT configs (#1526)
* qa: Enforce JSON lint for GCT configs

* Makefile/GHA: Make output more verbose on success, clean duplicate PHONY and rid excess newlines

* Makefile: Use printf for OS cross-compatibility output
2024-04-30 16:02:09 +10:00
Adrian Gallagher
08da42ddb7 CI/build: Update Go version, linters and fix minor issues (#1473)
* CI/build: Update Go version, linters and fix minor issues

* Bump golangci-lint to v1.56.1

* BinanceUS: Make uint usage consistent

* Throw blank identifiers into the trash
2024-02-14 11:02:06 +11:00
Adrian Gallagher
e8e0ff3a35 CI: Add macOS, Windows x64 and Linux arm64 test support (#1422)
* CI: Add macOS, Windows and Linux arm64 support

* Modify arm64 job name to be inline with the others

* linter: Prevent cache from causing "Cannot open: File exists" issues

* Use setup-go and setup-node's inbuilt caching
2023-12-28 14:55:50 +11:00
Adrian Gallagher
b25037dabc CI: Bump Go and golangci-lint versions (#1313) 2023-08-11 14:06:56 +10:00
Samuael A
969669cb0d linters: Bump golangci-lint version to v1.52.2 and fix issues (#1169)
* fix linter issues

* linter issue suppressing and fixing

* change linter version and linter issues fix

* Bump version
2023-04-27 10:13:46 +10:00
Adrian Gallagher
7e08e483fb CI: Bump go version, linters and fix minor issues (#1130)
* CI: Bump go version, linters and fix minor issues

* Bump version, fix loop variables

* Revert

* Rid TODOs now that 1.51 has been released
2023-02-03 15:56:59 +11:00
Adrian Gallagher
68588560e3 CI: Bump go version, linters and fix minor issues (#1010)
* Bump golang, golangci-lint versions and fix issues

* Add -fno-stack-protector

* Fix AppVeyor golangci-lint ver

* Nitters

* Nitters round 2
2022-08-17 11:37:22 +10:00
Scott
f929b4d51e backtester: Futures handling & FTX Cash and Carry example strategy (#930)
* implements futures functions and GRPC functions on new branch

* lint and test fixes

* Fix uneven split pnl. Adds collateral weight test. docs. New clear func

* Test protection if someone has zero collateral

* Uses string instead of double for accuracy

* Fixes old code panic

* context, match, docs

* Addresses Shazniterinos, var names, expanded tests

* Returns subaccount name, provides USD values when offlinecalc

* Fixes oopsie

* Fixes cool bug which allowed made up subaccount results

* Subaccount override on FTX, subaccount results for collateral

* Strenghten collateral account info checks. Improve FTX test

* English is my first language

* Fixes oopsies

* Adds some conceptual futures order details to track PNL

* Initial design of future order processing in the backtester

* Introduces futures concept for collateral and spot/futures config diffs

* Fixes most tests

* Simple designs for collateral funding pair concept

* Expands interface use so much it hurts

* Implements more collateral interfaces

* Adds liquidation, adds strategy, struggles with Binance

* Attempts at getting FTX to work

* Adds calculatePNL as a wrapper function and adds an `IsFutures` asset check

* Successfully loads backtester with collateral currency

* Fails to really get much going for supporting futures

* Merges master changes

* Fleshes out how FTX processes collateral

* Further FTX collateral workings

* hooks up more ftx collateral and pnl calculations

* more funcs to flesh out handling

* Adds more links, just can't fit the pieces together :(

* Greatly expands futures order processing

* Fleshes out position tracker to also handle asset and exchange +testing

* RM linkedOrderID. rn positioncontroller, unexport

* Successfully tracks futures order positions

* Fails to calculate PNL

* Calculates pnl from orders accurately with exception to flipping orders

* Calculates PNL from orders

* Adds another controller layer to make it ez from orderstore

* Backtester now compiles. Adds test coverage

* labels things add scaling collateral test

* Calculates pnl in line with fees

* Mostly accurate PNL, with exception to appending with diff prices

* Adds locks, adds rpc function

* grpc implementations

* Gracefully handles rpc function

* beautiful tests!

* rejiggles tests to polish

* Finishes FTX testing, adds comments

* Exposes collateral calculations to rpc

* Adds commands and testing for rpcserver.go functions

* Increase testing and fix up backtester code

* Returns cool changes to original branch

* end of day fixes

* Fixing some tests

* Fixing tests 🎉

* Fixes all the tests

* Splits the backtester setup and running into different files

* Merge, minor fixes

* Messing with some strategy updates

* Failed understanding at collateral usage

* Begins the creation of cash and carry strategy

* Adds underlying pair, adds filldependentevent for futures

* Completes fill prerequsite event implementation. Can't short though

* Some bug fixes

* investigating funds

* CAN NOW CREATE A SHORT ORDER

* Minor change in short size

* Fixes for unrealised PNL & collateral rendering

* Fixes lint and tests

* Adds some verbosity

* Updates to pnl calc

* Tracks pnl for short orders, minor update to strategy

* Close and open event based on conditions

* Adds pnl data for currency statistics

* Working through PNL calculation automatically. Now panics

* Adds tracking, is blocked from design

* Work to flesh out closing a position

* vain attempts at tracking zeroing out bugs

* woww, super fun new subloggers 🎉

* Begins attempt at automatically handling contracts and collateral based on direction

* Merge master + fixes

* Investigating issues with pnl and holdings

* Minor pnl fixes

* Fixes future position sizing, needs contract sizing

* Can render pnl results, focussing on funding statistics

* tracking candles for futures, but why not btc

* Improves funding statistics

* Colours and stats

* Fixes collateral and snapshot bugs

* Completes test

* Fixes totals bug

* Fix double buy, expand stats, fixes usd totals, introduce interface

* Begins report formatting and calculations

* Appends pnl to receiving curr. Fixes map[time]. accurate USD

* Improves report output rendering

* PNL stats in report. New tests for futures

* Fixes existing tests before adding new coverage

* Test coverage

* Completes portfolio coverage

* Increase coverage exchange, portfolio. fix size bug. NEW CHART

* WHAT IS GOING ON WITH PNL

* Fixes PNL calculation. Adds ability to skip om futures tracking

* minor commit before merge

* Adds basic liquidation to backtester

* Changes liquidation to order based

* Liquidationnnnnn

* Further fleshes out liquidations

* Completes liquidations in a honorable manner. Adds AppendReasonf

* Beginnings of spot futures gap chart. Needs to link currencies to render difference

* Removes fake liquidation. Adds cool new chart

* Fixes somet tests,allows for zero fee value v nil distinction,New tests

* Some annoying test fixes that took too long

* portfolio coverage

* holding coverage, privatisation funding

* Testwork

* boring tests

* engine coverage

* More backtesting coverage

* Funding, strategy, report test coverage

* Completes coverage of report package

* Documentation, fixes some assumptions on asset errors

* Changes before master merge

* Lint and Tests

* defaults to non-coloured rendering

* Chart rendering

* Fixes surprise non-local-lints

* Niterinos to the extremeos

* Fixes merge problems

* The linter splintered across the glinting plinths

* Many nits addressed. Now sells spot position on final candle

* Adds forgotten coverage

* Adds ability to size futures contracts to match spot positions.

* fixes order sell sizing

* Adds tests to sizing. Fixes charting issue

* clint splintered the linters with flint

* Improves stats, stat rendering

* minifix

* Fixes tests and fee bug

* Merge fixeroos

* Microfixes

* Updates orderPNL on first Correctly utilises fees. Adds committed funds

* New base funcs. New order summary

* Fun test updates

* Fix logo colouring

* Fixes niteroonies

* Fix report

* BAD COMMIT

* Fixes funding issues.Updates default fee rates.Combines cashcarry case

* doc regen

* Now returns err

* Fixes sizing bug issue introduced in PR

* Fixes fun fee/total US value bug

* Fix chart bug. Show log charts with disclaimer

* sellside fee

* fixes fee and slippage view

* Fixed slippage price issue

* Fixes calculation and removes rendering

* Fixes stats and some rendering

* Merge fix

* Fixes merge issues

* go mod tidy, lint updates

* New linter attempt

* Version bump in appveyor and makefile

* Regex filename, config fixes, template h2 fixes

* Removes bad stats.

* neatens config builder. Moves filename generator

* Fixes issue where linter wants to fix my spelling

* Fixes pointers and starts
2022-06-30 15:43:41 +10:00
Adrian Gallagher
9a4eb9de84 CI: Fix golangci-lint linter issues, add prealloc linter and bump version depends for Go 1.18 (#915)
* Bump CI versions

* Specifically set go version as 1.17.x bumps it to 1.18

* Another

* Adjust AppVeyor

* Part 1 of linter issues

* Part 2

* Fix various linters and improvements

* Part 3

* Finishing touches

* Tests and EqualFold

* Fix nitterinos plus bonus requester jobs bump for exchanges with large number of tests

* Fix nitterinos and bump golangci-lint timeout for AppVeyor

* Address nits, ensure all books are returned on err due to syncer regression

* Fix the wiggins

* Fix duplication

* Fix nitterinos
2022-04-20 13:45:15 +10:00
Adrian Gallagher
f0d45aa1d2 golangci-lint/CI: Bump versions and introduce new linters (#798)
* golangci-lint/CI: Bump versions

Fix remaining linter issues

* Specifically set AppVeyor version

* Fix the infamous typos 👀

* Add go env cmd to AppVeyor

* Add go version cmd to AppVeyor

* Specify AppVeyor image, adjust linters

* Update go get to go install due to deprecation

* Bump golangci-lint timeout time for AppVeyor

* Change NW contract to NQ

* Address nitters

* GetRandomPair -> Pair{}

* Address nits

* Address time nitterinos plus additional tweaks

* More time inception upgrades!

* Bending time and space
2021-10-14 16:38:53 +11:00
Rauno Ots
45d0973137 gen_sqlboiler: fix incorrect database path for non-default sqlite location (#603)
* the generator should use the same path as the database/drivers/sqlite3
* remove unused data directory
* generate sqlboiler.json for model generation
* ignore target folder
2020-12-02 10:11:10 +11:00
Mikhail Shogin
ba4ac4f3d6 engine/gRPC: Add update account info to grpc interface (#602)
* Add update account info to grpc interface

* Fix lbank tests

* Review corrections

* Review corrections

* Fix linter
2020-11-30 11:29:31 +11:00
Adrian Gallagher
d9bcf8246f Bump golangci-lint, Docker and Go CI versions (#564)
* Bump golangci-lint and Go CI versions

* Bump golangci-lint version

* Address nitterinos
2020-09-24 09:52:37 +10:00
Adrian Gallagher
0d787bc259 Bump golangci-lint to v1.24.0, linter fixes and general code improvements (#478)
* Bump golangci-lint version, update Go version deps and generic code improvements

* Fix wesbocket resp nil check and zip closures

* Update pprof path
2020-04-09 13:07:32 +10:00
Adrian Gallagher
c27b8657e2 Add 32-bit linux build target for Travis (#389)
* Add 32bit build matrix test

* Adjust travis/Makefile

* Set dist back to xenial

* export CGO_ENABLED=1

* Add gcc-multilib

* Sudo installerino

* make check -> make test
2019-12-03 08:44:57 +11:00
Adrian Gallagher
da81ea6c08 Merge branch 'master' into engine 2019-11-11 17:11:11 +11:00
Adrian Gallagher
e9b3e3d328 Bump golangci-lint and AppVeyor versions now that 1.13.x is supported (#375) 2019-11-11 15:15:29 +11:00
Andrew
92147cdc5f (Engine): Database system improvements (#358)
* Migrated to goose & sqlboiler

* create tests with sqlboiler

* code clean up

* Added gct -> sqlboiler config gen

* dropped pgx support

* dropped pgx support because who needs connection pools

* reenable sqlite audit tests

* first pass of migration changes

* stuff is broken :D

* sqlboiler :D

* end of date commit

* Added comments code clean up

* revert go module files back to upstream

* bug fix

* pushed go.mod update to use correc goose version

* renamed sqlite to sqlite3 for consistency across codebase and PR feedback changes

* makefile updates

* things are broken end of day commit

* added postgresql test

* use correct database name

* travis fixes for env vars

* travis fixes for env vars

* test fixes

* run migration on test setup

* test adding postgres support to appveyor

* Skip tests on appveyor due to issues with missing binaries

* oh yeah i have to support windows don't i

* bumped goose version up

* add postgres to osx

* fix travis config as osx does not support services move spin up to before_script

* added PGDATA path fix

* pass PG_DATA to pg_ctl

* added initdb to before install

* fixes to wording and bumps up goose version

* who needs ssl anyway

* moved ssl to correct section :D

* bumped goose version up

* unbreak travis

* unbreak travis

* fix if database is disabled in config

* move strings to consts

* converted more strings to const

* improvements to sqlboiler mmodel gen

* Added contrib\sqlboiler file

* sqlboiler windows contrib fixes

* bumped goose version up

* :D whoops

* further fixes to sql models

* further fixes to sql models

* database type fix for config gen

* README update

* go.mod clean up

* added config details for appveyor

* appveyor ordering fix

* force psql9.6

* appveyor config changes

* all the environmen vars

* model changes for psql

* model changes for psql

* sqlite model fixes

* attempt at osx fix

* added error check for migration

* typos and check against goose error instead of string :D

* updated sqlboiler commit id

* bump sqlboiler version again

* set decimal package to @0bb1631

* readme and makefile updates

* bump goose version update readme and add override flag to config gen

* README typo fix and lowered inserts in test down to 20 as we are only testing that inserts work running 200 was unnecessary

* added gctcli command for audit event

* Added debug output toggle to config added both postgres & sqlite support to gctcli command

* Wording changes on errors

* set sqlite to 1 connection to stop locke database issues

* Usage update for order

* README updates with config examples

* go.mod/sum tidy

* removed lines in import second

* removed lines in imports

* convert local time to utc for database and display output

* go mod clean up and error checking to time

* renamed all packages to sqlite3

* added windows command output for sql model gen

* time conversion fix

* time conversion on gctcli
2019-10-08 15:28:31 +11:00
Adrian Gallagher
7f576d0d4f Merge branch 'master' into engine 2019-09-13 14:29:48 +10:00
Adrian Gallagher
b1ed894d99 Bump golangci-lint to v1.18.0 (#354)
* Bump golangci-lint to v0.18.0

* Bump AppVeyor golangci-lint to v1.18

For science

* Bump golangci deadline

* Fix notifications typo

* Fix grammar

* Update DisableNTPCheck test string comparison
2019-09-13 13:36:52 +10:00
Adrian Gallagher
2dc813b5f3 Merge branch 'master' into engine 2019-08-23 21:59:01 +10:00
Ryan O'Hara-Reid
6d8ba0a96a Initial implementation of HTTP mock testing framework (#310)
* Initial implementation of HTTP mock testing framework

Convert to VCR testing server. Segregate live testing via build tags.

Converted Binance to VCR server

Convert Bitstamp to VCR mocking tests

Added VCR mock testing for localbitcoins

* Add server generation for concurrent testing

* Fix linter issues

* Fix linter issue

* fix race - potentially

* revert auto assigning of host vals

* Fix requested changes

* Adds mock testing for ANX
Switch to using TestMain functionality
Added cron job usage for travis-ci to live testing
Added appveyor scheduled build check for live testing

* WOOPS

* silly correction

* Fixes fantastic linter issues

* fixed another whoopsie

* WOOO!

* Adds gemini mock testing with additional fixes

* Add docs and sharedvalue

* Added tls using httptest package

* Fixed issues

* added explicit mock recording reference to error

* Fix requested changes

* strip port from mock files as they are not needed on tls server

* Change incorrect names

* fix requested changes

* lbank update

* Fix another issue

* Updated readme
2019-08-23 15:20:02 +10:00
Andrew
0c76789b0d Database interface & auditing feature (#332)
* added audit manager

* Basic database DOA setup

* Added base config file

* added sqlite support and creation of schema

* added basic tests and config entry

* corrected issues of database is disabled

* fixed path for test

* WIP

* Added tests fixed config checking

* reverted files back to upstream

* reverted go.mod files

* no more test test test

* removed local testing details for psql

* hello

* added comments

* increased ping to 30 seconds

* renamed database table and added additional condition around test

* removed database test details

* goimport ran on all files

* WIP

* first attempt at migration

* fixes for migration system

* Migration system logger interface implemented

* fixes to print functions

* added write pooling pass

* gofmt :D

* formatted imports correctly

* removed old code

* added creation of migration

* gofmt

* :D Hello

*  🏎️

* maybe one day i will remember to revert go mod files

* checked err return condition correctly

* first changes for PR feedback

* code clean up

* protect Connected with RWmutex & event with mutex

* : D

* we can just pretend like it never happened

* MOved migrations back to source directory and added README

* readme formatting update

* Addd command line override for datadir

* use correct var when creating a migration and confirm folder is created

* Check if database version is newer than latest migration and also you know make migrations work.....

* uses filepath instead of manual path to use correct path seperator

* Add connection message and lower timeout

* Added support for sslmode for psql

* no longer force Close of database instead allow driver to maage

* Added closer func to test output

* sslmode added to example config
2019-08-20 16:35:06 +10:00
Adrian Gallagher
2078ba907f Update URLs for transfer into org repo (#338)
* Update URLs for transfer into org repo

* Update codecov, travis and othe remaining links

* Update appveyor paths
2019-08-09 11:50:39 +10:00
Ryan O'Hara-Reid
e8bc3b5e76 Bump golangci-lint version to 1.16.0 (#311) 2019-05-27 14:10:54 +10:00
Andrew
58bd0a301b Added config option to enable pprof (#253)
* Added config option to enable pprof

* fixed typo in Makefile
2019-03-14 14:56:30 +11:00
Adrian Gallagher
7dcb1ab553 Migrate from gometalinter.v2 to golangci-lint (#249)
* Migrate from gometalinter.v2 to golangci-lint
2019-03-01 16:10:29 +11:00
Adrian Gallagher
5e5ca8a887 Linter fixes (#246)
Linter fixes
2019-02-05 16:26:04 +11:00
Adrian Gallagher
291e404a4a Makefile: add new recipes and linter features (#244)
* Makefile: add new recipes and linter features

* expand linter coverage and fix issues

* Update makefile

* address PR nitterinos
2019-01-31 14:53:24 +11:00
Anton Zhukov
ae199860d2 Add godep dependency manager and makefile 2018-02-01 21:15:06 +01:00