* codebase: Remove web frontend and related services
* refactor: Update StartPPROF to accept context and adjust related tests
* refactor: Simplify SetIfZero functions and update related tests
* config: Clarify DowngradeConfig method documentation regarding permanent removal of deprecated fields
* refactor: Rename setIfZeroAndWarn to setDefaultIfZeroWarn for clarity and update related calls
* refactor: Update error handling in DataHistoryManager and remove redundant error variable
* tests: Replace !errors.Is(err, target) with testify equivalents
* codebase: Manual !errors.Is(err, target) replacements
* typo: Replace errMisMatchedEvent with errMismatchedEvent
* tests: Enhance error messages for better output
* tests: Refactor error assertions in various test cases to use require and improve clarity
* misc linter: Fix assert should wording
* tests: Simplify assertions in TestCreateSignals for clarity and conciseness
* tests: Enhance assertion message in TestCreateSignals
* GHA, tests: Add additional checks for common issues
These checks include:
- Ensuring that all testify funcs use their formatted variants (e.g., `assert.Equalf(t, expected, actual)` instead of `assert.Equal(t, expected, actual)`).
- Replacing `%s` with %q
- Enforcing consistent usage of should/must wording for testify assert/require messages
* Add support for checking backticked string format specifiers and fix issues
* tests: Fix error comparisons
* tests: Replace errors.Is(err, nil) usage with testify and automate check
* refactor: Rename ExtractPort to ExtractPortOrDefault
* tests: Replace assert with require for error handling in multiple test files
* tests: Replace assert with require for error handling and improve assertions in data tests
* tests: Fix typo in assertion message for StreamVol test
* OKX: Fix GetOpenInterestAndVolumeStrike test with instrument selection and improved assertions
* OKX: Revert intentional error check
* Improve error message for expiry time check in GetOpenInterestAndVolumeStrike test
* Tests: Use currency.NewUSD and NewUSDT
Simple refactor to use the provided shortcut methods
* Github: Add CI check to ensure NewPair not used
Add a step to ensure NewPair(BTC, USD*) isn't used
* adds funding rate implementations and improvements
* merge fixes x1
* lint
* kucoin funding rates func make
* migrate sync-manager to keys
* some kucoin work
* adds some kucoin wrapper funcs
* ehhh, todo
* kucoin position
* start of orders
* adds the kucoin tests yay
* multiplier
* nits, EWS includes order limits
* NotYetImplemented, IsPerp improvements, cleaning
* lint, test fix, huobi time
* fixes issues, improves testing
* fixes linters I WRECKED
* local lint but remote lint, lint, lint, lint
* fixes err
* skip CI
* lint
* Supported rates, binance endpoints
* fixes weird mocktest problems
* no, CZ is invalid
* fixes some new EWS test errors
* allows sync manager customisation for values and logs
* config-example add
* who doesnt like more coverage?
* ensures you can actually disable it via config el oh el
* less ifs, better control
* fix verbose
* sync trades default false
* fix summary being printed when not enabled
* fixes config checker and output
* nits
* I can put this behind me now
* Fixed logCaSiNg
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
* combines if statements
---------
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
* SyncManager: Optimise and fixes
This is a fairly invasive change which addresses the amount of work the
sync manager does each cycle and the cycle intervals.
We switch to using discrete locks for each type of work on each pair,
so each worker can take a discrete chunk of work safely.
For performance and simplicity we now use a map for the currencyPairs.
* fix reporting when a websocket is reconnected
* fix not switching REST off after websocket available again
* fix race condition in isProcessing flag
This PR still could go further by avoiding cycling through everything
each time, and by pushing some one-time work for adding enabled pairs
down to a later stage off the hot path.
This was the smallest chunk of refactoring I felt could address
everything without changing too much.
Significant manual testing done with a variety of Timeouts to test for
edgecases and handling.
* SyncManager: Fix ticker/orderbook tracker linked
* SyncManager: Fix sync complete logging in update
* SyncManager: Fix pair format breaking sync key
Kraken seems to always switch to XBT_USDT format, but websockets still
pass around XBTUSDT format. Just to be safe this just removes the
delimiter to avoid any such issues
* SyncManager: Remove unused error
* SyncManager: Remove unused IsProcessing flag
* SyncManager: Fix Update test add() pair format
We had to unify pair format inside sync manager, so test needs to do the
same
* engine: shutdown and unload exchange when engine is stopped
* linter: fixes
* engine/exchMan: add nil check
* engine/exchanges: add shutdown method to exchanges, rm len check lock not needed, expanded code coverage, address some nits
* exchMan: report all failed shutdowns across exchanges, implement timer and monitoring routines.
* exchMan: improve shutdown sequence and aloc.
* further improvement
* exchman: log from warn to error
* websockconnection: Suppress error return when closure is caused by library
* linter: fix
* fix racies
* add note on why not parallel tests
* glorious: nits
* spelling kween
* thrasher: nits
* engine: change print of setting using reflection, I keep forgetting to implement this so program around forgetfulness
* engine/exchange_management: remove wait group and just rely on intermediary lock
* glorious: nits
* Update common/common.go
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
* Update main.go
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
---------
Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
* engine: make sure upper and lower case names standardize from config to default and return error if sync item never matches
* nits: Add tests for @gloriouscode
Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
* currency: Add method to derive pair
* currency: Add method to lower entire charset but used the slice copy and returned that. This will change the original, just gotta see if this is an issue, but the slice usually goes out of scope anyway.
* currency/pairs: add filter method
* currency: add function to derive select currencies from currency pairs
* currency/engine: slight adjustments
* currency: fix linter issue also shift burden of proof to caller instead of repair, more performant.
* currency: more linter
* pairs: optimize; reduce allocs/op and B/op
* currency: Add in function 'NewPairsFromString' for testing purposes
* currency: don't suppress error
* currency: stop panic on empty currency code
* currency: Add helper method to match currencies between exchanges
* currency: fixed my bad spelling
* currency: Implement stable coin checks, refactored base code methods, optimized upper and lower case strings for currency code/pairs
* currency: add pairs method to derive stable coins from internal list.
* Currency: Cleanup, fix tests.
* engine/exchanges/currency: fix whoops
* Currency: force govet no copy on Item datatype
* Currency: fix naughty linter issues
* exchange: revert change
* currency/config: fix config upgrade mistake
* currency: re-implement currency sub-systems
* *RetrieveConfigCurrencyPairs removed
*CheckCurrencyConfigValues to only provide warnings, add additional support when, disable when support is lost or not available and set default values.
*Drop Cryptocurrencies from configuration as this is not needed.
*Drop REST Poll delay field as this was unused.
*Update default values for currencyFileUpdateDuration & foreignExchangeUpdateDuration.
*Allow Role to be marshalled for file type.
*Refactor RunUpdater to verify and check config values and set default running foreign exchange provider.
* currency: cleanup
* currency: change match -> equal for comparison which is more of a standard and little easier to find
* currency: address nits
* currency: fix whoops
* currency: Add some more pairs methods
* currency: linter issues
* currency: RM unused field
* currency: rm verbose
* currency: fix word
* currency: gocritic
* currency: fix another whoopsie
* example_config: default to show log system name
* Currency: Force all support packages to use Equal method for comparison as there is a small comparison bug when checking upper and lower casing, this has a more of a pronounced impact between exchanges and client instances of currency generation
* currency: fix log name
* ordermanager: fix potential panic
* currency: small optim.
* engine: display correct bool and force shutdown
* currency: add function and fix regression
* Change ConvertCurrency -> ConvertFiat to be more precise
* ADD GetForeignExchangeRate to get specific exchange rate for fiat pair
* Fix currency display and formatting regression and tied in with config.Currency fields
* engine: fix tests
* currency: return the amount when no conversion needs to take place
* currency: reduce method name
* currency: Address nits glorious nits
* currency: fix linter
* currency: addr nits
* currency: check underlying role in test
* gct: change to EMPTYCODE and EMPTYPAIR across codebase
* currency: fix nits
* currency: this fixes test race but this issue has not been resolved. Please see: https://trello.com/c/54eizOIo/143-currency-package-upgrades
* currency: Add temp dir for testing
* Update engine/engine.go
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
* documentation: update and regen
* currency: Address niterinos
* currency: Add test case for config upgrade when falling over to exchange rate host as default from exchangeRates provider
* currency: addr nits
* currency: fix whoops
Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
Co-authored-by: Adrian Gallagher <adrian.gallagher@thrasher.io>
* orderbook: export orderbook nodes for external strategy inspection
* orderbook: Add in methods for locking and unlocking multiple books at the same time e.g. book1.LockWith(book2); defer book1.UnlockWith(book2)
* include waiting functionality for depth change alert
* backtester: add word.
* log: include logger changes to impl with downstream integration
* engine: reduce params for loading exchange
* assort: rm verbose in tests, change wording in ob, expose sync.waitgroup for ext. sync options
* ticker: reduce map look ups and contention when using RW mutex when there are over 80% writes adds find last function to get the latest rate
* engine/syncmanager: add in waitgroup for step over for external package calls
* cleaup
* engine: linter fix
* currency/fx: include all references to fiat currencies to default
* orderbook: Add in fields to Unsafe type for strategies to detect potential out of sync book operations
* syncmanager: changed config variable to display correct time
* ordermanager: Add time when none provided
* currency/manager: update getasset param to get enabled assets for minor optimizations
* ftx: use get all wallet balances for a better accounts breakdown
* orderbook: unlock in reverse order
* bithumb: fixes bug on market buy and sell orders
* bithumb: fix bug for nonce is also time window sensitive
* bithumb: get orders add required parameter
* bithumb: Add asset type to account struct
* currency: improve log output when checking currency and it fails
* bithumb: Add error return on incomplete pair
* ticker:unexport all service related methods
* ticker/currency: fixes
* orderbook: fix comment
* engine: revert variable name in LoadExchange method
* sync_manager: fix panic when enabling disabling manager
* engine: fix naming convention of exported function and comments
* engine: update comment
* orderbook: fix comment for unsafe type
* Speeds up tests
* Reduces time.Sleeps, lowers CreateTestBot complexity. Breaks things
* Removal of unecessary config reads. Parallel tests. Lower times
* Speeds up recent trades results
* mini update
* zoooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooom
* Removes the dupes
* Lint
* post cherrypick
* Fix rare kraken data race
* Fixes banking global issues. Fixes postgres trades
* rmline for appveyor test
* Expands timeout in event that channel is closed before send
* Fix data race
* No rows, no bows and definitely no shows
* Removes parallel from createsnapshot tests
* Extends timedmutext test a smidge. Exchange fatality
* Shorter end timeframe and bigger candle
* Initial codes for a trade tracker
* Moving everything in a broken fashion
* Removes tradetracker. Removes some errors for subsystems
* Cleans up some subsystems, renames stuttering types. Removes some global Bot usage
* More basic subsystem renaming and file moving
* Removes engine dependency from events,ntpserver,ordermanager,comms manager
* Exports eventManager, fixes rpcserver. puts rpcserver back for now
* Removes redundant error message, further removes engine dependencies
* experimental end of day interface usage
* adds ability to build the application
* Withdraw and event manager handling
* cleans up apiserver and communications manager
* Cleans up some start/setup processes. Though should separate
* More consistency with Setup Start Stop IsRunning funcs
* Final consistency pass before testing phase
* Fixes engine tests. Fixes stop nil issue
* api server tests
* Communications manager testing
* Connection manager tests and nilsubsystem error
* End of day currencypairsyncer tests
* Adds databaseconnection/databaseconnection_test.go
* Adds withdrawal manager tests
* Deposit address testing. Moved orderbook sync first as its more important
* Adds test for event manager
* More full eventmanager testing
* Adds testfile. Enables skipped test.
* ntp manager tests
* Adds ordermanager tests, Extracts a whole new subsystem from engine and fanangles import cycles
* Adds websocket routine manager tests
* Basic portfolio manager testing
* Fixes issue with currency pair sync startup
* Fixes issue with event manager startup
* Starts the order manager before backtester starts
* Fixes fee tests. Expands testing. Doesnt fix races
* Fixes most test races
* Resolves data races
* Fixes subsystem test issues
* currency pair syncer coverage tests
* Refactors portfolio. Fixes tests. Withdraw validation
Portfolio didn't need to exist with a portfolio manager. Now the porfolio manager
is in charge how the portfolio is handled and all portfolio functions are attached
to the base instead of just exported at the package level
Withdrawal validation occurred at the exchange level when it can just be run at the
withdrawal manager level. All withdrawal requests go through that endpoint
* lint -fix
* golang lint fixes
* lints and comments everything
* Updates GCT logo, adds documentation for some subsystems
* More documentation and more logo updates
* Fixes backtesting and apiserver errors encountered
* Fixes errors and typos from reviewing
* More minor fixes
* Changes %h verb to %w
* reverbs to %s
* Humbly begins reverting to more flat engine package
The main reasoning for this is that the subsystem split doesn't make sense
in a golang environment. The subsystems are only meant to be used with engine
and so by placing them in a non-engine area, it does not work and is
inconsistent with the rest of the application's package layout.
This will begin salvaging the changes made by reverting to a flat
engine package, but maintaining the consistent designs introduced.
Further, I will look to remove any TestMains and decrease the scope
of testing to be more local and decrease the issues that have been
caused from our style of testing.
* Manages to re-flatten things. Everything is within its own file
* mini fixes
* Fixes tests and data races and lints
* Updates docs tool for engine to create filename readmes
* os -> ioutil
* remove err
* Appveyor version increase test
* Removes tCleanup as its unsupported on appveyor
* Adds stuff that I thought was in previous merge master commit
* Removes cancel from test
* Fixes really fun test-exclusive data race
* minor nit fixes
* niterinos
* docs gen
* rm;rf test
* Remove typoline. expands startstop helper. Splits apiserver
* Removes accidental folder
* Uses update instead of replace for order upsert
* addresses nits. Renames files. Regenerates documentation.
* lint and removal of comments
* Add new test for default scenario
* Fixes typo
* regen docs
