Files
gocryptotrader/engine/engine_types.go
Scott 197ef2df21 Feature: Data history manager engine subsystem (#693)
* Adds lovely initial concept for historical data doer

* Adds ability to save tasks. Adds config. Adds startStop to engine

* Has a database microservice without use of globals! Further infrastructure design. Adds readme

* Commentary to help design

* Adds migrations for database

* readme and adds database models

* Some modelling that doesn't work end of day

* Completes datahistoryjob sql.Begins datahistoryjobresult

* Adds datahistoryjob functions to retreive job results. Adapts subsystem

* Adds process for upserting jobs and job results to the database

* Broken end of day weird sqlboiler crap

* Fixes issue with SQL generation.

* RPC generation and addition of basic upsert command

* Renames types

* Adds rpc functions

* quick commit before context swithc. Exchanges aren't being populated

* Begin the tests!

* complete sql tests. stop failed jobs. CLI command creation

* Defines rpc commands

* Fleshes out RPC implementation

* Expands testing

* Expands testing, removes double remove

* Adds coverage of data history subsystem, expands errors and nil checks

* Minor logic improvement

* streamlines datahistory test setup

* End of day minor linting

* Lint, convert simplify, rpc expansion, type expansion, readme expansion

* Documentation update

* Renames for consistency

* Completes RPC server commands

* Fixes tests

* Speeds up testing by reducing unnecessary actions. Adds maxjobspercycle config

* Comments for everything

* Adds missing result string. checks interval supported. default start end cli

* Fixes ID problem. Improves binance trade fetch. job ranges are processed

* adds dbservice coverage. adds rpcserver coverage

* docs regen, uses dbcon interface, reverts binance, fixes races, toggle manager

* Speed up tests, remove bad global usage, fix uuid check

* Adds verbose. Updates docs. Fixes postgres

* Minor changes to logging and start stop

* Fixes postgres db tests, fixes postgres column typo

* Fixes old string typo,removes constraint,error parsing for nonreaders

* prevents dhm running when table doesn't exist. Adds prereq documentation

* Adds parallel, rmlines, err fix, comment fix, minor param fixes

* doc regen, common time range check and test updating

* Fixes job validation issues. Updates candle range checker.

* Ensures test cannot fail due to time.Now() shenanigans

* Fixes oopsie, adds documentation and a warn

* Fixes another time test, adjusts copy

* Drastically speeds up data history manager tests via function overrides

* Fixes summary bug and better logs

* Fixes local time test, fixes websocket tests

* removes defaults and comment,updates error messages,sets cli command args

* Fixes FTX trade processing

* Fixes issue where jobs got stuck if data wasn't returned but retrieval was successful

* Improves test speed. Simplifies trade verification SQL. Adds command help

* Fixes the oopsies

* Fixes use of query within transaction. Fixes trade err

* oopsie, not needed

* Adds missing data status. Properly ends job even when data is missing

* errors are more verbose and so have more words to describe them

* Doc regen for new status

* tiny test tinkering

* str := string("Removes .String()").String()

* Merge fixups

* Fixes a data race discovered during github actions

* Allows websocket test to pass consistently

* Fixes merge issue preventing datahistorymanager from starting via config

* Niterinos cmd defaults and explanations

* fixes default oopsie

* Fixes lack of nil protection

* Additional oopsie

* More detailed error for validating job exchange
2021-07-01 16:21:48 +10:00

108 lines
2.9 KiB
Go

package engine
import (
"sync"
"time"
)
// Settings stores engine params
type Settings struct {
ConfigFile string
DataDir string
MigrationDir string
LogFile string
GoMaxProcs int
CheckParamInteraction bool
// Core Settings
EnableDryRun bool
EnableAllExchanges bool
EnableAllPairs bool
EnableCoinmarketcapAnalysis bool
EnablePortfolioManager bool
EnableDataHistoryManager bool
PortfolioManagerDelay time.Duration
EnableGRPC bool
EnableGRPCProxy bool
EnableWebsocketRPC bool
EnableDeprecatedRPC bool
EnableCommsRelayer bool
EnableExchangeSyncManager bool
EnableDepositAddressManager bool
EnableEventManager bool
EnableOrderManager bool
EnableConnectivityMonitor bool
EnableDatabaseManager bool
EnableGCTScriptManager bool
EnableNTPClient bool
EnableWebsocketRoutine bool
EventManagerDelay time.Duration
Verbose bool
// Exchange syncer settings
EnableTickerSyncing bool
EnableOrderbookSyncing bool
EnableTradeSyncing bool
SyncWorkers int
SyncContinuously bool
SyncTimeoutREST time.Duration
SyncTimeoutWebsocket time.Duration
// Forex settings
EnableCurrencyConverter bool
EnableCurrencyLayer bool
EnableFixer bool
EnableOpenExchangeRates bool
EnableExchangeRateHost bool
// Exchange tuning settings
EnableExchangeHTTPRateLimiter bool
EnableExchangeHTTPDebugging bool
EnableExchangeVerbose bool
ExchangePurgeCredentials bool
EnableExchangeAutoPairUpdates bool
DisableExchangeAutoPairUpdates bool
EnableExchangeRESTSupport bool
EnableExchangeWebsocketSupport bool
MaxHTTPRequestJobsLimit int
TradeBufferProcessingInterval time.Duration
RequestMaxRetryAttempts int
// Global HTTP related settings
GlobalHTTPTimeout time.Duration
GlobalHTTPUserAgent string
GlobalHTTPProxy string
// Exchange HTTP related settings
HTTPTimeout time.Duration
HTTPUserAgent string
HTTPProxy string
// Dispatch system settings
EnableDispatcher bool
DispatchMaxWorkerAmount int
DispatchJobsLimit int
// GCTscript settings
MaxVirtualMachines uint
// Withdraw settings
WithdrawCacheSize uint64
}
const (
// MsgStatusOK message to display when status is "OK"
MsgStatusOK string = "ok"
// MsgStatusSuccess message to display when status is successful
MsgStatusSuccess string = "success"
// MsgStatusError message to display when failure occurs
MsgStatusError string = "error"
grpcName string = "grpc"
grpcProxyName string = "grpc_proxy"
)
// newConfigMutex only locks and unlocks on engine creation functions
// as engine modifies global files, this protects the main bot creation
// functions from interfering with eachother
var newEngineMutex sync.Mutex