* orderbook/buffer: data integrity and resubscription pass
* btcmarkets: REMOVE THAT LIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIINE!!!!!!!!!!!!!!!!!
* buffer: reinstate publish, refaactor, invalidate more and comments
* buffer/orderbook: improve update and snapshot performance. Move Update type to orderbook package to util. pointer through entire function calls. (cleanup). Change action string to uint8 for easier comparison. Add parsing helper. Update current test benchmark comments.
* dispatch: change publish func to variadic id param
* dispatch: remove sender receiver wait time as this adds overhead and complexity. update tests.
* dispatch: don't create pointers for every job container
* rpcserver: fix assertion issues with data publishing change
* linter: fixes
* glorious: nits addr
* depth: change validation handling to incorporate and store err
* linter: fix more issues
* dispatch: fix race
* travis: update before fetching
* depth: wrap and return wrapped error in invalidate call and fix tests
* btcmarkets: fix commenting
* workflow: check
* workflow: check
* orderbook: check error
* buffer/depth: return invalidation error and fix tests
* gctcli: display errors on orderbook streams
* buffer: remove unused types
* orderbook/bitmex: shift function to bitmex
* orderbook: Add specific comments to unexported functions that don't have locking require locking.
* orderbook: restrict published data functionality to orderbook.Outbound interface
* common: add assertion failure helper for error
* dispatch: remove atomics, add mutex protection, remove add/remove worker, redo main tests
* dispatch: export function
* engine: revert and change sub logger to manager
* engine: remove old test
* dispatch: add common variable ;)
* btcmarket: don't overflow int in tests on 32bit systems
* ci: force 1.17.7 usage for go
* Revert "ci: force 1.17.7 usage for go"
This reverts commit af2f95563bf218cf2b9f36a9fcf3258e2c6a2d91.
* golangci: bump version add and remove linter items
* Revert "golangci: bump version add and remove linter items"
This reverts commit 3c98bffc9d030e39faca0387ea40c151df2ab06b.
* dispatch: remove unsused mutex from mux
* order: slight optimizations
* nits: glorious
* dispatch: fix regression on uuid generation and input inline with master
* linter: fix
* linter: fix
* glorious: nit - rm slice segration
* account: fix test after merge
* coinbasepro: revert change
* account: close channel instead of needing a receiver, push alert in routine to prepare for waiter.
Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
* Bump CI versions
* Specifically set go version as 1.17.x bumps it to 1.18
* Another
* Adjust AppVeyor
* Part 1 of linter issues
* Part 2
* Fix various linters and improvements
* Part 3
* Finishing touches
* Tests and EqualFold
* Fix nitterinos plus bonus requester jobs bump for exchanges with large number of tests
* Fix nitterinos and bump golangci-lint timeout for AppVeyor
* Address nits, ensure all books are returned on err due to syncer regression
* Fix the wiggins
* Fix duplication
* Fix nitterinos
* gct: phase one context awareness pass
* exchanges: context propagation pass
* common/requester: force context requirement
* gctcli/exchanges: linter fix
* rpcserver: fix test using dummy rpc server
* backtester: fix comments
* grpc: add correct cancel and timeout for commands
* rpcserver_test: add comment on dummy server
* common: deprecated SendHTTPGetRequest
* linter: fix
* linter: turn on no context check
* apichecker: fix context linter issue
* binance: use param context
* common: remove checks as this gets executed before main
* common: change mutex to RW as clients can be used by multiple go routines.
* common: remove init and JIT default client. Unexport global variables and add protection.
* common: Add comments
* bithumb: after dinner mints fix
* websocket: select case error if no receiver, add in functionality to reset to initial sync for books on a new websocket connection
* websocket: fix tests
* websocket: log error instead of losing it
* websocket: fix whoopsie
* exchanges: fix test
* websocket: force requirement of specific functionality
* exchanges: fix tests
* exchanges/websocket: move waitgroup add before scheduling across exchanges
* gateio: add feature subscribe
* bithumb/bittrex: include connection state reset, fix reconnection bug for Bithumb
* huobi: Add listen to shutdown to routine so it actually returns and stops being a naughty boy.
* huobi: add missing waitgroup add.
* exchanges: bleed comms channels
* binance: fix reconnection bug with buffer
* bithumb: fix reconnection bug with ws orderbook when websocket is diabled/enabled
* bithumb/bittrex: add bleeders for ws orderbook jobs
* linter: fix
* kraken: reduce code block from double assertion
* This bug ruined my day.
* glorious: error checking
* zb: add correct path for websocket connection
* exchange: Add verbosity when config conflicts and overwrites default values
* zb: add https to path
* exchanges: glorious nits
* stream: Add checkAndSetMonitoring to reduce potential routine bundling, increase timeout and check state in tests
* stream: remove check that is not needed.
* glorious: nits addr.
* lint: test
* golangci: Enable err checking linter to expose unchecked errors.
* gct: handle errors across the board
* gct: handle errors NOTE: Found bug in FTX (WIP)
* linter: fix issues
* ftx/exchanges: fix bug where error was being returned when setting pair management variables to an already enabled state
* bitmex: fix bug where a dangly supported asset in config danglied up the place.
* linter: fix more linter issues
* linter: fix my terrible spelling.
* currency: fix test
* exchanges: fix tests
* logger: fix test
* exchanges: fix tests
* glorious: nits
* vm: revert rm variable and instigate test
* [FIX] Enable ws orderbook sync recovery by:
- Testing if books have been cleared
- Assigning options when loading snapshot
* orderbooks: remove setlastupdate method and on select depth method that updates linked list, this reduced lock contention across code base and fixes buffer bug on applying buffered updates
* WS - Introduce signaling for the need to fetch the orderbook
* Address nits
* Update error messages to include exchange name
Co-authored-by: shazbert <oharareid.ryan@gmail.com>
* Adds lovely initial concept for historical data doer
* Adds ability to save tasks. Adds config. Adds startStop to engine
* Has a database microservice without use of globals! Further infrastructure design. Adds readme
* Commentary to help design
* Adds migrations for database
* readme and adds database models
* Some modelling that doesn't work end of day
* Completes datahistoryjob sql.Begins datahistoryjobresult
* Adds datahistoryjob functions to retreive job results. Adapts subsystem
* Adds process for upserting jobs and job results to the database
* Broken end of day weird sqlboiler crap
* Fixes issue with SQL generation.
* RPC generation and addition of basic upsert command
* Renames types
* Adds rpc functions
* quick commit before context swithc. Exchanges aren't being populated
* Begin the tests!
* complete sql tests. stop failed jobs. CLI command creation
* Defines rpc commands
* Fleshes out RPC implementation
* Expands testing
* Expands testing, removes double remove
* Adds coverage of data history subsystem, expands errors and nil checks
* Minor logic improvement
* streamlines datahistory test setup
* End of day minor linting
* Lint, convert simplify, rpc expansion, type expansion, readme expansion
* Documentation update
* Renames for consistency
* Completes RPC server commands
* Fixes tests
* Speeds up testing by reducing unnecessary actions. Adds maxjobspercycle config
* Comments for everything
* Adds missing result string. checks interval supported. default start end cli
* Fixes ID problem. Improves binance trade fetch. job ranges are processed
* adds dbservice coverage. adds rpcserver coverage
* docs regen, uses dbcon interface, reverts binance, fixes races, toggle manager
* Speed up tests, remove bad global usage, fix uuid check
* Adds verbose. Updates docs. Fixes postgres
* Minor changes to logging and start stop
* Fixes postgres db tests, fixes postgres column typo
* Fixes old string typo,removes constraint,error parsing for nonreaders
* prevents dhm running when table doesn't exist. Adds prereq documentation
* Adds parallel, rmlines, err fix, comment fix, minor param fixes
* doc regen, common time range check and test updating
* Fixes job validation issues. Updates candle range checker.
* Ensures test cannot fail due to time.Now() shenanigans
* Fixes oopsie, adds documentation and a warn
* Fixes another time test, adjusts copy
* Drastically speeds up data history manager tests via function overrides
* Fixes summary bug and better logs
* Fixes local time test, fixes websocket tests
* removes defaults and comment,updates error messages,sets cli command args
* Fixes FTX trade processing
* Fixes issue where jobs got stuck if data wasn't returned but retrieval was successful
* Improves test speed. Simplifies trade verification SQL. Adds command help
* Fixes the oopsies
* Fixes use of query within transaction. Fixes trade err
* oopsie, not needed
* Adds missing data status. Properly ends job even when data is missing
* errors are more verbose and so have more words to describe them
* Doc regen for new status
* tiny test tinkering
* str := string("Removes .String()").String()
* Merge fixups
* Fixes a data race discovered during github actions
* Allows websocket test to pass consistently
* Fixes merge issue preventing datahistorymanager from starting via config
* Niterinos cmd defaults and explanations
* fixes default oopsie
* Fixes lack of nil protection
* Additional oopsie
* More detailed error for validating job exchange
* Update Bittrex API from v1.1 to v3
V1.1 has been retired as of 9/30/2020
- Update REST to V3
- Add initial websocket support
* Bittrex update - enable websockets in testdata config file
* Update Bittrex - add Websocket capability to docs
* Update Bittrex connector
- AppVeyor warnings
- Update tests
- Generate documentation
- Fix nits
* Update Bittrex - add websocket order processing
* Update Bittrex connector
* Bittrex connector - fix ineffectual err assignment
* Fix nits
* Orderbook synchronization
* Remove redundant nil
* Log WS fetch orderbook message as debug message instead of as warning
* Update after rebase
* Add tests
* Add allowed candle interval values
* Replace literals with declared constants
* Replace variable name 'request' with 'req'
* Add check and update for deprecated REST URL
* Nits and some cleaning up
* Change ParseInt bit size to 64
* [FIX] Remove several shadow declarations
* Do not export constructTicker
* Remove parseTime()
* Update GetHistoricCandles()
* [FIX] Address gocritic nits
* [FIX] Address gocritic nits
* Use SendMessageReturnResponse() instead of local map
* Rate limit subscribing and unsubscribing
* [FIX] use go routine for subscribing and unsubscribing
* [FIX] Set correct index for map
* [FIX] Address unused vars, literals, time format
* Adjusted timing when subscribing to many order books
* Cache partial updates to tickers instead of calling REST function
* [FIX] Update sequence nr when multiple updates are queued
* Address golint issues
* Fix nits
