* backtester/request: trying to fix panic (WIP)
* request: fix race for transport layer
* request: linter issue fix
* request: more linter issues
* requester: Add function to remove the tracking of underlying http client and add to engine unload exchange.
* request: add more context to error return
* request: Fix after cherry pick issues
* request: fix niterinos
* exchanges: change return to package variable
* request: changed named
Co-authored-by: Ryan O'Hara-Reid <ryan.oharareid@thrasher.io>
* golangci-lint/CI: Bump versions
Fix remaining linter issues
* Specifically set AppVeyor version
* Fix the infamous typos 👀
* Add go env cmd to AppVeyor
* Add go version cmd to AppVeyor
* Specify AppVeyor image, adjust linters
* Update go get to go install due to deprecation
* Bump golangci-lint timeout time for AppVeyor
* Change NW contract to NQ
* Address nitters
* GetRandomPair -> Pair{}
* Address nits
* Address time nitterinos plus additional tweaks
* More time inception upgrades!
* Bending time and space
* gct: phase one context awareness pass
* exchanges: context propagation pass
* common/requester: force context requirement
* gctcli/exchanges: linter fix
* rpcserver: fix test using dummy rpc server
* backtester: fix comments
* grpc: add correct cancel and timeout for commands
* rpcserver_test: add comment on dummy server
* common: deprecated SendHTTPGetRequest
* linter: fix
* linter: turn on no context check
* apichecker: fix context linter issue
* binance: use param context
* common: remove checks as this gets executed before main
* common: change mutex to RW as clients can be used by multiple go routines.
* common: remove init and JIT default client. Unexport global variables and add protection.
* common: Add comments
* bithumb: after dinner mints fix
* rate limits: Make context aware
* binance: rate limit allow for cancellation of reservation when deadline is exceeded
* request: add context.done() before initiating any bulk work.
* binance: update error return for rate limiting
* request: updated dealine check to remove after time.Now procedure as this will obfuscate a deadline which will be limited by the context check on every attempt, so no need to sleep with delay.
* requester: defer execution of request.Item generation in closure.
* bithumb: fix issue
* coinut/itbit: fix linter issues
* binance: fix bug on recvWindow setting
* requester: standardize sendpayload + add readme update
* nonce: remove inc() function
* request: defer unlockiflocked
* binance: revert changes for open orders
* btcmarkets: defer auth generation functionality, rm context deadline as this will be created just before sending HTTP request.
* binance: move const to top
* exmo: remove debug output as its generated in the requester function
* ftx: defer auth functionality
* requester: move error to top
* bittrex: defer auth functionality
* bitmex: defer auth functionality and remove deadline as generation occurs after rate limiting.
* btse: defer auth functionality
* coinbasepro: defer auth functionality and removed context deadline as this is generated after rate limiting
* coinbene: defer auth functionality and remove context deadline as this is generated after rate limiting
* huobi: defer auth functionality and remove context deadline as this is generated after rate limiting
* huobi-futures: defer auth functionality and remove context deadline as this is generated after rate limiting
* kraken: defer auth functionality and remove context deadline as this is generated after rate limiting
* kraken: remove deadline protection for timestamp generation
* okgroup: defer auth functionality and remove context deadline as this is generated after rate limiting
* poloniex: defer auth functionality
* zb: defer auth functionality and remove context deadline as this is generated after rate limiting
* exchanges: clean up log output which are done and inspected in the requester package
* binance: fix path bug on every retry, rm timeout context as this is not needed
* coinbene: fix path bug on retry
* binance: consolidate functionality
* coinbene: fix linter issues
* poloniex: linter fix
* kraken: change add -> set
* bitstamp: fix path bug for retry
* BTSE: fix retry path bug
* coinbene: fix path bug whoopsie by me
* gateio: fix bug where on retry it does not reset reader
* localbitcoins: fix path bug on retry
* zb: change domain to land
* exchanges: make sure io.Reader is generated every request
* exchanges: move reader generation into function scope
* wrapper_issues: setup exchange manager
* engine: expand withdraw manager test
* engine: dont look for environment
* bitstamp: fix pathing bug (@thrasher-)
* engine/withdraw_manager: purge tests as this is covered in repository withdraw
* Remove old concept. Introduce new job types and candle scaling
* Adds extra processing, commands
* new concept for queued jobs. Jobs can pause. New commands to manage status
* =End of day commit designing tables and implementing prerequisites further.
* Adds postgres data history relations
* Fixes table design for sqlite. Fixes all issues from merge
* Fixes craziness of database design. Adds some functions to get related jobs
* Fixes errors
* Updates some documentation, manages prerequisite jobs a little better, adds rpc funcs
* Fixes database design and adjust repo functions
* Tests database relationship
* Test coverage of new job functions
* Finishes coverage of new functions
* Commands and RPC coverage
* New database modifications for new job types
* Adds db support of new columns. Adds conversion validation. lint
* command blurb changes
* Allows websocket test to pass consistently
* Fixes merge issue preventing datahistorymanager from starting via config
* Minor fixes for different job type processing
* Fixes rangeholder issue, fixes validation, does not address jobs not starting or wrong status
* Fixes database tests, but at what cost. Fixes dhm tests
* Fixes dhj completion issue. Adds prerequisite by nickname
* Fixes validation processing. Adds db tests and validation
* Fixes validation job processing range
* Fixes trade sql. Reduces defaults. Validation processing and errors
* Updates cli job commands. adds validation decimal. fix job validation
* Expands run job handling and tests
* Validation work
* Fixes validation processing
* candle relations. new job type. updating database design
* Adds secondary exchange support. Sets stage for candle override
* Re adds accidentally deleted relationship
* Updates loading and saving candles to have relationship data when relevant
* Now validates and replaces candle data appropriately
* Fixes getting and setting datahistory data. Neatens DHM
* Test coverage
* Updates proto for new db types. New test coverage. Secondary exchange work
* Investigation into never-ending validation jobs. Now that intervals are ruled out, now need to complete the job....
* Fixes issues with validation job completion. Fixes validation volume issue for secondary exchange
* Adds candle warning support to the backtester
* Fixes warnings
* lint and begin docs
* Documentation updates. Final testing changes
* Minor fixes
* docs, prerequisite checks, more testing
* Fixes binance trade test. Rename err
* Documentation fixes. Figure fixes
* documentation update
* Fixes remote PSQL tests
* Fix binance mock test
* Remove unnecessary JSON
* regen proto
* Some minor nit fixes
* Var usage, query sorting, log improving, sql mirroring
* Extra coverage
* Experimental removal of m.jobs and mutex. Fix messaging
* Fixes error
* Lint fixes, command description improvements. More isRunning gates
* description improvements
* Lint
* BUFF regenerate
* Rough concept to fix insertions taking up long periods of time
* New calculation for trade data. Adds batch saving
This also adds an experimental request feature to shut down lingering requests. However, its uncertain whether or not this is having any impact. Initially thought it was the trades that was taking time and not SQL. Will investigate further
* Removes experimental requester. Adds documentation. Fixes typo
* rm unused error
* re-adds more forgotten contributors
* Now with proper commit count
* exchanges/request: Requester.doRequest() now always parses returned response body into JSON even in the case of an artificial error after the request itself
* exchanges/ftx: consider order cancellation successful under two new conditions, reported by the exchange: (1) order is already closed or (2) order is already queued for cancellation
* exchanges/ftx: fix a typo in a comment
* exchanges/request: keep the same behavior of doRequest() when there is an unmarshaling error
* exchanges/ftx: FTX.DeleteOrderByClientID now also reports no errors when requesting the cancellation of orders that are already canceled on the exchange
* exchanges/ftx: order deletion methods are now unified
* exchanges/ftx: DeleteOrder* methods now check if the given ID is not empty
* Initial codes for a trade tracker
* Moving everything in a broken fashion
* Removes tradetracker. Removes some errors for subsystems
* Cleans up some subsystems, renames stuttering types. Removes some global Bot usage
* More basic subsystem renaming and file moving
* Removes engine dependency from events,ntpserver,ordermanager,comms manager
* Exports eventManager, fixes rpcserver. puts rpcserver back for now
* Removes redundant error message, further removes engine dependencies
* experimental end of day interface usage
* adds ability to build the application
* Withdraw and event manager handling
* cleans up apiserver and communications manager
* Cleans up some start/setup processes. Though should separate
* More consistency with Setup Start Stop IsRunning funcs
* Final consistency pass before testing phase
* Fixes engine tests. Fixes stop nil issue
* api server tests
* Communications manager testing
* Connection manager tests and nilsubsystem error
* End of day currencypairsyncer tests
* Adds databaseconnection/databaseconnection_test.go
* Adds withdrawal manager tests
* Deposit address testing. Moved orderbook sync first as its more important
* Adds test for event manager
* More full eventmanager testing
* Adds testfile. Enables skipped test.
* ntp manager tests
* Adds ordermanager tests, Extracts a whole new subsystem from engine and fanangles import cycles
* Adds websocket routine manager tests
* Basic portfolio manager testing
* Fixes issue with currency pair sync startup
* Fixes issue with event manager startup
* Starts the order manager before backtester starts
* Fixes fee tests. Expands testing. Doesnt fix races
* Fixes most test races
* Resolves data races
* Fixes subsystem test issues
* currency pair syncer coverage tests
* Refactors portfolio. Fixes tests. Withdraw validation
Portfolio didn't need to exist with a portfolio manager. Now the porfolio manager
is in charge how the portfolio is handled and all portfolio functions are attached
to the base instead of just exported at the package level
Withdrawal validation occurred at the exchange level when it can just be run at the
withdrawal manager level. All withdrawal requests go through that endpoint
* lint -fix
* golang lint fixes
* lints and comments everything
* Updates GCT logo, adds documentation for some subsystems
* More documentation and more logo updates
* Fixes backtesting and apiserver errors encountered
* Fixes errors and typos from reviewing
* More minor fixes
* Changes %h verb to %w
* reverbs to %s
* Humbly begins reverting to more flat engine package
The main reasoning for this is that the subsystem split doesn't make sense
in a golang environment. The subsystems are only meant to be used with engine
and so by placing them in a non-engine area, it does not work and is
inconsistent with the rest of the application's package layout.
This will begin salvaging the changes made by reverting to a flat
engine package, but maintaining the consistent designs introduced.
Further, I will look to remove any TestMains and decrease the scope
of testing to be more local and decrease the issues that have been
caused from our style of testing.
* Manages to re-flatten things. Everything is within its own file
* mini fixes
* Fixes tests and data races and lints
* Updates docs tool for engine to create filename readmes
* os -> ioutil
* remove err
* Appveyor version increase test
* Removes tCleanup as its unsupported on appveyor
* Adds stuff that I thought was in previous merge master commit
* Removes cancel from test
* Fixes really fun test-exclusive data race
* minor nit fixes
* niterinos
* docs gen
* rm;rf test
* Remove typoline. expands startstop helper. Splits apiserver
* Removes accidental folder
* Uses update instead of replace for order upsert
* addresses nits. Renames files. Regenerates documentation.
* lint and removal of comments
* Add new test for default scenario
* Fixes typo
* regen docs
* Adds potential fix for websocket reconnection failure
* Addr tests, we now don't return an error, this allows us to reuse existing if still in operation.
* update depends && go mod tidy
* adds in channel direction for parameter
* Add full subscriber function, increased test coverage, initiate go routine after calling routine instance check in connection monitor
* fix linter issue
* use protected methods for setting field variables
* removed function, added tests
* lock sub manipulation
* fix linter issue
* Added in transport idleconnection timeout to fix MACOS reconnection issue when all idle connections are consuming resources
* used protected methods to set underlying fields
* set variable via time.Duration param
* Added in lock around field variable in test
* Addr thrasher nits and expanded exchange tests
* Fix test
* Addr glorious nits
* go mod tidy
* Add a larger timeout for traffic monitor if the test runs slow
* Add in initial header pass back for package side inspection
* change to ptr to point to correct ptr value to ensure len and cap values; dereferences so we don't need to string join the value iteration coming from main header
* Add tests
* More tests added for edge case
* change field name
* remove unnecessary coversion
* changed return and comment
Refactoring the timeout retries into a more general 'retry policy' with
support for retrying on HTTP 429 (Too Many Requests) and other responses
with a `Retry-After` header
The delay between requests is controlled by a combination of a 'backoff'
(currently only a simple linear backoff), and honouring the
`Retry-After` value (longest delay wins)
This makes the 'rate limiter' an optional argument as well, removing the
use of `nil` when one isn't supplied
Signed-off-by: David Ackroyd <daveo.ackroyd@gmail.com>
Request types / variations contribute different weights towards the limit
Binance enforces. These can be considerably more than 1 per request,
which results in the server side limits being hit, producing 429 and 418
responses and bans
* Initial changes, removing exchange name as an arg and puts it in the pointer struct. Adds case to ws routines
* Adds CancelAllOrders func, adds GetByExchangeAndID. Adds modify handler in routines.go
* initial poor attempts to have bitmex work with new datahandler handlers. fixes ordersides
* bitmex Completes new order
* Better bitmex handling, but not complete. Begins a gargantuan task of unifying order data structs. Sometimes an order update will contain lot's of information, so its best to be able to update all fields of our orders, rather than just an arbitrary subset. As a result, everything will be broken for the foreseeable future :glitch_crab:
* Removes old order handler which did nothing. Updates order properties for everything everywhere - now consistent. Changes order status. Adds asset type and wallet address to all order types
* Adds order updater to update only relevant fields since the object is generic, we don't know what fields are passed from what exchanges. Adds "lastupdated" field to order.Detail. Expands order cancellation for engine orders.
* Ensures that new orders are added to the ordermanager's order store. Saaa many comments. Internalises orderStore get func. Adds internalOrderID to orderdetail and adds websocket support for it
* Fixes a cancelAllOrders oopsie doopsie
* Adds potential func to update orderdetails from an orderdetail struct. Unsure if will keep.
* Begins btcmarkets implementation. Expands order "stringToOrder" funcs to allow for some more flexible string coversions. Removes order.Submit via websocket as it would cause unlimited order place issues :D
* Finishes btc markets without testing
* Adds untested ws auth func to btse
* Finises btse, fixes btcmarkets bug
* Adds coinbasepro support
* Fixes a few more fields in coinbase pro and readds the extra subs
* Begins work on coinbene. Plus theyve added a new ws connection yeee
* Wasted a bunch of time adding support to an additional websocket that isn't needed ;_; Fixed a bug in coinbasepro. Fully kitted out coinbene support. Updates order types with all fields
* Removes extra websocket connection ;_;
* Finishes gemini. Fixes order side unknown
* Adds okgroup support. Moves byte reading to another function to allow for unit testing. Updates routines to use pointers. Updates date update handling for order details
* Finishes order data for okgroup websocket, but starts the STRANGE process of converting all other websocket endpoints to be a little less silly
* Cleans up okroup websocket implementation. Fixes bug in Gemini
* Adds poloniex support. Updates ws order handling
* new bitmex support. Adds some tests now that its all in its own func. Fixes poloniex bug
* Begins work on authenticated binance websocket
* Attempts to track user data via binance websocket
* Maybe finishes Binance websocket support
* Begins adding test coverage to orders.go. Updates names of script properties to match updated
* Begins an experiment with code coverage. Fixes more rebase issues
* Completes orders coverage. Botches a few other things though. Fixes more scripting stuff
* All tests in engine package pass
* Adds some loevely routine tests
* Moves ordermanager to test Bot ordermanager
Adds lovely routine tests to ensure things that get sent to be handled the data handler are handled by the data handler by handling them
* Replaces "wsHandleData" with "wsReadData" as that's what its going to do now.
* Splits all wsHandleData into wsReadData and wsHandleData to allow for easy testing via sending []byte json examples to test proper functionality. Breaks so many tests
* Fixes majority of test issues. But data races which are tough on the engine package
* "Fixes" test by removing shutdown test. It interferes with too many things. Requires some thought
* Tests all the binance websocket points
* Adds better bitfinex websocket support.
* Adds testing for bitfinex, bitstamp and btcmarkets. Fixes websocket bugs encountered
* Adds BTSE ws tests. Fixes bugs in ws
* Adds coinbase pro tests. Fixes any issues
* Coinbene tests
* Starts to handle coinut. Runs into a problem conceptually regarding websocket roundtrip and orders. Both events need to happen without impacting eachother/racing
* Addresses a data race issue regarding websocket and bot order management submission - order submission locks at an earlier point to prevent routines.go from creating an order before order submission creates it. Updates rpcserver to use order management bot to submit orders.
* Finishes the hectic coinut testing
* Adds tests for gateio
* Fixes rebase issues. Updates tests to work without being overloaded
* Begins testing of gemini. fixes up minor issues
* ginishes gemini tests and fixes
* Adds hitbtc tests. Fixes all the many issues with hitbtc websocket
* Adds remaining tests. Increases default test channel limit again
* Begins work towards huobi tests
* Finishes huobi tests
* Fixed all mythical rebase adventures
* Begins kraken transformation
* Finishes kraken. Fixes coinbene leverage now that its changed
* Begins okgroup testing
* Adds okgroup ws tests
* Does some poloniex
* Fixes basic curreny issue by extracting to func
* Begins redesign of poloniex websocket datahandling. Completes authenticated handling, now onto unauth
* Finishes poloniex revision
* Finishes ZB additions
* Fixes data races
* Fixes rebase issues. Fixes bad kraken logic
* Fixes after reviewing code
* lint everywhere
* Fixes lingering lints
* lint
* Adds test coverage to order detail and modify updating
* Fixes linting
* Fixes huge int, fixes date tests
* Adds GetByExchange, adds test for it. Protects fakepass echange. Renames DisplayQty to DisplayQuantity. Removes verbose. Adds some websocket properties. Updates bitmex asset type in test
* Addresses timestamps, type abbreviations, verbosity. Expands binance kline switch cases. Updates some websocket capabilities.
* Adds coverage to the stringToOrderType/Status functions introduced in PR
* Minor fixes addressing some time, error text and use of StringDataCompareInsensitive
* Introduces shiny new system which checks if there is an awaiting ID, if found, processes via wrapper method, else, goes through wsHandleData method. Removes weird locking system from wrapper/websocket data race. Updates bitfinex to properly handle websocket order requests and notifications
* Moves fakePassingExchange to test_helper. Fixes some order side implementations for trades. Botches a new error type
* Adds new error type to track and handle order classification errors separately
* Fully fleshes out ClassificationError for all instances of status conversion. Even in order trades and some wrapper functions
* Introduces common.SimpleTimeFormat for "2006-01-02 15:04:05". Fixes binance and bitfinex issues with auth endpoint use, map casting. Expands more order.ClassificationError usage. Fixes some more generic websocket response errors
* Future proofs order updating by utilising asset types. Expands testing to accomodate. Adds shiny new time type. Expands wrapper websocket functionality definitions
* minty linty
* Broken end of day code addressing basic nits on comments, returns and currency conversion
* Adds testing to btcmarkets websocket. Also updates websocket orderbook to use update instead
* Fixes fun rebase fun fun so fun
* Addresses minor nits regarding changed interface and comments
* Creates new function `GetRequestFormattedPairAndAssetType` to retrieve a currency pair and asset type based on a string. It will iterate over enabled pairs and compare them to formatted pairs and then return that pair if found.
* Fixes test
* Adds a single line to the end of the file, because that would be really bad if it wasn't there
* Updates fakepassexchange to not use params, updates test params, uses fatal in some tests where its important, updates order manager to have a rwmutex, removes some returns, improves ws key test for binance, updates properties to reflect their actual values, adds some more websocket properties
* Addresses binance switch linting
* Updates leverage property to int64
* Fixes what was broken
* renamed package to log to stop side import requirement
* reverted comment changes
* reverted comment changes
* one more reverting wording back to logger
* wording changes on comments
* Initial rework of rework of requester - WIP
* Implementing and checking rate limits - WIP
* implemented coinbene rate limiting shenanigans
* add in remaining WIP
* fixy
* use authenticated rate limit
* drop ceiling as this can be done with a counter later
* add functionality to struct
* purge config options for rate limiting so as to keep things minimal
* prepare futures and swap rate limiting for implementation
* Address linter issues
* Addressed nits, fixed race
* fix linter issue
* remove global var as this was only setting when newrequester was called
* moved rate limit functionality into its own file
* Update Bitfinex with correct rate limit and test endpoints (WIP)
* finish off bitfinex adjustments
* fixes
* fix linter issues
* slowed rate for coinbasepro
* drop rate limit for huobi as the doc times have intermittent 429 issues.
* Set MACOSX_DEPLOYMENT_TARGET to remove linking warning
* Addr Thrasher nits
* Addr glorious nits
* unexport do request function
* fixed nitorinos
* Fixed something I missed
* move disabled rate limiter into loadexchange and use interface functionality
* Add temp quick fix
* expose auth validator functionality for wrapper
* Add REST validation after keys set, package account types for future syncing
* Add transient error checking for initial creddemtial validation
* fix command types
* Addressed nits from glorious person
* Amalgamate body within error when not between 2xx status, added btcmarket specific auth error check
* nit fix for glorious person
* Format fix
* removed unused code
* check transient first then validate if its an exchange specific authentication error, all others will be disregarded
* Addressed glorious nits
* Addressed glorious nits
* Moved account processing to updateaccountinfo func and added in fetch account info
* Add GRPC Account streaming (NOTE: could not complete until sync item added)
* RM exchange check
* Address xtda nits
* RM comment code
* Fix linter issues
* used most recent protoc version
* lbank linter issues fixed
* Addressed nits and changed len check to range in for loops
* Fixed timeout issue
* thrasher nits addressed
* add string holdings
* drop common uuid v4 func and imported package as needed
* removed common functions regarding json marshal and unmarshal and used the json package directly. WRT unmarshal it was calling reflect and converted to string which is also checked in the JSON package so it was doing a double up, this will be a tiny gain as it was directly used in the requester package for all our outbound requests.
* add in string
* explicitly throw away return error value
* atleast return the error that websocket initialise returns
* return error when not connected
* fix comment
* Adds comments
* move package declarations
* drop append whenever we call supported
* remove unused import
* Change incorrect spelling
* fix tests
* fix go import issue
* Improved error message when no config is set on startup
* Change inccorect error wording
* bump Bitfinex websocket orderbook return length to max
* temporary fix of incorrect orderbook updates, limit to bid and ask len of 100, will be extended later if needed
* Fixed issue in binance websocket that appended 0 volume bid/ask items
* Fix panic when unmarshalling an empty pair from config
* Add get pair asset method for exchange base
Fix Bitmex orderbook stream
Unbuffer Bitmex orderbook stream
* force syncer to update ticker instead of fetch, which allows a stream
* Fix websocket last price for coinbasepro
* fix websocket ticker for coinut
* Fix websocket orderbook stream Huobi
* increase orderbook depth REST for Huobi
* Fix websocket support and ensure data integrity
* Fix time parsing issue after error checks
* check error, only process enabled currency pairs, signal websocket data processing
* expanded websocket functionality for okgroup
* Add logic to not process zero length slice for orderbooks
* fix websocket ticker only updating enabled and individual book updates
* ZB fixes to order submission/retrieval/cancellation w/ general fixes
* Quiet unnecessary warning
* updated config entry values for REST and websocket (initial hack until I come up with a better solution for asset types)
* Ch GetName function to field access modifyer & rm useless code
* Add in error I missed
* Nits addressed
* some more fixes
* Turned kraken default websocket to true and some small changes
* fixes linter issues
* Ensured okgroup books and sent update through to datahandler. Zb update as well.
* Add test case to get asset type from pair
* Add test for pairs unmarshal
* Add testing and addressed nits
* FIX linter issue
* Addressed Gees nits
* Thanks glorious spotter
* more nitorinos
* Addres even more nits
* Add stringerino 4000
* Fix for panic cause by sort slice out of range, also nits addressed
* fix linter issues
* Changed from function to field access
* Changed from function to field access
* fix for orderbook update panic, removes quick fix - caused by sync item fetching through same protocol
* Add new test and update random generator
* pass in invalid string to future ob fetching, due to futures contract expire and a http 400 error is returned
* Removes lock unlock timer and instead sets unlocks between getting a nonce and sending a payload. Increases dispatch channel buffer to deal with len(enabledCurrencies) > ~100
* Adds additional comments to help explain the situation
* Fixes bug that could unlock mutex too early
* Fixes LIES where Gemini gets a nonce and then proceeds to declare it doesn't get a nonce causing an unrecoverable lock
* Fun new concept! The creation of a tested timed mutex. Unlocking an unlocked mutex cannot occur and response can be checked to verify whether the mutex was unlocked from timeout or command.
* Adds new cmd parameter "dispatchjobbuffer"
* Expands comments and renames benchmark. Makes `Timer` property private
* Happy little linters
* Renames jobBuffer and all related instances to jobs limit
* Tiny error message update
* Grammatical fix and setting dispatch.Start to use defaults
* First pass at adding new logging system
* NewLogger
* NewLogger
* WIP
* silly bug fix
* :D removed files
* removed old logging interface
* added tests
* added tests
* Started to add new lines to all f calls
* Added subsystem log types
* Logger improvements
* Further performance improvements
* changes to logger and sublogger creation
* Renamed Logging types
* removed old print statement
* changes based on feedback
* moved sublogger types to own file
* :)
* added console as output type
* added get level command
* added get/set log level via grpc command
* added check for output being empty for migration support
* first pass at log rotation
* added log rotation
* :D derp fixed
* added tests
* changes based on feedback
* changed log type
* comments
* renamed file -> fileSettings
* typo fix
* changes based on feedback
* gofmt ran on additional files
* gofmt ran on additional files
* Step one: Sets up connection handler for websockets to always be connected until a shutdown event is received.
Sets up a vague subscription handler to ensure subscriptions are subscribed
* Adds support for resubscriptions for bitfinex, bitstamp, bitmex and btcc. Adds subscription params for special websocket subscription requirements. Removes subscription monitor from wait group so that it can exist despite a shutdown and continuously check
* Adds channel subscription support to bitmex, btse, coibasepro, coinut, gateio, gemini, hitbtc, huobi, hadax, kraken, okgroup, poloniex and zb
* Implements unsubscribe for bitfinex, btcc, btse, coinbasepro, gateio, gitbtc, huobi, hadax
* ManageSubscriptions now called from WSConnect and made private instead of inside individual exchanges. ManageSubscriptions can now unsubscribe. exchange_websocket_types.go now contains all exchange_websocket.go types to avoid clutter
* Adds it to websocket functionality so managesubscriptions will close when not supported
* Separates functions into testable functions to ensure logic works. Adds tests. Updates websocket setup to include verbosity (inherited from exchange). Adds no connection tolerance to fatal on failed reconnects
* More exchange_websocket tests. Updating to use pointers. Creation of equals func to make comparison easier
* Fixes okex, okcoin tests. Fixes race conditions. Removes pointer usage again.
* Adds subscribe and unsubscribe to wrappers
* Fixes deadlock. Fixes ws verbosity.
* Updates all exchanges to properly support subscription/connection feature. Also reintroduces race conditions....
* Moves connection varialbes to struct from package to allow each websocket to have their own reconnection checks. Neatens up logs
* Fixes lint/critic issues. Fixes tests. Removes unused function.
* Moves websocket ratelimiter to their own const variables. Fixes more race conditions with connecting variable
* Removes redundant subscribe functions. Ensuring only the exchange_websocket.go can manage subscriptions. Fixes debug logs to be verbose wrapped
* Fixes issue with slice copying. Re-adds okgroup default channels
* Adds nolint to append
* Adds comments and adds support for gateio auth request subscriptions
* Adds new test to ensure slices dont point to the same vars
* removes fatals. gofmt goimports
* more gofmts
* Addresses PR comments, removing empty and redundant lines
* Addresses PR comments. Ensures that writing to the websocket is single-threaded by adding a mutex to exchanges. Minimises wrapper code and moves subscription loops to exchange_websocket. Privatises ChannelsToSubscribe, Connecting properties and removeChannelToSubscribe func to prevent unnecessary tampering.
* Removes unused mutex. FMTS and IMPORTS
* Fixes request lock time change
* More specific logs
* Renames ws mutex. Fixes bitmex subscriptions. Increased gateio ratelimiter to 120ms. Removes ratelimiter from bitfinex, bitmex, bitstamp, btcc, btse, coibasepro, hitbtc, huobi, hadax, poloniex and zb
* changes recieved typo due to not being well received
* Fixes parsing issue with Huobi and hadax
* Fixes data race with more locks
* removes defer locks. fixes huobi/hadax verbose output
* Fixes double JSONEncode for coinut. Fixes verbose output for coinut
* gofmt,goimport for coinut
* Fixes issue where multiple connection monitors can spawn
* Removes defer exchange.WebsocketConn.Close() in defer handledata exit as connectionmonitor handles connections instead
* gofmt and go import
* More fmts
* First pass adding DumpResponse
* WIP
* Added httpDebug flag to SendPayload
* anx -> coinbase converted
* HTTPDebug flag added to all exchanges
* Updated Alphapoint:
* reworded error message
* Corrected itbit and huobihadax sendpayload
* Removed setting of default value
* don't return body for response as we already get this in verbose mode
* body is now included based on verbose setting
* WIP
* Due to issues with DumpResponse and gzip decoding use raw response from exchange for body output Also adds gzip decoding back as bug fix for OK group exchange
* removed additional new line
* Added URL path to dumpbody display as well
* Updates nonce generation to adhere to fifo channel buffer before request executes by routine
* removed unused variables, lns etc
* Fix requested changes and added in timer that disengages lock if out of scope error occurs
* Fixed woopsy daisy issue
* Add benchmark, reduce time in force to unlock before stack insertion, add nil check for edge case
* Remove unusued waitgroup field
* use return nonce.Value and method, rm redundant nonce code, fix tests.
* Fix linter issue: unnecessary conversion
* Updates requester package to allow unpacking of zipped files and defaults to warn if no JSON is present
* Initial addition of coinmarketcap functionality
* fix requested changes
* Fix issue with displaying false positive in request.go && reorder plan list
* Rename CurrencyProvider -> CryptocurrencyProvider
Skip seeding currency data if not enabled
Rm line in main.go
* Update test procedures and relevant json files
* Fix const issue within config.go
* Added new base logger
* updated example and test configs
* updated exchange helpers restful router & server
* logPath is now passed to the logger to remove dependency on common package
* updated everything besides exchanges to use new logger
* alphapoint to bitmex done
* updated bitmex bitstamp bittrex btcc and also performance changes to logger
* btcmarkets coinbase coinut exmo gateio wrappers updated
* gateio and gemini logger updated
* hitbtc huobi itbit & kraken updated
* All exchanges updatd
* return correct error for disabled websocket
* don't disconnect client on invalid json
* updated router internal logging
* log.Fatal to t.Error for tests
* Changed from fatal to error failure to set maxprocs
* output ANSI codes for everything but windows for now due to lack of windows support
* added error handling to logger and unit tests
* clear wording on print -> log.print
* added benchmark test
* cleaned up import sections
* Updated logger based on PR requests (added default config options on failure/setting errors)
* ah this should fix travici enc config issue
* Load entire config and clear out logging to hopefully fix travisci issue
* wording & test error handling
* fixed formatting issues based on feedback
* fixed formatting issues based on feedback
* changed CheckDir to use mkdirall instead of mkdir and other changes based on feedback
* Changed IBotExchange interface ModifyOrder function paramater and return value to exchange type for easier addition or retraction of variables.
* Function ModifyOrder for Binance not supported via API
* Change to unsupported function for exchange ZB
* Change to unsupported function for exchange Yobit
* Add modify order support for Poloniex
* Change to unsupported function for exchange Okex
* Change to unsupported function for exchange Localbitcoins
* Change to unsupported function for exchange Liqui
* Change to unsupported function for exchange LakeBTC
* Change to unsupported function for exchange Kraken
* Change to unsupported function for exchange Itbit
* Change to unsupported function for exchange HuobiHadax
* Change to unsupported function for exchange Huobi
* Change to unsupported function for exchange HitBTC
* Change to unsupported function for exchange Gemini
* Change to unsupported function for exchange GateIO
* Change to unsupported function for exchange Exmo
* Change to unsupported function for exchange Coinut
* Change to unsupported function for exchange Coinbase
* Change to unsupported function for exchange BTCMarkets
* Change to unsupported function for exchange Bittrex
* Change to unsupported function for exchange Bitstamp
* Add modify order support for Bitmex
* Add verbose header information in request package
* Add modify order support for Bithumb exchange
* Change to unsupported function for exchange Bitflyer
* Change to unsupported function for exchange Bitfinex
* Change to unsupported function for exchange ANX
* Change interface function signature
* Rm redundant code for authenticated requests in Bithumb
* Add error check if decimal values supplied for create or modifying an order on Bitmex
* Added test functions across the exchanges
* Rm comment for modify order on Alphapoint exchange
* Update tmpl file for exchange wrapper
* Reimplements order cancellation for alphapoint, anx, binance, bitfinex, bithumb, bitmex, bitstamp, bittrex, btcmarkets, coinbasepro, coinut, exmo, gateio, gemini, gitbtc, huobi, hadax, itbit, kraken, lakebtc, liqui, okcoin, okex, poloniex, wex, yobit and zb wrappers. Adds new order cancellation struct type. Updates old tests that pointed to the wrong unrenamed methods
* Sets up tests for all supported exchanges. request.DoRequest errors when response status is not 200
* Updates alphapoint, coinut, hitbtc, lakebtc cancel order implementations. Finishes testing
* Adds localbitcoin cancel order wrapper support
* Fixes tests and build issues. Adds WexIssue flag for tests
* Changes CancelOrder signature to only return error. Allows exchange to format currency pairs with delimiters
* Initial overhaul of websocket connection and feeds
* Added proxy support
* Piped to routines.go
* Added new websocket file in exchanges
Refactored orderbook handling into exchange_websocket.go
Added better error responses for binance_websocket.go
General clean for binance_websocket.go
* General fixes - bitfinex_websocket.go
Refactored orderbook cache code - bitfinex_websocket.go
Removed fatal error with unhandled type - routines.go
* Added general improvements to bitmex_websocket.go
Refactored orderbook handling to exchange_websocket.go
Added variable in Item struct in orderbook.go for looking up orders by ID
* Fix issue when routines are blocked due to Data Handler not started
Updated traffic handler
General fixes for bitstamp_websocket.go
* General fixes for coinbasepro_websocket.go
* General fixes for coinut_websocket.go
Fixed error return in exchange_websocket.go
* Removed comments in coinut_wrapper.go
Refactor orderbook logic from hitbtc_websocket.go to exchange_websocket.go
* General fixes
* Removed comments
General fixes
* Updated routines.go
* After rebase fix
* Fixed update config pairs in okcoin.go
* fixed config currency issue in okcoin.go for okcoin China
* exchange_websocket.go
*Removed unused const dec
*Removed state change routine
*Improved trafficMonitor routine
*Increased verbosity for error returns
*Removed uneeded mutex locks
exchange_websocket_test.go
*Added new tests for websocket and orderbook updating
routines.go
*Removed string cased
* Fixed race conditions on sync.waitgroup in exchanges_websocket.go
* Changes variable name in config.go
* Removes unnecessary comment
* Removes indefinite lock on error return
* Removes unnecessary comment
* Adds support for BTCC websocket
Drops support for BTCC REST
* Rewords comment in exchange_websocket.go
Moves types to poloniex_types.go
* Moves types to coinut_types.go
* Removes uneeded range for accessing array variables for coinbase_websocket.go
Removes comments in coinut_types.go
* Adds verbosity flag to GCT
Suppresses verbose output from routines.go
* Fixes setting proxy for REST and Websocket per exchange
Upgrades error handling
Drops unused *url.Url variable in exchange type
* Adds test for setting proxy
* Fixes bug that closes connection due to incorrect timeout time through a proxy connection
* Clarify verbose flag message
- Modifications made to the request package. Planned improvements will be
sending requests on intervals, rate limiter back off support, dynamic tuning
and requests packaged into a request job group.
- Can modify each exchanges individual HTTP client (e.g timeout and
transport settings).
- Bot now uses an exchange config HTTP timeout value.
- Bot now uses a global HTTP timeout (configurable).
- Batched ticker request support for exchanges.
- Ticker and Orderbook fetching now are spanned accross multiple
go routines and regulated by a sync wait group.
- Fixes hack used to load exchanges, now uses a sync wait group.
- Ticker and Orderbook storage and fetching now uses mutex locks.
- New pair function for finding different pairs between two supplied
pair arrays. This is used for currency pair updates for exchange which
support dynamic updating.
- Shows removal/additions of dynamic updates currencies.
