Commit Graph

23 Commits

Author SHA1 Message Date
Scott
5ea5245afb Improvement: Subsystem separation (#664)
* Initial codes for a trade tracker

* Moving everything in a broken fashion

* Removes tradetracker. Removes some errors for subsystems

* Cleans up some subsystems, renames stuttering types. Removes some global Bot usage

* More basic subsystem renaming and file moving

* Removes engine dependency from events,ntpserver,ordermanager,comms manager

* Exports eventManager, fixes rpcserver. puts rpcserver back for now

* Removes redundant error message, further removes engine dependencies

* experimental end of day interface usage

* adds ability to build the application

* Withdraw and event manager handling

* cleans up apiserver and communications manager

* Cleans up some start/setup processes. Though should separate

* More consistency with Setup Start Stop IsRunning funcs

* Final consistency pass before testing phase

* Fixes engine tests. Fixes stop nil issue

* api server tests

* Communications manager testing

* Connection manager tests and nilsubsystem error

* End of day currencypairsyncer tests

* Adds databaseconnection/databaseconnection_test.go

* Adds withdrawal manager tests

* Deposit address testing. Moved orderbook sync first as its more important

* Adds test for event manager

* More full eventmanager testing

* Adds testfile. Enables skipped test.

* ntp manager tests

* Adds ordermanager tests, Extracts a whole new subsystem from engine and fanangles import cycles

* Adds websocket routine manager tests

* Basic portfolio manager testing

* Fixes issue with currency pair sync startup

* Fixes issue with event manager startup

* Starts the order manager before backtester starts

* Fixes fee tests. Expands testing. Doesnt fix races

* Fixes most test races

* Resolves data races

* Fixes subsystem test issues

* currency pair syncer coverage tests

* Refactors portfolio. Fixes tests. Withdraw validation

Portfolio didn't need to exist with a portfolio manager. Now the porfolio manager
is in charge how the portfolio is handled and all portfolio functions are attached
to the base instead of just exported at the package level

Withdrawal validation occurred at the exchange level when it can just be run at the
withdrawal manager level. All withdrawal requests go through that endpoint

* lint -fix

* golang lint fixes

* lints and comments everything

* Updates GCT logo, adds documentation for some subsystems

* More documentation and more logo updates

* Fixes backtesting and apiserver errors encountered

* Fixes errors and typos from reviewing

* More minor fixes

* Changes %h verb to %w

* reverbs to %s

* Humbly begins reverting to more flat engine package

The main reasoning for this is that the subsystem split doesn't make sense
in a golang environment. The subsystems are only meant to be used with engine
and so by placing them in a non-engine area, it does not work and is
inconsistent with the rest of the application's package layout.

This will begin salvaging the changes made by reverting to a flat
engine package, but maintaining the consistent designs introduced.
Further, I will look to remove any TestMains and decrease the scope
of testing to be more local and decrease the issues that have been
caused from our style of testing.

* Manages to re-flatten things. Everything is within its own file

* mini fixes

* Fixes tests and data races and lints

* Updates docs tool for engine to create filename readmes

* os -> ioutil

* remove err

* Appveyor version increase test

* Removes tCleanup as its unsupported on appveyor

* Adds stuff that I thought was in previous merge master commit

* Removes cancel from test

* Fixes really fun test-exclusive data race

* minor nit fixes

* niterinos

* docs gen

* rm;rf test

* Remove typoline. expands startstop helper. Splits apiserver

* Removes accidental folder

* Uses update instead of replace for order upsert

* addresses nits. Renames files. Regenerates documentation.

* lint and removal of comments

* Add new test for default scenario

* Fixes typo

* regen docs
2021-05-31 10:17:12 +10:00
Ryan O'Hara-Reid
1e7cef96dc Portfolio: Add support for XRP balance fetching and fix ETH address balance 404 issue (#485)
* Added in balance fetching for ripple and fixed an eth bug

* Add test and fix other test

* Addressed thrasher nits
2020-04-22 14:33:56 +10:00
Andrew
b26ec86d43 Withdraw additional functionality (validation/submission/tracking) (#409)
* reworked request struct and exchange response started work on validation system

* removed import cycle until work around

* Added intial withdraw support via CLI added

* Added Crypto command to gctcli

* moved var declartion to single line

* Test updates for binance and anx

* All exchange tests have been updated test coverage added to validate

* First pass at adding withdrawl select from database

* started adding basic lru cache system

* Added basic LRU cache including Add Get Remove Contains ContainsOrAdd Clear

* wording changes on comments

* removed exported var's in strut as they are not required

* Added README

* README updates

* corrected ID on commands

* rm line :D

* merged in origin/cache

* linter fixes (gofmt)

* Added basic cache lookup to events

* swapped to mutex over rwmutex updated comments

* unexported getNewest & getOldest

* unexported getNewest & getOldest

* Updated comments and cited references in source

* updated comments

* WIP

* Migrated exchange WithdrawFiat wrapper to new struct response

* Migrated exchange WithdrawFiat wrapper to new struct response

* started work on bank management

* Added exchange level banking details back with migration to banking package

* Removed broken tests for now

* Added validation to bank accounts

* removed duplicate bank details from withdraw struct

* Test coverage increased

* gofmt

* merged upstream/master with clean up

* First pass at adding command line linking to gctcli

* added validation for crypto address, added gctcli support to retreive previous withdrawal requests

* general cleanup

* general cleanup

* reordered imports

* Increased test coverage moved to database sublogger

* Pass incorrect currency no longer return error from c.CheckBankAccountConfig

* remove TestMain() for now as other tests in this package will need to be reworked

* Happy little race car

* Reverted to upstream tests

* Added test covarege for validation method, corrected response on cli protobuf

* query clean up and removal of duplicated code

* cleaned up queries into singlem ethod increased test coverage

* Migrated international fund withdraw to new exchange response and added cache size override

* Migrated international fund withdraw to new exchange response and added cache size override

* Extended gctcli commands

* lowered default cache to 25

* small code clean up

* added get by date method

* add returned error

* cli commands cleaing return error on nil results to fix out of bounds

* merged write & read helpers into one for test coverage and increased engine/withdraw test coverage

* gofmt

* Added test coverage for valid ID

* removed unused param

* converted to use timestamp package from protobuf

* gofmt

* use built in RFC3339 timestamp

* remove setting of CreatedAt & UpdatedAt and allow ORm to take care of it

* also use ptype on byid

* code flow improvements

* remove inverse conditional check and linters run

* removed test data

* removed comment

* removed comment

* also write failures to database for auditing

* converted to use default time for start & end

* Default to time.Now() minus 30 days

* Default to time.Now() minus 30 days

* small code clean up

* fixed missing semicolon on migrations, code clean up

* updated sqlite migrations

* Added additonal check for exchange level bank account if global is not found

* case sensativity fix for currency names

* use correct compare

* test coverage fixed

* removed space

* return pointer to banking.Account

* return pointer to banking.Account

* added else check back to validate()

* Added empty string as default to migration over NULL due to retrivial of data
2020-02-26 15:45:13 +11:00
Adrian Gallagher
4e05ad41e3 Exchanges: Remove ANX from codebase (#408)
* Remove ANX from codebase

* Doc changes
2020-01-03 14:12:52 +11:00
Ryan O'Hara-Reid
98a277a4c3 Bugfixes: Bitfinex websocket, ZB market response and portfolio (#397)
* bug fix for websocket orderbook processing

* Fix more panics

* fix linter issue

* kick panic can down the road

* temp fix for issue with a 404 returned error as chainz.cryptoid dropped eth support

* Address nits and fixed orderbook updating

* Fix trade data, rm'd event time from struct

* fix time conversion for huobi

* Actually process kline data and fix time stamps

* btse time conversion fix and RM log, as it seems that the gain is reflecting transaction side. Drop ticker fetching support because there does not seem to be support on docs. And added trade fetching support.

* revert huobi println

* Adressed suggestion

* rm unnecessary assignment

* rm unnecessary check and assign

* fix conversion mishap

* fix currency conversion bug

* update websocket logging

* RM websocket type which stops conversion and copy

* fix linter issue, add in unknown side type
2019-12-19 13:40:30 +11:00
Adrian Gallagher
da81ea6c08 Merge branch 'master' into engine 2019-11-11 17:11:11 +11:00
Adrian Gallagher
e9b3e3d328 Bump golangci-lint and AppVeyor versions now that 1.13.x is supported (#375) 2019-11-11 15:15:29 +11:00
Scott
ccfcdf26aa Engine: Protocol Features, coverage, types, BTC markets websocket (#368)
* Attempts to update orderbook so it doesn't need to sort

* Reverts the ws ob stuff. Gets rid of sorting because it happens later. Adds some exchange features

* update existing feature lists. Expands list definition to match my emotions

* Adds bithumb bitmex and bitstamp. adds a couple more types

* Features for you, features for me, features for bittrex, btcmarkets, btse, coinbasepro, coinut, exmo, gateio and gemini

* Features for hitbtc, huobi, itbit, kraken, lakebtc, lbank, localbitcoins, okcoin, okex, poloniex, yobit, zb

* Who can forget good old alphapoint?

* Adds btcmarksets websocket :glitch_crab: fixes alphapoint features

* Adds extra data not in the documentation :/

* Replaces websocket features by using protocol features. However, it breaks it due to import cycles. I'm not sure what I'll do just yet

* Removes import cycle via duplicate structs.

* Increases coverage of config with `TestCheckCurrencyConfigValues`. Moves all currency pair package types into their own files or places it at the bottom of files if necessary

* Increase coverage in code.go

* One way of determining a test has failed, is when to it fails. Removed redundant explanation

* Increases code coverage of conversion

* Lint fixes

* Fixes orderbook tests

* Re-adds sorting because its important to still have the internal pre-processed orderbook to be representative of a real orderbook

* Secret lints that did not show up via Windows linting

* Adds protocol package to contain exchange features

* Fixes protocol implementation

* Fixes ws tests

* Addresses the following: Removes st-st-stutters in config types, changes GetAvailableForexProviders -> GetSupportedForexProviders, removes errors from tests where error is nil, removes orderbook setup when not necessary, removes import newlines, removes false bools from declaration, changes should of to should have

* imports and casing

* Fixes two more nil error checks
2019-10-22 10:56:20 +11:00
Adrian Gallagher
8ecd8a0c16 Merge branch 'master' into engine 2019-08-09 15:46:24 +10:00
Adrian Gallagher
2078ba907f Update URLs for transfer into org repo (#338)
* Update URLs for transfer into org repo

* Update codecov, travis and othe remaining links

* Update appveyor paths
2019-08-09 11:50:39 +10:00
Adrian Gallagher
bd8dc47c38 daily progress build 2019-06-06 17:20:40 +10:00
Adrian Gallagher
f5914e8c10 Engine changes 2019-05-22 17:06:38 +10:00
Ryan O'Hara-Reid
0990f9d118 Currency package update (#247)
* Initial currency overhaul before service system implementation

* Remove redundant currency string in orderbook.Base
Unexport lastupdated field in orderbook.Base as it was being instantiated multiple times
Add error handling for process orderbook

*  Remove redundant currency string in ticker.Price
 Unexport lastupdated field in ticker.Price
 Add error handling for process ticker function and fix tests

* Phase Two Update

* Update translations to use map type - thankyou to kempeng for spotting this

* Change pair method name from Display -> Format for better readability

* Fixes misspelling and tests

* Implement requested changes from GloriousCode

* Remove reduntant function and streamlined return in currency_translation.go

* Revert pair method naming conventions

* Change currency naming conventions

* Changed code type to exported Item type with underlying string to reduce complexity

* Added interim orderbook process method to orderbook.Base type

* Changed feebuilder struct field to currency.Pair

* Adds fall over system for backup fx providers

* deprecate function and children and fix linter issue with btcmarkets

* Fixed requested changes

* Fix bug and move mtx for rates

* Fixed after rebase oopsies

* Fix linter issues

* Fixes race conditions in testing functions

* Final phase coinmarketcap update

* fix linter issues

* Implement requested changes

* Adds configuration variables to increase/decrease time durations between updating currency file and fetching new currency rates

* Add a collection of tests to improve codecov

* After rebase oopsy fixes for btse

* Fix requested changes

* fix after rebase oopsies and add more efficient comparison checks within currency pair

* Fix linter issues
2019-03-19 11:49:05 +11:00
Adrian Gallagher
7dcb1ab553 Migrate from gometalinter.v2 to golangci-lint (#249)
* Migrate from gometalinter.v2 to golangci-lint
2019-03-01 16:10:29 +11:00
Adrian Gallagher
5e5ca8a887 Linter fixes (#246)
Linter fixes
2019-02-05 16:26:04 +11:00
Adrian Gallagher
d3edd2845f Reorganise portfolio and prevent saving nil portfolio if not seeded 2018-05-31 14:07:23 +10:00
Adrian Gallagher
c0ccc901dc Fix GoReport issues 2018-03-26 12:18:19 +11:00
Adrian Gallagher
d571773680 Update Ethereum balance updating via Ethplorer 2018-01-17 14:33:17 +11:00
Adrian Gallagher
6e9bda83a1 Link up websocket handler to routes after refactor and various improvements 2017-09-14 15:08:19 +10:00
Adrian Gallagher
3ad9c3dfbe Use cryptoID instead of blockr (currently down) 2017-09-14 15:08:19 +10:00
Adrian Gallagher
4f34b58d55 Improve portfolio, coverage and tool 2017-08-17 11:56:54 +10:00
Ryan O'Hara-Reid
f11f83fe50 Formatting, linter and test for portfolio 2017-07-31 11:44:28 +10:00
Ryan O'Hara-Reid
3e4fb1660d Fixed test race conditions 2017-07-31 11:44:12 +10:00