Commit Graph

29 Commits

Author SHA1 Message Date
Ryan O'Hara-Reid
db317a2447 Engine: Dispatch service (#346)
* Added dispatch service

* Added orderbook streaming capabilities

* Assigned correct orderbook.base exchange name

* Fixed Requested niterinos
Add in cli orderbook QA tool to gctcli
Add exchange orderbook streaming

* Add ticker streaming support through dispatch package

* Added in some more info on error returns for orderbook.go

* fix linter issues

* Fix some issues

* Update

* Fix requested

* move dispatch out of exchanges folder to its own independant folder

* Fix requested

* change orderbook string to tickers

* Limit orderbooks to 50 and made dispatch system more stateless in operation

* lower cases for update/retrieve/sub exchange name

* Adds in asset validation and lower case conversion on cli

* Remove comment

* Moved timer to a higher scope so its not constantly initialised just reset per instance and removed returning unused channel on error

* Rm unused release function in dispatch.go
Reset timer and bleed buffered timer chan if needed in dispatch.go
Added in ticker.Stop() and timer.Stop() functions for worker routine return in dispatch.go
Index aggregated bid and ask functions for orderbook.go
Added in dummy slice for wsorderbook_test.go

* Moved drain to above Reset so potential race would not occur in dispatch.go
Fix various linter issues dispatch.go

* Fix some issues

* change to start/stop service, added in service state change via cli, updated logger

* fix requested

* Add worker amount init spawning

* fix linter issues

* Fix more linter issues

* More fixes

* Fix race issue on releasing pipe channel on a close after shutting down dispatcher system

* Moved all types to dispatch_types.go && remove panic

* Moved types into serperate file && improve test coverage

* RM unnecessary select case for draining channel && fixed error string

* Added orderbook_types file and improved code coverage

* gofmt file

* reinstated select cases on drain because I am silly

* Remove error for drop worker

* Added more test cases

* not checking error issue fix

* remove func causing race in test, this has required protection via an exported function

* set Gemini websocket orderbook exchange name
2019-10-03 09:47:37 +10:00
Ryan O'Hara-Reid
e2d57540a6 Config overwrite bugfix (#363)
* Fix bug where on parsing an alternate new config it will overwrite main config.json in gct dir

* Stop movement of config.json file from root dir when a new config is parsed in

* Stop overiding config.json at gct dir with new config.json from root directory

* RM LN :D

* Fix bug where promptforconfig in config_encryption.go overwrites default config
Ensure periphery command packages do not interact or save over configuration
Ensure tests to not save over or change current testdata/config
2019-09-27 16:03:41 +10:00
Adrian Gallagher
6bdbe236c0 Rid GCT os.Exit's in packages outside of main 2019-09-27 15:18:54 +10:00
Adrian Gallagher
3b1c0825f4 Merge branch 'master' into engine 2019-09-26 11:35:18 +10:00
Scott
0493a215a6 Exchange wrapper test tool (#353)
* Initial commit which runs through all wrapper funcs for one of each currency pair for each asset type for each exchange and then outputs it all into a JSON file

* Fixes up data holding responses to allow for good json output after completion

* Starting work for reading credentials from a config. Some variable names are more appropriate

* Keys can now be read from keys.json and applied to exchanges. Fixes naming bug for console output

* Cleans up implementation. Uses templating to output some pretty html report

* Changes config to include a bank account to customise for withdrawals. Updates template for warnings for not implemented. Updates template to include jsonified parameters that are sent to the wrapper. Jsonifies the response so you actually understand whats returned.

* Adds bank for withdrawals. Adds wallet address for crypto withdrawals

* Adds ordersubmission configuration to config. Sets command line overrides. Adds lbank to config. Adds okgroup clientid to config.

* Adds missing comma

* Adds config to gitignore

* Removes because thats not how it works...

* Revert go mod changes

* Formatting for prettiness

* Adds asset type override

* Adds verbose, exchangesToExclude, filename CL flags. Removes wg redeclaration. Creates func to check to load exchange before its loaded. Removes double variables and uses flag.Stringvar instead

* Prettifies the JSON output

* Stringifies the []byteified jsonified wrapper response in the template

* Puts types in their own EXCLUSIVE types.go. LOWERCASES COMMAND LINE ARGUMENTS. Wraps vars in var so theres less vars to look at.

* Generates wrapperconfig.json on the fly if not present. Adds missing config fields and exchanges when not present. Saves config file on finish. Fixes some formatting on output when its just a number

* micro format update

* Changes printfs to printlns. Handles config error. Comments on types. Fixes lint issue with shadow err declaration

* Fixes linting issue, removes returns after fatals, evaluates assetType, formats template

* Stringifies byte output when console output is selected. Verbose mode now includes wrapper responses in console output
2019-09-24 18:28:07 +10:00
Adrian Gallagher
132d2d03c2 Merge branch 'master' into engine 2019-09-24 17:05:58 +10:00
Adrian Gallagher
7c49ada7f6 Add bash/zsh autocomplete files and minor fixes 2019-09-18 10:22:52 +10:00
Adrian Gallagher
e8b517ef0a Daily engine changes
1) Although gRPC does server side validation currently, validate basic things
on gctcli before relaying the request to the gRPC server
2) Make pair format consistent for the exchange sycner
3) Fix OKEX ticker failure due to thinking futures info is authenticated
4) Start filling out config tests
5) Extend timeout for golangci config so that AppVeyor has time to
complete (Travis is fine)
6) Add IsSupported exchange func for easy lookup
2019-09-10 17:07:00 +10:00
Adrian Gallagher
0fcf867697 Add support for get/set/rm of exchange pairs via gRPC 2019-09-06 11:47:58 +10:00
Adrian Gallagher
5e9d13f7be Expand RetrieveConfigCurrencyPairs to support different asset types 2019-08-27 08:54:10 +10:00
Adrian Gallagher
f6afeee800 Minor improvements
1) gen_otp supports single use OTP secrets
2) improve database config check logic
3) reconnect websocket routine to engine (apart from the exchange pair syncer)
4) ImPrOvE CoNsIsTeNcY wItH LoG OuTpUt
2019-08-26 12:46:53 +10:00
Adrian Gallagher
2dc813b5f3 Merge branch 'master' into engine 2019-08-23 21:59:01 +10:00
Adrian Gallagher
94a784d63f Merge branch 'master' into engine 2019-08-22 18:32:57 +10:00
Andrew
0c76789b0d Database interface & auditing feature (#332)
* added audit manager

* Basic database DOA setup

* Added base config file

* added sqlite support and creation of schema

* added basic tests and config entry

* corrected issues of database is disabled

* fixed path for test

* WIP

* Added tests fixed config checking

* reverted files back to upstream

* reverted go.mod files

* no more test test test

* removed local testing details for psql

* hello

* added comments

* increased ping to 30 seconds

* renamed database table and added additional condition around test

* removed database test details

* goimport ran on all files

* WIP

* first attempt at migration

* fixes for migration system

* Migration system logger interface implemented

* fixes to print functions

* added write pooling pass

* gofmt :D

* formatted imports correctly

* removed old code

* added creation of migration

* gofmt

* :D Hello

*  🏎️

* maybe one day i will remember to revert go mod files

* checked err return condition correctly

* first changes for PR feedback

* code clean up

* protect Connected with RWmutex & event with mutex

* : D

* we can just pretend like it never happened

* MOved migrations back to source directory and added README

* readme formatting update

* Addd command line override for datadir

* use correct var when creating a migration and confirm folder is created

* Check if database version is newer than latest migration and also you know make migrations work.....

* uses filepath instead of manual path to use correct path seperator

* Add connection message and lower timeout

* Added support for sslmode for psql

* no longer force Close of database instead allow driver to maage

* Added closer func to test output

* sslmode added to example config
2019-08-20 16:35:06 +10:00
Adrian Gallagher
8ecd8a0c16 Merge branch 'master' into engine 2019-08-09 15:46:24 +10:00
Andrew
3de1d94e5f New logging system (#319)
* First pass at adding new logging system

* NewLogger

* NewLogger

* WIP

* silly bug fix

* :D removed files

* removed old logging interface

* added tests

* added tests

* Started to add new lines to all f calls

* Added subsystem log types

* Logger improvements

* Further performance improvements

* changes to logger and sublogger creation

* Renamed Logging types

* removed old print statement

* changes based on feedback

* moved sublogger types to own file

* :)

* added console as output type

* added get level command

* added get/set log level via grpc command

* added check for output being empty for migration support

* first pass at log rotation

* added log rotation

* :D derp fixed

* added tests

* changes based on feedback

* changed log type

* comments

* renamed file -> fileSettings

* typo fix

* changes based on feedback

* gofmt ran on additional files

* gofmt ran on additional files
2019-07-07 05:20:31 +10:00
Adrian Gallagher
7112a89491 Add orderbook calculator and verify func 2019-07-04 07:22:12 +10:00
Adrian Gallagher
9ff4471a45 Merge branch 'master' into engine 2019-06-21 18:10:55 +10:00
Ryan O'Hara-Reid
897bcfd9a4 Documentation Update (#318)
* Initial update

* update

* Fix linter issues

* Add new documentation template and fix
2019-06-21 12:38:35 +10:00
Ryan O'Hara-Reid
20c24601fb Asset update to fix minor stutter (#316) 2019-06-17 09:02:07 +10:00
Adrian Gallagher
b901c4b670 Engine improvements
Add back events tests
Fill out SMTP comms handler
Add getcommunicationrelayers gRPC command
2019-06-14 18:00:42 +10:00
Adrian Gallagher
6b2cfe7905 Daily engine improvements
Link up various subsystems to be managed atomically with the ability to start/stop them
New subsystem APIs
Comms changes
2019-06-13 17:30:50 +10:00
Adrian Gallagher
33085318c4 Switch connchecker to service 2019-06-12 17:52:40 +10:00
Adrian Gallagher
2ad808e70c Daily engine improvements:
New GetExchangeOTPs API
CLI validation
Standardised pairs for GCTCLI
Expand test coverage
Trim SMS global from name is len > 11
Linter fixes
2019-06-11 17:02:00 +10:00
Adrian Gallagher
f777e68716 Engine improvements 2019-06-10 20:02:09 +10:00
Adrian Gallagher
04c7c4895f Split common package more and QA 2019-06-07 20:52:44 +10:00
Adrian Gallagher
e965e54e09 Split up common.go, file path fixes and much more 2019-06-04 17:04:32 +10:00
Adrian Gallagher
6ebafb0f33 Merge branch 'master' into engine 2019-06-04 12:25:40 +10:00
Adrian Gallagher
f5914e8c10 Engine changes 2019-05-22 17:06:38 +10:00