34 Commits

Author SHA1 Message Date
Adrian Gallagher
bb122dcafa codebase: Remove web frontend and related services (#2067)
* codebase: Remove web frontend and related services

* refactor: Update StartPPROF to accept context and adjust related tests

* refactor: Simplify SetIfZero functions and update related tests

* config: Clarify DowngradeConfig method documentation regarding permanent removal of deprecated fields

* refactor: Rename setIfZeroAndWarn to setDefaultIfZeroWarn for clarity and update related calls

* refactor: Update error handling in DataHistoryManager and remove redundant error variable
2025-09-30 13:32:09 +10:00
Adrian Gallagher
ba92ba3254 docs: Add CODING_GUIDELINES.md and other adjustments (#1988)
* docs: Create new CODING_GUIDELINES doc

Rids excess contribution instructions from other packages
Adds AGENTS.md file for the AI overlords
Rids unused templates
Updates ADD_NEW_EXCHANGE.md with minor fixes

* docs: Fix linter issues and minor adjustments based on Copilot feedback

* docs: Update coding guidelines for API parameters and testing practices

* docs: Remove redundant GoDoc references

Adds copilot-instructions.md

* docs: Update CODING_GUIDELINES with export recommendations and test commentary

* docs: Fix formatting inconsistencies in ADD_NEW_EXCHANGE.md links

* docs: Update struct naming conventions for request and response types

* docs: Improve clarity and consistency in ADD_NEW_EXCHANGE.md and CODING_GUIDELINES.md

* refactor: Simplify error handling in QueryOrder method
2025-08-08 12:10:53 +10:00
Adrian Gallagher
16c1ff3bdf Slack: Update shared invite link (#1967)
* Slack: Update shared invite link

* fix: add rel="noopener noreferrer" to Slack invite links for security
2025-07-08 13:19:46 +10:00
Adrian Gallagher
4651af5767 modernise: Run new gopls modernise tool against the codebase and fix minor issues (#1826)
* modernise: Run new gopls modernise tool against codebase

* Address shazbert's nits

* apichecker, gctcli: Simplify HTML scraping functions and improve depth limit handling

* refactor: Create minSyncInterval const and update order book limit handling for binance and binanceUS

* refactor: Various slice usage improvements and rename TODO

* tranches: Revert deleteByID changes due to performance decrease

Shazbert was a F1 driver in a past lifetime 🏎️

* tranches: Simply retrieve copy

Thanks to shazbert

* documentation: Sort contributors list by contributions

* tranches: Remove deadcode in deleteByID
2025-03-21 09:17:10 +11:00
Adrian Gallagher
69bb645fc2 docs: Migrate development tracking from Trello to GitHub Projects (#1696)
* docs: Migrate development tracking from Trello to GitHub Projects

* docs: Replace non-template based references and correct templates
2024-11-08 10:10:49 +11:00
Adrian Gallagher
7e08e483fb CI: Bump go version, linters and fix minor issues (#1130)
* CI: Bump go version, linters and fix minor issues

* Bump version, fix loop variables

* Revert

* Rid TODOs now that 1.51 has been released
2023-02-03 15:56:59 +11:00
Adrian Gallagher
9a4eb9de84 CI: Fix golangci-lint linter issues, add prealloc linter and bump version depends for Go 1.18 (#915)
* Bump CI versions

* Specifically set go version as 1.17.x bumps it to 1.18

* Another

* Adjust AppVeyor

* Part 1 of linter issues

* Part 2

* Fix various linters and improvements

* Part 3

* Finishing touches

* Tests and EqualFold

* Fix nitterinos plus bonus requester jobs bump for exchanges with large number of tests

* Fix nitterinos and bump golangci-lint timeout for AppVeyor

* Address nits, ensure all books are returned on err due to syncer regression

* Fix the wiggins

* Fix duplication

* Fix nitterinos
2022-04-20 13:45:15 +10:00
Adrian Gallagher
f0d45aa1d2 golangci-lint/CI: Bump versions and introduce new linters (#798)
* golangci-lint/CI: Bump versions

Fix remaining linter issues

* Specifically set AppVeyor version

* Fix the infamous typos 👀

* Add go env cmd to AppVeyor

* Add go version cmd to AppVeyor

* Specify AppVeyor image, adjust linters

* Update go get to go install due to deprecation

* Bump golangci-lint timeout time for AppVeyor

* Change NW contract to NQ

* Address nitters

* GetRandomPair -> Pair{}

* Address nits

* Address time nitterinos plus additional tweaks

* More time inception upgrades!

* Bending time and space
2021-10-14 16:38:53 +11:00
Ryan O'Hara-Reid
d636049fb2 exchanges: Initial context propagation (#744)
* gct: phase one context awareness pass

* exchanges: context propagation pass

* common/requester: force context requirement

* gctcli/exchanges: linter fix

* rpcserver: fix test using dummy rpc server

* backtester: fix comments

* grpc: add correct cancel and timeout for commands

* rpcserver_test: add comment on dummy server

* common: deprecated SendHTTPGetRequest

* linter: fix

* linter: turn on no context check

* apichecker: fix context linter issue

* binance: use param context

* common: remove checks as this gets executed before main

* common: change mutex to RW as clients can be used by multiple go routines.

* common: remove init and JIT default client. Unexport global variables and add protection.

* common: Add comments

* bithumb: after dinner mints fix
2021-09-11 13:52:07 +10:00
Scott
63257ce4ca Improvement: Speeding up slow tests (#707)
* Speeds up tests

* Reduces time.Sleeps, lowers CreateTestBot complexity. Breaks things

* Removal of unecessary config reads. Parallel tests. Lower times

* Speeds up recent trades results

* mini update

* zoooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooom

* Removes the dupes

* Lint

* post cherrypick

* Fix rare kraken data race

* Fixes banking global issues. Fixes postgres trades

* rmline for appveyor test

* Expands timeout in event that channel is closed before send

* Fix data race

* No rows, no bows and definitely no shows

* Removes parallel from createsnapshot tests

* Extends timedmutext test a smidge. Exchange fatality

* Shorter end timeframe and bigger candle
2021-07-07 12:42:03 +10:00
Scott
5ea5245afb Improvement: Subsystem separation (#664)
* Initial codes for a trade tracker

* Moving everything in a broken fashion

* Removes tradetracker. Removes some errors for subsystems

* Cleans up some subsystems, renames stuttering types. Removes some global Bot usage

* More basic subsystem renaming and file moving

* Removes engine dependency from events,ntpserver,ordermanager,comms manager

* Exports eventManager, fixes rpcserver. puts rpcserver back for now

* Removes redundant error message, further removes engine dependencies

* experimental end of day interface usage

* adds ability to build the application

* Withdraw and event manager handling

* cleans up apiserver and communications manager

* Cleans up some start/setup processes. Though should separate

* More consistency with Setup Start Stop IsRunning funcs

* Final consistency pass before testing phase

* Fixes engine tests. Fixes stop nil issue

* api server tests

* Communications manager testing

* Connection manager tests and nilsubsystem error

* End of day currencypairsyncer tests

* Adds databaseconnection/databaseconnection_test.go

* Adds withdrawal manager tests

* Deposit address testing. Moved orderbook sync first as its more important

* Adds test for event manager

* More full eventmanager testing

* Adds testfile. Enables skipped test.

* ntp manager tests

* Adds ordermanager tests, Extracts a whole new subsystem from engine and fanangles import cycles

* Adds websocket routine manager tests

* Basic portfolio manager testing

* Fixes issue with currency pair sync startup

* Fixes issue with event manager startup

* Starts the order manager before backtester starts

* Fixes fee tests. Expands testing. Doesnt fix races

* Fixes most test races

* Resolves data races

* Fixes subsystem test issues

* currency pair syncer coverage tests

* Refactors portfolio. Fixes tests. Withdraw validation

Portfolio didn't need to exist with a portfolio manager. Now the porfolio manager
is in charge how the portfolio is handled and all portfolio functions are attached
to the base instead of just exported at the package level

Withdrawal validation occurred at the exchange level when it can just be run at the
withdrawal manager level. All withdrawal requests go through that endpoint

* lint -fix

* golang lint fixes

* lints and comments everything

* Updates GCT logo, adds documentation for some subsystems

* More documentation and more logo updates

* Fixes backtesting and apiserver errors encountered

* Fixes errors and typos from reviewing

* More minor fixes

* Changes %h verb to %w

* reverbs to %s

* Humbly begins reverting to more flat engine package

The main reasoning for this is that the subsystem split doesn't make sense
in a golang environment. The subsystems are only meant to be used with engine
and so by placing them in a non-engine area, it does not work and is
inconsistent with the rest of the application's package layout.

This will begin salvaging the changes made by reverting to a flat
engine package, but maintaining the consistent designs introduced.
Further, I will look to remove any TestMains and decrease the scope
of testing to be more local and decrease the issues that have been
caused from our style of testing.

* Manages to re-flatten things. Everything is within its own file

* mini fixes

* Fixes tests and data races and lints

* Updates docs tool for engine to create filename readmes

* os -> ioutil

* remove err

* Appveyor version increase test

* Removes tCleanup as its unsupported on appveyor

* Adds stuff that I thought was in previous merge master commit

* Removes cancel from test

* Fixes really fun test-exclusive data race

* minor nit fixes

* niterinos

* docs gen

* rm;rf test

* Remove typoline. expands startstop helper. Splits apiserver

* Removes accidental folder

* Uses update instead of replace for order upsert

* addresses nits. Renames files. Regenerates documentation.

* lint and removal of comments

* Add new test for default scenario

* Fixes typo

* regen docs
2021-05-31 10:17:12 +10:00
Adrian Gallagher
051675255d CI: Add support for GitHub actions (#680)
* Add initial workflows config

* Add PSQL service

* Add 32bit and frontend jobs

* Add gcc-multilib and fix typo

* Fix backtester test for linux/unix filesystem errors and npm commands

* 32-bit Add -race to purposefully throw an error to ensure envs are set correctly (will revert)

* Revert "32-bit Add -race to purposefully throw an error to ensure envs are set correctly (will revert)"

This reverts commit b97e66c49fca859f8738e01107f96f9ca4040b2e.

* Bump workers count for orderbook processing

* Remove comments to trigger 32-bit cache hit after job success

* Cancel addition plus version bump, spawn more overlords

* Update build badge to Github Actions

* Brach -> Branch
2021-05-06 14:35:05 +10:00
Scott
0da62b7fbf Feature: Add mock testing to ZB (#569)
* Adds mock testing to ZB

* STEALS improved time validation code from the original STOLEN validation code :D

* Mini fixes from review

* happy fun comment stealing

* Moves the loop checker earlier to ensure no double appendages

* Fixes sneaky test

* Fixes the important part where mock tests work instead of live tests

* Skips authenticated endpoints for mock testing.

* lint

* Updates candle wrapper functions to respect design

* basic linting fix

* Reverts configtest.json, updates readme to be way better, adds coverage to validateCandlesRequest

* Tiniest grammatical fix

* Fixes more outdated code references

* Closing out a high

* Fixes spacing

* Replaces all instances of 4 spaces in tmpl files with a tab

* fixes spacing and tab related readme issues once and for all 🤞

* tidy

* indentation violation identification situation
2020-10-07 11:59:08 +11:00
Adam
7dd3f13c8d Exchange API modification monitor (#404)
* WIP

* Variable Name Changes and other PR Changes

* Name Changes and Error Checks

* WIP

* Some new functionality added and nits fixed

* WIP on Nits

* More trello functionality added and some PR changes

* Minor changes and linter fixes

* Errors Fixed and some PR Changes

* PR Changes

* WIP

* WIP

* Addition of Config

* WIP of Readme file

* Readme WIP

* Addition of ReadMe

* Linter Issues fixed

* Minor Errors fixed

* fixed test timeout issues and added api keys check for tests

* Errors fixed

* Nits WIP

* Nits fixed

* More efficient mutex usage

* Consts cleaned up

* Readme file fixed

* linter issues fixed

* format issues:

* readme updates and nits

* remove backup file and address nits

* backup file removed for real

* OkexOkcoin updates and linter fixes

* errors fixed

* Nits

* errors fixed

* formatting issues

* race conditions fixed

* files fixed

* go mod n go sum

* WIP adding rate limits

* Added rate limits

* Remove test vars

* go mod tidy

* improving readme file and 2digits for trello auto update

* Added space

* removed space

* addressing nits

* documentation update

* more nits

* board id check

* linter fixes

* test race fixed

* test file match

* errors fixed

* nitz

* LOWERCASE flags

* more nitz

* broken wip

* flag addition fixed

* error fixed

* documentation updated for adding exchange using flags

* something

* add comment

* nitz fixed

* pr changes

* added gemini exchange

* testfile saving changes

* nitz

* fixing done

* pr changes

* nits

* WIP

* minor error fix

* wip

* wip

* WIP

* pr changes

* pr changes

* pr changes

* nits

* nits fixed

* t.Fail removed

* nitz

* disable lakeBTC

* contributors fixed

contributors fixed

ALL THE NITZ

OMG

* OMGGG

ALL THE NITZ

OMG

* name change errors fixed

* linter issues fixed

* linterzz

* Added disabling feature for exchanges

* bug fix

* small feature added

* fixed

bug fix:

no keys

* nits

* test

* test2

* test 3

* linter fixed

* testing

* testing

* remove logs

* OMG

* bugfix

* cyam88888

* nits

* error handled

* minor nits

fixed createnewcheck issue

fixed createnewcheck issue

* functionality change

some functionality changes

no api keys 4u :D

* trello names thro config

* err check
2020-04-01 11:15:10 +11:00
Andrew
7fccc03164 (Logger) Rename package to log (#444)
* renamed package to log to stop side import requirement

* reverted comment changes

* reverted comment changes

* one more reverting wording back to logger

* wording changes on comments
2020-02-12 18:09:56 +11:00
Adrian Gallagher
b949388994 General engine improvements (#437)
* Add exchange manager to engine

* Several improvements for engine and friends

1) New file.Exists func
2) gRPC TLS cert expiration date check and regeneration
3) New donation var for use across the codebase
4) Use Go log package until the logger is initialised

* Add cert tests and create dir tree if it doesn't exist for file.Write

* Link up donation address to documentation tool plus minor adjustments

* Fix remaining donation addrs

* Move non-needed reload exchange funcs

* Revert accidental config_example.json changes 🕯️

* Use go logger for logging until the logger has initiliased, otherwise no output will be seen

* Link up portfolio delay val and other fixes

* Run go mod tidy after dependabot PR

* Address nitterinos
2020-02-06 12:32:01 +11:00
Ryan O'Hara-Reid
24bddcc090 Add directories to exclusion list && change default repo string (#387)
* Add directories to exclusion list && change default repo string

* Add in .idea folder to directory exclusion list

* cleaned

* Added support for calling the tool outside of its file

* fix formatting

* changed strings
2019-11-29 11:14:19 +11:00
Scott
ccfcdf26aa Engine: Protocol Features, coverage, types, BTC markets websocket (#368)
* Attempts to update orderbook so it doesn't need to sort

* Reverts the ws ob stuff. Gets rid of sorting because it happens later. Adds some exchange features

* update existing feature lists. Expands list definition to match my emotions

* Adds bithumb bitmex and bitstamp. adds a couple more types

* Features for you, features for me, features for bittrex, btcmarkets, btse, coinbasepro, coinut, exmo, gateio and gemini

* Features for hitbtc, huobi, itbit, kraken, lakebtc, lbank, localbitcoins, okcoin, okex, poloniex, yobit, zb

* Who can forget good old alphapoint?

* Adds btcmarksets websocket :glitch_crab: fixes alphapoint features

* Adds extra data not in the documentation :/

* Replaces websocket features by using protocol features. However, it breaks it due to import cycles. I'm not sure what I'll do just yet

* Removes import cycle via duplicate structs.

* Increases coverage of config with `TestCheckCurrencyConfigValues`. Moves all currency pair package types into their own files or places it at the bottom of files if necessary

* Increase coverage in code.go

* One way of determining a test has failed, is when to it fails. Removed redundant explanation

* Increases code coverage of conversion

* Lint fixes

* Fixes orderbook tests

* Re-adds sorting because its important to still have the internal pre-processed orderbook to be representative of a real orderbook

* Secret lints that did not show up via Windows linting

* Adds protocol package to contain exchange features

* Fixes protocol implementation

* Fixes ws tests

* Addresses the following: Removes st-st-stutters in config types, changes GetAvailableForexProviders -> GetSupportedForexProviders, removes errors from tests where error is nil, removes orderbook setup when not necessary, removes import newlines, removes false bools from declaration, changes should of to should have

* imports and casing

* Fixes two more nil error checks
2019-10-22 10:56:20 +11:00
Ryan O'Hara-Reid
e2d57540a6 Config overwrite bugfix (#363)
* Fix bug where on parsing an alternate new config it will overwrite main config.json in gct dir

* Stop movement of config.json file from root dir when a new config is parsed in

* Stop overiding config.json at gct dir with new config.json from root directory

* RM LN :D

* Fix bug where promptforconfig in config_encryption.go overwrites default config
Ensure periphery command packages do not interact or save over configuration
Ensure tests to not save over or change current testdata/config
2019-09-27 16:03:41 +10:00
Adrian Gallagher
132d2d03c2 Merge branch 'master' into engine 2019-09-24 17:05:58 +10:00
Adrian Gallagher
bbdf532b16 Update Slack shared invite link 2019-09-24 17:01:32 +10:00
Adrian Gallagher
8ecd8a0c16 Merge branch 'master' into engine 2019-08-09 15:46:24 +10:00
Adrian Gallagher
2078ba907f Update URLs for transfer into org repo (#338)
* Update URLs for transfer into org repo

* Update codecov, travis and othe remaining links

* Update appveyor paths
2019-08-09 11:50:39 +10:00
Andrew
3de1d94e5f New logging system (#319)
* First pass at adding new logging system

* NewLogger

* NewLogger

* WIP

* silly bug fix

* :D removed files

* removed old logging interface

* added tests

* added tests

* Started to add new lines to all f calls

* Added subsystem log types

* Logger improvements

* Further performance improvements

* changes to logger and sublogger creation

* Renamed Logging types

* removed old print statement

* changes based on feedback

* moved sublogger types to own file

* :)

* added console as output type

* added get level command

* added get/set log level via grpc command

* added check for output being empty for migration support

* first pass at log rotation

* added log rotation

* :D derp fixed

* added tests

* changes based on feedback

* changed log type

* comments

* renamed file -> fileSettings

* typo fix

* changes based on feedback

* gofmt ran on additional files

* gofmt ran on additional files
2019-07-07 05:20:31 +10:00
Adrian Gallagher
f777e68716 Engine improvements 2019-06-10 20:02:09 +10:00
Adrian Gallagher
3010b62ac1 Use string.EqualFold where necessary 2019-06-04 17:27:00 +10:00
Adrian Gallagher
e965e54e09 Split up common.go, file path fixes and much more 2019-06-04 17:04:32 +10:00
Adrian Gallagher
5d891e70e9 Cleanup: rm/adjust non-needed println's/warnings 2019-03-26 09:50:22 +11:00
Adrian Gallagher
7dcb1ab553 Migrate from gometalinter.v2 to golangci-lint (#249)
* Migrate from gometalinter.v2 to golangci-lint
2019-03-01 16:10:29 +11:00
Adrian Gallagher
928a332220 Use official Slack join link instead of herokuapp 2019-02-12 13:45:43 +11:00
Adrian Gallagher
5e5ca8a887 Linter fixes (#246)
Linter fixes
2019-02-05 16:26:04 +11:00
Ryan O'Hara-Reid
92534249bf Remove unwanted wrapper name stuttering and standardise common error returns (#213) 2018-11-23 12:55:00 +11:00
Ryan O'Hara-Reid
3b8591bcc8 Updated documentation tool (#155)
* Updated documentation tool
Added templates
Updated documentation using tool

* Fixed incorrect version in web README

* Added new templates to tool.
Updated documents in templates across the code base.
Used tool to regenerate documentation.
2018-07-19 16:02:24 +10:00
Ryan O'Hara-Reid
9d0616d8cf New communications package
Support for Slack, SMSGlobal, SMTP and Telegram

Supersedes: https://github.com/thrasher-/gocryptotrader/pull/126
2018-06-08 14:09:36 +10:00