Improvement: Subsystem separation (#664)

* Initial codes for a trade tracker

* Moving everything in a broken fashion

* Removes tradetracker. Removes some errors for subsystems

* Cleans up some subsystems, renames stuttering types. Removes some global Bot usage

* More basic subsystem renaming and file moving

* Removes engine dependency from events,ntpserver,ordermanager,comms manager

* Exports eventManager, fixes rpcserver. puts rpcserver back for now

* Removes redundant error message, further removes engine dependencies

* experimental end of day interface usage

* adds ability to build the application

* Withdraw and event manager handling

* cleans up apiserver and communications manager

* Cleans up some start/setup processes. Though should separate

* More consistency with Setup Start Stop IsRunning funcs

* Final consistency pass before testing phase

* Fixes engine tests. Fixes stop nil issue

* api server tests

* Communications manager testing

* Connection manager tests and nilsubsystem error

* End of day currencypairsyncer tests

* Adds databaseconnection/databaseconnection_test.go

* Adds withdrawal manager tests

* Deposit address testing. Moved orderbook sync first as its more important

* Adds test for event manager

* More full eventmanager testing

* Adds testfile. Enables skipped test.

* ntp manager tests

* Adds ordermanager tests, Extracts a whole new subsystem from engine and fanangles import cycles

* Adds websocket routine manager tests

* Basic portfolio manager testing

* Fixes issue with currency pair sync startup

* Fixes issue with event manager startup

* Starts the order manager before backtester starts

* Fixes fee tests. Expands testing. Doesnt fix races

* Fixes most test races

* Resolves data races

* Fixes subsystem test issues

* currency pair syncer coverage tests

* Refactors portfolio. Fixes tests. Withdraw validation

Portfolio didn't need to exist with a portfolio manager. Now the porfolio manager
is in charge how the portfolio is handled and all portfolio functions are attached
to the base instead of just exported at the package level

Withdrawal validation occurred at the exchange level when it can just be run at the
withdrawal manager level. All withdrawal requests go through that endpoint

* lint -fix

* golang lint fixes

* lints and comments everything

* Updates GCT logo, adds documentation for some subsystems

* More documentation and more logo updates

* Fixes backtesting and apiserver errors encountered

* Fixes errors and typos from reviewing

* More minor fixes

* Changes %h verb to %w

* reverbs to %s

* Humbly begins reverting to more flat engine package

The main reasoning for this is that the subsystem split doesn't make sense
in a golang environment. The subsystems are only meant to be used with engine
and so by placing them in a non-engine area, it does not work and is
inconsistent with the rest of the application's package layout.

This will begin salvaging the changes made by reverting to a flat
engine package, but maintaining the consistent designs introduced.
Further, I will look to remove any TestMains and decrease the scope
of testing to be more local and decrease the issues that have been
caused from our style of testing.

* Manages to re-flatten things. Everything is within its own file

* mini fixes

* Fixes tests and data races and lints

* Updates docs tool for engine to create filename readmes

* os -> ioutil

* remove err

* Appveyor version increase test

* Removes tCleanup as its unsupported on appveyor

* Adds stuff that I thought was in previous merge master commit

* Removes cancel from test

* Fixes really fun test-exclusive data race

* minor nit fixes

* niterinos

* docs gen

* rm;rf test

* Remove typoline. expands startstop helper. Splits apiserver

* Removes accidental folder

* Uses update instead of replace for order upsert

* addresses nits. Renames files. Regenerates documentation.

* lint and removal of comments

* Add new test for default scenario

* Fixes typo

* regen docs

.appveyor.yml

@@ -26,7 +26,7 @@ environment:
   PSQL_SSLMODE: disable
   PSQL_SKIPSQLCMD: true
   PSQL_TESTDBNAME: gct_dev_ci
-stack: go 1.15.x
+stack: go 1.16.x
 
 services:
   - postgresql96

CONTRIBUTORS

@@ -10,6 +10,7 @@ vazha | https://github.com/vazha
 ermalguni | https://github.com/ermalguni
 MadCozBadd | https://github.com/MadCozBadd
 vadimzhukck | https://github.com/vadimzhukck
+dependabot[bot] | https://github.com/apps/dependabot
 140am | https://github.com/140am
 marcofranssen | https://github.com/marcofranssen
 dackroyd | https://github.com/dackroyd
@@ -22,7 +23,6 @@ bretep | https://github.com/bretep
 Christian-Achilli | https://github.com/Christian-Achilli
 gam-phon | https://github.com/gam-phon
 cornelk | https://github.com/cornelk
-dependabot[bot] | https://github.com/apps/dependabot
 if1live | https://github.com/if1live
 lozdog245 | https://github.com/lozdog245
 soxipy | https://github.com/soxipy

README.md

@@ -1,4 +1,4 @@
-<img src="https://github.com/thrasher-corp/gocryptotrader/blob/master/web/src/assets/page-logo.png?raw=true" width="350px" height="350px" hspace="70">
+<img src="/common/gctlogo.png?raw=true" width="350px" height="350px" hspace="70">
 
 [![Build Status](https://github.com/thrasher-corp/gocryptotrader/actions/workflows/tests.yml/badge.svg?branch=master)](https://github.com/thrasher-corp/gocryptotrader/actions/workflows/tests.yml)
 [![Software License](https://img.shields.io/badge/License-MIT-orange.svg?style=flat-square)](https://github.com/thrasher-corp/gocryptotrader/blob/master/LICENSE)
@@ -143,16 +143,17 @@ Binaries will be published once the codebase reaches a stable condition.
 
 |User|Contribution Amount|
 |--|--|
-| [thrasher-](https://github.com/thrasher-) | 650 |
-| [shazbert](https://github.com/shazbert) | 202 |
-| [gloriousCode](https://github.com/gloriousCode) | 176 |
-| [dependabot-preview[bot]](https://github.com/apps/dependabot-preview) | 87 |
+| [thrasher-](https://github.com/thrasher-) | 654 |
+| [shazbert](https://github.com/shazbert) | 207 |
+| [gloriousCode](https://github.com/gloriousCode) | 179 |
+| [dependabot-preview[bot]](https://github.com/apps/dependabot-preview) | 88 |
 | [xtda](https://github.com/xtda) | 47 |
 | [Rots](https://github.com/Rots) | 15 |
 | [vazha](https://github.com/vazha) | 15 |
 | [ermalguni](https://github.com/ermalguni) | 14 |
-| [MadCozBadd](https://github.com/MadCozBadd) | 10 |
+| [MadCozBadd](https://github.com/MadCozBadd) | 12 |
 | [vadimzhukck](https://github.com/vadimzhukck) | 10 |
+| [dependabot[bot]](https://github.com/apps/dependabot) | 10 |
 | [140am](https://github.com/140am) | 8 |
 | [marcofranssen](https://github.com/marcofranssen) | 8 |
 | [dackroyd](https://github.com/dackroyd) | 5 |
@@ -165,7 +166,6 @@ Binaries will be published once the codebase reaches a stable condition.
 | [Christian-Achilli](https://github.com/Christian-Achilli) | 2 |
 | [gam-phon](https://github.com/gam-phon) | 2 |
 | [cornelk](https://github.com/cornelk) | 2 |
-| [dependabot[bot]](https://github.com/apps/dependabot) | 2 |
 | [if1live](https://github.com/if1live) | 2 |
 | [lozdog245](https://github.com/lozdog245) | 2 |
 | [soxipy](https://github.com/soxipy) | 2 |

backtester/backtest/backtest.go

@@ -40,6 +40,7 @@ import (
 	gctexchange "github.com/thrasher-corp/gocryptotrader/exchanges"
 	"github.com/thrasher-corp/gocryptotrader/exchanges/asset"
 	gctkline "github.com/thrasher-corp/gocryptotrader/exchanges/kline"
+	gctorder "github.com/thrasher-corp/gocryptotrader/exchanges/order"
 	"github.com/thrasher-corp/gocryptotrader/log"
 )
 
@@ -281,7 +282,7 @@ func (bt *BackTest) setupExchangeSettings(cfg *config.Config) (exchange.Exchange
 		sellRule.Validate()
 
 		limits, err := exch.GetOrderExecutionLimits(a, pair)
-		if err != nil {
+		if err != nil && !errors.Is(err, gctorder.ErrExchangeLimitNotLoaded) {
 			return resp, err
 		}
 
@@ -361,15 +362,26 @@ func (bt *BackTest) setupBot(cfg *config.Config, bot *engine.Engine) error {
 	if err != nil {
 		return err
 	}
-
+	bt.Bot.ExchangeManager = engine.SetupExchangeManager()
 	for i := range cfg.CurrencySettings {
 		err = bt.Bot.LoadExchange(cfg.CurrencySettings[i].ExchangeName, false, nil)
 		if err != nil && !errors.Is(err, engine.ErrExchangeAlreadyLoaded) {
 			return err
 		}
 	}
-	if !bt.Bot.OrderManager.Started() {
-		return bt.Bot.OrderManager.Start(bt.Bot)
+	if !bt.Bot.OrderManager.IsRunning() {
+		bt.Bot.OrderManager, err = engine.SetupOrderManager(
+			bt.Bot.ExchangeManager,
+			bt.Bot.CommunicationsManager,
+			&bt.Bot.ServicesWG,
+			bot.Settings.Verbose)
+		if err != nil {
+			return err
+		}
+		err = bt.Bot.OrderManager.Start()
+		if err != nil {
+			return err
+		}
 	}
 
 	return nil
@@ -470,18 +482,26 @@ func (bt *BackTest) loadData(cfg *config.Config, exch gctexchange.IBotExchange,
 		if cfg.DataSettings.DatabaseData.ConfigOverride != nil {
 			bt.Bot.Config.Database = *cfg.DataSettings.DatabaseData.ConfigOverride
 			gctdatabase.DB.DataPath = filepath.Join(gctcommon.GetDefaultDataDir(runtime.GOOS), "database")
-			gctdatabase.DB.Config = cfg.DataSettings.DatabaseData.ConfigOverride
-			err = bt.Bot.DatabaseManager.Start(bt.Bot)
+			err = gctdatabase.DB.SetConfig(cfg.DataSettings.DatabaseData.ConfigOverride)
 			if err != nil {
 				return nil, err
 			}
-			defer func() {
-				err = bt.Bot.DatabaseManager.Stop()
-				if err != nil {
-					log.Error(log.BackTester, err)
-				}
-			}()
 		}
+		bt.Bot.DatabaseManager, err = engine.SetupDatabaseConnectionManager(gctdatabase.DB.GetConfig())
+		if err != nil {
+			return nil, err
+		}
+
+		err = bt.Bot.DatabaseManager.Start(&bt.Bot.ServicesWG)
+		if err != nil {
+			return nil, err
+		}
+		defer func() {
+			stopErr := bt.Bot.DatabaseManager.Stop()
+			if stopErr != nil {
+				log.Error(log.BackTester, stopErr)
+			}
+		}()
 		resp, err = loadDatabaseData(cfg, exch.GetName(), fPair, a, dataType)
 		if err != nil {
 			return nil, fmt.Errorf("unable to retrieve data from GoCryptoTrader database. Error: %v. Please ensure the database is setup correctly and has data before use", err)
@@ -768,7 +788,7 @@ func (bt *BackTest) processDataEvent(e common.DataEventHandler) error {
 // updateStatsForDataEvent makes various systems aware of price movements from
 // data events
 func (bt *BackTest) updateStatsForDataEvent(e common.DataEventHandler) {
-	// update portfolio with latest price
+	// update portfoliomanager with latest price
 	err := bt.Portfolio.Update(e)
 	if err != nil {
 		log.Error(log.BackTester, err)

backtester/backtest/backtest_test.go

@@ -30,7 +30,7 @@ import (
 	gctkline "github.com/thrasher-corp/gocryptotrader/exchanges/kline"
 )
 
-const testExchange = "binance"
+const testExchange = "Bitstamp"
 
 func newBotWithExchange() (*engine.Engine, gctexchange.IBotExchange) {
 	bot, err := engine.NewFromSettings(&engine.Settings{
@@ -40,6 +40,7 @@ func newBotWithExchange() (*engine.Engine, gctexchange.IBotExchange) {
 	if err != nil {
 		log.Fatal(err)
 	}
+	bot.ExchangeManager = engine.SetupExchangeManager()
 	err = bot.LoadExchange(testExchange, false, nil)
 	if err != nil {
 		log.Fatal(err)
@@ -103,7 +104,7 @@ func TestNewFromConfig(t *testing.T) {
 	}
 
 	cfg.CurrencySettings[0].Base = "BTC"
-	cfg.CurrencySettings[0].Quote = "USDT"
+	cfg.CurrencySettings[0].Quote = "USD"
 
 	cfg.DataSettings.APIData = &config.APIData{
 		StartDate: time.Time{},
@@ -162,7 +163,7 @@ func TestLoadData(t *testing.T) {
 			Quote:        "test",
 		},
 	}
-	cfg.CurrencySettings[0].ExchangeName = testExchange
+	cfg.CurrencySettings[0].ExchangeName = "binance"
 	cfg.CurrencySettings[0].Asset = asset.Spot.String()
 	cfg.CurrencySettings[0].Base = "BTC"
 	cfg.CurrencySettings[0].Quote = "USDT"
@@ -193,7 +194,7 @@ func TestLoadData(t *testing.T) {
 		Reports: &report.Data{},
 	}
 
-	cp := currency.NewPair(currency.BTC, currency.USDT)
+	cp := currency.NewPair(currency.BTC, currency.USD)
 	_, err = bt.loadData(cfg, exch, cp, asset.Spot)
 	if err != nil {
 		t.Error(err)
@@ -241,7 +242,7 @@ func TestLoadData(t *testing.T) {
 
 func TestLoadDatabaseData(t *testing.T) {
 	t.Parallel()
-	cp := currency.NewPair(currency.BTC, currency.USDT)
+	cp := currency.NewPair(currency.BTC, currency.USD)
 	_, err := loadDatabaseData(nil, "", cp, "", -1)
 	if err != nil && !strings.Contains(err.Error(), "nil config data received") {
 		t.Error(err)

backtester/common/common_types.go

@@ -41,6 +41,8 @@ var (
 	ErrNilArguments = errors.New("received nil argument(s)")
 	// ErrNilEvent is a common error for whenever a nil event occurs when it shouldn't have
 	ErrNilEvent = errors.New("nil event received")
+	// ErrInvalidDataType occurs when an invalid data type is defined in the config
+	ErrInvalidDataType = errors.New("invalid datatype received")
 )
 
 // EventHandler interface implements required GetTime() & Pair() return

backtester/config/configbuilder/main.go

@@ -395,7 +395,10 @@ func parseDatabase(reader *bufio.Reader, cfg *config.Config) error {
 			}
 		}
 		cfg.DataSettings.DatabaseData.ConfigOverride.Port = uint16(port)
-		database.DB.Config = cfg.DataSettings.DatabaseData.ConfigOverride
+		err = database.DB.SetConfig(cfg.DataSettings.DatabaseData.ConfigOverride)
+		if err != nil {
+			return fmt.Errorf("database failed to set config: %w", err)
+		}
 		if cfg.DataSettings.DatabaseData.ConfigOverride.Driver == database.DBPostgreSQL {
 			_, err = dbPSQL.Connect()
 			if err != nil {

backtester/data/kline/api/api.go

@@ -44,7 +44,7 @@ func LoadData(dataType int64, startDate, endDate time.Time, interval time.Durati
 			return nil, fmt.Errorf("could not convert trade data to candles for %v %v %v, %v", exch.GetName(), a, fPair, err)
 		}
 	default:
-		return nil, fmt.Errorf("could not retrieve data for %v %v %v, invalid data type received", exch.GetName(), a, fPair)
+		return nil, fmt.Errorf("could not retrieve data for %v %v %v, %w", exch.GetName(), a, fPair, common.ErrInvalidDataType)
 	}
 	candles.Exchange = strings.ToLower(candles.Exchange)
 

backtester/data/kline/api/api_test.go

@@ -1,10 +1,10 @@
 package api
 
 import (
+	"errors"
 	"log"
 	"os"
 	"path/filepath"
-	"strings"
 	"testing"
 	"time"
 
@@ -33,6 +33,7 @@ func TestMain(m *testing.M) {
 		log.Fatal(err)
 	}
 
+	bot.ExchangeManager = engine.SetupExchangeManager()
 	err = bot.LoadExchange(testExchange, false, nil)
 	if err != nil {
 		log.Fatal(err)
@@ -62,15 +63,15 @@ func TestLoadCandles(t *testing.T) {
 	}
 
 	_, err = LoadData(-1, tt1, tt2, interval.Duration(), exch, p, a)
-	if err != nil && !strings.Contains(err.Error(), "could not retrieve data for Binance spot BTCUSDT, invalid data type received") {
-		t.Error(err)
+	if !errors.Is(err, common.ErrInvalidDataType) {
+		t.Errorf("expected '%v' received '%v'", err, common.ErrInvalidDataType)
 	}
 }
 
 func TestLoadTrades(t *testing.T) {
 	t.Parallel()
 	interval := gctkline.FifteenMin
-	tt1 := time.Now().Add(-time.Minute * 60).Round(interval.Duration())
+	tt1 := time.Now().Add(-time.Minute * 15).Round(interval.Duration())
 	tt2 := time.Now().Round(interval.Duration())
 	a := asset.Spot
 	p := currency.NewPair(currency.BTC, currency.USDT)

backtester/data/kline/csv/csv.go

@@ -149,7 +149,7 @@ func LoadData(dataType int64, filepath, exchangeName string, interval time.Durat
 			return nil, fmt.Errorf("could not read csv trade data for %v %v %v, %v", exchangeName, a, fPair, err)
 		}
 	default:
-		return nil, fmt.Errorf("could not process csv data for %v %v %v, invalid data type received", exchangeName, a, fPair)
+		return nil, fmt.Errorf("could not process csv data for %v %v %v, %w", exchangeName, a, fPair, common.ErrInvalidDataType)
 	}
 	resp.Item.Exchange = strings.ToLower(exchangeName)
 	resp.Item.Pair = fPair

backtester/data/kline/csv/csv_test.go

@@ -1,8 +1,8 @@
 package csv
 
 import (
+	"errors"
 	"path/filepath"
-	"strings"
 	"testing"
 
 	"github.com/thrasher-corp/gocryptotrader/backtester/common"
@@ -56,7 +56,7 @@ func TestLoadDataInvalid(t *testing.T) {
 		gctkline.FifteenMin.Duration(),
 		p,
 		a)
-	if err != nil && !strings.Contains(err.Error(), "could not process csv data for binance spot BTCUSDT, invalid data type received") {
-		t.Error(err)
+	if !errors.Is(err, common.ErrInvalidDataType) {
+		t.Errorf("expected '%v' received '%v'", err, common.ErrInvalidDataType)
 	}
 }

backtester/data/kline/database/database.go

@@ -48,7 +48,7 @@ func LoadData(startDate, endDate time.Time, interval time.Duration, exchangeName
 		}
 		resp.Item = klineItem
 	default:
-		return nil, fmt.Errorf("could not retrieve database data for %v %v %v, invalid data type received", exchangeName, a, fPair)
+		return nil, fmt.Errorf("could not retrieve database data for %v %v %v, %w", exchangeName, a, fPair, common.ErrInvalidDataType)
 	}
 	resp.Item.Exchange = strings.ToLower(resp.Item.Exchange)
 

backtester/data/kline/database/database_test.go

@@ -1,11 +1,11 @@
 package database
 
 import (
+	"errors"
 	"fmt"
 	"io/ioutil"
 	"os"
 	"path/filepath"
-	"strings"
 	"testing"
 	"time"
 
@@ -86,7 +86,11 @@ func TestLoadDataCandles(t *testing.T) {
 		t.Error(err)
 	}
 
-	err = bot.DatabaseManager.Start(bot)
+	bot.DatabaseManager, err = engine.SetupDatabaseConnectionManager(&bot.Config.Database)
+	if err != nil {
+		t.Error(err)
+	}
+	err = bot.DatabaseManager.Start(&bot.ServicesWG)
 	if err != nil {
 		t.Error(err)
 	}
@@ -161,7 +165,11 @@ func TestLoadDataTrades(t *testing.T) {
 		t.Error(err)
 	}
 
-	err = bot.DatabaseManager.Start(bot)
+	bot.DatabaseManager, err = engine.SetupDatabaseConnectionManager(&bot.Config.Database)
+	if err != nil {
+		t.Error(err)
+	}
+	err = bot.DatabaseManager.Start(&bot.ServicesWG)
 	if err != nil {
 		t.Error(err)
 	}
@@ -202,7 +210,7 @@ func TestLoadDataInvalid(t *testing.T) {
 	dStart := time.Date(2020, 1, 0, 0, 0, 0, 0, time.UTC)
 	dEnd := time.Date(2020, 1, 1, 0, 0, 0, 0, time.UTC)
 	_, err := LoadData(dStart, dEnd, gctkline.FifteenMin.Duration(), exch, -1, p, a)
-	if err != nil && !strings.Contains(err.Error(), "could not retrieve database data for binance spot BTCUSDT, invalid data type received") {
-		t.Error(err)
+	if !errors.Is(err, common.ErrInvalidDataType) {
+		t.Errorf("expected '%v' received '%v'", err, common.ErrInvalidDataType)
 	}
 }

backtester/data/kline/live/live.go

@@ -58,7 +58,7 @@ func LoadData(exch exchange.IBotExchange, dataType int64, interval time.Duration
 			}
 		}
 	default:
-		return nil, fmt.Errorf("could not retrieve live data for %v %v %v, invalid data type received", exch.GetName(), a, fPair)
+		return nil, fmt.Errorf("could not retrieve live data for %v %v %v, %w", exch.GetName(), a, fPair, common.ErrInvalidDataType)
 	}
 	candles.Exchange = strings.ToLower(exch.GetName())
 	return &candles, nil

backtester/data/kline/live/live_test.go

@@ -1,40 +1,38 @@
 package live
 
 import (
+	"errors"
+	"log"
 	"path/filepath"
-	"strings"
 	"testing"
 
 	"github.com/thrasher-corp/gocryptotrader/backtester/common"
+	"github.com/thrasher-corp/gocryptotrader/config"
 	"github.com/thrasher-corp/gocryptotrader/currency"
 	"github.com/thrasher-corp/gocryptotrader/engine"
 	"github.com/thrasher-corp/gocryptotrader/exchanges/asset"
 	gctkline "github.com/thrasher-corp/gocryptotrader/exchanges/kline"
 )
 
-const testExchange = "binance"
+const testExchange = "FTX"
 
 func TestLoadCandles(t *testing.T) {
 	t.Parallel()
 	interval := gctkline.FifteenMin
-	bot, err := engine.NewFromSettings(&engine.Settings{
-		ConfigFile:   filepath.Join("..", "..", "..", "..", "testdata", "configtest.json"),
-		EnableDryRun: true,
-	}, nil)
+	bot := new(engine.Engine)
+	bot.Config = &config.Config{}
+	err := bot.Config.LoadConfig(filepath.Join("..", "..", "..", "..", "testdata", "configtest.json"), true)
 	if err != nil {
-		t.Error(err)
+		t.Fatalf("SetupTest: Failed to load config: %s", err)
 	}
-
+	bot.ExchangeManager = engine.SetupExchangeManager()
 	err = bot.LoadExchange(testExchange, false, nil)
 	if err != nil {
-		t.Fatal(err)
-	}
-	exch := bot.GetExchangeByName(testExchange)
-	if exch == nil {
-		t.Fatal("expected binance")
+		log.Fatal(err)
 	}
+	exch := bot.ExchangeManager.GetExchangeByName(testExchange)
 	a := asset.Spot
-	p := currency.NewPair(currency.BTC, currency.USDT)
+	p := currency.NewPair(currency.BTC, currency.USD)
 	var data *gctkline.Item
 	data, err = LoadData(exch, common.DataCandle, interval.Duration(), p, a)
 	if err != nil {
@@ -45,8 +43,8 @@ func TestLoadCandles(t *testing.T) {
 	}
 
 	_, err = LoadData(exch, -1, interval.Duration(), p, a)
-	if err != nil && !strings.Contains(err.Error(), "could not retrieve live data for Binance spot BTCUSDT, invalid data type received") {
-		t.Error(err)
+	if !errors.Is(err, common.ErrInvalidDataType) {
+		t.Errorf("expected '%v' received '%v'", err, common.ErrInvalidDataType)
 	}
 }
 
@@ -60,6 +58,7 @@ func TestLoadTrades(t *testing.T) {
 	if err != nil {
 		t.Error(err)
 	}
+	bot.ExchangeManager = engine.SetupExchangeManager()
 
 	err = bot.LoadExchange(testExchange, false, nil)
 	if err != nil {

backtester/eventhandlers/exchange/exchange_test.go

@@ -139,7 +139,13 @@ func TestPlaceOrder(t *testing.T) {
 	if err != nil {
 		t.Fatal(err)
 	}
-	err = bot.OrderManager.Start(bot)
+	em := engine.SetupExchangeManager()
+	bot.ExchangeManager = em
+	bot.OrderManager, err = engine.SetupOrderManager(em, &engine.CommunicationManager{}, &bot.ServicesWG, false)
+	if err != nil {
+		t.Error(err)
+	}
+	err = bot.OrderManager.Start()
 	if err != nil {
 		t.Error(err)
 	}
@@ -187,7 +193,13 @@ func TestExecuteOrder(t *testing.T) {
 		t.Fatal(err)
 	}
 
-	err = bot.OrderManager.Start(bot)
+	em := engine.SetupExchangeManager()
+	bot.ExchangeManager = em
+	bot.OrderManager, err = engine.SetupOrderManager(em, &engine.CommunicationManager{}, &bot.ServicesWG, false)
+	if err != nil {
+		t.Error(err)
+	}
+	err = bot.OrderManager.Start()
 	if err != nil {
 		t.Error(err)
 	}
@@ -287,8 +299,14 @@ func TestExecuteOrderBuySellSizeLimit(t *testing.T) {
 	if err != nil {
 		t.Fatal(err)
 	}
+	em := engine.SetupExchangeManager()
+	bot.ExchangeManager = em
+	bot.OrderManager, err = engine.SetupOrderManager(em, &engine.CommunicationManager{}, &bot.ServicesWG, false)
+	if err != nil {
+		t.Error(err)
+	}
 
-	err = bot.OrderManager.Start(bot)
+	err = bot.OrderManager.Start()
 	if err != nil {
 		t.Error(err)
 	}

cmd/apichecker/README.md

@@ -1,6 +1,6 @@
 # GoCryptoTrader package Apichecker
 
-<img src="https://github.com/thrasher-corp/gocryptotrader/blob/master/web/src/assets/page-logo.png?raw=true" width="350px" height="350px" hspace="70">
+<img src="/common/gctlogo.png?raw=true" width="350px" height="350px" hspace="70">
 
 
 [![Build Status](https://github.com/thrasher-corp/gocryptotrader/actions/workflows/tests.yml/badge.svg?branch=master)](https://github.com/thrasher-corp/gocryptotrader/actions/workflows/tests.yml)

cmd/config/config_test.go

@@ -6,13 +6,13 @@ func TestEncryptOrDecrypt(t *testing.T) {
 	reValue := EncryptOrDecrypt(true)
 	if reValue != "encrypted" {
 		t.Error(
-			"Tools/Config/Config_test.go - EncryptOrDecrypt Error",
+			"expected encrypted",
 		)
 	}
 	reValue = EncryptOrDecrypt(false)
 	if reValue != "decrypted" {
 		t.Error(
-			"Tools/Config/Config_test.go - EncryptOrDecrypt Error",
+			"expected decrypted",
 		)
 	}
 }

cmd/documentation/README.md

@@ -1,6 +1,6 @@
 # GoCryptoTrader package Documentation
 
-<img src="https://github.com/thrasher-corp/gocryptotrader/blob/master/web/src/assets/page-logo.png?raw=true" width="350px" height="350px" hspace="70">
+<img src="/common/gctlogo.png?raw=true" width="350px" height="350px" hspace="70">
 
 
 [![Build Status](https://github.com/thrasher-corp/gocryptotrader/actions/workflows/tests.yml/badge.svg?branch=master)](https://github.com/thrasher-corp/gocryptotrader/actions/workflows/tests.yml)

cmd/documentation/documentation.go

@@ -63,7 +63,8 @@ var (
 	repoDir string
 	// is a broken down version of the documentation tool dir for cross platform
 	// checking
-	ref = []string{"gocryptotrader", "cmd", "documentation"}
+	ref          = []string{"gocryptotrader", "cmd", "documentation"}
+	engineFolder = "engine"
 )
 
 // Contributor defines an account associated with this code base by doing
@@ -121,8 +122,8 @@ func main() {
 	}
 
 	if strings.Contains(wd, filepath.Join(ref...)) {
-		rootdir := filepath.Dir(filepath.Dir(wd))
-		repoDir = rootdir
+		rootDir := filepath.Dir(filepath.Dir(wd))
+		repoDir = rootDir
 		toolDir = wd
 	} else {
 		if toolDir == "" {
@@ -132,7 +133,7 @@ func main() {
 		repoDir = wd
 	}
 
-	fmt.Println(core.Banner)
+	fmt.Print(core.Banner)
 	fmt.Println("This will update and regenerate documentation for the different packages in your repo.")
 	fmt.Println()
 
@@ -272,15 +273,11 @@ func main() {
 		fmt.Println("All core systems fetched, updating documentation...")
 	}
 
-	err = UpdateDocumentation(DocumentationDetails{
+	UpdateDocumentation(DocumentationDetails{
 		dirList,
 		tmpl,
 		contributors,
 		&config})
-	if err != nil {
-		log.Fatalf("Documentation Generation Tool - UpdateDocumentation error %s",
-			err)
-	}
 
 	fmt.Println("\nDocumentation Generation Tool - Finished")
 }
@@ -355,7 +352,7 @@ func GetProjectDirectoryTree(c *Config) ([]string, error) {
 		directoryData = append(directoryData, filepath.Join(repoDir, ContributorFile))
 	}
 
-	walkfn := func(path string, info os.FileInfo, err error) error {
+	walkFn := func(path string, info os.FileInfo, err error) error {
 		if err != nil {
 			return err
 		}
@@ -376,7 +373,7 @@ func GetProjectDirectoryTree(c *Config) ([]string, error) {
 		return nil
 	}
 
-	return directoryData, filepath.Walk(repoDir, walkfn)
+	return directoryData, filepath.Walk(repoDir, walkFn)
 }
 
 // GetTemplateFiles parses and returns all template files in the documentation
@@ -384,7 +381,7 @@ func GetProjectDirectoryTree(c *Config) ([]string, error) {
 func GetTemplateFiles() (*template.Template, error) {
 	tmpl := template.New("")
 
-	walkfn := func(path string, info os.FileInfo, err error) error {
+	walkFn := func(path string, info os.FileInfo, err error) error {
 		if err != nil {
 			return err
 		}
@@ -406,7 +403,7 @@ func GetTemplateFiles() (*template.Template, error) {
 		return nil
 	}
 
-	return tmpl, filepath.Walk(toolDir, walkfn)
+	return tmpl, filepath.Walk(toolDir, walkFn)
 }
 
 // GetContributorList fetches a list of contributors from the github api
@@ -458,14 +455,14 @@ func GetGoDocURL(name string) string {
 
 // UpdateDocumentation generates or updates readme/documentation files across
 // the codebase
-func UpdateDocumentation(details DocumentationDetails) error {
+func UpdateDocumentation(details DocumentationDetails) {
 	for i := range details.Directories {
-		cutset := details.Directories[i][len(repoDir):]
-		if cutset != "" && cutset[0] == os.PathSeparator {
-			cutset = cutset[1:]
+		cutSet := details.Directories[i][len(repoDir):]
+		if cutSet != "" && cutSet[0] == os.PathSeparator {
+			cutSet = cutSet[1:]
 		}
 
-		data := strings.Split(cutset, string(os.PathSeparator))
+		data := strings.Split(cutSet, string(os.PathSeparator))
 
 		var temp []string
 		for x := range data {
@@ -491,40 +488,66 @@ func UpdateDocumentation(details DocumentationDetails) error {
 			}
 			continue
 		}
-
+		if name == engineFolder {
+			d, err := ioutil.ReadDir(details.Directories[i])
+			if err != nil {
+				fmt.Println("Excluding file:", err)
+			}
+			for x := range d {
+				nameSplit := strings.Split(d[x].Name(), ".go")
+				engineTemplateName := engineFolder + " " + nameSplit[0]
+				if details.Tmpl.Lookup(engineTemplateName) == nil {
+					fmt.Printf("Template not found for path %s create new template with {{define \"%s\" -}} TEMPLATE HERE {{end}}\n",
+						details.Directories[i],
+						name)
+					continue
+				}
+				err = runTemplate(details, filepath.Join(details.Directories[i], nameSplit[0]+".md"), engineTemplateName)
+				if err != nil {
+					fmt.Println(err)
+				}
+			}
+			continue
+		}
 		if details.Tmpl.Lookup(name) == nil {
 			fmt.Printf("Template not found for path %s create new template with {{define \"%s\" -}} TEMPLATE HERE {{end}}\n",
 				details.Directories[i],
 				name)
 			continue
 		}
-
 		var mainPath string
-		if name == LicenseFile || name == ContributorFile {
+		switch {
+		case name == LicenseFile || name == ContributorFile:
 			mainPath = details.Directories[i]
-		} else {
+		default:
 			mainPath = filepath.Join(details.Directories[i], "README.md")
 		}
 
-		err := os.Remove(mainPath)
-		if err != nil && !(strings.Contains(err.Error(), "no such file or directory") ||
-			strings.Contains(err.Error(), "The system cannot find the file specified.")) {
-			return err
+		if err := runTemplate(details, mainPath, name); err != nil {
+			log.Println(err)
+			continue
 		}
-
-		file, err := os.Create(mainPath)
-		if err != nil {
-			return err
-		}
-
-		attr := GetDocumentationAttributes(name, details.Contributors)
-
-		err = details.Tmpl.ExecuteTemplate(file, name, attr)
-		if err != nil {
-			file.Close()
-			return err
-		}
-		file.Close()
 	}
-	return nil
+}
+
+func runTemplate(details DocumentationDetails, mainPath, name string) error {
+	err := os.Remove(mainPath)
+	if err != nil && !(strings.Contains(err.Error(), "no such file or directory") ||
+		strings.Contains(err.Error(), "The system cannot find the file specified.")) {
+		return err
+	}
+
+	f, err := os.Create(mainPath)
+	if err != nil {
+		return err
+	}
+	defer func(f *os.File) {
+		err := f.Close()
+		if err != nil {
+			log.Printf("could not close file %s: %v", mainPath, err)
+		}
+	}(f)
+
+	attr := GetDocumentationAttributes(name, details.Contributors)
+	return details.Tmpl.ExecuteTemplate(f, name, attr)
 }

cmd/documentation/engine_templates/apiserver.tmpl

@@ -0,0 +1,28 @@
+{{define "engine apiserver" -}}
+{{template "header" .}}
+## Current Features for {{.CapitalName}}
++ The API server subsystem is a deprecated service used to host a REST or websocket server to interact with some functions of GoCryptoTrader
++ This subsystem is no longer maintained and it is highly encouraged to interact with GRPC endpoints directly where possible
++ In order to modify the behaviour of the API server subsystem, you can edit the following inside your config file:
+
+### deprecatedRPC
+
+| Config | Description | Example |
+| ------ | ----------- | ------- |
+| enabled | If enabled will create a REST server which will listen to commands on the listen address | `true` |
+| listenAddress | If enabled will listen for REST requests on this address and return a JSON response | `localhost:9050` |
+
+### websocketRPC
+
+| Config | Description | Example |
+| ------ | ----------- | ------- |
+| enabled | If enabled will create a REST server which will listen to commands on the listen address | `true` |
+| listenAddress | If enabled will listen for requests on this address and return a JSON response | `localhost:9051` |
+| connectionLimit | Defines how many connections the websocket RPC server can handle simultanesoly | `1` |
+| maxAuthFailures | For authenticated endpoints, the amount of failed attempts allowed before disconnection | `3` |
+| allowInsecureOrigin | Allows use of insecure connections | `true` |
+
+### Please click GoDocs chevron above to view current GoDoc information for this package
+{{template "contributions"}}
+{{template "donations" .}}
+{{end}}

cmd/documentation/engine_templates/communication_manager.tmpl

@@ -0,0 +1,56 @@
+{{define "engine communication_manager" -}}
+{{template "header" .}}
+## Current Features for {{.CapitalName}}
++ The communication manager subsystem is used to push events raised in GoCryptoTrader to any enabled communication system such as a Slack server
++ In order to modify the behaviour of the communication manager subsystem, you can edit the following inside your config file under `communications`:
+
+### slack
+
+| Config | Description | Example |
+| ------ | ----------- | ------- |
+| enabled | Determines whether the push communications to a Slack server | `true` |
+| verbose | If enabled will log more details to your logger output | `false` |
+| targetChannel | The channel to send communications to | `announcements` |
+| verificationToken | The token generated by Slack to allow interactions with the server and channel | `iamafaketoken` |
+
+### smsGlobal
+
+| Config | Description | Example |
+| ------ | ----------- | ------- |
+| name | The name of the SMS sender | `SMSGlobal` |
+| from | Who the text name is from | `Skynet` |
+| enabled | Determines whether the push communications to the SMS service | `true` |
+| verbose | If enabled will log more details to your logger output | `false` |
+| username | The username to use with the SMS provider | `username` |
+| password | The username to use with the SMS provider | `password` |
+| contacts | The `name` `number` of the user people you wish to send SMS to and whether it is `enabled` | `"name": "StyleGherkin", "number": "1231424", "enabled": true` |
+
+### smtp
+
+| Config | Description | Example |
+| ------ | ----------- | ------- |
+| name | The name of the service | `SMTP` |
+| enabled | Determines whether the push communications to a email server | `true` |
+| verbose | If enabled will log more details to your logger output | `false` |
+| host | The SMTP host | `smtp.google.com` |
+| port | The port to use | `537` |
+| accountName | Your username | `username` |
+| accountPassword | Your password | `password` |
+| from | The display name of the sender | `Jeff Bezos` |
+| recipientList | A comma delimited list of addresses to send alerts to | `bill@gates.com` |
+
+### telegram
+
+| Config | Description | Example |
+| ------ | ----------- | ------- |
+| name | The name to be displayed | `Telegram` |
+| enabled | Determines whether the push communications to a Telegram server | `true` |
+| verbose | If enabled will log more details to your logger output | `false` |
+| verificationToken | The token generated by Telegram to allow you to send messages | `iamafaketoken` |
+
+
+
+### Please click GoDocs chevron above to view current GoDoc information for this package
+{{template "contributions"}}
+{{template "donations" .}}
+{{end}}

cmd/documentation/engine_templates/connection_manager.tmpl

@@ -0,0 +1,19 @@
+{{define "engine connection_manager" -}}
+{{template "header" .}}
+## Current Features for {{.CapitalName}}
++ The connection manager subsystem is used to periodically check whether the application is connected to the internet and will provide alerts of any changes
++ In order to modify the behaviour of the connection manager subsystem, you can edit the following inside your config file under `connectionMonitor`:
+
+### connectionMonitor
+
+| Config | Description | Example |
+| ------ | ----------- | ------- |
+| perferredDNSList | Is a string array of DNS servers to periodically verify whether GoCryptoTrader is connected to the internet |  `["8.8.8.8","8.8.4.4","1.1.1.1","1.0.0.1"]` |
+| preferredDomainList |  Is a string array of domains to periodically verify whether GoCryptoTrader is connected to the internet |  `["www.google.com","www.cloudflare.com","www.facebook.com"]` |
+| checkInterval | A time period in golang `time.Duration` format to check whether GoCryptoTrader is connected to the internet | `1000000000` |
+
+
+### Please click GoDocs chevron above to view current GoDoc information for this package
+{{template "contributions"}}
+{{template "donations" .}}
+{{end}}

cmd/documentation/engine_templates/database_connection.tmpl

@@ -0,0 +1,30 @@
+{{define "engine database_connection" -}}
+{{template "header" .}}
+## Current Features for {{.CapitalName}}
++ The database connection manager subsystem is used to periodically check whether the application is connected to the database and will provide alerts of any changes
++ In order to modify the behaviour of the database connection manager subsystem, you can edit the following inside your config file under `database`:
+
+### database
+
+| Config | Description | Example |
+| ------ | ----------- | ------- |
+| enabled | Enabled or disables the database connection subsystem |  `true` |
+| verbose | Displays more information to the logger which can be helpful for debugging | `false` |
+| driver | The SQL driver to use. Can be `postgres` or `sqlite`. | `sqlite` |
+| connectionDetails | See below |  |
+
+### connectionDetails
+
+| Config | Description | Example |
+| ------ | ----------- | ------- |
+| host | The host address of the database |  `localhost` |
+| port |  The port used to connect to the database |  `5432` |
+| username | An optional username to connect to the database | `username` |
+| password | An optional password to connect to the database | `password` |
+| database | The name of the database | `database.db` |
+| sslmode | The connection type of the database for Postgres databases only | `disable` |
+
+### Please click GoDocs chevron above to view current GoDoc information for this package
+{{template "contributions"}}
+{{template "donations" .}}
+{{end}}

cmd/documentation/engine_templates/depositaddress.tmpl

@@ -0,0 +1,11 @@
+{{define "engine depositaddress" -}}
+{{template "header" .}}
+## Current Features for {{.CapitalName}}
++ The deposit address manager subsystem stores Exchange deposit addresses.
++ On start of the application the engine Bot will retrieve deposit addresses from exchanges if you have API keys set
+
+
+### Please click GoDocs chevron above to view current GoDoc information for this package
+{{template "contributions"}}
+{{template "donations" .}}
+{{end}}

cmd/documentation/engine_templates/event_manager.tmpl

@@ -0,0 +1,18 @@
+{{define "engine event_manager" -}}
+{{template "header" .}}
+## Current Features for {{.CapitalName}}
++ The event manager subsystem is used to push events to communication systems such as Slack
++ The only configurable aspects of the event manager are the delays between receiving an event and pushing it and enabling verbose:
+
+### connectionMonitor
+
+| Config | Description | Example |
+| ------ | ----------- | ------- |
+| eventmanagerdelay | Sets the event managers sleep delay between event checking by a Golang `time.Duration` |  `0` |
+| verbose | Outputs debug messaging allowing for greater transparency for what the event manager is doing |  `false` |
+
+
+### Please click GoDocs chevron above to view current GoDoc information for this package
+{{template "contributions"}}
+{{template "donations" .}}
+{{end}}

cmd/documentation/engine_templates/exchange_manager.tmpl

@@ -0,0 +1,11 @@
+{{define "engine exchange_manager" -}}
+{{template "header" .}}
+## Current Features for {{.CapitalName}}
++ The exchange manager subsystem is used load and store exchanges so that the engine Bot can use them to track orderbooks, submit orders etc etc
++ The exchange manager itself is not customisable, it is always enabled.
++ The exchange manager by default will load all exchanges that are enabled in your config, however, it will also load exchanges by request via GRPC commands
+
+### Please click GoDocs chevron above to view current GoDoc information for this package
+{{template "contributions"}}
+{{template "donations" .}}
+{{end}}

cmd/documentation/engine_templates/ntp_manager.tmpl

@@ -0,0 +1,22 @@
+{{define "engine ntp_manager" -}}
+{{template "header" .}}
+## Current Features for {{.CapitalName}}
++ The NTP manager subsystem is used highlight discrepancies between your system time and specified NTP server times
++ It is useful for debugging and understanding why a request to an exchange may be rejected
++ The NTP manager cannot update your system clock, so when it does alert you of issues, you must take it upon yourself to change your system time in the event your requests are being rejected for being too far out of sync
++ In order to modify the behaviour of the NTP manager subsystem, you can edit the following inside your config file under `ntpclient`:
+
+### ntpclient
+
+| Config | Description | Example |
+| ------ | ----------- | ------- |
+| enabled | An integer value representing whether the NTP manager is enabled. It will warn you of time sync discrepancies on startup with a value of 0 and will alert you periodically with a value of 1. A value of -1 will disable the manager  |  `1` |
+| pool | A string array of the NTP servers to check for time discrepancies |  `["0.pool.ntp.org:123","pool.ntp.org:123"]` |
+| allowedDifference | A Golang time.Duration representation of the allowable time discrepancy between NTP server and your system time. Any discrepancy greater than this allowance will display an alert to your logging output |  `50000000` |
+| allowedNegativeDifference | A Golang time.Duration representation of the allowable negative time discrepancy between NTP server and your system time. Any discrepancy greater than this allowance will display an alert to your logging output |  `50000000` |
+
+
+### Please click GoDocs chevron above to view current GoDoc information for this package
+{{template "contributions"}}
+{{template "donations" .}}
+{{end}}

cmd/documentation/engine_templates/order_manager.tmpl

@@ -0,0 +1,11 @@
+{{define "engine order_manager" -}}
+{{template "header" .}}
+## Current Features for {{.CapitalName}}
++ The order manager subsystem stores and monitors all orders from enabled exchanges with API keys and `authenticatedSupport` enabled
++ It can be enabled or disabled via runtime command `-ordermanager=false` and defaults to true
++ All orders placed via GoCryptoTrader will be added to the order manager store
+
+### Please click GoDocs chevron above to view current GoDoc information for this package
+{{template "contributions"}}
+{{template "donations" .}}
+{{end}}

cmd/documentation/engine_templates/portfolio_manager.tmpl

@@ -0,0 +1,32 @@
+{{define "engine portfolio_manager" -}}
+{{template "header" .}}
+## Current Features for {{.CapitalName}}
++ The portfolio manager subsystem is used to synchronise and monitor wallet addresses
++ It can read addresses specified in your config file
++ If you have set API keys for an enabled exchange and enabled `authenticatedSupport`, it will store your exchange addresses
++ In order to modify the behaviour of the portfolio manager subsystem, you can edit the following inside your config file under `portfolioAddresses`:
+
+### portfolioAddresses
+
+| Config | Description | Example |
+| ------ | ----------- | ------- |
+| Verbose | Enabling this will output more detailed logs to your logging output  |  `false` |
+| addresses | An array of portfolio wallet addresses to monitor, see below table |   |
+
+### addresses
+
+| Config | Description | Example |
+| ------ | ----------- | ------- |
+| Address | The wallet address  |  `{{.DonationAddress}}` |
+| CoinType | The coin for the wallet address | `BTC` |
+| Balance | The balance of the wallet |   |
+| Description | A customisable description  | `My secret billion stash`  |
+| WhiteListed | Determines whether GoCryptoTrader withdraw manager subsystem can make withdrawals from this address | `true` |
+| ColdStorage | Describes whether the wallet address is a cold storage wallet eg Ledger | `false`  |
+| SupportedExchanges | A comma delimited string of which exchanges are allowed to interact with this wallet | `"Binance"`  |
+
+
+### Please click GoDocs chevron above to view current GoDoc information for this package
+{{template "contributions"}}
+{{template "donations" .}}
+{{end}}

cmd/documentation/engine_templates/subsystem_types.tmpl

@@ -0,0 +1,13 @@
+{{define "engine subsystem_types" -}}
+{{template "header" .}}
+## Current Features for {{.CapitalName}}
++ Subsystem contains subsystems that are used at run time by an `engine.Engine`, however they can be setup and run individually.
++ Subsystems are designed to be self contained
++ All subsystems have a public `Setup(...) (..., error)` function to return a valid subsystem ready for use
+  + Subsystems which are designed to be switched off also have `Start(...) error`, `IsRunning() bool` and `Stop(...) error` functions to allow the main `engine.Engine` instance to manage them
++ Common subsystem types such as errors can be found within the `subsystem.go` file
+
+### Please click GoDocs chevron above to view current GoDoc information for this package
+{{template "contributions"}}
+{{template "donations" .}}
+{{end}}

cmd/documentation/engine_templates/sync_manager.tmpl

@@ -0,0 +1,22 @@
+{{define "engine sync_manager" -}}
+{{template "header" .}}
+## Current Features for {{.CapitalName}}
++ The currency pair syncer subsystem is used to keep all trades, tickers and orderbooks up to date for all enabled exchange asset currency pairs
++ It can sync data via a websocket connection or REST and will switch between them if there has been no updates
++ In order to modify the behaviour of the currency pair syncer subsystem, you can change runtime parameters as detailed below:
+
+| Config | Description | Example |
+| ------ | ----------- | ------- |
+| syncmanager | Determines whether the subsystem is enabled | `true` |
+| tickersync |  Enables ticker syncing for all enabled exchanges |   `true`|
+| orderbooksync | Enables orderbook syncing for all enabled exchanges |  `true` |
+| tradesync | Enables trade syncing for all enabled exchanges |  `true` |
+| syncworkers | The amount of workers (goroutines) to use for syncing exchange data | `15` |
+| synccontinuously | Whether to sync exchange data continuously (ticker, orderbook and trades) | `true` |
+| synctimeout | The amount of time in golang `time.Duration` format before the syncer will switch from one protocol to the other (e.g. from REST to websocket) | `15000000000` |
+
+
+### Please click GoDocs chevron above to view current GoDoc information for this package
+{{template "contributions"}}
+{{template "donations" .}}
+{{end}}

cmd/documentation/engine_templates/websocketroutine_manager.tmpl

@@ -0,0 +1,14 @@
+{{define "engine websocketroutine_manager" -}}
+{{template "header" .}}
+## Current Features for {{.CapitalName}}
++ The websocket routine manager subsystem is used process websocket data in a unified manner across enabled exchanges with websocket support
++ It can help process orders to the order manager subsystem when it receives new data
++ Logs output of ticker and orderbook updates
++ The websocket routine manager subsystem can be enabled or disabled via runtime command `-websocketroutine=false` defaulting to true
++ Logs can be customised to display values the config value `fiatDisplayCurrency` under `currencyConfig`
+
+
+### Please click GoDocs chevron above to view current GoDoc information for this package
+{{template "contributions"}}
+{{template "donations" .}}
+{{end}}

cmd/documentation/engine_templates/withdraw_manager.tmpl

@@ -0,0 +1,15 @@
+{{define "engine withdraw_manager" -}}
+{{template "header" .}}
+## Current Features for {{.CapitalName}}
++ The withdraw manager subsystem is responsible for the processing of withdrawal requests and submitting them to exchanges
++ The withdraw manager can be interacted with via GRPC commands such as `WithdrawFiatRequest` and `WithdrawCryptoRequest`
++ Supports caching of responses to allow for quick viewing of withdrawal events via GRPC
++ If the database is enabled, withdrawal events are stored to the database for later viewing
++ Will not process withdrawal events if `dryrun` is true
++ The withdraw manager subsystem is always enabled
+
+
+### Please click GoDocs chevron above to view current GoDoc information for this package
+{{template "contributions"}}
+{{template "donations" .}}
+{{end}}

cmd/documentation/root_templates/root_readme.tmpl

@@ -1,5 +1,5 @@
 {{define "root" -}}
-<img src="https://github.com/thrasher-corp/gocryptotrader/blob/master/web/src/assets/page-logo.png?raw=true" width="350px" height="350px" hspace="70">
+<img src="/common/gctlogo.png?raw=true" width="350px" height="350px" hspace="70">
 
 [![Build Status](https://github.com/thrasher-corp/gocryptotrader/actions/workflows/tests.yml/badge.svg?branch=master)](https://github.com/thrasher-corp/gocryptotrader/actions/workflows/tests.yml)
 [![Software License](https://img.shields.io/badge/License-MIT-orange.svg?style=flat-square)](https://github.com/thrasher-corp/gocryptotrader/blob/master/LICENSE)

cmd/documentation/sub_templates/header.tmpl

@@ -1,7 +1,7 @@
 {{define "header" -}}
 # GoCryptoTrader package {{.CapitalName}}
 
-<img src="https://github.com/thrasher-corp/gocryptotrader/blob/master/web/src/assets/page-logo.png?raw=true" width="350px" height="350px" hspace="70">
+<img src="/common/gctlogo.png?raw=true" width="350px" height="350px" hspace="70">
 
 {{template "status" .NameURL}}
 

cmd/exchange_template/readme_file.tmpl

@@ -1,7 +1,7 @@
 {{- define "readme"}}
 # GoCryptoTrader {{.CapitalName}} Exchange Wrapper
 
-<img src="https://github.com/thrasher-corp/gocryptotrader/blob/master/web/src/assets/page-logo.png?raw=true" width="350px" height="350px" hspace="70">
+<img src="/common/gctlogo.png?raw=true" width="350px" height="350px" hspace="70">
 
 An exchange interface wrapper for the GoCryptoTrader application.
 

cmd/exchange_wrapper_issues/main.go

@@ -730,13 +730,17 @@ func testWrappers(e exchange.IBotExchange, base *exchange.Base, config *Config)
 			},
 			Amount: config.OrderSubmission.Amount,
 		}
-		var withdrawCryptocurrencyFundsResponse *withdraw.ExchangeResponse
-		withdrawCryptocurrencyFundsResponse, err = e.WithdrawCryptocurrencyFunds(&withdrawRequest)
 		msg = ""
+		err = withdrawRequest.Validate()
 		if err != nil {
 			msg = err.Error()
 			responseContainer.ErrorCount++
 		}
+		withdrawCryptocurrencyFundsResponse, err := e.WithdrawCryptocurrencyFunds(&withdrawRequest)
+		if err != nil {
+			msg += ", " + err.Error()
+			responseContainer.ErrorCount++
+		}
 		responseContainer.EndpointResponses = append(responseContainer.EndpointResponses, EndpointResponse{
 			SentParams: jsonifyInterface([]interface{}{withdrawRequest}),
 			Function:   "WithdrawCryptocurrencyFunds",
@@ -796,8 +800,7 @@ func testWrappers(e exchange.IBotExchange, base *exchange.Base, config *Config)
 				IntermediaryBankCode:          config.BankDetails.IntermediaryBankCode,
 			},
 		}
-		var withdrawFiatFundsResponse *withdraw.ExchangeResponse
-		withdrawFiatFundsResponse, err = e.WithdrawFiatFunds(&withdrawRequestFiat)
+		withdrawFiatFundsResponse, err := e.WithdrawFiatFunds(&withdrawRequestFiat)
 		msg = ""
 		if err != nil {
 			msg = err.Error()
@@ -810,8 +813,7 @@ func testWrappers(e exchange.IBotExchange, base *exchange.Base, config *Config)
 			Response:   withdrawFiatFundsResponse,
 		})
 
-		var withdrawFiatFundsInternationalResponse *withdraw.ExchangeResponse
-		withdrawFiatFundsInternationalResponse, err = e.WithdrawFiatFundsToInternationalBank(&withdrawRequestFiat)
+		withdrawFiatFundsInternationalResponse, err := e.WithdrawFiatFundsToInternationalBank(&withdrawRequestFiat)
 		msg = ""
 		if err != nil {
 			msg = err.Error()

cmd/gctcli/commands.go

@@ -2225,11 +2225,11 @@ var addEventCommand = cli.Command{
 		},
 		cli.BoolFlag{
 			Name:  "check_bids",
-			Usage: "whether to check the bids (if false, asks will be used)",
+			Usage: "whether to check the bids",
 		},
 		cli.BoolFlag{
-			Name:  "check_bids_and_asks",
-			Usage: "the wallet address",
+			Name:  "check_asks",
+			Usage: "whether to check the asks",
 		},
 		cli.Float64Flag{
 			Name:  "orderbook_amount",
@@ -2260,7 +2260,7 @@ func addEvent(c *cli.Context) error {
 	var condition string
 	var price float64
 	var checkBids bool
-	var checkBidsAndAsks bool
+	var checkAsks bool
 	var orderbookAmount float64
 	var currencyPair string
 	var assetType string
@@ -2296,8 +2296,8 @@ func addEvent(c *cli.Context) error {
 		checkBids = c.Bool("check_bids")
 	}
 
-	if c.IsSet("check_bids_and_asks") {
-		checkBids = c.Bool("check_bids_and_asks")
+	if c.IsSet("check_asks") {
+		checkAsks = c.Bool("check_asks")
 	}
 
 	if c.IsSet("orderbook_amount") {
@@ -2345,11 +2345,11 @@ func addEvent(c *cli.Context) error {
 		Exchange: exchangeName,
 		Item:     item,
 		ConditionParams: &gctrpc.ConditionParams{
-			Condition:        condition,
-			Price:            price,
-			CheckBids:        checkBids,
-			CheckBidsAndAsks: checkBidsAndAsks,
-			OrderbookAmount:  orderbookAmount,
+			Condition:       condition,
+			Price:           price,
+			CheckBids:       checkBids,
+			CheckAsks:       checkAsks,
+			OrderbookAmount: orderbookAmount,
 		},
 		Pair: &gctrpc.CurrencyPair{
 			Delimiter: p.Delimiter,

cmd/gen_sqlboiler_config/main.go

@@ -93,5 +93,6 @@ func convertGCTtoSQLBoilerConfig(c *database.Config) {
 
 // getLoadedDBPath gets the path loaded by 'database/drivers/sqlite3'
 func getLoadedDBPath() string {
-	return filepath.Join(database.DB.DataPath, database.DB.Config.Database)
+	cfg := database.DB.GetConfig()
+	return filepath.Join(database.DB.DataPath, cfg.Database)
 }

common/README.md

@@ -1,6 +1,6 @@
 # GoCryptoTrader package Common
 
-<img src="https://github.com/thrasher-corp/gocryptotrader/blob/master/web/src/assets/page-logo.png?raw=true" width="350px" height="350px" hspace="70">
+<img src="/common/gctlogo.png?raw=true" width="350px" height="350px" hspace="70">
 
 
 [![Build Status](https://github.com/thrasher-corp/gocryptotrader/actions/workflows/tests.yml/badge.svg?branch=master)](https://github.com/thrasher-corp/gocryptotrader/actions/workflows/tests.yml)

common/common.go

@@ -15,6 +15,7 @@ import (
 	"regexp"
 	"strconv"
 	"strings"
+	"sync"
 	"time"
 
 	"github.com/thrasher-corp/gocryptotrader/log"
@@ -32,6 +33,7 @@ var (
 	// ErrFunctionNotSupported defines a standardised error for an unsupported
 	// wrapper function by an API
 	ErrFunctionNotSupported = errors.New("unsupported wrapper function")
+	m                       sync.Mutex
 )
 
 // Const declarations for common.go operations
@@ -50,10 +52,12 @@ const (
 )
 
 func initialiseHTTPClient() {
+	m.Lock()
 	// If the HTTPClient isn't set, start a new client with a default timeout of 15 seconds
 	if HTTPClient == nil {
 		HTTPClient = NewHTTPClientWithTimeout(time.Second * 15)
 	}
+	m.Unlock()
 }
 
 // NewHTTPClientWithTimeout initialises a new HTTP client and its underlying
@@ -270,8 +274,11 @@ func ExtractHost(address string) string {
 
 // ExtractPort returns the port name out of a string
 func ExtractPort(host string) int {
-	portStr := strings.Split(host, ":")[1]
-	port, _ := strconv.Atoi(portStr)
+	portStrs := strings.Split(host, ":")
+	if len(portStrs) == 1 {
+		return 80
+	}
+	port, _ := strconv.Atoi(portStrs[1])
 	return port
 }
 

common/common_test.go

@@ -313,6 +313,14 @@ func TestExtractPort(t *testing.T) {
 		t.Errorf(
 			"Expected '%d'. Actual '%d'.", expectedOutput, actualResult)
 	}
+
+	address = "localhost"
+	expectedOutput = 80
+	actualResult = ExtractPort(address)
+	if expectedOutput != actualResult {
+		t.Errorf(
+			"Expected '%d'. Actual '%d'.", expectedOutput, actualResult)
+	}
 }
 
 func TestGetURIPath(t *testing.T) {

communications/base/README.md

@@ -1,6 +1,6 @@
 # GoCryptoTrader package Base
 
-<img src="https://github.com/thrasher-corp/gocryptotrader/blob/master/web/src/assets/page-logo.png?raw=true" width="350px" height="350px" hspace="70">
+<img src="/common/gctlogo.png?raw=true" width="350px" height="350px" hspace="70">
 
 
 [![Build Status](https://github.com/thrasher-corp/gocryptotrader/actions/workflows/tests.yml/badge.svg?branch=master)](https://github.com/thrasher-corp/gocryptotrader/actions/workflows/tests.yml)

communications/base/base.go

@@ -4,18 +4,13 @@ import (
 	"time"
 )
 
-// global vars contain staged update data that will be sent to the communication
-// mediums
-var (
-	ServiceStarted time.Time
-)
-
 // Base enforces standard variables across communication packages
 type Base struct {
-	Name      string
-	Enabled   bool
-	Verbose   bool
-	Connected bool
+	Name           string
+	Enabled        bool
+	Verbose        bool
+	Connected      bool
+	ServiceStarted time.Time
 }
 
 // Event is a generalise event type
@@ -50,5 +45,80 @@ func (b *Base) GetName() string {
 func (b *Base) GetStatus() string {
 	return `
 	GoCryptoTrader Service: Online
-	Service Started: ` + ServiceStarted.String()
+	Service Started: ` + b.ServiceStarted.String()
+}
+
+// SetServiceStarted sets the time the service started
+func (b *Base) SetServiceStarted(t time.Time) {
+	b.ServiceStarted = t
+}
+
+// CommunicationsConfig holds all the information needed for each
+// enabled communication package
+type CommunicationsConfig struct {
+	SlackConfig     SlackConfig     `json:"slack"`
+	SMSGlobalConfig SMSGlobalConfig `json:"smsGlobal"`
+	SMTPConfig      SMTPConfig      `json:"smtp"`
+	TelegramConfig  TelegramConfig  `json:"telegram"`
+}
+
+// IsAnyEnabled returns whether or any any comms relayers
+// are enabled
+func (c *CommunicationsConfig) IsAnyEnabled() bool {
+	if c.SMSGlobalConfig.Enabled ||
+		c.SMTPConfig.Enabled ||
+		c.SlackConfig.Enabled ||
+		c.TelegramConfig.Enabled {
+		return true
+	}
+	return false
+}
+
+// SlackConfig holds all variables to start and run the Slack package
+type SlackConfig struct {
+	Name              string `json:"name"`
+	Enabled           bool   `json:"enabled"`
+	Verbose           bool   `json:"verbose"`
+	TargetChannel     string `json:"targetChannel"`
+	VerificationToken string `json:"verificationToken"`
+}
+
+// SMSContact stores the SMS contact info
+type SMSContact struct {
+	Name    string `json:"name"`
+	Number  string `json:"number"`
+	Enabled bool   `json:"enabled"`
+}
+
+// SMSGlobalConfig structure holds all the variables you need for instant
+// messaging and broadcast used by SMSGlobal
+type SMSGlobalConfig struct {
+	Name     string       `json:"name"`
+	From     string       `json:"from"`
+	Enabled  bool         `json:"enabled"`
+	Verbose  bool         `json:"verbose"`
+	Username string       `json:"username"`
+	Password string       `json:"password"`
+	Contacts []SMSContact `json:"contacts"`
+}
+
+// SMTPConfig holds all variables to start and run the SMTP package
+type SMTPConfig struct {
+	Name            string `json:"name"`
+	Enabled         bool   `json:"enabled"`
+	Verbose         bool   `json:"verbose"`
+	Host            string `json:"host"`
+	Port            string `json:"port"`
+	AccountName     string `json:"accountName"`
+	AccountPassword string `json:"accountPassword"`
+	From            string `json:"from"`
+	RecipientList   string `json:"recipientList"`
+}
+
+// TelegramConfig holds all variables to start and run the Telegram package
+type TelegramConfig struct {
+	Name              string `json:"name"`
+	Enabled           bool   `json:"enabled"`
+	Verbose           bool   `json:"verbose"`
+	VerificationToken string `json:"verificationToken"`
 }

communications/base/base_interface.go

@@ -4,7 +4,6 @@ import (
 	"errors"
 	"time"
 
-	"github.com/thrasher-corp/gocryptotrader/config"
 	"github.com/thrasher-corp/gocryptotrader/log"
 )
 
@@ -13,18 +12,18 @@ type IComm []ICommunicate
 
 // ICommunicate enforces standard functions across communication packages
 type ICommunicate interface {
-	Setup(config *config.CommunicationsConfig)
+	Setup(config *CommunicationsConfig)
 	Connect() error
 	PushEvent(Event) error
 	IsEnabled() bool
 	IsConnected() bool
 	GetName() string
+	SetServiceStarted(time.Time)
 }
 
-// Setup sets up communication variables and intiates a connection to the
+// Setup sets up communication variables and initiates a connection to the
 // communication mediums
 func (c IComm) Setup() {
-	ServiceStarted = time.Now()
 	for i := range c {
 		if c[i].IsEnabled() && !c[i].IsConnected() {
 			err := c[i].Connect()
@@ -33,6 +32,7 @@ func (c IComm) Setup() {
 				continue
 			}
 			log.Debugf(log.CommunicationMgr, "Communications: %v is enabled and online.", c[i].GetName())
+			c[i].SetServiceStarted(time.Now())
 		}
 	}
 }

communications/base/base_test.go

@@ -2,6 +2,7 @@ package base
 
 import (
 	"testing"
+	"time"
 )
 
 var (
@@ -35,13 +36,26 @@ func TestGetName(t *testing.T) {
 	}
 }
 
+func TestSetServiceStarted(t *testing.T) {
+	b = Base{}
+	tt := time.Now()
+	if b.ServiceStarted.Equal(tt) {
+		t.Errorf("expected '%v', received '%v'", time.Time{}, tt)
+	}
+	b.SetServiceStarted(tt)
+	if !b.ServiceStarted.Equal(tt) {
+		t.Errorf("expected '%v', received '%v'", tt, b.ServiceStarted)
+	}
+}
+
 type CommunicationProvider struct {
 	ICommunicate
 
-	isEnabled       bool
-	isConnected     bool
-	ConnectCalled   bool
-	PushEventCalled bool
+	isEnabled        bool
+	isConnected      bool
+	ConnectCalled    bool
+	PushEventCalled  bool
+	ServiceStartTime time.Time
 }
 
 func (p *CommunicationProvider) IsEnabled() bool {
@@ -66,6 +80,10 @@ func (p *CommunicationProvider) GetName() string {
 	return "someTestProvider"
 }
 
+func (p *CommunicationProvider) SetServiceStarted(t time.Time) {
+	p.ServiceStartTime = t
+}
+
 func TestSetup(t *testing.T) {
 	var ic IComm
 	testConfigs := []struct {

communications/communications.go

@@ -8,7 +8,6 @@ import (
 	"github.com/thrasher-corp/gocryptotrader/communications/smsglobal"
 	"github.com/thrasher-corp/gocryptotrader/communications/smtpservice"
 	"github.com/thrasher-corp/gocryptotrader/communications/telegram"
-	"github.com/thrasher-corp/gocryptotrader/config"
 )
 
 // Communications is the overarching type across the communications packages
@@ -16,10 +15,13 @@ type Communications struct {
 	base.IComm
 }
 
+// ErrNoCommunicationRelayersEnabled returns when no relayers enabled
+var ErrNoCommunicationRelayersEnabled = errors.New("no communication relayers enabled")
+
 // NewComm sets up and returns a pointer to a Communications object
-func NewComm(cfg *config.CommunicationsConfig) (*Communications, error) {
+func NewComm(cfg *base.CommunicationsConfig) (*Communications, error) {
 	if !cfg.IsAnyEnabled() {
-		return nil, errors.New("no communication relayers enabled")
+		return nil, ErrNoCommunicationRelayersEnabled
 	}
 
 	var comm Communications

communications/communications_test.go

@@ -3,11 +3,11 @@ package communications
 import (
 	"testing"
 
-	"github.com/thrasher-corp/gocryptotrader/config"
+	"github.com/thrasher-corp/gocryptotrader/communications/base"
 )
 
 func TestNewComm(t *testing.T) {
-	var cfg config.CommunicationsConfig
+	var cfg base.CommunicationsConfig
 	_, err := NewComm(&cfg)
 	if err == nil {
 		t.Error("NewComm should have failed on no enabled communication mediums")

communications/slack/README.md

@@ -1,6 +1,6 @@
 # GoCryptoTrader package Slack
 
-<img src="https://github.com/thrasher-corp/gocryptotrader/blob/master/web/src/assets/page-logo.png?raw=true" width="350px" height="350px" hspace="70">
+<img src="/common/gctlogo.png?raw=true" width="350px" height="350px" hspace="70">
 
 
 [![Build Status](https://github.com/thrasher-corp/gocryptotrader/actions/workflows/tests.yml/badge.svg?branch=master)](https://github.com/thrasher-corp/gocryptotrader/actions/workflows/tests.yml)

communications/slack/slack.go

@@ -15,7 +15,6 @@ import (
 	"github.com/gorilla/websocket"
 	"github.com/thrasher-corp/gocryptotrader/common"
 	"github.com/thrasher-corp/gocryptotrader/communications/base"
-	"github.com/thrasher-corp/gocryptotrader/config"
 	"github.com/thrasher-corp/gocryptotrader/log"
 )
 
@@ -57,7 +56,7 @@ func (s *Slack) IsConnected() bool {
 
 // Setup takes in a slack configuration, sets bots target channel and
 // sets verification token to access workspace
-func (s *Slack) Setup(cfg *config.CommunicationsConfig) {
+func (s *Slack) Setup(cfg *base.CommunicationsConfig) {
 	s.Name = cfg.SlackConfig.Name
 	s.Enabled = cfg.SlackConfig.Enabled
 	s.Verbose = cfg.SlackConfig.Verbose
@@ -180,7 +179,7 @@ func (s *Slack) NewConnection() error {
 			s.TargetChannel)
 		return s.WebsocketConnect()
 	}
-	return errors.New("slack.go NewConnection() Already Connected")
+	return errors.New("newConnection() Already Connected")
 }
 
 // WebsocketConnect creates a websocket dialer amd initiates a websocket

communications/smsglobal/README.md

@@ -1,6 +1,6 @@
 # GoCryptoTrader package Smsglobal
 
-<img src="https://github.com/thrasher-corp/gocryptotrader/blob/master/web/src/assets/page-logo.png?raw=true" width="350px" height="350px" hspace="70">
+<img src="/common/gctlogo.png?raw=true" width="350px" height="350px" hspace="70">
 
 
 [![Build Status](https://github.com/thrasher-corp/gocryptotrader/actions/workflows/tests.yml/badge.svg?branch=master)](https://github.com/thrasher-corp/gocryptotrader/actions/workflows/tests.yml)

communications/smsglobal/smsglobal.go

@@ -10,7 +10,6 @@ import (
 
 	"github.com/thrasher-corp/gocryptotrader/common"
 	"github.com/thrasher-corp/gocryptotrader/communications/base"
-	"github.com/thrasher-corp/gocryptotrader/config"
 	"github.com/thrasher-corp/gocryptotrader/log"
 )
 
@@ -34,7 +33,7 @@ type SMSGlobal struct {
 
 // Setup takes in a SMSGlobal configuration, sets username, password and
 // and recipient list
-func (s *SMSGlobal) Setup(cfg *config.CommunicationsConfig) {
+func (s *SMSGlobal) Setup(cfg *base.CommunicationsConfig) {
 	s.Name = cfg.SMSGlobalConfig.Name
 	s.Enabled = cfg.SMSGlobalConfig.Enabled
 	s.Verbose = cfg.SMSGlobalConfig.Verbose

communications/smtpservice/smtpservice.go

@@ -7,7 +7,6 @@ import (
 	"strings"
 
 	"github.com/thrasher-corp/gocryptotrader/communications/base"
-	"github.com/thrasher-corp/gocryptotrader/config"
 	"github.com/thrasher-corp/gocryptotrader/log"
 )
 
@@ -29,7 +28,7 @@ type SMTPservice struct {
 
 // Setup takes in a SMTP configuration and sets SMTP server details and
 // recipient list
-func (s *SMTPservice) Setup(cfg *config.CommunicationsConfig) {
+func (s *SMTPservice) Setup(cfg *base.CommunicationsConfig) {
 	s.Name = cfg.SMTPConfig.Name
 	s.Enabled = cfg.SMTPConfig.Enabled
 	s.Verbose = cfg.SMTPConfig.Verbose

communications/telegram/README.md

@@ -1,6 +1,6 @@
 # GoCryptoTrader package Telegram
 
-<img src="https://github.com/thrasher-corp/gocryptotrader/blob/master/web/src/assets/page-logo.png?raw=true" width="350px" height="350px" hspace="70">
+<img src="/common/gctlogo.png?raw=true" width="350px" height="350px" hspace="70">
 
 
 [![Build Status](https://github.com/thrasher-corp/gocryptotrader/actions/workflows/tests.yml/badge.svg?branch=master)](https://github.com/thrasher-corp/gocryptotrader/actions/workflows/tests.yml)

communications/telegram/telegram.go

@@ -14,7 +14,6 @@ import (
 
 	"github.com/thrasher-corp/gocryptotrader/common"
 	"github.com/thrasher-corp/gocryptotrader/communications/base"
-	"github.com/thrasher-corp/gocryptotrader/config"
 	"github.com/thrasher-corp/gocryptotrader/log"
 )
 
@@ -59,7 +58,7 @@ type Telegram struct {
 func (t *Telegram) IsConnected() bool { return t.Connected }
 
 // Setup takes in a Telegram configuration and sets verification token
-func (t *Telegram) Setup(cfg *config.CommunicationsConfig) {
+func (t *Telegram) Setup(cfg *base.CommunicationsConfig) {
 	t.Name = cfg.TelegramConfig.Name
 	t.Enabled = cfg.TelegramConfig.Enabled
 	t.Token = cfg.TelegramConfig.VerificationToken

config/README.md

@@ -1,6 +1,6 @@
 # GoCryptoTrader package Config
 
-<img src="https://github.com/thrasher-corp/gocryptotrader/blob/master/web/src/assets/page-logo.png?raw=true" width="350px" height="350px" hspace="70">
+<img src="/common/gctlogo.png?raw=true" width="350px" height="350px" hspace="70">
 
 
 [![Build Status](https://github.com/thrasher-corp/gocryptotrader/actions/workflows/tests.yml/badge.svg?branch=master)](https://github.com/thrasher-corp/gocryptotrader/actions/workflows/tests.yml)

config/config.go

@@ -18,6 +18,7 @@ import (
 	"github.com/thrasher-corp/gocryptotrader/common"
 	"github.com/thrasher-corp/gocryptotrader/common/convert"
 	"github.com/thrasher-corp/gocryptotrader/common/file"
+	"github.com/thrasher-corp/gocryptotrader/communications/base"
 	"github.com/thrasher-corp/gocryptotrader/connchecker"
 	"github.com/thrasher-corp/gocryptotrader/currency"
 	"github.com/thrasher-corp/gocryptotrader/currency/forexprovider"
@@ -171,7 +172,7 @@ func (c *Config) PurgeExchangeAPICredentials() {
 }
 
 // GetCommunicationsConfig returns the communications configuration
-func (c *Config) GetCommunicationsConfig() CommunicationsConfig {
+func (c *Config) GetCommunicationsConfig() base.CommunicationsConfig {
 	m.Lock()
 	comms := c.Communications
 	m.Unlock()
@@ -180,7 +181,7 @@ func (c *Config) GetCommunicationsConfig() CommunicationsConfig {
 
 // UpdateCommunicationsConfig sets a new updated version of a Communications
 // configuration
-func (c *Config) UpdateCommunicationsConfig(config *CommunicationsConfig) {
+func (c *Config) UpdateCommunicationsConfig(config *base.CommunicationsConfig) {
 	m.Lock()
 	c.Communications = *config
 	m.Unlock()
@@ -211,7 +212,7 @@ func (c *Config) CheckCommunicationsConfig() {
 	// with example settings
 
 	if c.Communications.SlackConfig.Name == "" {
-		c.Communications.SlackConfig = SlackConfig{
+		c.Communications.SlackConfig = base.SlackConfig{
 			Name:              "Slack",
 			TargetChannel:     "general",
 			VerificationToken: "testtest",
@@ -221,7 +222,7 @@ func (c *Config) CheckCommunicationsConfig() {
 	if c.Communications.SMSGlobalConfig.Name == "" {
 		if c.SMS != nil {
 			if c.SMS.Contacts != nil {
-				c.Communications.SMSGlobalConfig = SMSGlobalConfig{
+				c.Communications.SMSGlobalConfig = base.SMSGlobalConfig{
 					Name:     "SMSGlobal",
 					Enabled:  c.SMS.Enabled,
 					Verbose:  c.SMS.Verbose,
@@ -232,13 +233,13 @@ func (c *Config) CheckCommunicationsConfig() {
 				// flush old SMS config
 				c.SMS = nil
 			} else {
-				c.Communications.SMSGlobalConfig = SMSGlobalConfig{
+				c.Communications.SMSGlobalConfig = base.SMSGlobalConfig{
 					Name:     "SMSGlobal",
 					From:     c.Name,
 					Username: "main",
 					Password: "test",
 
-					Contacts: []SMSContact{
+					Contacts: []base.SMSContact{
 						{
 							Name:    "bob",
 							Number:  "1234",
@@ -248,12 +249,12 @@ func (c *Config) CheckCommunicationsConfig() {
 				}
 			}
 		} else {
-			c.Communications.SMSGlobalConfig = SMSGlobalConfig{
+			c.Communications.SMSGlobalConfig = base.SMSGlobalConfig{
 				Name:     "SMSGlobal",
 				Username: "main",
 				Password: "test",
 
-				Contacts: []SMSContact{
+				Contacts: []base.SMSContact{
 					{
 						Name:    "bob",
 						Number:  "1234",
@@ -279,7 +280,7 @@ func (c *Config) CheckCommunicationsConfig() {
 	}
 
 	if c.Communications.SMTPConfig.Name == "" {
-		c.Communications.SMTPConfig = SMTPConfig{
+		c.Communications.SMTPConfig = base.SMTPConfig{
 			Name:            "SMTP",
 			Host:            "smtp.google.com",
 			Port:            "537",
@@ -290,7 +291,7 @@ func (c *Config) CheckCommunicationsConfig() {
 	}
 
 	if c.Communications.TelegramConfig.Name == "" {
-		c.Communications.TelegramConfig = TelegramConfig{
+		c.Communications.TelegramConfig = base.TelegramConfig{
 			Name:              "Telegram",
 			VerificationToken: "testest",
 		}
@@ -749,7 +750,7 @@ func (c *Config) GetExchangeConfig(name string) (*ExchangeConfig, error) {
 			return &c.Exchanges[i], nil
 		}
 	}
-	return nil, fmt.Errorf(ErrExchangeNotFound, name)
+	return nil, fmt.Errorf("%s %w", name, ErrExchangeNotFound)
 }
 
 // GetForexProvider returns a forex provider configuration by its name
@@ -794,7 +795,7 @@ func (c *Config) UpdateExchangeConfig(e *ExchangeConfig) error {
 			return nil
 		}
 	}
-	return fmt.Errorf(ErrExchangeNotFound, e.Name)
+	return fmt.Errorf("%s %w", e.Name, ErrExchangeNotFound)
 }
 
 // CheckExchangeConfigValues returns configuation values for all enabled
@@ -1345,9 +1346,7 @@ func (c *Config) checkDatabaseConfig() error {
 		database.DB.DataPath = databaseDir
 	}
 
-	database.DB.Config = &c.Database
-
-	return nil
+	return database.DB.SetConfig(&c.Database)
 }
 
 // CheckNTPConfig checks for missing or incorrectly configured NTPClient and recreates with known safe defaults
@@ -1371,14 +1370,14 @@ func (c *Config) CheckNTPConfig() {
 	}
 }
 
-// DisableNTPCheck allows the user to change how they are prompted for timesync alerts
-func (c *Config) DisableNTPCheck(input io.Reader) (string, error) {
+// SetNTPCheck allows the user to change how they are prompted for timesync alerts
+func (c *Config) SetNTPCheck(input io.Reader) (string, error) {
 	m.Lock()
 	defer m.Unlock()
 
 	reader := bufio.NewReader(input)
 	log.Warnln(log.ConfigMgr, "Your system time is out of sync, this may cause issues with trading")
-	log.Warnln(log.ConfigMgr, "How would you like to show future notifications? (a)lert / (w)arn / (d)isable")
+	log.Warnln(log.ConfigMgr, "How would you like to show future notifications? (a)lert at startup / (w)arn periodically / (d)isable")
 
 	var resp string
 	answered := false
@@ -1458,9 +1457,9 @@ func GetAndMigrateDefaultPath(configFile string) (string, error) {
 
 // GetFilePath returns the desired config file or the default config file name
 // and whether it was loaded from a default location (rather than explicitly specified)
-func GetFilePath(configfile string) (configPath string, isImplicitDefaultPath bool, err error) {
-	if configfile != "" {
-		return configfile, false, nil
+func GetFilePath(configFile string) (configPath string, isImplicitDefaultPath bool, err error) {
+	if configFile != "" {
+		return configFile, false, nil
 	}
 
 	exePath, err := common.GetExecutablePath()
@@ -1477,16 +1476,16 @@ func GetFilePath(configfile string) (configPath string, isImplicitDefaultPath bo
 
 	for _, p := range defaultPaths {
 		if file.Exists(p) {
-			configfile = p
+			configFile = p
 			break
 		}
 	}
-	if configfile == "" {
+	if configFile == "" {
 		return "", false, fmt.Errorf("config.json file not found in %s, please follow README.md in root dir for config generation",
 			newDir)
 	}
 
-	return configfile, true, nil
+	return configFile, true, nil
 }
 
 // migrateConfig will move the config file to the target

config/config_test.go

@@ -11,13 +11,13 @@ import (
 	"github.com/thrasher-corp/gocryptotrader/common"
 	"github.com/thrasher-corp/gocryptotrader/common/convert"
 	"github.com/thrasher-corp/gocryptotrader/common/file"
+	"github.com/thrasher-corp/gocryptotrader/communications/base"
 	"github.com/thrasher-corp/gocryptotrader/connchecker"
 	"github.com/thrasher-corp/gocryptotrader/currency"
 	"github.com/thrasher-corp/gocryptotrader/database"
 	"github.com/thrasher-corp/gocryptotrader/exchanges/asset"
 	gctscript "github.com/thrasher-corp/gocryptotrader/gctscript/vm"
 	"github.com/thrasher-corp/gocryptotrader/log"
-	"github.com/thrasher-corp/gocryptotrader/ntpclient"
 	"github.com/thrasher-corp/gocryptotrader/portfolio/banking"
 )
 
@@ -302,7 +302,7 @@ func TestUpdateCommunicationsConfig(t *testing.T) {
 	if err != nil {
 		t.Error("UpdateCommunicationsConfig LoadConfig error", err)
 	}
-	cfg.UpdateCommunicationsConfig(&CommunicationsConfig{SlackConfig: SlackConfig{Name: testString}})
+	cfg.UpdateCommunicationsConfig(&base.CommunicationsConfig{SlackConfig: base.SlackConfig{Name: testString}})
 	if cfg.Communications.SlackConfig.Name != testString {
 		t.Error("UpdateCommunicationsConfig LoadConfig error")
 	}
@@ -340,7 +340,7 @@ func TestCheckCommunicationsConfig(t *testing.T) {
 		t.Error("CheckCommunicationsConfig LoadConfig error", err)
 	}
 
-	cfg.Communications = CommunicationsConfig{}
+	cfg.Communications = base.CommunicationsConfig{}
 	cfg.CheckCommunicationsConfig()
 	if cfg.Communications.SlackConfig.Name != "Slack" ||
 		cfg.Communications.SMSGlobalConfig.Name != "SMSGlobal" ||
@@ -350,14 +350,14 @@ func TestCheckCommunicationsConfig(t *testing.T) {
 			cfg.Communications)
 	}
 
-	cfg.SMS = &SMSGlobalConfig{}
+	cfg.SMS = &base.SMSGlobalConfig{}
 	cfg.Communications.SMSGlobalConfig.Name = ""
 	cfg.CheckCommunicationsConfig()
 	if cfg.Communications.SMSGlobalConfig.Password != testString {
 		t.Error("CheckCommunicationsConfig error:", err)
 	}
 
-	cfg.SMS.Contacts = append(cfg.SMS.Contacts, SMSContact{
+	cfg.SMS.Contacts = append(cfg.SMS.Contacts, base.SMSContact{
 		Name:    "Bobby",
 		Number:  "4321",
 		Enabled: false,
@@ -380,7 +380,7 @@ func TestCheckCommunicationsConfig(t *testing.T) {
 		t.Error("CheckCommunicationsConfig From value should have been trimmed to 11 characters")
 	}
 
-	cfg.SMS = &SMSGlobalConfig{}
+	cfg.SMS = &base.SMSGlobalConfig{}
 	cfg.CheckCommunicationsConfig()
 	if cfg.SMS != nil {
 		t.Error("CheckCommunicationsConfig unexpected data:",
@@ -1883,7 +1883,7 @@ func TestDisableNTPCheck(t *testing.T) {
 
 	var c Config
 
-	warn, err := c.DisableNTPCheck(strings.NewReader("w\n"))
+	warn, err := c.SetNTPCheck(strings.NewReader("w\n"))
 	if err != nil {
 		t.Fatalf("to create ntpclient failed reason: %v", err)
 	}
@@ -1891,17 +1891,17 @@ func TestDisableNTPCheck(t *testing.T) {
 	if warn != "Time sync has been set to warn only" {
 		t.Errorf("failed expected %v got %v", "Time sync has been set to warn only", warn)
 	}
-	alert, _ := c.DisableNTPCheck(strings.NewReader("a\n"))
+	alert, _ := c.SetNTPCheck(strings.NewReader("a\n"))
 	if alert != "Time sync has been set to alert" {
 		t.Errorf("failed expected %v got %v", "Time sync has been set to alert", alert)
 	}
 
-	disable, _ := c.DisableNTPCheck(strings.NewReader("d\n"))
+	disable, _ := c.SetNTPCheck(strings.NewReader("d\n"))
 	if disable != "Future notifications for out of time sync has been disabled" {
 		t.Errorf("failed expected %v got %v", "Future notifications for out of time sync has been disabled", disable)
 	}
 
-	_, err = c.DisableNTPCheck(strings.NewReader(" "))
+	_, err = c.SetNTPCheck(strings.NewReader(" "))
 	if err.Error() != "EOF" {
 		t.Errorf("failed expected EOF got: %v", err)
 	}
@@ -1960,7 +1960,6 @@ func TestCheckNTPConfig(t *testing.T) {
 	c.NTPClient.AllowedDifference = nil
 
 	c.CheckNTPConfig()
-	_ = ntpclient.NTPClient(c.NTPClient.Pool)
 
 	if c.NTPClient.Pool[0] != "pool.ntp.org:123" {
 		t.Error("ntpclient with no valid pool should default to pool.ntp.org")

config/config_types.go

@@ -1,9 +1,11 @@
 package config
 
 import (
+	"errors"
 	"sync"
 	"time"
 
+	"github.com/thrasher-corp/gocryptotrader/communications/base"
 	"github.com/thrasher-corp/gocryptotrader/currency"
 	"github.com/thrasher-corp/gocryptotrader/database"
 	"github.com/thrasher-corp/gocryptotrader/exchanges/protocol"
@@ -39,17 +41,9 @@ const (
 // Constants here hold some messages
 const (
 	ErrExchangeNameEmpty                       = "exchange #%d name is empty"
-	ErrExchangeAvailablePairsEmpty             = "exchange %s available pairs is empty"
-	ErrExchangeEnabledPairsEmpty               = "exchange %s enabled pairs is empty"
-	ErrExchangeBaseCurrenciesEmpty             = "exchange %s base currencies is empty"
-	ErrExchangeNotFound                        = "exchange %s not found"
 	ErrNoEnabledExchanges                      = "no exchanges enabled"
-	ErrCryptocurrenciesEmpty                   = "cryptocurrencies variable is empty"
 	ErrFailureOpeningConfig                    = "fatal error opening %s file. Error: %s"
 	ErrCheckingConfigValues                    = "fatal error checking config values. Error: %s"
-	ErrSavingConfigBytesMismatch               = "config file %q bytes comparison doesn't match, read %s expected %s"
-	WarningWebserverCredentialValuesEmpty      = "webserver support disabled due to empty Username/Password values"
-	WarningWebserverListenAddressInvalid       = "webserver support disabled due to invalid listen address"
 	WarningExchangeAuthAPIDefaultOrEmptyValues = "exchange %s authenticated API support disabled due to default/empty APIKey/Secret/ClientID values"
 	WarningPairsLastUpdatedThresholdExceeded   = "exchange %s last manual update of available currency pairs has exceeded %d days. Manual update required!"
 )
@@ -67,37 +61,38 @@ const (
 
 // Variables here are used for configuration
 var (
-	Cfg Config
-	m   sync.Mutex
+	Cfg                 Config
+	m                   sync.Mutex
+	ErrExchangeNotFound = errors.New("exchange not found")
 )
 
 // Config is the overarching object that holds all the information for
 // prestart management of Portfolio, Communications, Webserver and Enabled
 // Exchanges
 type Config struct {
-	Name              string                  `json:"name"`
-	DataDirectory     string                  `json:"dataDirectory"`
-	EncryptConfig     int                     `json:"encryptConfig"`
-	GlobalHTTPTimeout time.Duration           `json:"globalHTTPTimeout"`
-	Database          database.Config         `json:"database"`
-	Logging           log.Config              `json:"logging"`
-	ConnectionMonitor ConnectionMonitorConfig `json:"connectionMonitor"`
-	Profiler          Profiler                `json:"profiler"`
-	NTPClient         NTPClientConfig         `json:"ntpclient"`
-	GCTScript         gctscript.Config        `json:"gctscript"`
-	Currency          CurrencyConfig          `json:"currencyConfig"`
-	Communications    CommunicationsConfig    `json:"communications"`
-	RemoteControl     RemoteControlConfig     `json:"remoteControl"`
-	Portfolio         portfolio.Base          `json:"portfolioAddresses"`
-	Exchanges         []ExchangeConfig        `json:"exchanges"`
-	BankAccounts      []banking.Account       `json:"bankAccounts"`
+	Name              string                    `json:"name"`
+	DataDirectory     string                    `json:"dataDirectory"`
+	EncryptConfig     int                       `json:"encryptConfig"`
+	GlobalHTTPTimeout time.Duration             `json:"globalHTTPTimeout"`
+	Database          database.Config           `json:"database"`
+	Logging           log.Config                `json:"logging"`
+	ConnectionMonitor ConnectionMonitorConfig   `json:"connectionMonitor"`
+	Profiler          Profiler                  `json:"profiler"`
+	NTPClient         NTPClientConfig           `json:"ntpclient"`
+	GCTScript         gctscript.Config          `json:"gctscript"`
+	Currency          CurrencyConfig            `json:"currencyConfig"`
+	Communications    base.CommunicationsConfig `json:"communications"`
+	RemoteControl     RemoteControlConfig       `json:"remoteControl"`
+	Portfolio         portfolio.Base            `json:"portfolioAddresses"`
+	Exchanges         []ExchangeConfig          `json:"exchanges"`
+	BankAccounts      []banking.Account         `json:"bankAccounts"`
 
 	// Deprecated config settings, will be removed at a future date
 	Webserver           *WebserverConfig          `json:"webserver,omitempty"`
 	CurrencyPairFormat  *CurrencyPairFormatConfig `json:"currencyPairFormat,omitempty"`
 	FiatDisplayCurrency *currency.Code            `json:"fiatDispayCurrency,omitempty"`
 	Cryptocurrencies    *currency.Currencies      `json:"cryptocurrencies,omitempty"`
-	SMS                 *SMSGlobalConfig          `json:"smsGlobal,omitempty"`
+	SMS                 *base.SMSGlobalConfig     `json:"smsGlobal,omitempty"`
 	// encryption session values
 	storedSalt []byte
 	sessionDK  []byte
@@ -251,76 +246,6 @@ type CryptocurrencyProvider struct {
 	AccountPlan string `json:"accountPlan"`
 }
 
-// CommunicationsConfig holds all the information needed for each
-// enabled communication package
-type CommunicationsConfig struct {
-	SlackConfig     SlackConfig     `json:"slack"`
-	SMSGlobalConfig SMSGlobalConfig `json:"smsGlobal"`
-	SMTPConfig      SMTPConfig      `json:"smtp"`
-	TelegramConfig  TelegramConfig  `json:"telegram"`
-}
-
-// IsAnyEnabled returns whether or any any comms relayers
-// are enabled
-func (c *CommunicationsConfig) IsAnyEnabled() bool {
-	if c.SMSGlobalConfig.Enabled ||
-		c.SMTPConfig.Enabled ||
-		c.SlackConfig.Enabled ||
-		c.TelegramConfig.Enabled {
-		return true
-	}
-	return false
-}
-
-// SlackConfig holds all variables to start and run the Slack package
-type SlackConfig struct {
-	Name              string `json:"name"`
-	Enabled           bool   `json:"enabled"`
-	Verbose           bool   `json:"verbose"`
-	TargetChannel     string `json:"targetChannel"`
-	VerificationToken string `json:"verificationToken"`
-}
-
-// SMSContact stores the SMS contact info
-type SMSContact struct {
-	Name    string `json:"name"`
-	Number  string `json:"number"`
-	Enabled bool   `json:"enabled"`
-}
-
-// SMSGlobalConfig structure holds all the variables you need for instant
-// messaging and broadcast used by SMSGlobal
-type SMSGlobalConfig struct {
-	Name     string       `json:"name"`
-	From     string       `json:"from"`
-	Enabled  bool         `json:"enabled"`
-	Verbose  bool         `json:"verbose"`
-	Username string       `json:"username"`
-	Password string       `json:"password"`
-	Contacts []SMSContact `json:"contacts"`
-}
-
-// SMTPConfig holds all variables to start and run the SMTP package
-type SMTPConfig struct {
-	Name            string `json:"name"`
-	Enabled         bool   `json:"enabled"`
-	Verbose         bool   `json:"verbose"`
-	Host            string `json:"host"`
-	Port            string `json:"port"`
-	AccountName     string `json:"accountName"`
-	AccountPassword string `json:"accountPassword"`
-	From            string `json:"from"`
-	RecipientList   string `json:"recipientList"`
-}
-
-// TelegramConfig holds all variables to start and run the Telegram package
-type TelegramConfig struct {
-	Name              string `json:"name"`
-	Enabled           bool   `json:"enabled"`
-	Verbose           bool   `json:"verbose"`
-	VerificationToken string `json:"verificationToken"`
-}
-
 // FeaturesSupportedConfig stores the exchanges supported features
 type FeaturesSupportedConfig struct {
 	REST                  bool              `json:"restAPI"`

connchecker/connchecker.go

@@ -79,6 +79,7 @@ type Checker struct {
 
 // Shutdown cleanly shutsdown monitor routine
 func (c *Checker) Shutdown() {
+	c.connected = false
 	close(c.shutdown)
 	c.wg.Wait()
 }

currency/README.md

@@ -1,6 +1,6 @@
 # GoCryptoTrader package Currency
 
-<img src="https://github.com/thrasher-corp/gocryptotrader/blob/master/web/src/assets/page-logo.png?raw=true" width="350px" height="350px" hspace="70">
+<img src="/common/gctlogo.png?raw=true" width="350px" height="350px" hspace="70">
 
 
 [![Build Status](https://github.com/thrasher-corp/gocryptotrader/actions/workflows/tests.yml/badge.svg?branch=master)](https://github.com/thrasher-corp/gocryptotrader/actions/workflows/tests.yml)

currency/forexprovider/README.md

@@ -1,6 +1,6 @@
 # GoCryptoTrader package Forexprovider
 
-<img src="https://github.com/thrasher-corp/gocryptotrader/blob/master/web/src/assets/page-logo.png?raw=true" width="350px" height="350px" hspace="70">
+<img src="/common/gctlogo.png?raw=true" width="350px" height="350px" hspace="70">
 
 
 [![Build Status](https://github.com/thrasher-corp/gocryptotrader/actions/workflows/tests.yml/badge.svg?branch=master)](https://github.com/thrasher-corp/gocryptotrader/actions/workflows/tests.yml)

currency/forexprovider/base/README.md

@@ -1,6 +1,6 @@
 # GoCryptoTrader package Base
 
-<img src="https://github.com/thrasher-corp/gocryptotrader/blob/master/web/src/assets/page-logo.png?raw=true" width="350px" height="350px" hspace="70">
+<img src="/common/gctlogo.png?raw=true" width="350px" height="350px" hspace="70">
 
 
 [![Build Status](https://github.com/thrasher-corp/gocryptotrader/actions/workflows/tests.yml/badge.svg?branch=master)](https://github.com/thrasher-corp/gocryptotrader/actions/workflows/tests.yml)

currency/forexprovider/currencyconverterapi/README.md

@@ -1,6 +1,6 @@
 # GoCryptoTrader package Currencyconverterapi
 
-<img src="https://github.com/thrasher-corp/gocryptotrader/blob/master/web/src/assets/page-logo.png?raw=true" width="350px" height="350px" hspace="70">
+<img src="/common/gctlogo.png?raw=true" width="350px" height="350px" hspace="70">
 
 
 [![Build Status](https://github.com/thrasher-corp/gocryptotrader/actions/workflows/tests.yml/badge.svg?branch=master)](https://github.com/thrasher-corp/gocryptotrader/actions/workflows/tests.yml)

currency/forexprovider/currencylayer/README.md

@@ -1,6 +1,6 @@
 # GoCryptoTrader package Currencylayer
 
-<img src="https://github.com/thrasher-corp/gocryptotrader/blob/master/web/src/assets/page-logo.png?raw=true" width="350px" height="350px" hspace="70">
+<img src="/common/gctlogo.png?raw=true" width="350px" height="350px" hspace="70">
 
 
 [![Build Status](https://github.com/thrasher-corp/gocryptotrader/actions/workflows/tests.yml/badge.svg?branch=master)](https://github.com/thrasher-corp/gocryptotrader/actions/workflows/tests.yml)

currency/forexprovider/exchangerate.host/README.md

@@ -1,6 +1,6 @@
 # GoCryptoTrader package Exchangerate.Host
 
-<img src="https://github.com/thrasher-corp/gocryptotrader/blob/master/web/src/assets/page-logo.png?raw=true" width="350px" height="350px" hspace="70">
+<img src="/common/gctlogo.png?raw=true" width="350px" height="350px" hspace="70">
 
 
 [![Build Status](https://github.com/thrasher-corp/gocryptotrader/actions/workflows/tests.yml/badge.svg?branch=master)](https://github.com/thrasher-corp/gocryptotrader/actions/workflows/tests.yml)

currency/forexprovider/exchangeratesapi.io/README.md

@@ -1,6 +1,6 @@
 # GoCryptoTrader package Exchangeratesapi.Io
 
-<img src="https://github.com/thrasher-corp/gocryptotrader/blob/master/web/src/assets/page-logo.png?raw=true" width="350px" height="350px" hspace="70">
+<img src="/common/gctlogo.png?raw=true" width="350px" height="350px" hspace="70">
 
 
 [![Build Status](https://github.com/thrasher-corp/gocryptotrader/actions/workflows/tests.yml/badge.svg?branch=master)](https://github.com/thrasher-corp/gocryptotrader/actions/workflows/tests.yml)

currency/forexprovider/exchangeratesapi.io/exchangeratesapi.go

@@ -166,11 +166,11 @@ func (e *ExchangeRates) GetTimeSeriesRates(startDate, endDate time.Time, baseCur
 	}
 
 	if startDate.IsZero() || endDate.IsZero() {
-		return nil, errors.New("startDate and endDate params must be set")
+		return nil, errStartEndDatesInvalid
 	}
 
 	if startDate.After(endDate) {
-		return nil, errors.New("startDate must be before endDate")
+		return nil, errStartAfterEnd
 	}
 
 	v := url.Values{}
@@ -197,11 +197,11 @@ func (e *ExchangeRates) GetFluctuations(startDate, endDate time.Time, baseCurren
 	}
 
 	if startDate.IsZero() || endDate.IsZero() {
-		return nil, errors.New("startDate and endDate must be set")
+		return nil, errStartEndDatesInvalid
 	}
 
 	if startDate.After(endDate) {
-		return nil, errors.New("startDate must be before endDate")
+		return nil, errStartAfterEnd
 	}
 
 	v := url.Values{}

currency/forexprovider/exchangeratesapi.io/exchangeratesapi_test.go

@@ -147,14 +147,14 @@ func TestGetTimeSeriesRates(t *testing.T) {
 	}
 
 	_, err := e.GetTimeSeriesRates(time.Time{}, time.Time{}, "USD", []string{"EUR", "USD"})
-	if err == nil {
-		t.Fatal("empty startDate endDate params should throw an error")
+	if !errors.Is(err, errStartEndDatesInvalid) {
+		t.Fatalf("received '%v' expected '%v'", err, errStartEndDatesInvalid)
 	}
 
 	tmNow := time.Now()
 	_, err = e.GetTimeSeriesRates(tmNow.AddDate(0, 1, 0), tmNow, "USD", []string{"EUR", "USD"})
-	if err == nil {
-		t.Fatal("future startTime should throw an error")
+	if !errors.Is(err, errStartAfterEnd) {
+		t.Fatalf("received '%v' expected '%v'", err, errStartAfterEnd)
 	}
 
 	_, err = e.GetTimeSeriesRates(tmNow.AddDate(0, -1, 0), tmNow, "EUR", []string{"AUD,USD"})

currency/forexprovider/exchangeratesapi.io/exchangeratesapi_types.go

@@ -28,6 +28,8 @@ const (
 var (
 	errCannotSetBaseCurrencyOnFreePlan = errors.New("base currency cannot be set on the free plan")
 	errAPIKeyLevelRestrictedAccess     = errors.New("apiKey level function access denied")
+	errStartEndDatesInvalid            = errors.New("startDate and endDate params must be set")
+	errStartAfterEnd                   = errors.New("startDate must be before endDate")
 )
 
 // ExchangeRates stores the struct for the ExchangeRatesAPI API

currency/forexprovider/fixer.io/README.md

@@ -1,6 +1,6 @@
 # GoCryptoTrader package Fixer.Io
 
-<img src="https://github.com/thrasher-corp/gocryptotrader/blob/master/web/src/assets/page-logo.png?raw=true" width="350px" height="350px" hspace="70">
+<img src="/common/gctlogo.png?raw=true" width="350px" height="350px" hspace="70">
 
 
 [![Build Status](https://github.com/thrasher-corp/gocryptotrader/actions/workflows/tests.yml/badge.svg?branch=master)](https://github.com/thrasher-corp/gocryptotrader/actions/workflows/tests.yml)

currency/forexprovider/openexchangerates/README.md

@@ -1,6 +1,6 @@
 # GoCryptoTrader package Openexchangerates
 
-<img src="https://github.com/thrasher-corp/gocryptotrader/blob/master/web/src/assets/page-logo.png?raw=true" width="350px" height="350px" hspace="70">
+<img src="/common/gctlogo.png?raw=true" width="350px" height="350px" hspace="70">
 
 
 [![Build Status](https://github.com/thrasher-corp/gocryptotrader/actions/workflows/tests.yml/badge.svg?branch=master)](https://github.com/thrasher-corp/gocryptotrader/actions/workflows/tests.yml)

database/database.go

@@ -0,0 +1,96 @@
+package database
+
+import (
+	"database/sql"
+	"time"
+
+	"github.com/thrasher-corp/sqlboiler/boil"
+)
+
+// SetConfig safely sets the global database instance's config with some
+// basic locks and checks
+func (i *Instance) SetConfig(cfg *Config) error {
+	if i == nil {
+		return errNilInstance
+	}
+	if cfg == nil {
+		return errNilConfig
+	}
+	i.m.Lock()
+	i.config = cfg
+	if i.config.Verbose {
+		boil.DebugMode = true
+		boil.DebugWriter = Logger{}
+	} else {
+		boil.DebugMode = false
+	}
+	i.m.Unlock()
+	return nil
+}
+
+// SetSQLiteConnection safely sets the global database instance's connection
+// to use SQLite
+func (i *Instance) SetSQLiteConnection(con *sql.DB) {
+	i.m.Lock()
+	defer i.m.Unlock()
+	i.SQL = con
+	i.SQL.SetMaxOpenConns(1)
+}
+
+// SetPostgresConnection safely sets the global database instance's connection
+// to use Postgres
+func (i *Instance) SetPostgresConnection(con *sql.DB) error {
+	if err := con.Ping(); err != nil {
+		return err
+	}
+	i.m.Lock()
+	defer i.m.Unlock()
+	i.SQL = con
+	i.SQL.SetMaxOpenConns(2)
+	i.SQL.SetMaxIdleConns(1)
+	i.SQL.SetConnMaxLifetime(time.Hour)
+	return nil
+}
+
+// SetConnected safely sets the global database instance's connected
+// status
+func (i *Instance) SetConnected(v bool) {
+	i.m.Lock()
+	i.connected = v
+	i.m.Unlock()
+}
+
+// CloseConnection safely disconnects the global database instance
+func (i *Instance) CloseConnection() error {
+	i.m.Lock()
+	defer i.m.Unlock()
+	return i.SQL.Close()
+}
+
+// IsConnected safely checks the SQL connection status
+func (i *Instance) IsConnected() bool {
+	i.m.RLock()
+	defer i.m.RUnlock()
+	return i.connected
+}
+
+// GetConfig safely returns a copy of the config
+func (i *Instance) GetConfig() *Config {
+	i.m.RLock()
+	defer i.m.RUnlock()
+	cpy := i.config
+	return cpy
+}
+
+// Ping pings the database
+func (i *Instance) Ping() error {
+	if i == nil {
+		return errNilInstance
+	}
+	i.m.RLock()
+	defer i.m.RUnlock()
+	if i.SQL == nil {
+		return errNilSQL
+	}
+	return i.SQL.Ping()
+}

database/database_types.go

@@ -13,9 +13,9 @@ import (
 type Instance struct {
 	SQL       *sql.DB
 	DataPath  string
-	Config    *Config
-	Connected bool
-	Mu        sync.RWMutex
+	config    *Config
+	connected bool
+	m         sync.RWMutex
 }
 
 // Config holds all database configurable options including enable/disabled & DSN settings
@@ -29,21 +29,21 @@ type Config struct {
 var (
 	// DB Global Database Connection
 	DB = &Instance{}
-
 	// MigrationDir which folder to look in for current migrations
 	MigrationDir = filepath.Join("..", "..", "database", "migrations")
-
 	// ErrNoDatabaseProvided error to display when no database is provided
 	ErrNoDatabaseProvided = errors.New("no database provided")
-
 	// ErrDatabaseSupportDisabled error to display when no database is provided
 	ErrDatabaseSupportDisabled = errors.New("database support is disabled")
-
 	// SupportedDrivers slice of supported database driver types
 	SupportedDrivers = []string{DBSQLite, DBSQLite3, DBPostgreSQL}
-
+	// ErrFailedToConnect for when a database fails to connect
+	ErrFailedToConnect = errors.New("database failed to connect")
 	// DefaultSQLiteDatabase is the default sqlite3 database name to use
 	DefaultSQLiteDatabase = "gocryptotrader.db"
+	errNilConfig          = errors.New("received nil config")
+	errNilInstance        = errors.New("database instance is nil")
+	errNilSQL             = errors.New("database SQL connection is nil")
 )
 
 const (

database/drivers/postgres/postgres.go

@@ -3,7 +3,6 @@ package postgres
 import (
 	"database/sql"
 	"fmt"
-	"time"
 
 	// import go libpq driver package
 	_ "github.com/lib/pq"
@@ -12,32 +11,27 @@ import (
 
 // Connect opens a connection to Postgres database and returns a pointer to database.DB
 func Connect() (*database.Instance, error) {
-	if database.DB.Config.SSLMode == "" {
-		database.DB.Config.SSLMode = "disable"
+	cfg := database.DB.GetConfig()
+
+	if cfg.SSLMode == "" {
+		cfg.SSLMode = "disable"
 	}
 
 	configDSN := fmt.Sprintf("postgres://%s:%s@%s:%d/%s?sslmode=%s",
-		database.DB.Config.Username,
-		database.DB.Config.Password,
-		database.DB.Config.Host,
-		database.DB.Config.Port,
-		database.DB.Config.Database,
-		database.DB.Config.SSLMode)
+		cfg.Username,
+		cfg.Password,
+		cfg.Host,
+		cfg.Port,
+		cfg.Database,
+		cfg.SSLMode)
 
 	db, err := sql.Open(database.DBPostgreSQL, configDSN)
 	if err != nil {
 		return nil, err
 	}
-
-	err = db.Ping()
+	err = database.DB.SetPostgresConnection(db)
 	if err != nil {
 		return nil, err
 	}
-
-	database.DB.SQL = db
-	database.DB.SQL.SetMaxOpenConns(2)
-	database.DB.SQL.SetMaxIdleConns(1)
-	database.DB.SQL.SetConnMaxLifetime(time.Hour)
-
 	return database.DB, nil
 }

database/drivers/sqlite3/sqlite3.go

@@ -11,19 +11,19 @@ import (
 
 // Connect opens a connection to sqlite database and returns a pointer to database.DB
 func Connect() (*database.Instance, error) {
-	if database.DB.Config.Database == "" {
+	cfg := database.DB.GetConfig()
+	if cfg.Database == "" {
 		return nil, database.ErrNoDatabaseProvided
 	}
 
-	databaseFullLocation := filepath.Join(database.DB.DataPath, database.DB.Config.Database)
+	databaseFullLocation := filepath.Join(database.DB.DataPath, cfg.Database)
 
 	dbConn, err := sql.Open("sqlite3", databaseFullLocation)
 	if err != nil {
 		return nil, err
 	}
 
-	database.DB.SQL = dbConn
-	database.DB.SQL.SetMaxOpenConns(1)
+	database.DB.SetSQLiteConnection(dbConn)
 
 	return database.DB, nil
 }

database/repository/repository.go

@@ -6,7 +6,8 @@ import (
 
 // GetSQLDialect returns current SQL Dialect based on enabled driver
 func GetSQLDialect() string {
-	switch database.DB.Config.Driver {
+	cfg := database.DB.GetConfig()
+	switch cfg.Driver {
 	case "sqlite", "sqlite3":
 		return database.DBSQLite3
 	case "psql", "postgres", "postgresql":

database/repository/repository_test.go

@@ -32,8 +32,11 @@ func TestGetSQLDialect(t *testing.T) {
 		test := testCases[x]
 
 		t.Run(test.driver, func(t *testing.T) {
-			database.DB.Config = &database.Config{
+			err := database.DB.SetConfig(&database.Config{
 				Driver: test.driver,
+			})
+			if err != nil {
+				t.Error(err)
 			}
 			ret := GetSQLDialect()
 			if ret != test.expectedReturn {

database/testhelpers/test_helpers.go

@@ -75,7 +75,10 @@ func GetConnectionDetails() *database.Config {
 
 // ConnectToDatabase opens connection to database and returns pointer to instance of database.DB
 func ConnectToDatabase(conn *database.Config) (dbConn *database.Instance, err error) {
-	database.DB.Config = conn
+	err = database.DB.SetConfig(conn)
+	if err != nil {
+		return nil, err
+	}
 	if conn.Driver == database.DBPostgreSQL {
 		dbConn, err = psqlConn.Connect()
 		if err != nil {

dispatch/dispatch.go

@@ -8,10 +8,12 @@ import (
 	"time"
 
 	"github.com/gofrs/uuid"
-	"github.com/thrasher-corp/gocryptotrader/engine/subsystem"
 	"github.com/thrasher-corp/gocryptotrader/log"
 )
 
+// Name is an exported subsystem name
+const Name = "dispatch"
+
 func init() {
 	dispatcher = &Dispatcher{
 		routes: make(map[uuid.UUID][]chan interface{}),
@@ -80,7 +82,7 @@ func SpawnWorker() error {
 // configuration, then spawns workers
 func (d *Dispatcher) start(workers, channelCapacity int) error {
 	if atomic.LoadUint32(&d.running) == 1 {
-		return fmt.Errorf("dispatcher %w", subsystem.ErrSubSystemAlreadyStarted)
+		return errors.New("dispatcher already running")
 	}
 
 	if workers < 1 {
@@ -115,7 +117,7 @@ func (d *Dispatcher) start(workers, channelCapacity int) error {
 // stop stops the service and shuts down all worker routines
 func (d *Dispatcher) stop() error {
 	if !atomic.CompareAndSwapUint32(&d.running, 1, 0) {
-		return fmt.Errorf("dispatcher %w", subsystem.ErrSubSystemNotStarted)
+		return errors.New("dispatcher not running")
 	}
 	close(d.shutdown)
 	ch := make(chan struct{})
@@ -154,6 +156,9 @@ func (d *Dispatcher) stop() error {
 
 // isRunning returns if the dispatch system is running
 func (d *Dispatcher) isRunning() bool {
+	if d == nil {
+		return false
+	}
 	return atomic.LoadUint32(&d.running) == 1
 }
 

dispatch/dispatch_types.go

@@ -66,7 +66,7 @@ type job struct {
 	ID   uuid.UUID
 }
 
-// Mux defines a new multiplexor for the dispatch system, these a generated
+// Mux defines a new multiplexer for the dispatch system, these a generated
 // per subsystem
 type Mux struct {
 	// Reference to the main running dispatch service
@@ -80,6 +80,6 @@ type Pipe struct {
 	C chan interface{}
 	// ID to tracked system
 	id uuid.UUID
-	// Reference to multiplexor
+	// Reference to multiplexer
 	m *Mux
 }

dispatch/mux.go

@@ -7,7 +7,7 @@ import (
 	"github.com/gofrs/uuid"
 )
 
-// GetNewMux returns a new multiplexor to track subsystem updates
+// GetNewMux returns a new multiplexer to track subsystem updates
 func GetNewMux() *Mux {
 	return &Mux{d: dispatcher}
 }

engine/addr_helpers.go

@@ -1,100 +0,0 @@
-package engine
-
-import (
-	"errors"
-	"strings"
-	"sync"
-
-	"github.com/thrasher-corp/gocryptotrader/currency"
-)
-
-// DepositAddressStore stores a list of exchange deposit addresses
-type DepositAddressStore struct {
-	m     sync.Mutex
-	Store map[string]map[string]string
-}
-
-// DepositAddressManager manages the exchange deposit address store
-type DepositAddressManager struct {
-	Store DepositAddressStore
-}
-
-// vars related to the deposit address helpers
-var (
-	ErrDepositAddressStoreIsNil = errors.New("deposit address store is nil")
-	ErrDepositAddressNotFound   = errors.New("deposit address does not exist")
-)
-
-// Seed seeds the deposit address store
-func (d *DepositAddressStore) Seed(coinData map[string]map[string]string) {
-	d.m.Lock()
-	defer d.m.Unlock()
-	if d.Store == nil {
-		d.Store = make(map[string]map[string]string)
-	}
-
-	for k, v := range coinData {
-		r := make(map[string]string)
-		for w, x := range v {
-			r[strings.ToUpper(w)] = x
-		}
-		d.Store[strings.ToUpper(k)] = r
-	}
-}
-
-// GetDepositAddress returns a deposit address based on the specified item
-func (d *DepositAddressStore) GetDepositAddress(exchName string, item currency.Code) (string, error) {
-	d.m.Lock()
-	defer d.m.Unlock()
-
-	if len(d.Store) == 0 {
-		return "", ErrDepositAddressStoreIsNil
-	}
-
-	r, ok := d.Store[strings.ToUpper(exchName)]
-	if !ok {
-		return "", ErrExchangeNotFound
-	}
-
-	addr, ok := r[strings.ToUpper(item.String())]
-	if !ok {
-		return "", ErrDepositAddressNotFound
-	}
-
-	return addr, nil
-}
-
-// GetDepositAddresses returns a list of stored deposit addresses
-func (d *DepositAddressStore) GetDepositAddresses(exchName string) (map[string]string, error) {
-	d.m.Lock()
-	defer d.m.Unlock()
-
-	if len(d.Store) == 0 {
-		return nil, ErrDepositAddressStoreIsNil
-	}
-
-	r, ok := d.Store[strings.ToUpper(exchName)]
-	if !ok {
-		return nil, ErrDepositAddressNotFound
-	}
-
-	return r, nil
-}
-
-// GetDepositAddressByExchange returns a deposit address for the specified exchange and cryptocurrency
-// if it exists
-func (d *DepositAddressManager) GetDepositAddressByExchange(exchName string, currencyItem currency.Code) (string, error) {
-	return d.Store.GetDepositAddress(exchName, currencyItem)
-}
-
-// GetDepositAddressesByExchange returns a list of cryptocurrency addresses for the specified
-// exchange if they exist
-func (d *DepositAddressManager) GetDepositAddressesByExchange(exchName string) (map[string]string, error) {
-	return d.Store.GetDepositAddresses(exchName)
-}
-
-// Sync synchronises all deposit addresses
-func (d *DepositAddressManager) Sync() {
-	result := Bot.GetExchangeCryptocurrencyDepositAddresses()
-	d.Store.Seed(result)
-}

engine/addr_helpers_test.go

@@ -1,64 +0,0 @@
-package engine
-
-import (
-	"testing"
-
-	"github.com/thrasher-corp/gocryptotrader/currency"
-)
-
-const (
-	testBTCAddress = "1F1tAaz5x1HUXrCNLbtMDqcw6o5GNn4xqX"
-)
-
-func TestSeed(t *testing.T) {
-	var d DepositAddressStore
-	u := map[string]map[string]string{
-		"BITSTAMP": {
-			"BTC": testBTCAddress,
-		},
-	}
-
-	d.Seed(u)
-	r, err := d.GetDepositAddress("BITSTAMP", currency.BTC)
-	if err != nil {
-		t.Error("unexpected result")
-	}
-
-	if r != testBTCAddress {
-		t.Error("unexpected result")
-	}
-}
-
-func TestGetDepositAddress(t *testing.T) {
-	var d DepositAddressStore
-	_, err := d.GetDepositAddress("", currency.BTC)
-	if err != ErrDepositAddressStoreIsNil {
-		t.Error("non-error on non-existent exchange")
-	}
-
-	d.Store = map[string]map[string]string{
-		"BITSTAMP": {
-			"BTC": testBTCAddress,
-		},
-	}
-
-	_, err = d.GetDepositAddress("", currency.BTC)
-	if err != ErrExchangeNotFound {
-		t.Error("non-error on non-existent exchange")
-	}
-
-	var r string
-	r, err = d.GetDepositAddress("BiTStAmP", currency.NewCode("bTC"))
-	if err != nil {
-		t.Error("unexpected err: ", err)
-	}
-
-	if r != testBTCAddress {
-		t.Error("unexpected BTC address: ", r)
-	}
-
-	_, err = d.GetDepositAddress("BiTStAmP", currency.LTC)
-	if err != ErrDepositAddressNotFound {
-		t.Error("unexpected err: ", err)
-	}
-}

engine/apiserver.go

@@ -0,0 +1,906 @@
+package engine
+
+import (
+	"context"
+	"encoding/json"
+	"errors"
+	"fmt"
+	"net/http"
+	"net/http/pprof"
+	"runtime"
+	"strconv"
+	"strings"
+	"sync/atomic"
+	"time"
+
+	"github.com/gorilla/mux"
+	"github.com/gorilla/websocket"
+	"github.com/thrasher-corp/gocryptotrader/common"
+	"github.com/thrasher-corp/gocryptotrader/common/crypto"
+	"github.com/thrasher-corp/gocryptotrader/config"
+	"github.com/thrasher-corp/gocryptotrader/currency"
+	"github.com/thrasher-corp/gocryptotrader/database/repository/exchange"
+	"github.com/thrasher-corp/gocryptotrader/exchanges/asset"
+	"github.com/thrasher-corp/gocryptotrader/log"
+)
+
+// setupAPIServerManager checks and creates an api server manager
+func setupAPIServerManager(remoteConfig *config.RemoteControlConfig, pprofConfig *config.Profiler, exchangeManager iExchangeManager, bot iBot, portfolioManager iPortfolioManager, configPath string) (*apiServerManager, error) {
+	if remoteConfig == nil {
+		return nil, errNilRemoteConfig
+	}
+	if pprofConfig == nil {
+		return nil, errNilPProfConfig
+	}
+	if exchangeManager == nil {
+		return nil, errNilExchangeManager
+	}
+	if bot == nil {
+		return nil, errNilBot
+	}
+	if configPath == "" {
+		return nil, errEmptyConfigPath
+	}
+	return &apiServerManager{
+		remoteConfig:           remoteConfig,
+		pprofConfig:            pprofConfig,
+		restListenAddress:      remoteConfig.DeprecatedRPC.ListenAddress,
+		websocketListenAddress: remoteConfig.WebsocketRPC.ListenAddress,
+		exchangeManager:        exchangeManager,
+		bot:                    bot,
+		gctConfigPath:          configPath,
+		portfolioManager:       portfolioManager,
+	}, nil
+}
+
+// IsRESTServerRunning safely checks whether the subsystem is running
+func (m *apiServerManager) IsRESTServerRunning() bool {
+	if m == nil {
+		return false
+	}
+	return atomic.LoadInt32(&m.restStarted) == 1
+}
+
+// IsWebsocketServerRunning safely checks whether the subsystem is running
+func (m *apiServerManager) IsWebsocketServerRunning() bool {
+	if m == nil {
+		return false
+	}
+	return atomic.LoadInt32(&m.websocketStarted) == 1
+}
+
+// StopRESTServer attempts to shutdown the subsystem
+func (m *apiServerManager) StopRESTServer() error {
+	if m == nil {
+		return fmt.Errorf("api server %w", ErrNilSubsystem)
+	}
+	if !atomic.CompareAndSwapInt32(&m.restStarted, 1, 0) {
+		return fmt.Errorf("apiserver deprecated server %w", ErrSubSystemNotStarted)
+	}
+	err := m.restHTTPServer.Shutdown(context.Background())
+	if err != nil && !errors.Is(err, http.ErrServerClosed) {
+		return err
+	}
+	m.wgRest.Wait()
+	m.restRouter = nil
+	return nil
+}
+
+func (m *apiServerManager) StopWebsocketServer() error {
+	if m == nil {
+		return fmt.Errorf("api server %w", ErrNilSubsystem)
+	}
+	if !atomic.CompareAndSwapInt32(&m.websocketStarted, 1, 0) {
+		return fmt.Errorf("apiserver websocket server %w", ErrSubSystemNotStarted)
+	}
+
+	err := m.websocketHTTPServer.Shutdown(context.Background())
+	if err != nil && !errors.Is(err, http.ErrServerClosed) {
+		return err
+	}
+	m.websocketRouter = nil
+	m.websocketHub = nil
+	m.wgWebsocket.Wait()
+	m.websocketHTTPServer = nil
+	return nil
+}
+
+// newRouter takes in the exchange interfaces and returns a new multiplexer
+// router
+func (m *apiServerManager) newRouter(isREST bool) *mux.Router {
+	router := mux.NewRouter().StrictSlash(true)
+	var routes []Route
+	if common.ExtractPort(m.websocketListenAddress) == 80 {
+		m.websocketListenAddress = common.ExtractHost(m.websocketListenAddress)
+	} else {
+		m.websocketListenAddress = strings.Join([]string{common.ExtractHost(m.websocketListenAddress),
+			strconv.Itoa(common.ExtractPort(m.websocketListenAddress))}, ":")
+	}
+
+	if isREST {
+		routes = []Route{
+			{"", http.MethodGet, "/", m.getIndex},
+			{"GetAllSettings", http.MethodGet, "/config/all", m.restGetAllSettings},
+			{"SaveAllSettings", http.MethodPost, "/config/all/save", m.restSaveAllSettings},
+			{"AllEnabledAccountInfo", http.MethodGet, "/exchanges/enabled/accounts/all", m.restGetAllEnabledAccountInfo},
+			{"AllActiveExchangesAndCurrencies", http.MethodGet, "/exchanges/enabled/latest/all", m.restGetAllActiveTickers},
+			{"GetPortfolio", http.MethodGet, "/portfolio/all", m.restGetPortfolio},
+			{"AllActiveExchangesAndOrderbooks", http.MethodGet, "/exchanges/orderbook/latest/all", m.restGetAllActiveOrderbooks},
+		}
+
+		if m.pprofConfig.Enabled {
+			if m.pprofConfig.MutexProfileFraction > 0 {
+				runtime.SetMutexProfileFraction(m.pprofConfig.MutexProfileFraction)
+			}
+			log.Debugf(log.RESTSys,
+				"HTTP Go performance profiler (pprof) endpoint enabled: http://%s:%d/debug/pprof/\n",
+				common.ExtractHost(m.websocketListenAddress),
+				common.ExtractPort(m.websocketListenAddress))
+			router.PathPrefix("/debug/pprof/").HandlerFunc(pprof.Index)
+		}
+	} else {
+		routes = []Route{
+			{"ws", http.MethodGet, "/ws", m.WebsocketClientHandler},
+		}
+	}
+
+	for _, route := range routes {
+		router.
+			Methods(route.Method).
+			Path(route.Pattern).
+			Name(route.Name).
+			Handler(restLogger(route.HandlerFunc, route.Name)).
+			Host(m.websocketListenAddress)
+	}
+	return router
+}
+
+// StartRESTServer starts a REST handler
+func (m *apiServerManager) StartRESTServer() error {
+	if !atomic.CompareAndSwapInt32(&m.restStarted, 0, 1) {
+		return fmt.Errorf("rest server %w", errAlreadyRunning)
+	}
+	if !m.remoteConfig.DeprecatedRPC.Enabled {
+		atomic.StoreInt32(&m.restStarted, 0)
+		return fmt.Errorf("rest %w", errServerDisabled)
+	}
+	log.Debugf(log.RESTSys,
+		"Deprecated RPC handler support enabled. Listen URL: http://%s:%d\n",
+		common.ExtractHost(m.restListenAddress), common.ExtractPort(m.restListenAddress))
+	m.restRouter = m.newRouter(true)
+	if m.restHTTPServer == nil {
+		m.restHTTPServer = &http.Server{
+			Addr:    m.restListenAddress,
+			Handler: m.restRouter,
+		}
+	}
+	m.wgRest.Add(1)
+	go func() {
+		defer m.wgRest.Done()
+		err := m.restHTTPServer.ListenAndServe()
+		if err != nil {
+			atomic.StoreInt32(&m.restStarted, 0)
+			if !errors.Is(err, http.ErrServerClosed) {
+				log.Error(log.APIServerMgr, err)
+			}
+		}
+	}()
+	return nil
+}
+
+// restLogger logs the requests internally
+func restLogger(inner http.Handler, name string) http.Handler {
+	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		start := time.Now()
+		inner.ServeHTTP(w, r)
+
+		log.Debugf(log.RESTSys,
+			"%s\t%s\t%s\t%s",
+			r.Method,
+			r.RequestURI,
+			name,
+			time.Since(start),
+		)
+	})
+}
+
+// writeResponse outputs a JSON response of the response interface
+func writeResponse(w http.ResponseWriter, response interface{}) error {
+	w.Header().Set("Content-Type", "application/json; charset=UTF-8")
+	w.WriteHeader(http.StatusOK)
+	return json.NewEncoder(w).Encode(response)
+}
+
+// handleError prints the REST method and error
+func handleError(method string, err error) {
+	log.Errorf(log.APIServerMgr, "RESTful %s: handler failed to send JSON response. Error %s\n",
+		method, err)
+}
+
+// restGetAllSettings replies to a request with an encoded JSON response about the
+// trading Bots configuration.
+func (m *apiServerManager) restGetAllSettings(w http.ResponseWriter, r *http.Request) {
+	err := writeResponse(w, config.GetConfig())
+	if err != nil {
+		handleError(r.Method, err)
+	}
+}
+
+// restSaveAllSettings saves all current settings from request body as a JSON
+// document then reloads state and returns the settings
+func (m *apiServerManager) restSaveAllSettings(w http.ResponseWriter, r *http.Request) {
+	// Get the data from the request
+	decoder := json.NewDecoder(r.Body)
+	var responseData config.Post
+	err := decoder.Decode(&responseData)
+	if err != nil {
+		handleError(r.Method, err)
+	}
+	// Save change the settings
+	cfg := config.GetConfig()
+	err = cfg.UpdateConfig(m.gctConfigPath, &responseData.Data, false)
+	if err != nil {
+		handleError(r.Method, err)
+	}
+
+	err = writeResponse(w, cfg)
+	if err != nil {
+		handleError(r.Method, err)
+	}
+	err = m.bot.SetupExchanges()
+	if err != nil {
+		handleError(r.Method, err)
+	}
+}
+
+// restGetAllActiveOrderbooks returns all enabled exchange orderbooks
+func (m *apiServerManager) restGetAllActiveOrderbooks(w http.ResponseWriter, r *http.Request) {
+	var response AllEnabledExchangeOrderbooks
+	response.Data = getAllActiveOrderbooks(m.exchangeManager)
+	err := writeResponse(w, response)
+	if err != nil {
+		handleError(r.Method, err)
+	}
+}
+
+// restGetPortfolio returns the Bot portfolio manager
+func (m *apiServerManager) restGetPortfolio(w http.ResponseWriter, r *http.Request) {
+	result := m.portfolioManager.GetPortfolioSummary()
+	err := writeResponse(w, result)
+	if err != nil {
+		handleError(r.Method, err)
+	}
+}
+
+// restGetAllActiveTickers returns all active tickers
+func (m *apiServerManager) restGetAllActiveTickers(w http.ResponseWriter, r *http.Request) {
+	var response AllEnabledExchangeCurrencies
+	response.Data = getAllActiveTickers(m.exchangeManager)
+	err := writeResponse(w, response)
+	if err != nil {
+		handleError(r.Method, err)
+	}
+}
+
+// restGetAllEnabledAccountInfo via get request returns JSON response of account
+// info
+func (m *apiServerManager) restGetAllEnabledAccountInfo(w http.ResponseWriter, r *http.Request) {
+	response := getAllActiveAccounts(m.exchangeManager)
+	err := writeResponse(w, response)
+	if err != nil {
+		handleError(r.Method, err)
+	}
+}
+
+// getIndex returns an HTML snippet for when a user requests the index URL
+func (m *apiServerManager) getIndex(w http.ResponseWriter, _ *http.Request) {
+	_, err := fmt.Fprint(w, restIndexResponse)
+	if err != nil {
+		log.Error(log.APIServerMgr, err)
+	}
+	w.WriteHeader(http.StatusOK)
+}
+
+// getAllActiveOrderbooks returns all enabled exchanges orderbooks
+func getAllActiveOrderbooks(m iExchangeManager) []EnabledExchangeOrderbooks {
+	var orderbookData []EnabledExchangeOrderbooks
+	exchanges := m.GetExchanges()
+	for x := range exchanges {
+		assets := exchanges[x].GetAssetTypes()
+		exchName := exchanges[x].GetName()
+		var exchangeOB EnabledExchangeOrderbooks
+		exchangeOB.ExchangeName = exchName
+
+		for y := range assets {
+			currencies, err := exchanges[x].GetEnabledPairs(assets[y])
+			if err != nil {
+				log.Errorf(log.APIServerMgr,
+					"Exchange %s could not retrieve enabled currencies. Err: %s\n",
+					exchName,
+					err)
+				continue
+			}
+			for z := range currencies {
+				ob, err := exchanges[x].FetchOrderbook(currencies[z], assets[y])
+				if err != nil {
+					log.Errorf(log.APIServerMgr,
+						"Exchange %s failed to retrieve %s orderbook. Err: %s\n", exchName,
+						currencies[z].String(),
+						err)
+					continue
+				}
+				exchangeOB.ExchangeValues = append(exchangeOB.ExchangeValues, *ob)
+			}
+			orderbookData = append(orderbookData, exchangeOB)
+		}
+		orderbookData = append(orderbookData, exchangeOB)
+	}
+	return orderbookData
+}
+
+// getAllActiveTickers returns all enabled exchanges tickers
+func getAllActiveTickers(m iExchangeManager) []EnabledExchangeCurrencies {
+	var tickers []EnabledExchangeCurrencies
+	exchanges := m.GetExchanges()
+	for x := range exchanges {
+		assets := exchanges[x].GetAssetTypes()
+		exchName := exchanges[x].GetName()
+		var exchangeTickers EnabledExchangeCurrencies
+		exchangeTickers.ExchangeName = exchName
+
+		for y := range assets {
+			currencies, err := exchanges[x].GetEnabledPairs(assets[y])
+			if err != nil {
+				log.Errorf(log.APIServerMgr,
+					"Exchange %s could not retrieve enabled currencies. Err: %s\n",
+					exchName,
+					err)
+				continue
+			}
+			for z := range currencies {
+				t, err := exchanges[x].FetchTicker(currencies[z], assets[y])
+				if err != nil {
+					log.Errorf(log.APIServerMgr,
+						"Exchange %s failed to retrieve %s ticker. Err: %s\n", exchName,
+						currencies[z].String(),
+						err)
+					continue
+				}
+				exchangeTickers.ExchangeValues = append(exchangeTickers.ExchangeValues, *t)
+			}
+			tickers = append(tickers, exchangeTickers)
+		}
+		tickers = append(tickers, exchangeTickers)
+	}
+	return tickers
+}
+
+// getAllActiveAccounts returns all enabled exchanges accounts
+func getAllActiveAccounts(m iExchangeManager) []AllEnabledExchangeAccounts {
+	var accounts []AllEnabledExchangeAccounts
+	exchanges := m.GetExchanges()
+	for x := range exchanges {
+		assets := exchanges[x].GetAssetTypes()
+		exchName := exchanges[x].GetName()
+		var exchangeAccounts AllEnabledExchangeAccounts
+		for y := range assets {
+			a, err := exchanges[x].FetchAccountInfo(assets[y])
+			if err != nil {
+				log.Errorf(log.APIServerMgr,
+					"Exchange %s failed to retrieve %s ticker. Err: %s\n",
+					exchName,
+					assets[y],
+					err)
+				continue
+			}
+			exchangeAccounts.Data = append(exchangeAccounts.Data, a)
+		}
+		accounts = append(accounts, exchangeAccounts)
+	}
+	return accounts
+}
+
+// StartWebsocketServer starts a Websocket handler
+func (m *apiServerManager) StartWebsocketServer() error {
+	if !atomic.CompareAndSwapInt32(&m.websocketStarted, 0, 1) {
+		return fmt.Errorf("websocket server %w", errAlreadyRunning)
+	}
+	if !m.remoteConfig.WebsocketRPC.Enabled {
+		atomic.StoreInt32(&m.websocketStarted, 0)
+		return fmt.Errorf("websocket %w", errServerDisabled)
+	}
+	log.Debugf(log.APIServerMgr,
+		"Websocket RPC support enabled. Listen URL: ws://%s:%d/ws\n",
+		common.ExtractHost(m.websocketListenAddress), common.ExtractPort(m.websocketListenAddress))
+	m.websocketRouter = m.newRouter(false)
+	if m.websocketHTTPServer == nil {
+		m.websocketHTTPServer = &http.Server{
+			Addr:    m.websocketListenAddress,
+			Handler: m.websocketRouter,
+		}
+	}
+
+	m.wgWebsocket.Add(1)
+	go func() {
+		defer m.wgWebsocket.Done()
+		err := m.websocketHTTPServer.ListenAndServe()
+		if err != nil {
+			atomic.StoreInt32(&m.websocketStarted, 0)
+			if !errors.Is(err, http.ErrServerClosed) {
+				log.Error(log.APIServerMgr, err)
+			}
+		}
+	}()
+	return nil
+}
+
+// newWebsocketHub Creates a new websocket hub
+func newWebsocketHub() *websocketHub {
+	return &websocketHub{
+		Broadcast:  make(chan []byte),
+		Register:   make(chan *websocketClient),
+		Unregister: make(chan *websocketClient),
+		Clients:    make(map[*websocketClient]bool),
+	}
+}
+
+func (h *websocketHub) run() {
+	for {
+		select {
+		case client := <-h.Register:
+			h.Clients[client] = true
+		case client := <-h.Unregister:
+			if _, ok := h.Clients[client]; ok {
+				log.Debugln(log.APIServerMgr, "websocket: disconnected client")
+				delete(h.Clients, client)
+				close(client.Send)
+			}
+		case message := <-h.Broadcast:
+			for client := range h.Clients {
+				select {
+				case client.Send <- message:
+				default:
+					log.Debugln(log.APIServerMgr, "websocket: disconnected client")
+					close(client.Send)
+					delete(h.Clients, client)
+				}
+			}
+		}
+	}
+}
+
+// SendWebsocketMessage sends a websocket event to the client
+func (c *websocketClient) SendWebsocketMessage(evt interface{}) error {
+	data, err := json.Marshal(evt)
+	if err != nil {
+		log.Errorf(log.APIServerMgr, "websocket: failed to send message: %s\n", err)
+		return err
+	}
+
+	c.Send <- data
+	return nil
+}
+
+func (c *websocketClient) read() {
+	defer func() {
+		c.Hub.Unregister <- c
+		conErr := c.Conn.Close()
+		if conErr != nil {
+			log.Error(log.APIServerMgr, conErr)
+		}
+	}()
+
+	for {
+		msgType, message, err := c.Conn.ReadMessage()
+		if err != nil {
+			if websocket.IsUnexpectedCloseError(err, websocket.CloseGoingAway, websocket.CloseAbnormalClosure) {
+				log.Errorf(log.APIServerMgr, "websocket: client disconnected, err: %s\n", err)
+			}
+			break
+		}
+
+		if msgType == websocket.TextMessage {
+			var evt WebsocketEvent
+			err := json.Unmarshal(message, &evt)
+			if err != nil {
+				log.Errorf(log.APIServerMgr, "websocket: failed to decode JSON sent from client %s\n", err)
+				continue
+			}
+
+			if evt.Event == "" {
+				log.Warnln(log.APIServerMgr, "websocket: client sent a blank event, disconnecting")
+				continue
+			}
+
+			dataJSON, err := json.Marshal(evt.Data)
+			if err != nil {
+				log.Errorln(log.APIServerMgr, "websocket: client sent data we couldn't JSON decode")
+				break
+			}
+
+			req := strings.ToLower(evt.Event)
+			log.Debugf(log.APIServerMgr, "websocket: request received: %s\n", req)
+
+			result, ok := wsHandlers[req]
+			if !ok {
+				log.Debugln(log.APIServerMgr, "websocket: unsupported event")
+				continue
+			}
+
+			if result.authRequired && !c.Authenticated {
+				log.Warnf(log.APIServerMgr, "Websocket: request %s failed due to unauthenticated request on an authenticated API\n", evt.Event)
+				err = c.SendWebsocketMessage(WebsocketEventResponse{Event: evt.Event, Error: "unauthorised request on authenticated API"})
+				if err != nil {
+					log.Error(log.APIServerMgr, err)
+				}
+				continue
+			}
+
+			err = result.handler(c, dataJSON)
+			if err != nil {
+				log.Errorf(log.APIServerMgr, "websocket: request %s failed. Error %s\n", evt.Event, err)
+				continue
+			}
+		}
+	}
+}
+
+func (c *websocketClient) write() {
+	defer func() {
+		err := c.Conn.Close()
+		if err != nil {
+			log.Error(log.APIServerMgr, err)
+		}
+	}()
+	for {
+		message, ok := <-c.Send
+		if !ok {
+			err := c.Conn.WriteMessage(websocket.CloseMessage, []byte{})
+			if err != nil {
+				log.Error(log.APIServerMgr, err)
+			}
+			log.Debugln(log.APIServerMgr, "websocket: hub closed the channel")
+			return
+		}
+
+		w, err := c.Conn.NextWriter(websocket.TextMessage)
+		if err != nil {
+			log.Errorf(log.APIServerMgr, "websocket: failed to create new io.writeCloser: %s\n", err)
+			return
+		}
+		_, err = w.Write(message)
+		if err != nil {
+			log.Error(log.APIServerMgr, err)
+		}
+
+		// Add queued chat messages to the current websocket message
+		n := len(c.Send)
+		for i := 0; i < n; i++ {
+			_, err = w.Write(<-c.Send)
+			if err != nil {
+				log.Error(log.APIServerMgr, err)
+			}
+		}
+
+		if err := w.Close(); err != nil {
+			log.Errorf(log.APIServerMgr, "websocket: failed to close io.WriteCloser: %s\n", err)
+			return
+		}
+	}
+}
+
+// StartWebsocketHandler starts the websocket hub and routine which
+// handles clients
+func StartWebsocketHandler() {
+	if !wsHubStarted {
+		wsHubStarted = true
+		wsHub = newWebsocketHub()
+		go wsHub.run()
+	}
+}
+
+// BroadcastWebsocketMessage meow
+func BroadcastWebsocketMessage(evt WebsocketEvent) error {
+	if !wsHubStarted {
+		return ErrWebsocketServiceNotRunning
+	}
+
+	data, err := json.Marshal(evt)
+	if err != nil {
+		return err
+	}
+
+	wsHub.Broadcast <- data
+	return nil
+}
+
+// WebsocketClientHandler upgrades the HTTP connection to a websocket
+// compatible one
+func (m *apiServerManager) WebsocketClientHandler(w http.ResponseWriter, r *http.Request) {
+	if !wsHubStarted {
+		StartWebsocketHandler()
+	}
+
+	connectionLimit := m.remoteConfig.WebsocketRPC.ConnectionLimit
+	numClients := len(wsHub.Clients)
+
+	if numClients >= connectionLimit {
+		log.Warnf(log.APIServerMgr,
+			"websocket: client rejected due to websocket client limit reached. Number of clients %d. Limit %d.\n",
+			numClients, connectionLimit)
+		w.WriteHeader(http.StatusForbidden)
+		return
+	}
+
+	upgrader := websocket.Upgrader{
+		WriteBufferSize: 1024,
+		ReadBufferSize:  1024,
+	}
+
+	// Allow insecure origin if the Origin request header is present and not
+	// equal to the Host request header. Default to false
+	if m.remoteConfig.WebsocketRPC.AllowInsecureOrigin {
+		upgrader.CheckOrigin = func(r *http.Request) bool { return true }
+	}
+
+	conn, err := upgrader.Upgrade(w, r, nil)
+	if err != nil {
+		log.Error(log.APIServerMgr, err)
+		return
+	}
+
+	client := &websocketClient{
+		Hub:              wsHub,
+		Conn:             conn,
+		Send:             make(chan []byte, 1024),
+		maxAuthFailures:  m.remoteConfig.WebsocketRPC.MaxAuthFailures,
+		username:         m.remoteConfig.Username,
+		password:         m.remoteConfig.Password,
+		configPath:       m.gctConfigPath,
+		exchangeManager:  m.exchangeManager,
+		bot:              m.bot,
+		portfolioManager: m.portfolioManager,
+	}
+
+	client.Hub.Register <- client
+	log.Debugf(log.APIServerMgr,
+		"websocket: client connected. Connected clients: %d. Limit %d.\n",
+		numClients+1, connectionLimit)
+
+	go client.read()
+	go client.write()
+}
+
+func wsAuth(client *websocketClient, data interface{}) error {
+	wsResp := WebsocketEventResponse{
+		Event: "auth",
+	}
+
+	var auth WebsocketAuth
+	err := json.Unmarshal(data.([]byte), &auth)
+	if err != nil {
+		wsResp.Error = err.Error()
+		sendErr := client.SendWebsocketMessage(wsResp)
+		if sendErr != nil {
+			log.Error(log.APIServerMgr, sendErr)
+		}
+		return err
+	}
+
+	hashPW := crypto.HexEncodeToString(crypto.GetSHA256([]byte(client.password)))
+	if auth.Username == client.username && auth.Password == hashPW {
+		client.Authenticated = true
+		wsResp.Data = WebsocketResponseSuccess
+		log.Debugln(log.APIServerMgr,
+			"websocket: client authenticated successfully")
+		return client.SendWebsocketMessage(wsResp)
+	}
+
+	wsResp.Error = "invalid username/password"
+	client.authFailures++
+	sendErr := client.SendWebsocketMessage(wsResp)
+	if sendErr != nil {
+		log.Error(log.APIServerMgr, sendErr)
+	}
+	if client.authFailures >= client.maxAuthFailures {
+		log.Debugf(log.APIServerMgr,
+			"websocket: disconnecting client, maximum auth failures threshold reached (failures: %d limit: %d)\n",
+			client.authFailures, client.maxAuthFailures)
+		wsHub.Unregister <- client
+		return nil
+	}
+
+	log.Debugf(log.APIServerMgr,
+		"websocket: client sent wrong username/password (failures: %d limit: %d)\n",
+		client.authFailures, client.maxAuthFailures)
+	return nil
+}
+
+func wsGetConfig(client *websocketClient, _ interface{}) error {
+	wsResp := WebsocketEventResponse{
+		Event: "GetConfig",
+		Data:  config.GetConfig(),
+	}
+	return client.SendWebsocketMessage(wsResp)
+}
+
+func wsSaveConfig(client *websocketClient, data interface{}) error {
+	wsResp := WebsocketEventResponse{
+		Event: "SaveConfig",
+	}
+	var respCfg config.Config
+	err := json.Unmarshal(data.([]byte), &respCfg)
+	if err != nil {
+		wsResp.Error = err.Error()
+		sendErr := client.SendWebsocketMessage(wsResp)
+		if sendErr != nil {
+			log.Error(log.APIServerMgr, sendErr)
+		}
+		return err
+	}
+
+	cfg := config.GetConfig()
+	err = cfg.UpdateConfig(client.configPath, &respCfg, false)
+	if err != nil {
+		wsResp.Error = err.Error()
+		sendErr := client.SendWebsocketMessage(wsResp)
+		if sendErr != nil {
+			log.Error(log.APIServerMgr, sendErr)
+		}
+		return err
+	}
+
+	err = client.bot.SetupExchanges()
+	if err != nil {
+		wsResp.Error = err.Error()
+		sendErr := client.SendWebsocketMessage(wsResp)
+		if sendErr != nil {
+			log.Error(log.APIServerMgr, sendErr)
+		}
+		return err
+	}
+	wsResp.Data = WebsocketResponseSuccess
+	return client.SendWebsocketMessage(wsResp)
+}
+
+func wsGetAccountInfo(client *websocketClient, data interface{}) error {
+	accountInfo := getAllActiveAccounts(client.exchangeManager)
+	wsResp := WebsocketEventResponse{
+		Event: "GetAccountInfo",
+		Data:  accountInfo,
+	}
+	return client.SendWebsocketMessage(wsResp)
+}
+
+func wsGetTickers(client *websocketClient, data interface{}) error {
+	wsResp := WebsocketEventResponse{
+		Event: "GetTickers",
+	}
+	wsResp.Data = getAllActiveTickers(client.exchangeManager)
+	return client.SendWebsocketMessage(wsResp)
+}
+
+func wsGetTicker(client *websocketClient, data interface{}) error {
+	wsResp := WebsocketEventResponse{
+		Event: "GetTicker",
+	}
+	var tickerReq WebsocketOrderbookTickerRequest
+	err := json.Unmarshal(data.([]byte), &tickerReq)
+	if err != nil {
+		wsResp.Error = err.Error()
+		sendErr := client.SendWebsocketMessage(wsResp)
+		if sendErr != nil {
+			log.Error(log.APIServerMgr, sendErr)
+		}
+		return err
+	}
+
+	p, err := currency.NewPairFromString(tickerReq.Currency)
+	if err != nil {
+		return err
+	}
+
+	a, err := asset.New(tickerReq.AssetType)
+	if err != nil {
+		return err
+	}
+
+	exch := client.exchangeManager.GetExchangeByName(tickerReq.Exchange)
+	if exch == nil {
+		wsResp.Error = exchange.ErrNoExchangeFound.Error()
+		sendErr := client.SendWebsocketMessage(wsResp)
+		if sendErr != nil {
+			log.Error(log.APIServerMgr, sendErr)
+		}
+		return err
+	}
+	tick, err := exch.FetchTicker(p, a)
+	if err != nil {
+		wsResp.Error = err.Error()
+		sendErr := client.SendWebsocketMessage(wsResp)
+		if sendErr != nil {
+			log.Error(log.APIServerMgr, sendErr)
+		}
+		return err
+	}
+	wsResp.Data = tick
+	return client.SendWebsocketMessage(wsResp)
+}
+
+func wsGetOrderbooks(client *websocketClient, data interface{}) error {
+	wsResp := WebsocketEventResponse{
+		Event: "GetOrderbooks",
+	}
+	wsResp.Data = getAllActiveOrderbooks(client.exchangeManager)
+	return client.SendWebsocketMessage(wsResp)
+}
+
+func wsGetOrderbook(client *websocketClient, data interface{}) error {
+	wsResp := WebsocketEventResponse{
+		Event: "GetOrderbook",
+	}
+	var orderbookReq WebsocketOrderbookTickerRequest
+	err := json.Unmarshal(data.([]byte), &orderbookReq)
+	if err != nil {
+		wsResp.Error = err.Error()
+		sendErr := client.SendWebsocketMessage(wsResp)
+		if sendErr != nil {
+			log.Error(log.APIServerMgr, sendErr)
+		}
+		return err
+	}
+
+	p, err := currency.NewPairFromString(orderbookReq.Currency)
+	if err != nil {
+		return err
+	}
+
+	a, err := asset.New(orderbookReq.AssetType)
+	if err != nil {
+		return err
+	}
+
+	exch := client.exchangeManager.GetExchangeByName(orderbookReq.Exchange)
+	if exch == nil {
+		wsResp.Error = exchange.ErrNoExchangeFound.Error()
+		sendErr := client.SendWebsocketMessage(wsResp)
+		if sendErr != nil {
+			log.Error(log.APIServerMgr, sendErr)
+		}
+		return err
+	}
+	ob, err := exch.FetchOrderbook(p, a)
+	if err != nil {
+		wsResp.Error = err.Error()
+		sendErr := client.SendWebsocketMessage(wsResp)
+		if sendErr != nil {
+			log.Error(log.APIServerMgr, sendErr)
+		}
+		return err
+	}
+	wsResp.Data = ob
+	return nil
+}
+
+func wsGetExchangeRates(client *websocketClient, data interface{}) error {
+	wsResp := WebsocketEventResponse{
+		Event: "GetExchangeRates",
+	}
+
+	var err error
+	wsResp.Data, err = currency.GetExchangeRates()
+	if err != nil {
+		return err
+	}
+
+	return client.SendWebsocketMessage(wsResp)
+}
+
+func wsGetPortfolio(client *websocketClient, data interface{}) error {
+	wsResp := WebsocketEventResponse{
+		Event: "GetPortfolio",
+	}
+
+	wsResp.Data = client.portfolioManager.GetPortfolioSummary()
+	return client.SendWebsocketMessage(wsResp)
+}

engine/apiserver.md

@@ -0,0 +1,62 @@
+# GoCryptoTrader package Apiserver
+
+<img src="/common/gctlogo.png?raw=true" width="350px" height="350px" hspace="70">
+
+
+[![Build Status](https://github.com/thrasher-corp/gocryptotrader/actions/workflows/tests.yml/badge.svg?branch=master)](https://github.com/thrasher-corp/gocryptotrader/actions/workflows/tests.yml)
+[![Software License](https://img.shields.io/badge/License-MIT-orange.svg?style=flat-square)](https://github.com/thrasher-corp/gocryptotrader/blob/master/LICENSE)
+[![GoDoc](https://godoc.org/github.com/thrasher-corp/gocryptotrader?status.svg)](https://godoc.org/github.com/thrasher-corp/gocryptotrader/engine/apiserver)
+[![Coverage Status](http://codecov.io/github/thrasher-corp/gocryptotrader/coverage.svg?branch=master)](http://codecov.io/github/thrasher-corp/gocryptotrader?branch=master)
+[![Go Report Card](https://goreportcard.com/badge/github.com/thrasher-corp/gocryptotrader)](https://goreportcard.com/report/github.com/thrasher-corp/gocryptotrader)
+
+
+This apiserver package is part of the GoCryptoTrader codebase.
+
+## This is still in active development
+
+You can track ideas, planned features and what's in progress on this Trello board: [https://trello.com/b/ZAhMhpOy/gocryptotrader](https://trello.com/b/ZAhMhpOy/gocryptotrader).
+
+Join our slack to discuss all things related to GoCryptoTrader! [GoCryptoTrader Slack](https://join.slack.com/t/gocryptotrader/shared_invite/enQtNTQ5NDAxMjA2Mjc5LTc5ZDE1ZTNiOGM3ZGMyMmY1NTAxYWZhODE0MWM5N2JlZDk1NDU0YTViYzk4NTk3OTRiMDQzNGQ1YTc4YmRlMTk)
+
+## Current Features for Apiserver
++ The API server subsystem is a deprecated service used to host a REST or websocket server to interact with some functions of GoCryptoTrader
++ This subsystem is no longer maintained and it is highly encouraged to interact with GRPC endpoints directly where possible
++ In order to modify the behaviour of the API server subsystem, you can edit the following inside your config file:
+
+### deprecatedRPC
+
+| Config | Description | Example |
+| ------ | ----------- | ------- |
+| enabled | If enabled will create a REST server which will listen to commands on the listen address | `true` |
+| listenAddress | If enabled will listen for REST requests on this address and return a JSON response | `localhost:9050` |
+
+### websocketRPC
+
+| Config | Description | Example |
+| ------ | ----------- | ------- |
+| enabled | If enabled will create a REST server which will listen to commands on the listen address | `true` |
+| listenAddress | If enabled will listen for requests on this address and return a JSON response | `localhost:9051` |
+| connectionLimit | Defines how many connections the websocket RPC server can handle simultanesoly | `1` |
+| maxAuthFailures | For authenticated endpoints, the amount of failed attempts allowed before disconnection | `3` |
+| allowInsecureOrigin | Allows use of insecure connections | `true` |
+
+### Please click GoDocs chevron above to view current GoDoc information for this package
+
+## Contribution
+
+Please feel free to submit any pull requests or suggest any desired features to be added.
+
+When submitting a PR, please abide by our coding guidelines:
+
++ Code must adhere to the official Go [formatting](https://golang.org/doc/effective_go.html#formatting) guidelines (i.e. uses [gofmt](https://golang.org/cmd/gofmt/)).
++ Code must be documented adhering to the official Go [commentary](https://golang.org/doc/effective_go.html#commentary) guidelines.
++ Code must adhere to our [coding style](https://github.com/thrasher-corp/gocryptotrader/blob/master/doc/coding_style.md).
++ Pull requests need to be based on and opened against the `master` branch.
+
+## Donations
+
+<img src="https://github.com/thrasher-corp/gocryptotrader/blob/master/web/src/assets/donate.png?raw=true" hspace="70">
+
+If this framework helped you in any way, or you would like to support the developers working on it, please donate Bitcoin to:
+
+***bc1qk0jareu4jytc0cfrhr5wgshsq8282awpavfahc***

engine/apiserver_test.go

@@ -0,0 +1,288 @@
+package engine
+
+import (
+	"encoding/json"
+	"errors"
+	"io/ioutil"
+	"net/http"
+	"net/http/httptest"
+	"os"
+	"reflect"
+	"sync"
+	"testing"
+	"time"
+
+	"github.com/thrasher-corp/gocryptotrader/config"
+)
+
+func TestSetupAPIServerManager(t *testing.T) {
+	t.Parallel()
+	_, err := setupAPIServerManager(nil, nil, nil, nil, nil, "")
+	if !errors.Is(err, errNilRemoteConfig) {
+		t.Errorf("error '%v', expected '%v'", err, errNilRemoteConfig)
+	}
+
+	_, err = setupAPIServerManager(&config.RemoteControlConfig{}, nil, nil, nil, nil, "")
+	if !errors.Is(err, errNilPProfConfig) {
+		t.Errorf("error '%v', expected '%v'", err, errNilPProfConfig)
+	}
+
+	_, err = setupAPIServerManager(&config.RemoteControlConfig{}, &config.Profiler{}, nil, nil, nil, "")
+	if !errors.Is(err, errNilExchangeManager) {
+		t.Errorf("error '%v', expected '%v'", err, errNilExchangeManager)
+	}
+
+	_, err = setupAPIServerManager(&config.RemoteControlConfig{}, &config.Profiler{}, &ExchangeManager{}, nil, nil, "")
+	if !errors.Is(err, errNilBot) {
+		t.Errorf("error '%v', expected '%v'", err, errNilBot)
+	}
+
+	_, err = setupAPIServerManager(&config.RemoteControlConfig{}, &config.Profiler{}, &ExchangeManager{}, &fakeBot{}, nil, "")
+	if !errors.Is(err, errEmptyConfigPath) {
+		t.Errorf("error '%v', expected '%v'", err, errEmptyConfigPath)
+	}
+
+	wd, _ := os.Getwd()
+	_, err = setupAPIServerManager(&config.RemoteControlConfig{}, &config.Profiler{}, &ExchangeManager{}, &fakeBot{}, nil, wd)
+	if !errors.Is(err, nil) {
+		t.Errorf("error '%v', expected '%v'", err, nil)
+	}
+}
+
+func TestStartRESTServer(t *testing.T) {
+	t.Parallel()
+	wd, _ := os.Getwd()
+	m, err := setupAPIServerManager(&config.RemoteControlConfig{}, &config.Profiler{}, &ExchangeManager{}, &fakeBot{}, nil, wd)
+	if !errors.Is(err, nil) {
+		t.Errorf("error '%v', expected '%v'", err, nil)
+	}
+	err = m.StartRESTServer()
+	if !errors.Is(err, errServerDisabled) {
+		t.Errorf("error '%v', expected '%v'", err, errServerDisabled)
+	}
+	m.remoteConfig.DeprecatedRPC.Enabled = true
+	var wg sync.WaitGroup
+	wg.Add(1)
+	// this is difficult to test as a webserver actually starts, so quit if an immediate error is not received
+	err = m.StartRESTServer()
+	if err != nil {
+		t.Fatal(err)
+	}
+	time.Sleep(time.Second)
+	wg.Done()
+}
+
+func TestStartWebsocketServer(t *testing.T) {
+	t.Parallel()
+	wd, _ := os.Getwd()
+	m, err := setupAPIServerManager(&config.RemoteControlConfig{}, &config.Profiler{}, &ExchangeManager{}, &fakeBot{}, nil, wd)
+	if !errors.Is(err, nil) {
+		t.Errorf("error '%v', expected '%v'", err, nil)
+	}
+	err = m.StartWebsocketServer()
+	if !errors.Is(err, errServerDisabled) {
+		t.Errorf("error '%v', expected '%v'", err, errServerDisabled)
+	}
+	m.remoteConfig.WebsocketRPC.Enabled = true
+	err = m.StartWebsocketServer()
+	if !errors.Is(err, nil) {
+		t.Errorf("error '%v', expected '%v'", err, nil)
+	}
+}
+
+func TestStopRESTServer(t *testing.T) {
+	t.Parallel()
+	wd, _ := os.Getwd()
+	m, err := setupAPIServerManager(&config.RemoteControlConfig{
+		DeprecatedRPC: config.DepcrecatedRPCConfig{
+			Enabled:       true,
+			ListenAddress: "localhost:9051",
+		},
+	}, &config.Profiler{}, &ExchangeManager{}, &fakeBot{}, nil, wd)
+	if !errors.Is(err, nil) {
+		t.Errorf("error '%v', expected '%v'", err, nil)
+	}
+
+	err = m.StopRESTServer()
+	if !errors.Is(err, ErrSubSystemNotStarted) {
+		t.Errorf("error '%v', expected '%v'", err, ErrSubSystemNotStarted)
+	}
+
+	err = m.StartRESTServer()
+	if !errors.Is(err, nil) {
+		t.Errorf("error '%v', expected '%v'", err, nil)
+	}
+	err = m.StopRESTServer()
+	if !errors.Is(err, nil) {
+		t.Errorf("error '%v', expected '%v'", err, nil)
+	}
+	// do it again to ensure things have reset appropriately and no errors occur starting
+	err = m.StartRESTServer()
+	if !errors.Is(err, nil) {
+		t.Errorf("error '%v', expected '%v'", err, nil)
+	}
+	err = m.StopRESTServer()
+	if !errors.Is(err, nil) {
+		t.Errorf("error '%v', expected '%v'", err, nil)
+	}
+}
+
+func TestWebsocketStop(t *testing.T) {
+	t.Parallel()
+	wd, _ := os.Getwd()
+	m, err := setupAPIServerManager(&config.RemoteControlConfig{
+		WebsocketRPC: config.WebsocketRPCConfig{
+			Enabled:       true,
+			ListenAddress: "localhost:9052",
+		},
+	}, &config.Profiler{}, &ExchangeManager{}, &fakeBot{}, nil, wd)
+	if !errors.Is(err, nil) {
+		t.Errorf("error '%v', expected '%v'", err, nil)
+	}
+
+	err = m.StopWebsocketServer()
+	if !errors.Is(err, ErrSubSystemNotStarted) {
+		t.Errorf("error '%v', expected '%v'", err, ErrSubSystemNotStarted)
+	}
+
+	err = m.StartWebsocketServer()
+	if !errors.Is(err, nil) {
+		t.Errorf("error '%v', expected '%v'", err, nil)
+	}
+	err = m.StopWebsocketServer()
+	if !errors.Is(err, nil) {
+		t.Errorf("error '%v', expected '%v'", err, nil)
+	}
+	// do it again to ensure things have reset appropriately and no errors occur starting
+	err = m.StartWebsocketServer()
+	if !errors.Is(err, nil) {
+		t.Errorf("error '%v', expected '%v'", err, nil)
+	}
+	err = m.StopWebsocketServer()
+	if !errors.Is(err, nil) {
+		t.Errorf("error '%v', expected '%v'", err, nil)
+	}
+}
+
+func TestIsRESTServerRunning(t *testing.T) {
+	t.Parallel()
+	m := &apiServerManager{}
+	if m.IsRESTServerRunning() {
+		t.Error("expected false")
+	}
+	m.restStarted = 1
+	if !m.IsRESTServerRunning() {
+		t.Error("expected true")
+	}
+	m = nil
+	if m.IsRESTServerRunning() {
+		t.Error("expected false")
+	}
+}
+
+func TestIsWebsocketServerRunning(t *testing.T) {
+	t.Parallel()
+	m := &apiServerManager{}
+	if m.IsWebsocketServerRunning() {
+		t.Error("expected false")
+	}
+	m.websocketStarted = 1
+	if !m.IsWebsocketServerRunning() {
+		t.Error("expected true")
+	}
+	m = nil
+	if m.IsWebsocketServerRunning() {
+		t.Error("expected false")
+	}
+}
+
+func TestGetAllActiveOrderbooks(t *testing.T) {
+	man := SetupExchangeManager()
+	bs, err := man.NewExchangeByName("Bitstamp")
+	if err != nil {
+		t.Fatal(err)
+	}
+	bs.SetDefaults()
+	man.Add(bs)
+	resp := getAllActiveOrderbooks(man)
+	if resp == nil {
+		t.Error("expected not nil")
+	}
+}
+
+func TestGetAllActiveTickers(t *testing.T) {
+	t.Parallel()
+	man := SetupExchangeManager()
+	bs, err := man.NewExchangeByName("Bitstamp")
+	if err != nil {
+		t.Fatal(err)
+	}
+	bs.SetDefaults()
+	man.Add(bs)
+	resp := getAllActiveTickers(man)
+	if resp == nil {
+		t.Error("expected not nil")
+	}
+}
+
+func TestGetAllActiveAccounts(t *testing.T) {
+	t.Parallel()
+	man := SetupExchangeManager()
+	bs, err := man.NewExchangeByName("Bitstamp")
+	if err != nil {
+		t.Fatal(err)
+	}
+	bs.SetDefaults()
+	man.Add(bs)
+	resp := getAllActiveAccounts(man)
+	if resp == nil {
+		t.Error("expected not nil")
+	}
+}
+
+func makeHTTPGetRequest(t *testing.T, response interface{}) *http.Response {
+	w := httptest.NewRecorder()
+
+	err := writeResponse(w, response)
+	if err != nil {
+		t.Error("Failed to make response.", err)
+	}
+	return w.Result()
+}
+
+// TestConfigAllJsonResponse test if config/all restful json response is valid
+func TestConfigAllJsonResponse(t *testing.T) {
+	t.Parallel()
+	var c config.Config
+	err := c.LoadConfig(config.TestFile, true)
+	if err != nil {
+		t.Error(err)
+	}
+	resp := makeHTTPGetRequest(t, c)
+	body, err := ioutil.ReadAll(resp.Body)
+	if err != nil {
+		t.Error("Body not readable", err)
+	}
+	err = resp.Body.Close()
+	if err != nil {
+		t.Error("Body not closable", err)
+	}
+
+	var responseConfig config.Config
+	jsonErr := json.Unmarshal(body, &responseConfig)
+	if jsonErr != nil {
+		t.Error("Response not parse-able as json", err)
+	}
+
+	if !reflect.DeepEqual(responseConfig, c) {
+		t.Error("Json not equal to config")
+	}
+}
+
+// fakeBot is a basic implementation of the iBot interface used for testing
+type fakeBot struct{}
+
+// SetupExchanges is a basic implementation of the iBot interface used for testing
+func (f *fakeBot) SetupExchanges() error {
+	return nil
+}

engine/apiserver_types.go

@@ -0,0 +1,169 @@
+package engine
+
+import (
+	"errors"
+	"net/http"
+	"sync"
+
+	"github.com/gorilla/mux"
+	"github.com/gorilla/websocket"
+	"github.com/thrasher-corp/gocryptotrader/config"
+	"github.com/thrasher-corp/gocryptotrader/exchanges/account"
+	"github.com/thrasher-corp/gocryptotrader/exchanges/orderbook"
+	"github.com/thrasher-corp/gocryptotrader/exchanges/ticker"
+)
+
+// Const vars for websocket
+const (
+	WebsocketResponseSuccess = "OK"
+	restIndexResponse        = "<html>GoCryptoTrader RESTful interface. For the web GUI, please visit the <a href=https://github.com/thrasher-corp/gocryptotrader/blob/master/web/README.md>web GUI readme.</a></html>"
+	DeprecatedName           = "deprecated_rpc"
+	WebsocketName            = "websocket_rpc"
+)
+
+var (
+	wsHub                   *websocketHub
+	wsHubStarted            bool
+	errNilRemoteConfig      = errors.New("received nil remote config")
+	errNilPProfConfig       = errors.New("received nil pprof config")
+	errNilBot               = errors.New("received nil engine bot")
+	errEmptyConfigPath      = errors.New("received empty config path")
+	errServerDisabled       = errors.New("server disabled")
+	errInvalidListenAddress = errors.New("invalid listen address")
+	errAlreadyRunning       = errors.New("already running")
+	// ErrWebsocketServiceNotRunning occurs when a message is sent to be broadcast via websocket
+	// and its not running
+	ErrWebsocketServiceNotRunning = errors.New("websocket service not started")
+)
+
+// apiServerManager holds all relevant fields to manage both REST and websocket
+// api servers
+type apiServerManager struct {
+	restStarted            int32
+	websocketStarted       int32
+	restListenAddress      string
+	websocketListenAddress string
+	gctConfigPath          string
+	restHTTPServer         *http.Server
+	websocketHTTPServer    *http.Server
+	wgRest                 sync.WaitGroup
+	wgWebsocket            sync.WaitGroup
+
+	restRouter      *mux.Router
+	websocketRouter *mux.Router
+	websocketHub    *websocketHub
+
+	remoteConfig     *config.RemoteControlConfig
+	pprofConfig      *config.Profiler
+	exchangeManager  iExchangeManager
+	bot              iBot
+	portfolioManager iPortfolioManager
+}
+
+// websocketClient stores information related to the websocket client
+type websocketClient struct {
+	Hub              *websocketHub
+	Conn             *websocket.Conn
+	Authenticated    bool
+	authFailures     int
+	Send             chan []byte
+	username         string
+	password         string
+	maxAuthFailures  int
+	exchangeManager  iExchangeManager
+	bot              iBot
+	portfolioManager iPortfolioManager
+	configPath       string
+}
+
+// websocketHub stores the data for managing websocket clients
+type websocketHub struct {
+	Clients    map[*websocketClient]bool
+	Broadcast  chan []byte
+	Register   chan *websocketClient
+	Unregister chan *websocketClient
+}
+
+// WebsocketEvent is the struct used for websocket events
+type WebsocketEvent struct {
+	Exchange  string `json:"exchange,omitempty"`
+	AssetType string `json:"assetType,omitempty"`
+	Event     string
+	Data      interface{}
+}
+
+// WebsocketEventResponse is the struct used for websocket event responses
+type WebsocketEventResponse struct {
+	Event string      `json:"event"`
+	Data  interface{} `json:"data"`
+	Error string      `json:"error"`
+}
+
+// WebsocketOrderbookTickerRequest is a struct used for ticker and orderbook
+// requests
+type WebsocketOrderbookTickerRequest struct {
+	Exchange  string `json:"exchangeName"`
+	Currency  string `json:"currency"`
+	AssetType string `json:"assetType"`
+}
+
+// WebsocketAuth is a struct used for
+type WebsocketAuth struct {
+	Username string `json:"username"`
+	Password string `json:"password"`
+}
+
+// Route is a sub type that holds the request routes
+type Route struct {
+	Name        string
+	Method      string
+	Pattern     string
+	HandlerFunc http.HandlerFunc
+}
+
+// AllEnabledExchangeOrderbooks holds the enabled exchange orderbooks
+type AllEnabledExchangeOrderbooks struct {
+	Data []EnabledExchangeOrderbooks `json:"data"`
+}
+
+// EnabledExchangeOrderbooks is a sub type for singular exchanges and respective
+// orderbooks
+type EnabledExchangeOrderbooks struct {
+	ExchangeName   string           `json:"exchangeName"`
+	ExchangeValues []orderbook.Base `json:"exchangeValues"`
+}
+
+// AllEnabledExchangeCurrencies holds the enabled exchange currencies
+type AllEnabledExchangeCurrencies struct {
+	Data []EnabledExchangeCurrencies `json:"data"`
+}
+
+// EnabledExchangeCurrencies is a sub type for singular exchanges and respective
+// currencies
+type EnabledExchangeCurrencies struct {
+	ExchangeName   string         `json:"exchangeName"`
+	ExchangeValues []ticker.Price `json:"exchangeValues"`
+}
+
+// AllEnabledExchangeAccounts holds all enabled accounts info
+type AllEnabledExchangeAccounts struct {
+	Data []account.Holdings `json:"data"`
+}
+
+var wsHandlers = map[string]wsCommandHandler{
+	"auth":             {authRequired: false, handler: wsAuth},
+	"getconfig":        {authRequired: true, handler: wsGetConfig},
+	"saveconfig":       {authRequired: true, handler: wsSaveConfig},
+	"getaccountinfo":   {authRequired: true, handler: wsGetAccountInfo},
+	"gettickers":       {authRequired: false, handler: wsGetTickers},
+	"getticker":        {authRequired: false, handler: wsGetTicker},
+	"getorderbooks":    {authRequired: false, handler: wsGetOrderbooks},
+	"getorderbook":     {authRequired: false, handler: wsGetOrderbook},
+	"getexchangerates": {authRequired: false, handler: wsGetExchangeRates},
+	"getportfolio":     {authRequired: true, handler: wsGetPortfolio},
+}
+
+type wsCommandHandler struct {
+	authRequired bool
+	handler      func(client *websocketClient, data interface{}) error
+}

engine/comms_relayer.go

@@ -1,95 +0,0 @@
-package engine
-
-import (
-	"errors"
-	"fmt"
-	"sync/atomic"
-
-	"github.com/thrasher-corp/gocryptotrader/communications"
-	"github.com/thrasher-corp/gocryptotrader/communications/base"
-	"github.com/thrasher-corp/gocryptotrader/engine/subsystem"
-	"github.com/thrasher-corp/gocryptotrader/log"
-)
-
-// commsManager starts the NTP manager
-type commsManager struct {
-	started  int32
-	shutdown chan struct{}
-	relayMsg chan base.Event
-	comms    *communications.Communications
-}
-
-func (c *commsManager) Started() bool {
-	return atomic.LoadInt32(&c.started) == 1
-}
-
-func (c *commsManager) Start() (err error) {
-	if !atomic.CompareAndSwapInt32(&c.started, 0, 1) {
-		return fmt.Errorf("communications manager %w", subsystem.ErrSubSystemAlreadyStarted)
-	}
-
-	defer func() {
-		if err != nil {
-			atomic.CompareAndSwapInt32(&c.started, 1, 0)
-		}
-	}()
-
-	log.Debugln(log.CommunicationMgr, "Communications manager starting...")
-	commsCfg := Bot.Config.GetCommunicationsConfig()
-	c.comms, err = communications.NewComm(&commsCfg)
-	if err != nil {
-		return err
-	}
-
-	c.shutdown = make(chan struct{})
-	c.relayMsg = make(chan base.Event)
-	go c.run()
-	log.Debugln(log.CommunicationMgr, "Communications manager started.")
-	return nil
-}
-
-func (c *commsManager) GetStatus() (map[string]base.CommsStatus, error) {
-	if !c.Started() {
-		return nil, errors.New("communications manager not started")
-	}
-	return c.comms.GetStatus(), nil
-}
-
-func (c *commsManager) Stop() error {
-	if atomic.LoadInt32(&c.started) == 0 {
-		return fmt.Errorf("communications manager %w", subsystem.ErrSubSystemNotStarted)
-	}
-	defer func() {
-		atomic.CompareAndSwapInt32(&c.started, 1, 0)
-	}()
-	close(c.shutdown)
-	log.Debugln(log.CommunicationMgr, "Communications manager shutting down...")
-	return nil
-}
-
-func (c *commsManager) PushEvent(evt base.Event) {
-	if !c.Started() {
-		return
-	}
-	select {
-	case c.relayMsg <- evt:
-	default:
-		log.Errorf(log.CommunicationMgr, "Failed to send, no receiver when pushing event [%v]", evt)
-	}
-}
-
-func (c *commsManager) run() {
-	defer func() {
-		// TO-DO shutdown comms connections for connected services (Slack etc)
-		log.Debugln(log.CommunicationMgr, "Communications manager shutdown.")
-	}()
-
-	for {
-		select {
-		case msg := <-c.relayMsg:
-			c.comms.PushEvent(msg)
-		case <-c.shutdown:
-			return
-		}
-	}
-}

engine/communication_manager.go

@@ -0,0 +1,117 @@
+package engine
+
+import (
+	"errors"
+	"fmt"
+	"sync/atomic"
+
+	"github.com/thrasher-corp/gocryptotrader/communications"
+	"github.com/thrasher-corp/gocryptotrader/communications/base"
+	"github.com/thrasher-corp/gocryptotrader/log"
+)
+
+// CommunicationsManagerName is an exported subsystem name
+const CommunicationsManagerName = "communications"
+
+// CommunicationManager ensures operations of communications
+type CommunicationManager struct {
+	started  int32
+	shutdown chan struct{}
+	relayMsg chan base.Event
+	comms    *communications.Communications
+}
+
+var errNilConfig = errors.New("received nil communications config")
+
+// SetupCommunicationManager creates a communications manager
+func SetupCommunicationManager(cfg *base.CommunicationsConfig) (*CommunicationManager, error) {
+	if cfg == nil {
+		return nil, errNilConfig
+	}
+	manager := &CommunicationManager{
+		shutdown: make(chan struct{}),
+		relayMsg: make(chan base.Event),
+	}
+	var err error
+	manager.comms, err = communications.NewComm(cfg)
+	if err != nil {
+		return nil, err
+	}
+	return manager, nil
+}
+
+// IsRunning safely checks whether the subsystem is running
+func (m *CommunicationManager) IsRunning() bool {
+	if m == nil {
+		return false
+	}
+	return atomic.LoadInt32(&m.started) == 1
+}
+
+// Start runs the subsystem
+func (m *CommunicationManager) Start() error {
+	if m == nil {
+		return fmt.Errorf("communications manager server %w", ErrNilSubsystem)
+	}
+	if !atomic.CompareAndSwapInt32(&m.started, 0, 1) {
+		return fmt.Errorf("communications manager %w", ErrSubSystemAlreadyStarted)
+	}
+	log.Debugf(log.CommunicationMgr, "Communications manager %s", MsgSubSystemStarting)
+	m.shutdown = make(chan struct{})
+	go m.run()
+	return nil
+}
+
+// GetStatus returns the status of communications
+func (m *CommunicationManager) GetStatus() (map[string]base.CommsStatus, error) {
+	if !m.IsRunning() {
+		return nil, fmt.Errorf("communications manager %w", ErrSubSystemNotStarted)
+	}
+	return m.comms.GetStatus(), nil
+}
+
+// Stop attempts to shutdown the subsystem
+func (m *CommunicationManager) Stop() error {
+	if m == nil {
+		return fmt.Errorf("communications manager server %w", ErrNilSubsystem)
+	}
+	if atomic.LoadInt32(&m.started) == 0 {
+		return fmt.Errorf("communications manager %w", ErrSubSystemNotStarted)
+	}
+	defer func() {
+		atomic.CompareAndSwapInt32(&m.started, 1, 0)
+	}()
+	close(m.shutdown)
+	log.Debugf(log.CommunicationMgr, "Communications manager %s", MsgSubSystemShuttingDown)
+	return nil
+}
+
+// PushEvent pushes an event to the communications relay
+func (m *CommunicationManager) PushEvent(evt base.Event) {
+	if !m.IsRunning() {
+		return
+	}
+	select {
+	case m.relayMsg <- evt:
+	default:
+		log.Errorf(log.CommunicationMgr, "Failed to send, no receiver when pushing event [%v]", evt)
+	}
+}
+
+// run takes awaiting messages and pushes them to be handled by communications
+func (m *CommunicationManager) run() {
+	log.Debugf(log.Global, "Communications manager %s", MsgSubSystemStarted)
+	defer func() {
+		// TO-DO shutdown comms connections for connected services (Slack etc)
+		log.Debugf(log.CommunicationMgr, "Communications manager %s", MsgSubSystemShutdown)
+	}()
+
+	for {
+		select {
+		case msg := <-m.relayMsg:
+			m.comms.PushEvent(msg)
+		case <-m.shutdown:
+			return
+		}
+	}
+}

engine/communication_manager.md

@@ -0,0 +1,90 @@
+# GoCryptoTrader package Communication_manager
+
+<img src="/common/gctlogo.png?raw=true" width="350px" height="350px" hspace="70">
+
+
+[![Build Status](https://github.com/thrasher-corp/gocryptotrader/actions/workflows/tests.yml/badge.svg?branch=master)](https://github.com/thrasher-corp/gocryptotrader/actions/workflows/tests.yml)
+[![Software License](https://img.shields.io/badge/License-MIT-orange.svg?style=flat-square)](https://github.com/thrasher-corp/gocryptotrader/blob/master/LICENSE)
+[![GoDoc](https://godoc.org/github.com/thrasher-corp/gocryptotrader?status.svg)](https://godoc.org/github.com/thrasher-corp/gocryptotrader/engine/communication_manager)
+[![Coverage Status](http://codecov.io/github/thrasher-corp/gocryptotrader/coverage.svg?branch=master)](http://codecov.io/github/thrasher-corp/gocryptotrader?branch=master)
+[![Go Report Card](https://goreportcard.com/badge/github.com/thrasher-corp/gocryptotrader)](https://goreportcard.com/report/github.com/thrasher-corp/gocryptotrader)
+
+
+This communication_manager package is part of the GoCryptoTrader codebase.
+
+## This is still in active development
+
+You can track ideas, planned features and what's in progress on this Trello board: [https://trello.com/b/ZAhMhpOy/gocryptotrader](https://trello.com/b/ZAhMhpOy/gocryptotrader).
+
+Join our slack to discuss all things related to GoCryptoTrader! [GoCryptoTrader Slack](https://join.slack.com/t/gocryptotrader/shared_invite/enQtNTQ5NDAxMjA2Mjc5LTc5ZDE1ZTNiOGM3ZGMyMmY1NTAxYWZhODE0MWM5N2JlZDk1NDU0YTViYzk4NTk3OTRiMDQzNGQ1YTc4YmRlMTk)
+
+## Current Features for Communication_manager
++ The communication manager subsystem is used to push events raised in GoCryptoTrader to any enabled communication system such as a Slack server
++ In order to modify the behaviour of the communication manager subsystem, you can edit the following inside your config file under `communications`:
+
+### slack
+
+| Config | Description | Example |
+| ------ | ----------- | ------- |
+| enabled | Determines whether the push communications to a Slack server | `true` |
+| verbose | If enabled will log more details to your logger output | `false` |
+| targetChannel | The channel to send communications to | `announcements` |
+| verificationToken | The token generated by Slack to allow interactions with the server and channel | `iamafaketoken` |
+
+### smsGlobal
+
+| Config | Description | Example |
+| ------ | ----------- | ------- |
+| name | The name of the SMS sender | `SMSGlobal` |
+| from | Who the text name is from | `Skynet` |
+| enabled | Determines whether the push communications to the SMS service | `true` |
+| verbose | If enabled will log more details to your logger output | `false` |
+| username | The username to use with the SMS provider | `username` |
+| password | The username to use with the SMS provider | `password` |
+| contacts | The `name` `number` of the user people you wish to send SMS to and whether it is `enabled` | `"name": "StyleGherkin", "number": "1231424", "enabled": true` |
+
+### smtp
+
+| Config | Description | Example |
+| ------ | ----------- | ------- |
+| name | The name of the service | `SMTP` |
+| enabled | Determines whether the push communications to a email server | `true` |
+| verbose | If enabled will log more details to your logger output | `false` |
+| host | The SMTP host | `smtp.google.com` |
+| port | The port to use | `537` |
+| accountName | Your username | `username` |
+| accountPassword | Your password | `password` |
+| from | The display name of the sender | `Jeff Bezos` |
+| recipientList | A comma delimited list of addresses to send alerts to | `bill@gates.com` |
+
+### telegram
+
+| Config | Description | Example |
+| ------ | ----------- | ------- |
+| name | The name to be displayed | `Telegram` |
+| enabled | Determines whether the push communications to a Telegram server | `true` |
+| verbose | If enabled will log more details to your logger output | `false` |
+| verificationToken | The token generated by Telegram to allow you to send messages | `iamafaketoken` |
+
+
+
+### Please click GoDocs chevron above to view current GoDoc information for this package
+
+## Contribution
+
+Please feel free to submit any pull requests or suggest any desired features to be added.
+
+When submitting a PR, please abide by our coding guidelines:
+
++ Code must adhere to the official Go [formatting](https://golang.org/doc/effective_go.html#formatting) guidelines (i.e. uses [gofmt](https://golang.org/cmd/gofmt/)).
++ Code must be documented adhering to the official Go [commentary](https://golang.org/doc/effective_go.html#commentary) guidelines.
++ Code must adhere to our [coding style](https://github.com/thrasher-corp/gocryptotrader/blob/master/doc/coding_style.md).
++ Pull requests need to be based on and opened against the `master` branch.
+
+## Donations
+
+<img src="https://github.com/thrasher-corp/gocryptotrader/blob/master/web/src/assets/donate.png?raw=true" hspace="70">
+
+If this framework helped you in any way, or you would like to support the developers working on it, please donate Bitcoin to:
+
+***bc1qk0jareu4jytc0cfrhr5wgshsq8282awpavfahc***

engine/communication_manager_test.go

@@ -0,0 +1,158 @@
+package engine
+
+import (
+	"errors"
+	"testing"
+	"time"
+
+	"github.com/thrasher-corp/gocryptotrader/communications"
+	"github.com/thrasher-corp/gocryptotrader/communications/base"
+)
+
+func TestSetup(t *testing.T) {
+	t.Parallel()
+	_, err := SetupCommunicationManager(nil)
+	if !errors.Is(err, errNilConfig) {
+		t.Errorf("error '%v', expected '%v'", err, errNilConfig)
+	}
+
+	_, err = SetupCommunicationManager(&base.CommunicationsConfig{})
+	if !errors.Is(err, communications.ErrNoCommunicationRelayersEnabled) {
+		t.Errorf("error '%v', expected '%v'", err, communications.ErrNoCommunicationRelayersEnabled)
+	}
+
+	m, err := SetupCommunicationManager(&base.CommunicationsConfig{
+		SlackConfig: base.SlackConfig{
+			Enabled: true,
+		},
+	})
+	if !errors.Is(err, nil) {
+		t.Errorf("error '%v', expected '%v'", err, nil)
+	}
+	if m == nil {
+		t.Error("expected manager")
+	}
+}
+
+func TestIsRunning(t *testing.T) {
+	t.Parallel()
+	m, err := SetupCommunicationManager(&base.CommunicationsConfig{
+		SMSGlobalConfig: base.SMSGlobalConfig{
+			Enabled: true,
+		},
+	})
+	if !errors.Is(err, nil) {
+		t.Errorf("error '%v', expected '%v'", err, nil)
+	}
+	err = m.Start()
+	if !errors.Is(err, nil) {
+		t.Errorf("error '%v', expected '%v'", err, nil)
+	}
+	if !m.IsRunning() {
+		t.Error("expected true")
+	}
+	m.started = 0
+	if m.IsRunning() {
+		t.Error("expected false")
+	}
+	m = nil
+	if m.IsRunning() {
+		t.Error("expected false")
+	}
+}
+
+func TestStart(t *testing.T) {
+	t.Parallel()
+	m, err := SetupCommunicationManager(&base.CommunicationsConfig{
+		SMTPConfig: base.SMTPConfig{
+			Enabled: true,
+		},
+	})
+	if !errors.Is(err, nil) {
+		t.Errorf("error '%v', expected '%v'", err, nil)
+	}
+	err = m.Start()
+	if !errors.Is(err, nil) {
+		t.Errorf("error '%v', expected '%v'", err, nil)
+	}
+	m.started = 1
+	err = m.Start()
+	if !errors.Is(err, ErrSubSystemAlreadyStarted) {
+		t.Errorf("error '%v', expected '%v'", err, ErrSubSystemAlreadyStarted)
+	}
+}
+
+func TestGetStatus(t *testing.T) {
+	t.Parallel()
+	m, err := SetupCommunicationManager(&base.CommunicationsConfig{
+		TelegramConfig: base.TelegramConfig{
+			Enabled: true,
+		},
+	})
+	if !errors.Is(err, nil) {
+		t.Errorf("error '%v', expected '%v'", err, nil)
+	}
+	err = m.Start()
+	if !errors.Is(err, nil) {
+		t.Errorf("error '%v', expected '%v'", err, nil)
+	}
+	_, err = m.GetStatus()
+	if !errors.Is(err, nil) {
+		t.Errorf("error '%v', expected '%v'", err, nil)
+	}
+	m.started = 0
+	_, err = m.GetStatus()
+	if !errors.Is(err, ErrSubSystemNotStarted) {
+		t.Errorf("error '%v', expected '%v'", err, ErrSubSystemNotStarted)
+	}
+}
+
+func TestStop(t *testing.T) {
+	t.Parallel()
+	m, err := SetupCommunicationManager(&base.CommunicationsConfig{
+		SlackConfig: base.SlackConfig{
+			Enabled: true,
+		},
+	})
+	if !errors.Is(err, nil) {
+		t.Errorf("error '%v', expected '%v'", err, nil)
+	}
+	err = m.Start()
+	if !errors.Is(err, nil) {
+		t.Errorf("error '%v', expected '%v'", err, nil)
+	}
+	err = m.Stop()
+	if !errors.Is(err, nil) {
+		t.Errorf("error '%v', expected '%v'", err, nil)
+	}
+	err = m.Stop()
+	if !errors.Is(err, ErrSubSystemNotStarted) {
+		t.Errorf("error '%v', expected '%v'", err, ErrSubSystemNotStarted)
+	}
+	m = nil
+	err = m.Stop()
+	if !errors.Is(err, ErrNilSubsystem) {
+		t.Errorf("error '%v', expected '%v'", err, ErrNilSubsystem)
+	}
+}
+
+func TestPushEvent(t *testing.T) {
+	t.Parallel()
+	m, err := SetupCommunicationManager(&base.CommunicationsConfig{
+		SlackConfig: base.SlackConfig{
+			Enabled: true,
+		},
+	})
+	if !errors.Is(err, nil) {
+		t.Errorf("error '%v', expected '%v'", err, nil)
+	}
+	err = m.Start()
+	if !errors.Is(err, nil) {
+		t.Errorf("error '%v', expected '%v'", err, nil)
+	}
+	m.PushEvent(base.Event{})
+	time.Sleep(time.Second)
+	m.PushEvent(base.Event{})
+	m = nil
+	m.PushEvent(base.Event{})
+}

engine/connection.go

@@ -1,67 +0,0 @@
-package engine
-
-import (
-	"fmt"
-	"sync/atomic"
-
-	"github.com/thrasher-corp/gocryptotrader/config"
-	"github.com/thrasher-corp/gocryptotrader/connchecker"
-	"github.com/thrasher-corp/gocryptotrader/engine/subsystem"
-	"github.com/thrasher-corp/gocryptotrader/log"
-)
-
-// connectionManager manages the connchecker
-type connectionManager struct {
-	started int32
-	conn    *connchecker.Checker
-}
-
-// Started returns if the connection manager has started
-func (c *connectionManager) Started() bool {
-	return atomic.LoadInt32(&c.started) == 1
-}
-
-// Start starts an instance of the connection manager
-func (c *connectionManager) Start(conf *config.ConnectionMonitorConfig) error {
-	if !atomic.CompareAndSwapInt32(&c.started, 0, 1) {
-		return fmt.Errorf("connection manager %w", subsystem.ErrSubSystemAlreadyStarted)
-	}
-
-	log.Debugln(log.ConnectionMgr, "Connection manager starting...")
-	var err error
-	c.conn, err = connchecker.New(conf.DNSList,
-		conf.PublicDomainList,
-		conf.CheckInterval)
-	if err != nil {
-		atomic.CompareAndSwapInt32(&c.started, 1, 0)
-		return err
-	}
-
-	log.Debugln(log.ConnectionMgr, "Connection manager started.")
-	return nil
-}
-
-// Stop stops the connection manager
-func (c *connectionManager) Stop() error {
-	if atomic.LoadInt32(&c.started) == 0 {
-		return fmt.Errorf("connection manager %w", subsystem.ErrSubSystemNotStarted)
-	}
-	defer func() {
-		atomic.CompareAndSwapInt32(&c.started, 1, 0)
-	}()
-
-	log.Debugln(log.ConnectionMgr, "Connection manager shutting down...")
-	c.conn.Shutdown()
-	log.Debugln(log.ConnectionMgr, "Connection manager stopped.")
-	return nil
-}
-
-// IsOnline returns if the connection manager is online
-func (c *connectionManager) IsOnline() bool {
-	if c.conn == nil {
-		log.Warnln(log.ConnectionMgr, "Connection manager: IsOnline called but conn is nil")
-		return false
-	}
-
-	return c.conn.IsConnected()
-}

engine/connection_manager.go

@@ -0,0 +1,100 @@
+package engine
+
+import (
+	"fmt"
+	"sync/atomic"
+
+	"github.com/thrasher-corp/gocryptotrader/config"
+	"github.com/thrasher-corp/gocryptotrader/connchecker"
+	"github.com/thrasher-corp/gocryptotrader/log"
+)
+
+// ConnectionManagerName is an exported subsystem name
+const ConnectionManagerName = "internet_monitor"
+
+// connectionManager manages the connchecker
+type connectionManager struct {
+	started int32
+	conn    *connchecker.Checker
+	cfg     *config.ConnectionMonitorConfig
+}
+
+// IsRunning safely checks whether the subsystem is running
+func (m *connectionManager) IsRunning() bool {
+	if m == nil {
+		return false
+	}
+	return atomic.LoadInt32(&m.started) == 1
+}
+
+// setupConnectionManager creates a connection manager
+func setupConnectionManager(cfg *config.ConnectionMonitorConfig) (*connectionManager, error) {
+	if cfg == nil {
+		return nil, errNilConfig
+	}
+	if cfg.DNSList == nil {
+		cfg.DNSList = connchecker.DefaultDNSList
+	}
+	if cfg.PublicDomainList == nil {
+		cfg.PublicDomainList = connchecker.DefaultDomainList
+	}
+	if cfg.CheckInterval == 0 {
+		cfg.CheckInterval = connchecker.DefaultCheckInterval
+	}
+	return &connectionManager{
+		cfg: cfg,
+	}, nil
+}
+
+// Start runs the subsystem
+func (m *connectionManager) Start() error {
+	if m == nil {
+		return fmt.Errorf("connection manager %w", ErrNilSubsystem)
+	}
+	if !atomic.CompareAndSwapInt32(&m.started, 0, 1) {
+		return fmt.Errorf("connection manager %w", ErrSubSystemAlreadyStarted)
+	}
+
+	log.Debugln(log.ConnectionMgr, "Connection manager starting...")
+	var err error
+	m.conn, err = connchecker.New(m.cfg.DNSList,
+		m.cfg.PublicDomainList,
+		m.cfg.CheckInterval)
+	if err != nil {
+		atomic.CompareAndSwapInt32(&m.started, 1, 0)
+		return err
+	}
+
+	log.Debugln(log.ConnectionMgr, "Connection manager started.")
+	return nil
+}
+
+// Stop stops the connection manager
+func (m *connectionManager) Stop() error {
+	if m == nil {
+		return fmt.Errorf("connection manager %w", ErrNilSubsystem)
+	}
+	if atomic.LoadInt32(&m.started) == 0 {
+		return fmt.Errorf("connection manager %w", ErrSubSystemNotStarted)
+	}
+	defer func() {
+		atomic.CompareAndSwapInt32(&m.started, 1, 0)
+	}()
+	log.Debugln(log.ConnectionMgr, "Connection manager shutting down...")
+	m.conn.Shutdown()
+	log.Debugln(log.ConnectionMgr, "Connection manager stopped.")
+	return nil
+}
+
+// IsOnline returns if the connection manager is online
+func (m *connectionManager) IsOnline() bool {
+	if m == nil {
+		return false
+	}
+	if m.conn == nil {
+		log.Warnln(log.ConnectionMgr, "Connection manager: IsOnline called but conn is nil")
+		return false
+	}
+
+	return m.conn.IsConnected()
+}

engine/connection_manager.md

@@ -0,0 +1,53 @@
+# GoCryptoTrader package Connection_manager
+
+<img src="/common/gctlogo.png?raw=true" width="350px" height="350px" hspace="70">
+
+
+[![Build Status](https://github.com/thrasher-corp/gocryptotrader/actions/workflows/tests.yml/badge.svg?branch=master)](https://github.com/thrasher-corp/gocryptotrader/actions/workflows/tests.yml)
+[![Software License](https://img.shields.io/badge/License-MIT-orange.svg?style=flat-square)](https://github.com/thrasher-corp/gocryptotrader/blob/master/LICENSE)
+[![GoDoc](https://godoc.org/github.com/thrasher-corp/gocryptotrader?status.svg)](https://godoc.org/github.com/thrasher-corp/gocryptotrader/engine/connection_manager)
+[![Coverage Status](http://codecov.io/github/thrasher-corp/gocryptotrader/coverage.svg?branch=master)](http://codecov.io/github/thrasher-corp/gocryptotrader?branch=master)
+[![Go Report Card](https://goreportcard.com/badge/github.com/thrasher-corp/gocryptotrader)](https://goreportcard.com/report/github.com/thrasher-corp/gocryptotrader)
+
+
+This connection_manager package is part of the GoCryptoTrader codebase.
+
+## This is still in active development
+
+You can track ideas, planned features and what's in progress on this Trello board: [https://trello.com/b/ZAhMhpOy/gocryptotrader](https://trello.com/b/ZAhMhpOy/gocryptotrader).
+
+Join our slack to discuss all things related to GoCryptoTrader! [GoCryptoTrader Slack](https://join.slack.com/t/gocryptotrader/shared_invite/enQtNTQ5NDAxMjA2Mjc5LTc5ZDE1ZTNiOGM3ZGMyMmY1NTAxYWZhODE0MWM5N2JlZDk1NDU0YTViYzk4NTk3OTRiMDQzNGQ1YTc4YmRlMTk)
+
+## Current Features for Connection_manager
++ The connection manager subsystem is used to periodically check whether the application is connected to the internet and will provide alerts of any changes
++ In order to modify the behaviour of the connection manager subsystem, you can edit the following inside your config file under `connectionMonitor`:
+
+### connectionMonitor
+
+| Config | Description | Example |
+| ------ | ----------- | ------- |
+| perferredDNSList | Is a string array of DNS servers to periodically verify whether GoCryptoTrader is connected to the internet |  `["8.8.8.8","8.8.4.4","1.1.1.1","1.0.0.1"]` |
+| preferredDomainList |  Is a string array of domains to periodically verify whether GoCryptoTrader is connected to the internet |  `["www.google.com","www.cloudflare.com","www.facebook.com"]` |
+| checkInterval | A time period in golang `time.Duration` format to check whether GoCryptoTrader is connected to the internet | `1000000000` |
+
+
+### Please click GoDocs chevron above to view current GoDoc information for this package
+
+## Contribution
+
+Please feel free to submit any pull requests or suggest any desired features to be added.
+
+When submitting a PR, please abide by our coding guidelines:
+
++ Code must adhere to the official Go [formatting](https://golang.org/doc/effective_go.html#formatting) guidelines (i.e. uses [gofmt](https://golang.org/cmd/gofmt/)).
++ Code must be documented adhering to the official Go [commentary](https://golang.org/doc/effective_go.html#commentary) guidelines.
++ Code must adhere to our [coding style](https://github.com/thrasher-corp/gocryptotrader/blob/master/doc/coding_style.md).
++ Pull requests need to be based on and opened against the `master` branch.
+
+## Donations
+
+<img src="https://github.com/thrasher-corp/gocryptotrader/blob/master/web/src/assets/donate.png?raw=true" hspace="70">
+
+If this framework helped you in any way, or you would like to support the developers working on it, please donate Bitcoin to:
+
+***bc1qk0jareu4jytc0cfrhr5wgshsq8282awpavfahc***

engine/connection_manager_test.go

@@ -0,0 +1,122 @@
+package engine
+
+import (
+	"errors"
+	"testing"
+
+	"github.com/thrasher-corp/gocryptotrader/config"
+)
+
+func TestSetupConnectionManager(t *testing.T) {
+	t.Parallel()
+	_, err := setupConnectionManager(nil)
+	if !errors.Is(err, errNilConfig) {
+		t.Errorf("error '%v', expected '%v'", err, errNilConfig)
+	}
+
+	m, err := setupConnectionManager(&config.ConnectionMonitorConfig{})
+	if !errors.Is(err, nil) {
+		t.Errorf("error '%v', expected '%v'", err, nil)
+	}
+	if m == nil {
+		t.Error("expected manager")
+	}
+}
+
+func TestConnectionMonitorIsRunning(t *testing.T) {
+	t.Parallel()
+	m, err := setupConnectionManager(&config.ConnectionMonitorConfig{})
+	if !errors.Is(err, nil) {
+		t.Errorf("error '%v', expected '%v'", err, nil)
+	}
+	err = m.Start()
+	if !errors.Is(err, nil) {
+		t.Errorf("error '%v', expected '%v'", err, nil)
+	}
+	if !m.IsRunning() {
+		t.Error("expected true")
+	}
+	m.started = 0
+	if m.IsRunning() {
+		t.Error("expected false")
+	}
+	m = nil
+	if m.IsRunning() {
+		t.Error("expected false")
+	}
+}
+
+func TestConnectionMonitorStart(t *testing.T) {
+	t.Parallel()
+	m, err := setupConnectionManager(&config.ConnectionMonitorConfig{})
+	if !errors.Is(err, nil) {
+		t.Errorf("error '%v', expected '%v'", err, nil)
+	}
+	err = m.Start()
+	if !errors.Is(err, nil) {
+		t.Errorf("error '%v', expected '%v'", err, nil)
+	}
+	err = m.Start()
+	if !errors.Is(err, ErrSubSystemAlreadyStarted) {
+		t.Errorf("error '%v', expected '%v'", err, ErrSubSystemAlreadyStarted)
+	}
+	m = nil
+	err = m.Start()
+	if !errors.Is(err, ErrNilSubsystem) {
+		t.Errorf("error '%v', expected '%v'", err, ErrNilSubsystem)
+	}
+}
+
+func TestConnectionMonitorStop(t *testing.T) {
+	t.Parallel()
+	m, err := setupConnectionManager(&config.ConnectionMonitorConfig{})
+	if !errors.Is(err, nil) {
+		t.Errorf("error '%v', expected '%v'", err, nil)
+	}
+	err = m.Start()
+	if !errors.Is(err, nil) {
+		t.Errorf("error '%v', expected '%v'", err, nil)
+	}
+	err = m.Stop()
+	if !errors.Is(err, nil) {
+		t.Errorf("error '%v', expected '%v'", err, nil)
+	}
+	err = m.Stop()
+	if !errors.Is(err, ErrSubSystemNotStarted) {
+		t.Errorf("error '%v', expected '%v'", err, ErrSubSystemNotStarted)
+	}
+	m = nil
+	err = m.Stop()
+	if !errors.Is(err, ErrNilSubsystem) {
+		t.Errorf("error '%v', expected '%v'", err, ErrNilSubsystem)
+	}
+}
+
+func TestConnectionMonitorIsOnline(t *testing.T) {
+	t.Parallel()
+	m, err := setupConnectionManager(&config.ConnectionMonitorConfig{})
+	if !errors.Is(err, nil) {
+		t.Errorf("error '%v', expected '%v'", err, nil)
+	}
+	err = m.Start()
+	if !errors.Is(err, nil) {
+		t.Errorf("error '%v', expected '%v'", err, nil)
+	}
+	// If someone runs this offline, who are we to fail them?
+	m.IsOnline()
+	err = m.Stop()
+	if err != nil {
+		t.Fatal(err)
+	}
+	if m.IsOnline() {
+		t.Error("expected false")
+	}
+	m.conn = nil
+	if m.IsOnline() {
+		t.Error("expected false")
+	}
+	m = nil
+	if m.IsOnline() {
+		t.Error("expected false")
+	}
+}

engine/database.go

@@ -1,132 +0,0 @@
-package engine
-
-import (
-	"errors"
-	"fmt"
-	"sync/atomic"
-	"time"
-
-	"github.com/thrasher-corp/gocryptotrader/database"
-	dbpsql "github.com/thrasher-corp/gocryptotrader/database/drivers/postgres"
-	dbsqlite3 "github.com/thrasher-corp/gocryptotrader/database/drivers/sqlite3"
-	"github.com/thrasher-corp/gocryptotrader/engine/subsystem"
-	"github.com/thrasher-corp/gocryptotrader/log"
-	"github.com/thrasher-corp/sqlboiler/boil"
-)
-
-var (
-	dbConn *database.Instance
-)
-
-type databaseManager struct {
-	started  int32
-	shutdown chan struct{}
-}
-
-func (a *databaseManager) Started() bool {
-	return atomic.LoadInt32(&a.started) == 1
-}
-
-func (a *databaseManager) Start(bot *Engine) (err error) {
-	if !atomic.CompareAndSwapInt32(&a.started, 0, 1) {
-		return fmt.Errorf("database manager %w", subsystem.ErrSubSystemAlreadyStarted)
-	}
-
-	defer func() {
-		if err != nil {
-			atomic.CompareAndSwapInt32(&a.started, 1, 0)
-		}
-	}()
-
-	log.Debugln(log.DatabaseMgr, "Database manager starting...")
-
-	a.shutdown = make(chan struct{})
-
-	if bot.Config.Database.Enabled {
-		if bot.Config.Database.Driver == database.DBPostgreSQL {
-			log.Debugf(log.DatabaseMgr,
-				"Attempting to establish database connection to host %s/%s utilising %s driver\n",
-				bot.Config.Database.Host,
-				bot.Config.Database.Database,
-				bot.Config.Database.Driver)
-			dbConn, err = dbpsql.Connect()
-		} else if bot.Config.Database.Driver == database.DBSQLite ||
-			bot.Config.Database.Driver == database.DBSQLite3 {
-			log.Debugf(log.DatabaseMgr,
-				"Attempting to establish database connection to %s utilising %s driver\n",
-				bot.Config.Database.Database,
-				bot.Config.Database.Driver)
-			dbConn, err = dbsqlite3.Connect()
-		}
-		if err != nil {
-			return fmt.Errorf("database failed to connect: %v Some features that utilise a database will be unavailable", err)
-		}
-		dbConn.Connected = true
-
-		DBLogger := database.Logger{}
-		if bot.Config.Database.Verbose {
-			boil.DebugMode = true
-			boil.DebugWriter = DBLogger
-		}
-
-		go a.run(bot)
-		return nil
-	}
-
-	return errors.New("database support disabled")
-}
-
-func (a *databaseManager) Stop() error {
-	if atomic.LoadInt32(&a.started) == 0 {
-		return fmt.Errorf("database manager %w", subsystem.ErrSubSystemNotStarted)
-	}
-	defer func() {
-		atomic.CompareAndSwapInt32(&a.started, 1, 0)
-	}()
-
-	err := dbConn.SQL.Close()
-	if err != nil {
-		log.Errorf(log.DatabaseMgr, "Failed to close database: %v", err)
-	}
-
-	close(a.shutdown)
-	return nil
-}
-
-func (a *databaseManager) run(bot *Engine) {
-	log.Debugln(log.DatabaseMgr, "Database manager started.")
-	bot.ServicesWG.Add(1)
-	t := time.NewTicker(time.Second * 2)
-
-	defer func() {
-		t.Stop()
-		bot.ServicesWG.Done()
-		log.Debugln(log.DatabaseMgr, "Database manager shutdown.")
-	}()
-
-	for {
-		select {
-		case <-a.shutdown:
-			return
-		case <-t.C:
-			go a.checkConnection()
-		}
-	}
-}
-
-func (a *databaseManager) checkConnection() {
-	dbConn.Mu.Lock()
-	defer dbConn.Mu.Unlock()
-
-	err := dbConn.SQL.Ping()
-	if err != nil {
-		log.Errorf(log.DatabaseMgr, "Database connection error: %v\n", err)
-		dbConn.Connected = false
-		return
-	}
-
-	if !dbConn.Connected {
-		log.Info(log.DatabaseMgr, "Database connection reestablished")
-		dbConn.Connected = true
-	}
-}

engine/database_connection.go

@@ -0,0 +1,191 @@
+package engine
+
+import (
+	"errors"
+	"fmt"
+	"sync"
+	"sync/atomic"
+	"time"
+
+	"github.com/thrasher-corp/gocryptotrader/database"
+	dbpsql "github.com/thrasher-corp/gocryptotrader/database/drivers/postgres"
+	dbsqlite3 "github.com/thrasher-corp/gocryptotrader/database/drivers/sqlite3"
+	"github.com/thrasher-corp/gocryptotrader/log"
+)
+
+// DatabaseConnectionManagerName is an exported subsystem name
+const DatabaseConnectionManagerName = "database"
+
+var (
+	errDatabaseDisabled = errors.New("database support disabled")
+)
+
+// DatabaseConnectionManager holds the database connection and its status
+type DatabaseConnectionManager struct {
+	started  int32
+	shutdown chan struct{}
+	enabled  bool
+	verbose  bool
+	host     string
+	username string
+	password string
+	database string
+	driver   string
+	wg       sync.WaitGroup
+	dbConn   *database.Instance
+}
+
+// IsRunning safely checks whether the subsystem is running
+func (m *DatabaseConnectionManager) IsRunning() bool {
+	if m == nil {
+		return false
+	}
+	return atomic.LoadInt32(&m.started) == 1
+}
+
+// SetupDatabaseConnectionManager creates a new database manager
+func SetupDatabaseConnectionManager(cfg *database.Config) (*DatabaseConnectionManager, error) {
+	if cfg == nil {
+		return nil, errNilConfig
+	}
+	m := &DatabaseConnectionManager{
+		shutdown: make(chan struct{}),
+		enabled:  cfg.Enabled,
+		verbose:  cfg.Verbose,
+		host:     cfg.Host,
+		username: cfg.Username,
+		password: cfg.Password,
+		database: cfg.Database,
+		driver:   cfg.Driver,
+		dbConn:   database.DB,
+	}
+	err := m.dbConn.SetConfig(cfg)
+	if err != nil {
+		return nil, err
+	}
+
+	return m, nil
+}
+
+// Start sets up the database connection manager to maintain a SQL connection
+func (m *DatabaseConnectionManager) Start(wg *sync.WaitGroup) (err error) {
+	if m == nil {
+		return fmt.Errorf("%s %w", DatabaseConnectionManagerName, ErrNilSubsystem)
+	}
+	if !atomic.CompareAndSwapInt32(&m.started, 0, 1) {
+		return fmt.Errorf("database manager %w", ErrSubSystemAlreadyStarted)
+	}
+	defer func() {
+		if err != nil {
+			atomic.CompareAndSwapInt32(&m.started, 1, 0)
+		}
+	}()
+
+	log.Debugln(log.DatabaseMgr, "Database manager starting...")
+
+	if m.enabled {
+		m.shutdown = make(chan struct{})
+		switch m.driver {
+		case database.DBPostgreSQL:
+			log.Debugf(log.DatabaseMgr,
+				"Attempting to establish database connection to host %s/%s utilising %s driver\n",
+				m.host,
+				m.database,
+				m.driver)
+			m.dbConn, err = dbpsql.Connect()
+		case database.DBSQLite,
+			database.DBSQLite3:
+			log.Debugf(log.DatabaseMgr,
+				"Attempting to establish database connection to %s utilising %s driver\n",
+				m.database,
+				m.driver)
+			m.dbConn, err = dbsqlite3.Connect()
+		default:
+			return database.ErrNoDatabaseProvided
+		}
+		if err != nil {
+			return fmt.Errorf("%w: %v Some features that utilise a database will be unavailable", database.ErrFailedToConnect, err)
+		}
+		m.dbConn.SetConnected(true)
+		wg.Add(1)
+		m.wg.Add(1)
+		go m.run(wg)
+		return nil
+	}
+
+	return errDatabaseDisabled
+}
+
+// Stop stops the database manager and closes the connection
+// Stop attempts to shutdown the subsystem
+func (m *DatabaseConnectionManager) Stop() error {
+	if m == nil {
+		return fmt.Errorf("%s %w", DatabaseConnectionManagerName, ErrNilSubsystem)
+	}
+	if atomic.LoadInt32(&m.started) == 0 {
+		return fmt.Errorf("%s %w", DatabaseConnectionManagerName, ErrSubSystemNotStarted)
+	}
+	defer func() {
+		atomic.CompareAndSwapInt32(&m.started, 1, 0)
+	}()
+
+	err := m.dbConn.CloseConnection()
+	if err != nil {
+		log.Errorf(log.DatabaseMgr, "Failed to close database: %v", err)
+	}
+
+	close(m.shutdown)
+	m.wg.Wait()
+	return nil
+}
+
+func (m *DatabaseConnectionManager) run(wg *sync.WaitGroup) {
+	log.Debugln(log.DatabaseMgr, "Database manager started.")
+	t := time.NewTicker(time.Second * 2)
+
+	defer func() {
+		t.Stop()
+		m.wg.Done()
+		wg.Done()
+		log.Debugln(log.DatabaseMgr, "Database manager shutdown.")
+	}()
+
+	for {
+		select {
+		case <-m.shutdown:
+			return
+		case <-t.C:
+			err := m.checkConnection()
+			if err != nil {
+				log.Error(log.DatabaseMgr, "Database connection error:", err)
+			}
+		}
+	}
+}
+
+func (m *DatabaseConnectionManager) checkConnection() error {
+	if m == nil {
+		return fmt.Errorf("%s %w", DatabaseConnectionManagerName, ErrNilSubsystem)
+	}
+	if atomic.LoadInt32(&m.started) == 0 {
+		return fmt.Errorf("%s %w", DatabaseConnectionManagerName, ErrSubSystemNotStarted)
+	}
+	if !m.enabled {
+		return database.ErrDatabaseSupportDisabled
+	}
+	if m.dbConn == nil {
+		return database.ErrNoDatabaseProvided
+	}
+
+	err := m.dbConn.Ping()
+	if err != nil {
+		m.dbConn.SetConnected(false)
+		return err
+	}
+
+	if !m.dbConn.IsConnected() {
+		log.Info(log.DatabaseMgr, "Database connection reestablished")
+		m.dbConn.SetConnected(true)
+	}
+	return nil
+}