Files
gocryptotrader/database
Scott fcc5ad4551 exchanges/qa: Add exchange wrapper testing suite (#1159)
* initial concept of a nice validation tester for exchanges

* adds some datahandler design

* expand testing

* more tests and fixes

* minor end of day fix for bithumb

* fixes implementation issues

* more test coverage and improvements, but not sure if i should continue

* fix more wrapper implementations

* adds error type, more fixes

* changes signature, fixes implementations

* fixes more wrapper implementations

* one more bit

* more cleanup

* WOW things work?

* lintle 1/1337

* mini bump

* fixes all linting

* neaten

* GetOrderInfo+ asset pair fixes+improvements

* adds new websocket test

* expand ws testing

* fix bug, expand tests, improve implementation

* code coverage of a lot of new codes

* fixes everything

* reverts accidental changes

* minor fixes from reviewing code

* removes Bitfinex cancelBatchOrder implementation

* fixes dumb baby typo for babies

* mini nit fixes

* so many nits to address

* addresses all the nits

* Titlecase

* switcheroo

* removes websocket testing for now

* fix appveyor, minor test fix

* fixes typo, re-kindles killed kode

* skip binance wrapper tests when running CI

* expired context, huobi okx fixes

* kodespull

* fix ordering

* time fix because why not

* fix exmo, others

* hopefully this fixes all of my life's problems

* last thing today

* huobi, more like hypotrophy

* golangci-lint, more like mypooroldknee-splint

* fix huobi times by removing them

* should fix okx currency issues

* blocks the application

* adds last little contingency for pairs

* addresses most nits and new problems

* lovely fixed before seeing why okx sucks

* fixes issues with okx websocket

* the classic receieieivaier

* lintle

* adds test and fixes existing tests

* expands error handling messages during setup

* fixes dumb okx bugs introduced

* quick fix for lint and exmo

* fixes nixes

* fix exmo deposit issue

* lint

* fixes issue with extra asset runs missing

* fix surprise race

* all the lint and merge fixes

* fixes surprise bugs in OKx

* fixes issues with times and chains

* fixing all the merge stuff

* merge fix

* rm logs and a panic potential

* lovely lint lament

* an easy demonstration of scenario, but not of initial purpose

* put it in the bin

* Revert "put it in the bin"

This reverts commit 15c6490f713233d43f10957367fcbf18e3818bdd.

* re-add after immediate error popup

* fix mini poor test design

* okx okay

* merge fixes

* fixes issues discovered in lovely test

* I FORGOT TO COMMIT THIS

* nit fixaroonaboo

* forgoetten test fix

* revert old okx asset intrument work

* fixes

* revert problems I didnt understand. update bybit

* fix merge bugs

* test cleanup

* further improvements

* reshuffle and lint

* rm redundant CI_TEST by rm the CI_TEST field that is redundant

* path fix

* move to its own section, dont run on 32 bit + appveyor

* lint

* fix lbank

* address nits

* let it rip

* fix failing test time range

* niteroo boogaloo

* mod tidy, use common.SimpleTimeFormat
2023-07-03 11:09:43 +10:00
..

GoCryptoTrader package Database

Build Status Software License GoDoc Coverage Status Go Report Card

This database package is part of the GoCryptoTrader codebase.

This is still in active development

You can track ideas, planned features and what's in progress on this Trello board: https://trello.com/b/ZAhMhpOy/gocryptotrader.

Join our slack to discuss all things related to GoCryptoTrader! GoCryptoTrader Slack

Current Features for database package

  • Establishes & Maintains database connection across program life cycle
  • Migration handed by Goose
  • Model generation handled by SQLBoiler

How to use

Prerequisites

SQLBoiler

go install github.com/thrasher-corp/sqlboiler

Postgres Driver

go install github.com/thrasher-corp/sqlboiler/drivers/sqlboiler-psql

SQLite Driver

go install github.com/thrasher-corp/sqlboiler-sqlite3
Configuration

The database configuration struct is currently:

type Config struct {
	Enabled                   bool   `json:"enabled"`
	Verbose                   bool   `json:"verbose"`
	Driver                    string `json:"driver"`
	drivers.ConnectionDetails `json:"connectionDetails"`
}

And Connection Details:

type ConnectionDetails struct {
	Host     string `json:"host"`
	Port     uint16 `json:"port"`
	Username string `json:"username"`
	Password string `json:"password"`
	Database string `json:"database"`
	SSLMode  string `json:"sslmode"`
}

With an example configuration being:

 "database": {
  "enabled": true,
  "verbose": true,
  "driver": "postgres",
  "connectionDetails": {
   "host": "localhost",
   "port": 5432,
   "username": "gct-dev",
   "password": "gct-dev",
   "database": "gct-dev",
   "sslmode": "disable"
  }
 },
Create and Run migrations

Migrations are created using a modified version of Goose

A helper tool sits in the ./cmd/dbmigrate folder that includes the following features:

  • Check current database version with the "status" command
dbmigrate -command status
  • Create a new migration
dbmigrate -command "create" -args "model"

This will create a folder in the ./database/migration folder that contains postgres.sql and sqlite.sql files

  • Run dbmigrate command with -command up
dbmigrate -command "up"

dbmigrate provides a -migrationdir flag override to tell it what path to look in for migrations

Adding a new model

Model's are generated using SQLBoiler A helper tool has been made located in gen_sqlboiler_config that will parse your GoCryptoTrader config and output a SQLBoiler config

gen_sqlboiler_config

By default this will look in your gocryptotrader data folder and default config, these can be overwritten along with the location of the sqlboiler generated config

-config "configname.json"
-datadir "~/.gocryptotrader/"
-outdir "~/.gocryptotrader/"

Generate a new model that gets placed in ./database/models/ folder

Linux:

sqlboiler -o database/models/postgres -p postgres --no-auto-timestamps --wipe psql 

Windows:

sqlboiler -o database\\models\\postgres -p postgres --no-auto-timestamps --wipe psql

Helpers have been provided in the Makefile for linux users

make gen_db_models

And in the contrib/sqlboiler.cmd for windows users

Adding a Repository
  • Create Repository directory in github.com/thrasher-corp/gocryptotrader/database/repository/
DBSeed helper

A helper tool cmd/dbseed has been created for assisting with data migration

Contribution

Please feel free to submit any pull requests or suggest any desired features to be added.

When submitting a PR, please abide by our coding guidelines:

  • Code must adhere to the official Go formatting guidelines (i.e. uses gofmt).
  • Code must be documented adhering to the official Go commentary guidelines.
  • Code must adhere to our coding style.
  • Pull requests need to be based on and opened against the master branch.

Donations

If this framework helped you in any way, or you would like to support the developers working on it, please donate Bitcoin to:

bc1qk0jareu4jytc0cfrhr5wgshsq8282awpavfahc