Files
gocryptotrader/gctscript/README.md
Andrew f6fd94ea69 Engine: Scripting support (#383)
* WIP

* updated appveyor and increased deadline 5 seconds due to increased linters being added

* revert files to upstream/engine

* WIP

* WIP

* mod file changes

* added script manager

* Added manager/and cli interfaces to scripting

* Added script task handler

* WIP - Added timer/repeat support and fleshed out wrapper further

* autoload support added + WIP

* WIP commit

* added account balance info

* btc markets temp work around

* WIP - merged with upstream for new order package BTC Markets responses broken

* Cancel order wrapper WIP

*  order wrapper update

* Added test coverage for VM

* moved to map for VM List shutdown of all VM now handled added gctcli commands for list and stop of running scripts

* added override to load/execute for path

* fixed incorrect channel shutdown added further test coverage and restructured gctcli commands into sub commands

* increased test coverage for packages

* Added docs cleaned up tests and example scripts

* Test coverage increased for module/gct/exchange package

* windows fixes

* merged upstream/engine

* WIP

* logger fixes - removed pointer to bool check removed duplicate test check for logger

* remove unused mutex

* added inital upload support

* fix linter issues for go-fmt

* added zip support for uploading and added base for fund withdrawing

* changed error return types and also log errors, fix zip path issue

* improved error outputs and code flow

* pairs response fix added protobuf defs for stop all and list all

* added stop all running scripts general clean up and moved across to OrderManager

* linter fixes (gofmt)

* added list all command

* rewrote zip handler to be cleaner also fixed file overwrite on upload

* added query command reworked tests

* added further error checking to compileandrun corrected use of pointers for accountinfo

* bumped tengo version

* Removed named returns reworded log messages removed unused falseptr

* WIP

* Added virutal machine limit improved config options

* added model for script event added upload validation

* script_event table has been completed, tests for wrapper functions implemented

* README updates

* reverted changes opened new PR to move withdraw struct outs

* intial work on adding withdraw support after merger of withdraw package

* started work on examples

* Added crypto withdraw support

* fix switch case assignment and gofmt project

* Reworking Fiat withdraw request pending #402

* removed double pointer call

* added withdraw support for fiat  currencies

* added tests for withdraw methods increased readme

* removed local tengo require and also fix linter issues

* Added default log size const added basic test for invalid script execution

* First pass at moving wrapper to validator package to allow proper validation of uploaded scripts

* Added script details to README added config test added test for no file extension

* moved tests to const and fixed incorrect pathing

* added test coverage to withdraw package

* corrected file close handling

* point to included configtest.json

* extended validator support when a script is uploaded

* Bug fix on bool logic

* Added mutex

* Don't create autit events on test execution

* reverted common to master

* moved file rename to unix timestamp format

* converted logger enabled back to pointer as i need nilness check also moved scriptid to text over blob

* started work on autoload add/remove support

* First round of PR fixes (mostly commented exports)

* Moved GCTScript load to last, removed unneeded error from cleanup()

* Comment clairty for AuitEventID

* added autoload add/remove command to cli

* added tests for autoload

* Test updates for Exchanges

* linter fixes (gofmt)

* Removed double check of engine pointer

* remove possible nil pointer on GetSpecificTicker

* Fixed not closing file handler on write that causes archive removal to fail

* file handler Close clean ups

* corrected spelling on error return and return invalid name n autoload

* moved strings to cosnt moved bool pointer creation to convert package

* new zip extractor added

* Validation has been added to archive uploads

* removed shadow var on err

* added ok check to conversion

* converted condition check

* basic test for zip extract added

* new zip handler

* reverted back to old atomic loading system

* removed shadow err

* lets add a new line

* added space to error return

* command line toggle for script now works properly

* readme updated

* set configLoaded to true

* check for configLoaded condition

* added mutex to allow for multiple access on virtual machine increased test coverage disable script manager if scripting is disabled

* linked up to enable/disablesubsystem commands

* added start/stop example to readme

* reworked logic on test as check should be done on Load()

* updated to tengo v2

* linters

* lower time on ntp client to stop slippage

* remove all fails if any fail validtion from an archive

* remove vm from list if timer is invalid

* removed shadow on err

* remove config creation from NTPCheck test

* WIP testing DB changes

* add unique constraint

* WIP: created has many model

* linters run

* basic sqlite3 support added for new database format

* linters run

* Added test coverage for script repo

* removed unused print

* updated env vars for CI instances

* updated env vars for CI instances

* Updated test packages

* Test updates for postgresql

* removed invalid tests from postgres

* remove duplication of struct and improved code flow

* general cleanup

* wording changes on log output

* use databasemgr logger and add support for autoload without file extension

* corrected test naming

* return correct error

* return correct error again version 82

* store scriptdata on creation

* Hello

* Errorln -> Errorf

* Removed unused vars

* Read me updates

* testing without parallel

* comment on exported type

* added nil check against VM for test

* add debugging information

* gofmt

* remove verbose and data sent to channel

* Added debug information

* linter fixes (gofmt)

* remove unused CompileAndRun() call

* test sleep to see if issue is timing related

* semi-concurrent map fixes

* one day i will run gofmt or setup precommit hooks

* new line :D

* increased test coverage

* added correct sleep time

* Moved over to sync map

* linter fixes (gofmt)

* goimports

* moved VM related methods to vm.go

* new line at end of file

* trying increased timeout on golangci-lint for appveyor

* add debugging information

* removed timeout

* reworked timeout logic

* linter fixes (gofmt)

* increased test coverage

* increased test coverage

* one day i will run gofmt or setup precommit hooks

* removed unused exchange test

* increased golangci-lint timeout

* Added nil check on shutdown and test coverage for it lowered timeout back to 1:30

* reworked ID system

* removed script hash as it was unused

* added comments on exported methods and read me update

* reorder code

* removed to atomic.value for test execution flag

* increased test coverage

* move add further up execution

* point to correct script file
2020-01-23 13:54:04 +11:00

263 lines
7.8 KiB
Markdown

# GoCryptoTrader package gctscript
<img src="https://github.com/thrasher-corp/gocryptotrader/blob/master/web/src/assets/page-logo.png?raw=true" width="350px" height="350px" hspace="70">
[![Build Status](https://travis-ci.org/thrasher-corp/gocryptotrader.svg?branch=master)](https://travis-ci.org/thrasher-corp/gocryptotrader)
[![Software License](https://img.shields.io/badge/License-MIT-orange.svg?style=flat-square)](https://github.com/thrasher-corp/gocryptotrader/blob/master/LICENSE)
[![GoDoc](https://godoc.org/github.com/thrasher-corp/gocryptotrader?status.svg)](https://godoc.org/github.com/thrasher-corp/gocryptotrader/portfolio)
[![Coverage Status](http://codecov.io/github/thrasher-corp/gocryptotrader/coverage.svg?branch=master)](http://codecov.io/github/thrasher-corp/gocryptotrader?branch=master)
[![Go Report Card](https://goreportcard.com/badge/github.com/thrasher-corp/gocryptotrader)](https://goreportcard.com/report/github.com/thrasher-corp/gocryptotrader)
This gctscript package is part of the GoCryptoTrader codebase.
## This is still in active development
You can track ideas, planned features and what's in progress on this Trello board: [https://trello.com/b/ZAhMhpOy/gocryptotrader](https://trello.com/b/ZAhMhpOy/gocryptotrader).
Join our slack to discuss all things related to GoCryptoTrader! [GoCryptoTrader Slack](https://join.slack.com/t/gocryptotrader/shared_invite/enQtNTQ5NDAxMjA2Mjc5LTc5ZDE1ZTNiOGM3ZGMyMmY1NTAxYWZhODE0MWM5N2JlZDk1NDU0YTViYzk4NTk3OTRiMDQzNGQ1YTc4YmRlMTk)
## Current Features for gctscript package
+ Execute scripts
+ Terminate scripts
+ Autoload scripts on bot startup
+ Current Exchange features supported:
+ Enabled Exchanges
+ Enabled currency pairs
+ Account information
+ Query Order
+ Submit Order
+ Cancel Order
+ Ticker
+ Orderbook
## How to use
##### Prerequisites
To Enable database logging support you must have an active migrated database by following the [database setup guide](../database/README.md)
##### Configuration
The gctscript configuration struct is currently:
```shell script
type Config struct {
Enabled bool `json:"enabled"`
ScriptTimeout time.Duration `json:"timeout"`
AllowImports bool `json:"allow_imports"`
AutoLoad []string `json:"auto_load"`
Verbose bool `json:"Verbose"`
}
```
With an example configuration being:
```sh
"gctscript": {
"enabled": true,
"timeout": 600000000,
"allow_imports": true,
"auto_load": [],
"debug": false
},
```
##### Script Control
+ You can autoload scripts on bot start up by placing their name in the "auto_load" config entry
```shell script
"auto_load": ["one","two"]
```
This will look in your GoCryptoTrader data directory in a folder called "scripts" for files one.gct and two.gct and autoload them
+ Manual control of scripts can be done via the gctcli command with support for the following:
- Enable/Disable GCTScript:
```shell script
gctcli enablesubsystem "gctscript"
gctcli disablesubsystem "gctscript"
```
- Start/Execute:
```shell script
gctcli gctscript execute <scriptname> <pathoverride>
gctcli gctscript execute "timer.gct" "~/gctscript"
{
"status": "ok",
"data": "timer.gct executed"
}
```
- Stop:
```shell script
gctcli gctscript stop <uuid>
gctcli gctscript stop 821bd73e-02b1-4974-9463-874cb49f130d
{
"status": "ok",
"data": "821bd73e-02b1-4974-9463-874cb49f130d terminated"
}
```
- Status:
```shell script
gctcli gctscript status
{
"status": "ok",
"scripts": [
{
"uuid": "821bd73e-02b1-4974-9463-874cb49f130d",
"name": "timer.gct",
"next_run": "2019-11-14 13:11:40.224919456 +1100 AEDT m=+91.062103259"
}
]
}
```
- Read file:
```shell script
gctcli gctscript read <filename>
gctcli gctscript read "timer.gct"
{
"status": "ok",
"script": {
"name": "timer.gct",
"path": "/home/x/.gocryptotrader/scripts"
},
"data": "fmt := import(\"fmt\")\nt := import(\"times\")\n\nname := \"run\"\ntimer := \"5s\"\n\nload := func() {\n\tfmt.printf(\"5s %s\\n\",t.now())\n}\n\nload()\n"
}
```
- Query running script:
```shell script
gctcli gctscript query <uuid>
gctcli gctscript query 821bd73e-02b1-4974-9463-874cb49f130d
{
"status": "ok",
"script": {
"UUID": "bf692e2d-fa1e-4d95-92fd-33d7634d3d77",
"name": "timer.gct",
"path": "/home/x/.gocryptotrader/scripts",
"next_run": "2019-12-12 07:44:19.747572406 +1100 AEDT m=+16.782773385"
},
"data": "fmt := import(\"fmt\")\nt := import(\"times\")\n\nname := \"run\"\ntimer := \"5s\"\n\nload := func() {\n\tfmt.printf(\"5s %s\\n\",t.now())\n}\n\nload()\n"
}
load()
```
- Add script to autoload:
```shell script
gctcli gctscript autoload add timer
{
"status": "success",
"data": "script timer added to autoload list"
}
```
- Remove script from autoload:
```shell script
gctcli gctscript autoload remove timer
{
"status": "success",
"data": "script timer removed from autoload list"
}
```
##### Scripting & Extending modules
The scripting engine utilises [tengo](https://github.com/d5/tengo) an intro tutorial for it can be found [here](https://github.com/d5/tengo/blob/master/docs/tutorial.md)
Modules have been written so far linking up common exchange features including
- Orderbook
- Ticker
- Order Management
- Account information
- Withdraw funds
- Get Deposit Addresses
Extending or creating new modules:
Extending an existing module the exchange module for example is simple
- Open required [module](modules/gct/exchange.go)
- Add to exchangeModule map
- Define function with signature ```(args ...objects.Object) (ret objects.Object, err error)```
Similar steps can be taken to add a new module with a few adjustments
- Open required [GCT](modules/gct/gct_types.go)
- Add module name to GCTModules map
##### GCT module methods
Current supported methods added and exposed to scripts are as follows:
```
accountinfo
-> exchange:string
depositaddress
-> exchange:string
-> currency:string
orderbook
-> exchange:string
-> currency pair:string
-> delimiter:string
-> asset:string
ticker
-> exchange:string
-> currency pair:string
-> delimiter:string
-> asset:string
pairs
-> exchange:string
-> enabled only:bool
-> asset:string
queryorder
-> exchange:string
-> order id:string
submitorder
-> exchange:string
-> currency pair:string
-> delimiter:string
-> order type:string
-> order side:string
-> price:float64
-> amount:float64
-> client_id:string
withdrawfiat
-> exchange:string
-> currency:string
-> description:string
-> amount:float64
-> bank id:string
withdrawcrypto
-> exchange:string
-> currency:string
-> address:string
-> address tag:string
-> amount:float64
-> fee:float64
-> description:string
```
## Contribution
Please feel free to submit any pull requests or suggest any desired features to be added.
When submitting a PR, please abide by our coding guidelines:
+ Code must adhere to the official Go [formatting](https://golang.org/doc/effective_go.html#formatting) guidelines (i.e. uses [gofmt](https://golang.org/cmd/gofmt/)).
+ Code must be documented adhering to the official Go [commentary](https://golang.org/doc/effective_go.html#commentary) guidelines.
+ Code must adhere to our [coding style](https://github.com/thrasher-corp/gocryptotrader/blob/master/doc/coding_style.md).
+ Pull requests need to be based on and opened against the `master` branch.
## Donations
<img src="https://github.com/thrasher-corp/gocryptotrader/blob/master/web/src/assets/donate.png?raw=true" hspace="70">
If this framework helped you in any way, or you would like to support the developers working on it, please donate Bitcoin to:
***1F5zVDgNjorJ51oGebSvNCrSAHpwGkUdDB***