Engine: Scripting support (#383)

* WIP * updated appveyor and increased deadline 5 seconds due to increased linters being added * revert files to upstream/engine * WIP * WIP * mod file changes * added script manager * Added manager/and cli interfaces to scripting * Added script task handler * WIP - Added timer/repeat support and fleshed out wrapper further * autoload support added + WIP * WIP commit * added account balance info * btc markets temp work around * WIP - merged with upstream for new order package BTC Markets responses broken * Cancel order wrapper WIP * order wrapper update * Added test coverage for VM * moved to map for VM List shutdown of all VM now handled added gctcli commands for list and stop of running scripts * added override to load/execute for path * fixed incorrect channel shutdown added further test coverage and restructured gctcli commands into sub commands * increased test coverage for packages * Added docs cleaned up tests and example scripts * Test coverage increased for module/gct/exchange package * windows fixes * merged upstream/engine * WIP * logger fixes - removed pointer to bool check removed duplicate test check for logger * remove unused mutex * added inital upload support * fix linter issues for go-fmt * added zip support for uploading and added base for fund withdrawing * changed error return types and also log errors, fix zip path issue * improved error outputs and code flow * pairs response fix added protobuf defs for stop all and list all * added stop all running scripts general clean up and moved across to OrderManager * linter fixes (gofmt) * added list all command * rewrote zip handler to be cleaner also fixed file overwrite on upload * added query command reworked tests * added further error checking to compileandrun corrected use of pointers for accountinfo * bumped tengo version * Removed named returns reworded log messages removed unused falseptr * WIP * Added virutal machine limit improved config options * added model for script event added upload validation * script_event table has been completed, tests for wrapper functions implemented * README updates * reverted changes opened new PR to move withdraw struct outs * intial work on adding withdraw support after merger of withdraw package * started work on examples * Added crypto withdraw support * fix switch case assignment and gofmt project * Reworking Fiat withdraw request pending #402 * removed double pointer call * added withdraw support for fiat currencies * added tests for withdraw methods increased readme * removed local tengo require and also fix linter issues * Added default log size const added basic test for invalid script execution * First pass at moving wrapper to validator package to allow proper validation of uploaded scripts * Added script details to README added config test added test for no file extension * moved tests to const and fixed incorrect pathing * added test coverage to withdraw package * corrected file close handling * point to included configtest.json * extended validator support when a script is uploaded * Bug fix on bool logic * Added mutex * Don't create autit events on test execution * reverted common to master * moved file rename to unix timestamp format * converted logger enabled back to pointer as i need nilness check also moved scriptid to text over blob * started work on autoload add/remove support * First round of PR fixes (mostly commented exports) * Moved GCTScript load to last, removed unneeded error from cleanup() * Comment clairty for AuitEventID * added autoload add/remove command to cli * added tests for autoload * Test updates for Exchanges * linter fixes (gofmt) * Removed double check of engine pointer * remove possible nil pointer on GetSpecificTicker * Fixed not closing file handler on write that causes archive removal to fail * file handler Close clean ups * corrected spelling on error return and return invalid name n autoload * moved strings to cosnt moved bool pointer creation to convert package * new zip extractor added * Validation has been added to archive uploads * removed shadow var on err * added ok check to conversion * converted condition check * basic test for zip extract added * new zip handler * reverted back to old atomic loading system * removed shadow err * lets add a new line * added space to error return * command line toggle for script now works properly * readme updated * set configLoaded to true * check for configLoaded condition * added mutex to allow for multiple access on virtual machine increased test coverage disable script manager if scripting is disabled * linked up to enable/disablesubsystem commands * added start/stop example to readme * reworked logic on test as check should be done on Load() * updated to tengo v2 * linters * lower time on ntp client to stop slippage * remove all fails if any fail validtion from an archive * remove vm from list if timer is invalid * removed shadow on err * remove config creation from NTPCheck test * WIP testing DB changes * add unique constraint * WIP: created has many model * linters run * basic sqlite3 support added for new database format * linters run * Added test coverage for script repo * removed unused print * updated env vars for CI instances * updated env vars for CI instances * Updated test packages * Test updates for postgresql * removed invalid tests from postgres * remove duplication of struct and improved code flow * general cleanup * wording changes on log output * use databasemgr logger and add support for autoload without file extension * corrected test naming * return correct error * return correct error again version 82 * store scriptdata on creation * Hello * Errorln -> Errorf * Removed unused vars * Read me updates * testing without parallel * comment on exported type * added nil check against VM for test * add debugging information * gofmt * remove verbose and data sent to channel * Added debug information * linter fixes (gofmt) * remove unused CompileAndRun() call * test sleep to see if issue is timing related * semi-concurrent map fixes * one day i will run gofmt or setup precommit hooks * new line :D * increased test coverage * added correct sleep time * Moved over to sync map * linter fixes (gofmt) * goimports * moved VM related methods to vm.go * new line at end of file * trying increased timeout on golangci-lint for appveyor * add debugging information * removed timeout * reworked timeout logic * linter fixes (gofmt) * increased test coverage * increased test coverage * one day i will run gofmt or setup precommit hooks * removed unused exchange test * increased golangci-lint timeout * Added nil check on shutdown and test coverage for it lowered timeout back to 1:30 * reworked ID system * removed script hash as it was unused * added comments on exported methods and read me update * reorder code * removed to atomic.value for test execution flag * increased test coverage * move add further up execution * point to correct script file
2026-05-13 23:16:45 +00:00 · 2020-01-23 13:54:04 +11:00
parent 4c33a0738a
commit f6fd94ea69
102 changed files with 15181 additions and 370 deletions
--- a/gctscript/vm/autoload.go
+++ b/gctscript/vm/autoload.go
@@ -0,0 +1,43 @@
+package vm
+
+import (
+	"fmt"
+	"os"
+	"path/filepath"
+
+	log "github.com/thrasher-corp/gocryptotrader/logger"
+)
+
+// Autoload remove entry from autoload slice
+func Autoload(name string, remove bool) error {
+	if filepath.Ext(name) != ".gct" {
+		name += ".gct"
+	}
+	if remove {
+		for x := range GCTScriptConfig.AutoLoad {
+			if GCTScriptConfig.AutoLoad[x] != name {
+				continue
+			}
+			GCTScriptConfig.AutoLoad = append(GCTScriptConfig.AutoLoad[:x], GCTScriptConfig.AutoLoad[x+1:]...)
+			if GCTScriptConfig.Verbose {
+				log.Debugf(log.GCTScriptMgr, "Removing script: %s from autoload", name)
+			}
+			return nil
+		}
+		return fmt.Errorf("%v - not found", name)
+	}
+
+	script := filepath.Join(ScriptPath, name)
+	_, err := os.Stat(script)
+	if err != nil {
+		if os.IsNotExist(err) {
+			return fmt.Errorf("%v - not found", script)
+		}
+		return err
+	}
+	GCTScriptConfig.AutoLoad = append(GCTScriptConfig.AutoLoad, name)
+	if GCTScriptConfig.Verbose {
+		log.Debugf(log.GCTScriptMgr, "Adding script: %s to autoload", name)
+	}
+	return nil
+}
--- a/gctscript/vm/gctscript.go
+++ b/gctscript/vm/gctscript.go
@@ -0,0 +1,81 @@
+package vm
+
+import (
+	"fmt"
+
+	"github.com/gofrs/uuid"
+	"github.com/thrasher-corp/gocryptotrader/gctscript/wrappers/validator"
+	log "github.com/thrasher-corp/gocryptotrader/logger"
+)
+
+// New returns a new instance of VM
+func New() *VM {
+	if VMSCount.Len() >= int32(GCTScriptConfig.MaxVirtualMachines) {
+		if GCTScriptConfig.Verbose {
+			log.Warnf(log.GCTScriptMgr, "GCTScript MaxVirtualMachines (%v) hit, unable to start further instances",
+				GCTScriptConfig.MaxVirtualMachines)
+		}
+		return nil
+	}
+	VMSCount.add()
+	vm := NewVM()
+	if vm == nil {
+		VMSCount.remove()
+	} else {
+		AllVMSync.Store(vm.ID, vm)
+	}
+	return vm
+}
+
+// Validate will attempt to execute a script in a test/non-live environment
+// to confirm it passes requirements for execution
+func Validate(file string) (err error) {
+	validator.IsTestExecution.Store(true)
+	defer validator.IsTestExecution.Store(false)
+	tempVM := NewVM()
+	err = tempVM.Load(file)
+	if err != nil {
+		return
+	}
+	err = tempVM.Compile()
+	if err != nil {
+		return
+	}
+	return tempVM.Run()
+}
+
+// ShutdownAll shutdown all
+func ShutdownAll() (err error) {
+	if GCTScriptConfig.Verbose {
+		log.Debugln(log.GCTScriptMgr, "Shutting down all Virtual Machines")
+	}
+
+	var errors []error
+	AllVMSync.Range(func(k, v interface{}) bool {
+		errShutdown := v.(*VM).Shutdown()
+		if err != nil {
+			errors = append(errors, errShutdown)
+		}
+		return true
+	})
+
+	if len(errors) > 0 {
+		err = fmt.Errorf("failed to shutdown the following Virtual Machines: %v", errors)
+	}
+
+	return err
+}
+
+// RemoveVM remove VM from list
+func RemoveVM(id uuid.UUID) error {
+	if _, f := AllVMSync.Load(id); !f {
+		return fmt.Errorf(ErrNoVMFound, id.String())
+	}
+
+	AllVMSync.Delete(id)
+	VMSCount.remove()
+	if GCTScriptConfig.Verbose {
+		log.Debugf(log.GCTScriptMgr, "VM %v removed from AllVMs", id)
+	}
+	return nil
+}
--- a/gctscript/vm/gctscript_types.go
+++ b/gctscript/vm/gctscript_types.go
@@ -0,0 +1,39 @@
+package vm
+
+import (
+	"errors"
+	"time"
+)
+
+const gctScript = "GCT Script"
+
+// Config user configurable options for gctscript
+type Config struct {
+	Enabled            bool          `json:"enabled"`
+	ScriptTimeout      time.Duration `json:"timeout"`
+	MaxVirtualMachines uint8         `json:"max_virtual_machines"`
+	AllowImports       bool          `json:"allow_imports"`
+	AutoLoad           []string      `json:"auto_load"`
+	Verbose            bool          `json:"verbose"`
+}
+
+// Error interface to meet error requirements
+type Error struct {
+	Script string
+	Action string
+	Cause  error
+}
+
+var (
+	// GCTScriptConfig initialised global copy of Config{}
+	GCTScriptConfig = &Config{}
+	// ScriptPath path to load/save scripts
+	ScriptPath string
+)
+
+var (
+	// ErrScriptingDisabled error message displayed when gctscript is disabled
+	ErrScriptingDisabled = errors.New("scripting is disabled")
+	// ErrNoVMLoaded error message displayed if a virtual machine has not been initialised
+	ErrNoVMLoaded = errors.New("no virtual machine loaded")
+)
--- a/gctscript/vm/vm.go
+++ b/gctscript/vm/vm.go
@@ -0,0 +1,302 @@
+package vm
+
+import (
+	"archive/zip"
+	"bytes"
+	"context"
+	"encoding/hex"
+	"io/ioutil"
+	"os"
+	"path/filepath"
+	"sync/atomic"
+	"time"
+
+	"github.com/d5/tengo/v2"
+	"github.com/gofrs/uuid"
+	"github.com/thrasher-corp/gocryptotrader/common/crypto"
+	scriptevent "github.com/thrasher-corp/gocryptotrader/database/repository/script"
+	"github.com/thrasher-corp/gocryptotrader/gctscript/modules/loader"
+	"github.com/thrasher-corp/gocryptotrader/gctscript/wrappers/validator"
+	log "github.com/thrasher-corp/gocryptotrader/logger"
+	"github.com/volatiletech/null"
+)
+
+// NewVM attempts to create a new Virtual Machine firstly from pool
+func NewVM() (vm *VM) {
+	newUUID, err := uuid.NewV4()
+	if err != nil {
+		log.Error(log.GCTScriptMgr, Error{
+			Action: "New: UUID",
+			Cause:  err,
+		})
+		return nil
+	}
+
+	if GCTScriptConfig.Verbose {
+		log.Debugln(log.GCTScriptMgr, "New GCTScript VM created")
+	}
+
+	vm = &VM{
+		ID:     newUUID,
+		Script: pool.Get().(*tengo.Script),
+	}
+	return
+}
+
+// Load parses and creates a new instance of tengo script vm
+func (vm *VM) Load(file string) error {
+	if vm == nil {
+		return ErrNoVMLoaded
+	}
+
+	if !GCTScriptConfig.Enabled {
+		return &Error{
+			Action: "Load",
+			Cause:  ErrScriptingDisabled,
+		}
+	}
+
+	if filepath.Ext(file) != ".gct" {
+		file += ".gct"
+	}
+
+	if GCTScriptConfig.Verbose {
+		log.Debugf(log.GCTScriptMgr, "Loading script: %s ID: %v", vm.ShortName(), vm.ID)
+	}
+
+	f, err := os.Open(file)
+	if err != nil {
+		return &Error{
+			Action: "Load: Open",
+			Script: file,
+			Cause:  err,
+		}
+	}
+
+	defer f.Close()
+	code, err := ioutil.ReadAll(f)
+	if err != nil {
+		return &Error{
+			Action: "Load: Read",
+			Script: file,
+			Cause:  err,
+		}
+	}
+
+	vm.File = file
+	vm.Path = filepath.Dir(file)
+	vm.Script = tengo.NewScript(code)
+	vm.Script.SetImports(loader.GetModuleMap())
+	vm.Hash = vm.getHash()
+
+	if GCTScriptConfig.AllowImports {
+		if GCTScriptConfig.Verbose {
+			log.Debugf(log.GCTScriptMgr, "File imports enabled for vm: %v", vm.ID)
+		}
+		vm.Script.EnableFileImport(true)
+	}
+	vm.event(StatusSuccess, TypeLoad)
+	return nil
+}
+
+// Compile compiles to byte code loaded copy of vm script
+func (vm *VM) Compile() (err error) {
+	vm.Compiled = new(tengo.Compiled)
+	vm.Compiled, err = vm.Script.Compile()
+	return
+}
+
+// Run runs byte code
+func (vm *VM) Run() (err error) {
+	if GCTScriptConfig.Verbose {
+		log.Debugf(log.GCTScriptMgr, "Running script: %s ID: %v", vm.ShortName(), vm.ID)
+	}
+
+	err = vm.Compiled.Run()
+	if err != nil {
+		vm.event(StatusFailure, TypeExecute)
+		return Error{
+			Action: "Run",
+			Cause:  err,
+		}
+	}
+	vm.event(StatusSuccess, TypeExecute)
+	return
+}
+
+// RunCtx runs compiled byte code with context.Context support.
+func (vm *VM) RunCtx() (err error) {
+	if vm.ctx == nil {
+		vm.ctx = context.Background()
+	}
+
+	ct, cancel := context.WithTimeout(vm.ctx, GCTScriptConfig.ScriptTimeout)
+	defer cancel()
+
+	if GCTScriptConfig.Verbose {
+		log.Debugf(log.GCTScriptMgr, "Running script: %s ID: %v", vm.ShortName(), vm.ID)
+	}
+
+	err = vm.Compiled.RunContext(ct)
+	if err != nil {
+		vm.event(StatusFailure, TypeExecute)
+		return Error{
+			Action: "RunCtx",
+			Cause:  err,
+		}
+	}
+	vm.event(StatusSuccess, TypeExecute)
+	return
+}
+
+// CompileAndRun Compile and Run script with support for task running
+func (vm *VM) CompileAndRun() {
+	if vm == nil {
+		return
+	}
+	err := vm.Compile()
+	if err != nil {
+		log.Error(log.GCTScriptMgr, err)
+		err = RemoveVM(vm.ID)
+		if err != nil {
+			log.Error(log.GCTScriptMgr, err)
+		}
+		return
+	}
+
+	err = vm.RunCtx()
+	if err != nil {
+		log.Error(log.GCTScriptMgr, err)
+		err = RemoveVM(vm.ID)
+		if err != nil {
+			log.Error(log.GCTScriptMgr, err)
+		}
+		return
+	}
+	if vm.Compiled.Get("timer").String() != "" {
+		vm.T, err = time.ParseDuration(vm.Compiled.Get("timer").String())
+		if err != nil {
+			log.Error(log.GCTScriptMgr, err)
+			err = vm.Shutdown()
+			if err != nil {
+				log.Error(log.GCTScriptMgr, err)
+			}
+			return
+		}
+		if vm.T < time.Nanosecond {
+			log.Error(log.GCTScriptMgr, "Repeat timer cannot be under 1 nano second")
+			err = vm.Shutdown()
+			if err != nil {
+				log.Errorln(log.GCTScriptMgr, err)
+			}
+			return
+		}
+		vm.runner()
+	} else {
+		err = vm.Shutdown()
+		if err != nil {
+			log.Error(log.GCTScriptMgr, err)
+		}
+		return
+	}
+}
+
+// Shutdown shuts down current VM
+func (vm *VM) Shutdown() error {
+	if vm == nil {
+		return ErrNoVMLoaded
+	}
+	if vm.S != nil {
+		close(vm.S)
+	}
+	if GCTScriptConfig.Verbose {
+		log.Debugf(log.GCTScriptMgr, "Shutting down script: %s ID: %v", vm.ShortName(), vm.ID)
+	}
+	vm.Script = nil
+	pool.Put(vm.Script)
+	vm.event(StatusSuccess, TypeStop)
+	return RemoveVM(vm.ID)
+}
+
+// Read contents of script back and create script event
+func (vm *VM) Read() ([]byte, error) {
+	vm.event(StatusSuccess, TypeRead)
+	return vm.read()
+}
+
+// Read contents of script back
+func (vm *VM) read() ([]byte, error) {
+	if GCTScriptConfig.Verbose {
+		log.Debugf(log.GCTScriptMgr, "Read script: %s ID: %v", vm.ShortName(), vm.ID)
+	}
+	return ioutil.ReadFile(vm.File)
+}
+
+// ShortName returns short (just filename.extension) of running script
+func (vm *VM) ShortName() string {
+	return filepath.Base(vm.File)
+}
+
+func (vm *VM) event(status, executionType string) {
+	if validator.IsTestExecution.Load() == true {
+		return
+	}
+
+	var data null.Bytes
+	if executionType == TypeLoad {
+		scriptData, err := vm.scriptData()
+		if err != nil {
+			log.Errorf(log.GCTScriptMgr, "Failed to retrieve scriptData: %v", err)
+		}
+		data.SetValid(scriptData)
+	}
+	scriptevent.Event(vm.getHash(), vm.ShortName(), vm.Path, data, executionType, status, time.Now())
+}
+
+func (vm *VM) scriptData() ([]byte, error) {
+	buf := new(bytes.Buffer)
+	w := zip.NewWriter(buf)
+
+	f, err := w.Create(vm.ShortName())
+	if err != nil {
+		return []byte{}, err
+	}
+	contents, err := vm.read()
+	if err != nil {
+		return []byte{}, err
+	}
+	_, err = f.Write(contents)
+	if err != nil {
+		return []byte{}, err
+	}
+	err = w.Close()
+	if err != nil {
+		return []byte{}, err
+	}
+	return buf.Bytes(), nil
+}
+
+func (vm *VM) getHash() string {
+	if vm.Hash != "" {
+		return vm.Hash
+	}
+	contents, err := vm.read()
+	if err != nil {
+		log.Errorln(log.GCTScriptMgr, err)
+	}
+	contents = append(contents, vm.ShortName()...)
+	return hex.EncodeToString(crypto.GetSHA256(contents))
+}
+
+func (vmc *vmscount) add() {
+	atomic.AddInt32((*int32)(vmc), 1)
+}
+
+func (vmc *vmscount) remove() {
+	atomic.AddInt32((*int32)(vmc), -1)
+}
+
+// Len() returns current length vmscount
+func (vmc *vmscount) Len() int32 {
+	return atomic.LoadInt32((*int32)(vmc))
+}
--- a/gctscript/vm/vm_error.go
+++ b/gctscript/vm/vm_error.go
@@ -0,0 +1,29 @@
+package vm
+
+import (
+	"fmt"
+	"path/filepath"
+)
+
+const (
+	// ErrNoVMFound error returned when no virtual machine is found
+	ErrNoVMFound = "VM %v not found"
+)
+
+func (e Error) Error() string {
+	var scriptName, action string
+	if e.Script != "" {
+		scriptName = fmt.Sprintf("(SCRIPT) %s ", filepath.Base(e.Script))
+	}
+
+	if e.Action != "" {
+		action = fmt.Sprintf("(ACTION) %s ", e.Action)
+	}
+
+	return fmt.Sprintf("%s: %s%s%s", gctScript, action, scriptName, e.Cause)
+}
+
+// Unwrap returns e.Cause meeting errors interface requirements.
+func (e Error) Unwrap() error {
+	return e.Cause
+}
--- a/gctscript/vm/vm_task.go
+++ b/gctscript/vm/vm_task.go
@@ -0,0 +1,30 @@
+package vm
+
+import (
+	"time"
+
+	log "github.com/thrasher-corp/gocryptotrader/logger"
+)
+
+func (vm *VM) runner() {
+	vm.S = make(chan struct{}, 1)
+	waitTime := time.NewTicker(vm.T)
+	vm.NextRun = time.Now().Add(vm.T)
+
+	go func() {
+		for {
+			select {
+			case <-waitTime.C:
+				vm.NextRun = time.Now().Add(vm.T)
+				err := vm.RunCtx()
+				if err != nil {
+					log.Error(log.GCTScriptMgr, err)
+					return
+				}
+			case <-vm.S:
+				waitTime.Stop()
+				return
+			}
+		}
+	}()
+}
--- a/gctscript/vm/vm_test.go
+++ b/gctscript/vm/vm_test.go
@@ -0,0 +1,478 @@
+package vm
+
+import (
+	"errors"
+	"os"
+	"path/filepath"
+	"reflect"
+	"testing"
+	"time"
+
+	"github.com/gofrs/uuid"
+	"github.com/thrasher-corp/gocryptotrader/common/convert"
+	"github.com/thrasher-corp/gocryptotrader/logger"
+)
+
+const (
+	maxTestVirtualMachines    uint8 = 30
+	testVirtualMachineTimeout       = time.Minute
+	scriptName                      = "1D01TH0RS3.gct"
+)
+
+var (
+	testScript               = filepath.Join("..", "..", "testdata", "gctscript", "once.gct")
+	testInvalidScript        = filepath.Join("..", "..", "testdata", "gctscript", "invalid.gct")
+	testBrokenScript         = filepath.Join("..", "..", "testdata", "gctscript", "broken.gct")
+	testScriptRunner         = filepath.Join("..", "..", "testdata", "gctscript", "timer.gct")
+	testScriptRunner1s       = filepath.Join("..", "..", "testdata", "gctscript", "1s_timer.gct")
+	testScriptRunnerInvalid  = filepath.Join("..", "..", "testdata", "gctscript", "invalid_timer.gct")
+	testScriptRunnerNegative = filepath.Join("..", "..", "testdata", "gctscript", "negative_timer.gct")
+)
+
+func TestMain(m *testing.M) {
+	c := logger.GenDefaultSettings()
+	c.Enabled = convert.BoolPtr(false)
+	logger.GlobalLogConfig = &c
+	GCTScriptConfig = configHelper(true, true, maxTestVirtualMachines)
+	os.Exit(m.Run())
+}
+
+func TestNewVM(t *testing.T) {
+	x := New()
+	xType := reflect.TypeOf(x).String()
+	if xType != "*vm.VM" {
+		t.Fatalf("vm.New should return pointer to VM instead received: %v", x)
+	}
+}
+
+func TestVMLoad(t *testing.T) {
+	GCTScriptConfig = configHelper(true, true, maxTestVirtualMachines)
+	testVM := New()
+	err := testVM.Load(testScript)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	testScript = testScript[0 : len(testScript)-4]
+	testVM = New()
+	err = testVM.Load(testScript)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	GCTScriptConfig = configHelper(false, false, maxTestVirtualMachines)
+	err = testVM.Load(testScript)
+	if err != nil {
+		if !errors.Is(err, ErrScriptingDisabled) {
+			t.Fatal(err)
+		}
+	}
+	GCTScriptConfig = configHelper(true, true, maxTestVirtualMachines)
+}
+
+func TestVMLoad1s(t *testing.T) {
+	testVM := New()
+	err := testVM.Load(testScriptRunner1s)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	testVM.CompileAndRun()
+	time.Sleep(5000)
+	err = testVM.Shutdown()
+	if err != nil {
+		if !errors.Is(err, ErrNoVMLoaded) {
+			t.Fatal(err)
+		}
+	}
+}
+
+func TestVMLoadNegativeTimer(t *testing.T) {
+	testVM := New()
+	err := testVM.Load(testScriptRunnerNegative)
+	if err != nil {
+		if !errors.Is(err, ErrNoVMLoaded) {
+			t.Fatal(err)
+		}
+	}
+	testVM.CompileAndRun()
+	err = testVM.Shutdown()
+	if err == nil {
+		t.Fatal("expect error on shutdown due to invalid VM")
+	}
+}
+
+func TestVMLoadNilVM(t *testing.T) {
+	testVM := New()
+	err := testVM.Load(testScript)
+	if err != nil {
+		if !errors.Is(err, ErrNoVMLoaded) {
+			t.Fatal(err)
+		}
+	}
+	testVM = nil
+	err = testVM.Load(testScript)
+	if err != nil {
+		if !errors.Is(err, ErrNoVMLoaded) {
+			t.Fatal(err)
+		}
+	}
+}
+
+func TestCompileAndRunNilVM(t *testing.T) {
+	vmcount := VMSCount.Len()
+	testVM := New()
+	err := testVM.Load(testScript)
+	if err != nil {
+		if !errors.Is(err, ErrNoVMLoaded) {
+			t.Fatal(err)
+		}
+	}
+	err = testVM.Load(testScript)
+	if err != nil {
+		if !errors.Is(err, ErrNoVMLoaded) {
+			t.Fatal(err)
+		}
+	}
+
+	testVM = nil
+	testVM.CompileAndRun()
+	err = testVM.Shutdown()
+	if err == nil {
+		t.Fatal("VM should not be running with invalid timer")
+	}
+	if VMSCount.Len() == vmcount-1 {
+		t.Fatal("expected VM count to decrease")
+	}
+}
+
+func TestVMLoadNoFile(t *testing.T) {
+	testVM := New()
+	err := testVM.Load("missing file")
+	if err != nil {
+		if !errors.Is(err, os.ErrNotExist) {
+			t.Fatal(err)
+		}
+	}
+}
+
+func TestVMCompile(t *testing.T) {
+	testVM := New()
+	err := testVM.Load(testScript)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	err = testVM.Compile()
+	if err != nil {
+		t.Fatal(err)
+	}
+}
+
+func TestVMRun(t *testing.T) {
+	testVM := NewVM()
+	err := testVM.Load(testScript)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	err = testVM.Compile()
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	err = testVM.Run()
+	if err != nil {
+		t.Fatal(err)
+	}
+}
+
+func TestVMRunTX(t *testing.T) {
+	testVM := NewVM()
+	err := testVM.Load(testScript)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	err = testVM.Compile()
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	err = testVM.RunCtx()
+	if err != nil {
+		t.Fatal(err)
+	}
+}
+
+func TestVMWithRunner(t *testing.T) {
+	vmCount := VMSCount.Len()
+	VM := New()
+	if VM == nil {
+		t.Fatal("Failed to allocate new VM exiting")
+	}
+	err := VM.Load(testScriptRunner)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if VMSCount.Len() == vmCount {
+		t.Fatal("expected VM count to increase")
+	}
+	VM.CompileAndRun()
+	err = VM.Shutdown()
+	if err != nil {
+		t.Fatal(err)
+	}
+	if VMSCount.Len() == vmCount-1 {
+		t.Fatal("expected VM count to decrease")
+	}
+}
+
+func TestVMWithRunnerOnce(t *testing.T) {
+	vmCount := VMSCount.Len()
+	VM := New()
+	if VM == nil {
+		t.Fatal("Failed to allocate new VM exiting")
+	}
+	err := VM.Load(testScript)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if VMSCount.Len() == vmCount {
+		t.Fatal("expected VM count to increase")
+	}
+	VM.CompileAndRun()
+	err = VM.Shutdown()
+	if err == nil {
+		t.Fatal("VM should not be running with invalid timer")
+	}
+}
+
+func TestVMWithRunnerNegativeTimer(t *testing.T) {
+	vmCount := VMSCount.Len()
+	VM := New()
+	if VM == nil {
+		t.Fatal("Failed to allocate new VM exiting")
+	}
+	err := VM.Load(testScriptRunnerNegative)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if VMSCount.Len() == vmCount {
+		t.Fatal("expected VM count to increase")
+	}
+	VM.CompileAndRun()
+	err = VM.Shutdown()
+	if err == nil {
+		t.Fatal("VM should not be running with invalid timer")
+	}
+	if VMSCount.Len() == vmCount-1 {
+		t.Fatal("expected VM count to decrease")
+	}
+}
+
+func TestShutdownAll(t *testing.T) {
+	vmCount := VMSCount.Len()
+	VM := New()
+	err := VM.Load(testScriptRunner)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	VM.CompileAndRun()
+
+	if VMSCount.Len() == vmCount {
+		t.Fatal("expected VM count to increase")
+	}
+	err = ShutdownAll()
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	if VMSCount.Len() == vmCount-1 {
+		t.Fatal("expected VM count to decrease")
+	}
+}
+
+func TestRead(t *testing.T) {
+	VM := NewVM()
+	err := VM.Load(testScriptRunner)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	ScriptPath = filepath.Join("..", "..", "testdata", "gctscript")
+	data, err := VM.Read()
+	if err != nil {
+		t.Fatal(err)
+	}
+	if len(data) < 1 {
+		t.Fatal("expected data to be returned")
+	}
+	_ = VM.Shutdown()
+}
+
+func TestRemoveVM(t *testing.T) {
+	id, _ := uuid.FromString("6f20c907-64a0-48f2-848a-7837dee61672")
+	err := RemoveVM(id)
+
+	if err != nil {
+		if err.Error() != "VM 6f20c907-64a0-48f2-848a-7837dee61672 not found" {
+			t.Fatal(err)
+		}
+	}
+}
+
+func TestError_Error(t *testing.T) {
+	x := Error{
+		Script: "noscript.gct",
+		Action: "test",
+		Cause:  errors.New("HELLO ERROR"),
+	}
+
+	if x.Error() != "GCT Script: (ACTION) test (SCRIPT) noscript.gct HELLO ERROR" {
+		t.Fatal(x.Error())
+	}
+}
+
+func TestVM_CompileInvalid(t *testing.T) {
+	testVM := New()
+	err := testVM.Load(testInvalidScript)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	err = testVM.Compile()
+	if err != nil {
+		t.Fatal(err)
+	}
+	err = testVM.Run()
+	if err == nil {
+		t.Fatal("unexpected result broken script compiled successfully ")
+	}
+
+	testVM = New()
+	err = testVM.Load(testInvalidScript)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	err = testVM.Compile()
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	err = testVM.RunCtx()
+	if err == nil {
+		t.Fatal("unexpected result broken script compiled successfully ")
+	}
+
+	testVM = New()
+	err = testVM.Load(testInvalidScript)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	testVM.CompileAndRun()
+	err = testVM.Shutdown()
+	if err == nil {
+		t.Fatal("Shutdown() passed successfully but expected to fail with invalid script")
+	}
+}
+
+func TestVM_CompileBroken(t *testing.T) {
+	testVM := New()
+	err := testVM.Load(testBrokenScript)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	err = testVM.Compile()
+	if err == nil {
+		t.Fatal("unexpected result broken script compiled successfully ")
+	}
+}
+
+func TestVM_CompileAndRunBroken(t *testing.T) {
+	testVM := New()
+	err := testVM.Load(testBrokenScript)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	testVM.CompileAndRun()
+	err = testVM.Shutdown()
+	if err == nil {
+		t.Fatal("expect error on shutdown due to invalid VM")
+	}
+}
+
+func TestValidate(t *testing.T) {
+	err := Validate(testBrokenScript)
+	if err == nil {
+		t.Fatal(err)
+	}
+	err = Validate(testScript)
+	if err != nil {
+		t.Fatal(err)
+	}
+}
+
+func TestVMLimit(t *testing.T) {
+	GCTScriptConfig = configHelper(true, false, 0)
+	testVM := New()
+	if testVM != nil {
+		t.Fatal("expected nil but received pointer to VM")
+	}
+	GCTScriptConfig = configHelper(true, true, maxTestVirtualMachines)
+}
+
+func TestAutoload(t *testing.T) {
+	GCTScriptConfig = &Config{
+		Enabled: true,
+		AutoLoad: []string{
+			scriptName,
+		},
+		Verbose: true,
+	}
+
+	ScriptPath = filepath.Join("..", "..", "testdata", "gctscript")
+	err := Autoload(scriptName, true)
+	if err != nil {
+		t.Fatal(err)
+	}
+	err = Autoload(scriptName, true)
+	if err == nil {
+		t.Fatal("expected err to be script not found received nil")
+	}
+	err = Autoload("once", false)
+	if err != nil {
+		t.Fatal(err)
+	}
+	err = Autoload(scriptName, false)
+	if err == nil {
+		t.Fatal("expected err to be script not found received nil")
+	}
+}
+
+func TestVMCount(t *testing.T) {
+	var c vmscount
+	c.add()
+	if c.Len() != 1 {
+		t.Fatalf("expect c len to be 1 instead received %v", c.Len())
+	}
+	c.remove()
+	if c.Len() != 0 {
+		t.Fatalf("expect c len to be 0 instead received %v", c.Len())
+	}
+}
+
+func configHelper(enabled, imports bool, max uint8) *Config {
+	return &Config{
+		Enabled:            enabled,
+		AllowImports:       imports,
+		ScriptTimeout:      testVirtualMachineTimeout,
+		MaxVirtualMachines: max,
+		Verbose:            true,
+	}
+}
--- a/gctscript/vm/vm_types.go
+++ b/gctscript/vm/vm_types.go
@@ -0,0 +1,61 @@
+package vm
+
+import (
+	"context"
+	"sync"
+	"time"
+
+	"github.com/d5/tengo/v2"
+	"github.com/gofrs/uuid"
+)
+
+const (
+	// DefaultTimeoutValue default timeout value for virtual machines
+	DefaultTimeoutValue = 30 * time.Second
+	// DefaultMaxVirtualMachines max number of virtual machines that can be loaded at one time
+	DefaultMaxVirtualMachines uint8 = 10
+
+	// TypeLoad text to display in script_event table when a VM is loaded
+	TypeLoad = "load"
+	// TypeCreate text to display in script_event table when a VM is created
+	TypeCreate = "create"
+	// TypeExecute text to display in script_event table when a script is executed
+	TypeExecute = "execute"
+	// TypeStop text to display in script_event table when a running script is stopped
+	TypeStop = "stop"
+	// TypeRead text to display in script_event table when a script contents is read
+	TypeRead = "read"
+
+	// StatusSuccess text to display in script_event table on successful execution
+	StatusSuccess = "success"
+	// StatusFailure text to display in script_event table when script execution fails
+	StatusFailure = "failure"
+)
+
+type vmscount int32
+
+var (
+	pool = &sync.Pool{
+		New: func() interface{} {
+			return new(tengo.Script)
+		},
+	}
+	// AllVMSync stores all current Virtual Machine instances
+	AllVMSync = &sync.Map{}
+	// VMSCount running total count of Virtual Machines
+	VMSCount vmscount
+)
+
+// VM contains a pointer to "script" (precompiled source) and "compiled" (compiled byte code) instances
+type VM struct {
+	ID       uuid.UUID
+	Hash     string
+	File     string
+	Path     string
+	Script   *tengo.Script
+	Compiled *tengo.Compiled
+	ctx      context.Context
+	T        time.Duration
+	NextRun  time.Time
+	S        chan struct{}
+}