Gareth Kirwan f9437dbd08 Bitfinex: Websocket subscription improvements (#1353)
* Websockets: Add keys to websocket subscriptions

* This switches all RO uses of the mutex to use a RLock method.
* The mutex used for discrete field access has had scope drift from
  name 'connectionMutex' so rename to more appropriate fieldsMutex
* The mutex used for Set/CanUseAuthEndpoints moves from the
  subscriptions endpoint to the fieldsMutex
* Add GetSubscription by key
* Expose stream.Matcher type

* Bitfinex: Subscribe and Unsubscribe atomicly

* Fix Auth failures ignored
* This change makes it so that Subscribe and Unsubscribe wait for success
** Tells the DataHandler about errors
** Errors are returned to consumers
* Subscribes concurrently to the channels
* It also simplifies the chanId to stream mapping
* Removes unable to locate chanID: %d errors which are just noise
* Paves the way for unified channelSubscription id handling
* Adds support for subId for Book subscriptions, which is more robust

* Vastly simplifies what we need to test TestWsSubscribedResponse
This test was working to ensure that the various fancy key parsing
mechanisms all worked. Now that we use subId, we just need a thorough
test of that
* Expose Match.Set in order to capture websocket incoming data
Can't see another way of doing this. Doesn't seem too bad

* Allow tests to run with auth or WS
These flags made it difficult to run the tests whilst working on
websockets

* Enable API auth and WS in testconfig
This change minimises the changes requires for a full test run against
live endpoints, so that new contributors have a clearer testing path.
I cannot see any reason to turn WS off and Auth endpoints off when we're
not going to run API tests without Creds being set, and we're not going
to do live fire tests without canManipulateRealOrders

* TestWsSubscribe and various fixes
** Enables the websocket for live non-authed integration tests by default
** Adds an integration test for subscriptions
** Changes the Ws tests to respect canManipulateRealOrders
** Uses WsConnect instead of setupWs; fixes seqNo config not sent for WS tests
** Allows api creds to live in config/testdata.json which might be
  less likely to accidentally commit, and less obtrusive

* Bitfinex: Support period and timeframe for Candles

* Fixes manual Subscribe() symbol or key formatting
* Unifies handling of params for DefaultSubscriptions and manual
  subsrciptions

* Bitfinex: Handle conf and info WS channel events

* Bitfinex: Better tests for subscriptions

* fixup! Websockets: Add keys to websocket subscriptions

* fixup! Bitfinex: Subscribe and Unsubscribe atomicly

* fixup! Websockets: Add keys to websocket subscriptions

* Websockets: Add Pending subscription status

Add a status tracker so that Sub/Unsub can prevent duplicates,
and also fixes when first message comes before we have added the sub
to the tracker

* Websockets: Add State instead of pending

This change allows more clarity about the current state and
checks for specifically already Unsubing

* Bitfinex: Fix first sub message maybe lost

The only link we have between a sub req and the sub resp is the subID.
And the only link we have between a sub message and the sub is the chanID.
We can't derive a link using Pair or anything else.

This meant that by sending the resp and its chanID down the IncomingData
channel, we allowed the channel reader to maybe process the next
message, the first message on the channel, before the runtime executed
the switch back to subscribeToChan waiting on the chan.

To fix this, we key initially on subId.(string), and then replace it
with chanId.(int64) when we have it *inside* the wsHandleData so we
know we've procedurally handled it before the next message.

subscribeToChan is then free to remove the subId keyed Sub regardless of
error or not

If there's an error, we don't need to inline handling because there
won't be any second update.

Expands test coverage to make sure those subId keyed subscriptions are
removed.

* Websocket: Validate state in SetChanState

* fixup! Bitfinex: Fix first sub message maybe lost

* Websockets: Rename RemoveUnsuccessfulSubs

Implementation doesn't imply Unsuccessful or need to.
This change supports the registering of Pending subs

* Bitfinex: Fix race in Tests
2023-11-02 12:10:43 +11:00
2023-10-03 16:07:03 +11:00
2023-05-10 17:52:53 +10:00
2019-10-04 19:31:04 +10:00
2020-09-17 09:40:01 +10:00
2019-11-22 16:07:30 +11:00

Build Status Software License GoDoc Coverage Status Go Report Card

A cryptocurrency trading bot supporting multiple exchanges written in Golang.

Please note that this bot is under development and is not ready for production!

Community

Join our slack to discuss all things related to GoCryptoTrader! GoCryptoTrader Slack

Exchange Support Table

Exchange REST API Streaming API FIX API
Alphapoint Yes Yes NA
Binance.US Yes Yes NA
Binance Yes Yes NA
Bitfinex Yes Yes NA
Bitflyer Yes No NA
Bithumb Yes Yes NA
BitMEX Yes Yes NA
Bitstamp Yes Yes No
Bittrex Yes Yes NA
BTCMarkets Yes Yes NA
BTSE Yes Yes NA
Bybit Yes Yes NA
CoinbasePro Yes Yes No
COINUT Yes Yes NA
Exmo Yes NA NA
GateIO Yes Yes NA
Gemini Yes Yes No
HitBTC Yes Yes No
Huobi.Pro Yes Yes NA
ItBit Yes NA No
Kraken Yes Yes NA
Kucoin Yes Yes NA
Lbank Yes No NA
Okcoin Yes Yes No
Okx Yes Yes NA
Poloniex Yes Yes NA
Yobit Yes NA NA
ZB.COM Yes Yes NA

We are aiming to support the top 30 exchanges sorted by average liquidity as ranked by CoinMarketCap. However, we welcome pull requests for any exchange which does not match this criterion. If you need help with this, please join us on Slack.

** NA means not applicable as the exchange does not support the feature.

Current Features

  • Support for all exchange fiat and digital currencies, with the ability to individually toggle them on/off.
  • AES256 encrypted config file.
  • REST API support for all exchanges.
  • Websocket support for applicable exchanges.
  • Ability to turn off/on certain exchanges.
  • Communication packages (Slack, SMS via SMSGlobal, Telegram and SMTP).
  • HTTP rate limiter package.
  • Unified API for exchange usage.
  • Customisation of HTTP client features including setting a proxy, user agent and adjusting transport settings.
  • NTP client package.
  • Database support (Postgres and SQLite3). See database.
  • OTP generation tool. See gen otp.
  • Connection monitor package.
  • gRPC service and JSON RPC proxy. See gRPC service.
  • gRPC client. See gctcli.
  • Forex currency converter packages (CurrencyConverterAPI, CurrencyLayer, Exchange Rates, Fixer.io, OpenExchangeRates, Exchange Rate Host).
  • Packages for handling currency pairs, tickers and orderbooks.
  • Portfolio management tool; fetches balances from supported exchanges and allows for custom address tracking.
  • Basic event trigger system.
  • OHLCV/Candle retrieval support. See OHLCV.
  • Scripting support. See gctscript.
  • Recent and historic trade processing. See trades.
  • Backtesting application. An event-driven backtesting tool to test and iterate trading strategies using historical or custom data. See backtester.
  • WebGUI (discontinued).
  • Exchange HTTP mock testing. See mock.
  • Exchange multichain deposits and withdrawals for specific exchanges. See multichain transfer support.

Planned Features

Planned features can be found on our community Trello page.

Contribution

Please feel free to submit any pull requests or suggest any desired features to be added.

When submitting a PR, please abide by our coding guidelines:

  • Code must adhere to the official Go formatting guidelines (i.e. uses gofmt).
  • Code must be documented adhering to the official Go commentary guidelines.
  • Code must adhere to our coding style.
  • Pull requests need to be based on and opened against the master branch.

Compiling instructions

Download and install Go from Go Downloads for your platform.

Linux/OSX

GoCryptoTrader is built using Go Modules and requires Go 1.11 or above Using Go Modules you now clone this repository outside your GOPATH

git clone https://github.com/thrasher-corp/gocryptotrader.git
cd gocryptotrader
go build
mkdir ~/.gocryptotrader
cp config_example.json ~/.gocryptotrader/config.json

Windows

git clone https://github.com/thrasher-corp/gocryptotrader.git
cd gocryptotrader
go build
copy config_example.json %APPDATA%\GoCryptoTrader\config.json
  • Make any necessary changes to the config.json file.
  • Run the gocryptotrader binary file inside your GOPATH bin folder.

Donations

If this framework helped you in any way, or you would like to support the developers working on it, please donate Bitcoin to:

bc1qk0jareu4jytc0cfrhr5wgshsq8282awpavfahc

Binaries

Binaries will be published once the codebase reaches a stable condition.

Contributor List

A very special thank you to all who have contributed to this program:

User Contribution Amount
thrasher- 683
shazbert 301
gloriousCode 219
dependabot[bot] 207
dependabot-preview[bot] 88
xtda 47
gbjk 35
lrascao 27
Rots 15
vazha 15
ydm 15
ermalguni 14
MadCozBadd 13
vadimzhukck 10
140am 8
marcofranssen 8
geseq 8
Beadko 6
TaltaM 6
samuael 6
dackroyd 5
cranktakular 5
khcchiu 5
yangrq1018 4
woshidama323 3
crackcomm 3
azhang 2
andreygrehov 2
bretep 2
Christian-Achilli 2
cornelk 2
gam-phon 2
herenow 2
if1live 2
lozdog245 2
MarkDzulko 2
mshogin 2
soxipy 2
tk42 2
blombard 1
cavapoo2 1
CodeLingoTeam 1
CodeLingoBot 1
Daanikus 1
daniel-cohen 1
DirectX 1
frankzougc 1
idoall 1
Jimexist 1
lookfirst 1
m1kola 1
mattkanwisher 1
merkeld 1
mKurrels 1
starit 1
zeldrinn 1
Description
A cryptocurrency trading bot and framework supporting multiple exchanges written in Golang.
Readme MIT 63 MiB
Languages
Go 99.9%