From 3fc40292b758f094d8f3c516ca00ace34cf8a703 Mon Sep 17 00:00:00 2001 From: Adrian Gallagher Date: Tue, 15 Apr 2025 15:08:02 +1000 Subject: [PATCH] cmd/documentation: Add pagination support for contributors (#1885) * cmd/documentation: Add pagination support for contributors * Drop break and return immediately * documentation: change defaultGithubAPIPerPageLimit to an integer and update usage in GetContributorList --- CONTRIBUTORS | 69 +++++++--- README.md | 69 +++++++--- cmd/documentation/documentation.go | 170 ++++-------------------- cmd/documentation/documentation_test.go | 15 +++ engine/currency_state_manager.md | 34 ++--- 5 files changed, 159 insertions(+), 198 deletions(-) create mode 100644 cmd/documentation/documentation_test.go diff --git a/CONTRIBUTORS b/CONTRIBUTORS index 2c9cf17a..3a46e585 100644 --- a/CONTRIBUTORS +++ b/CONTRIBUTORS @@ -27,33 +27,62 @@ khcchiu | https://github.com/khcchiu yangrq1018 | https://github.com/yangrq1018 woshidama323 | https://github.com/woshidama323 crackcomm | https://github.com/crackcomm -mshogin | https://github.com/mshogin -herenow | https://github.com/herenow -tk42 | https://github.com/tk42 -andreygrehov | https://github.com/andreygrehov azhang | https://github.com/azhang -bretep | https://github.com/bretep -Christian-Achilli | https://github.com/Christian-Achilli -cornelk | https://github.com/cornelk -gam-phon | https://github.com/gam-phon if1live | https://github.com/if1live lozdog245 | https://github.com/lozdog245 -MarkDzulko | https://github.com/MarkDzulko +Asalei | https://github.com/Asalei soxipy | https://github.com/soxipy -blombard | https://github.com/blombard +tk42 | https://github.com/tk42 +herenow | https://github.com/herenow +mshogin | https://github.com/mshogin +andreygrehov | https://github.com/andreygrehov +bretep | https://github.com/bretep +Christian-Achilli | https://github.com/Christian-Achilli +dsinuela-taurus | https://github.com/dsinuela-taurus +cornelk | https://github.com/cornelk +gam-phon | https://github.com/gam-phon +MarkDzulko | https://github.com/MarkDzulko +romanornr | https://github.com/romanornr +mortensorensen | https://github.com/mortensorensen cavapoo2 | https://github.com/cavapoo2 -CodeLingoTeam | https://github.com/CodeLingoTeam +tongxiaofeng | https://github.com/tongxiaofeng +tonywangcn | https://github.com/tonywangcn +varunbhat | https://github.com/varunbhat +idealhack | https://github.com/idealhack +hannut91 | https://github.com/hannut91 +vyloy | https://github.com/vyloy +arttobe | https://github.com/arttobe +shoman4eg | https://github.com/shoman4eg +cangqiaoyuzhuo | https://github.com/cangqiaoyuzhuo +whilei | https://github.com/whilei +yuhangcangqian | https://github.com/yuhangcangqian +phieudu241 | https://github.com/phieudu241 +snipesjr | https://github.com/snipesjr +snussik | https://github.com/snussik +taewdy | https://github.com/taewdy +threehonor | https://github.com/threehonor +xiiiew | https://github.com/xiiiew +antonzhukov | https://github.com/antonzhukov +blombard | https://github.com/blombard CodeLingoBot | https://github.com/CodeLingoBot +CodeLingoTeam | https://github.com/CodeLingoTeam Daanikus | https://github.com/Daanikus daniel-cohen | https://github.com/daniel-cohen -DirectX | https://github.com/DirectX -frankzougc | https://github.com/frankzougc -idoall | https://github.com/idoall -Jimexist | https://github.com/Jimexist -lookfirst | https://github.com/lookfirst -m1kola | https://github.com/m1kola -mattkanwisher | https://github.com/mattkanwisher merkeld | https://github.com/merkeld -mKurrels | https://github.com/mKurrels -starit | https://github.com/starit +shanhuhai5739 | https://github.com/shanhuhai5739 +DirectX | https://github.com/DirectX +dnldd | https://github.com/dnldd +Juneezee | https://github.com/Juneezee +fclairamb | https://github.com/fclairamb +frankzougc | https://github.com/frankzougc +gemscng | https://github.com/gemscng +Jdpurohit | https://github.com/Jdpurohit +jimexist | https://github.com/jimexist +lookfirst | https://github.com/lookfirst zeldrinn | https://github.com/zeldrinn +mattkanwisher | https://github.com/mattkanwisher +mgravitt | https://github.com/mgravitt +mKurrels | https://github.com/mKurrels +m1kola | https://github.com/m1kola +idoall | https://github.com/idoall +starit | https://github.com/starit diff --git a/README.md b/README.md index de4a63e0..77b5f352 100644 --- a/README.md +++ b/README.md @@ -183,33 +183,62 @@ Binaries will be published once the codebase reaches a stable condition. | [yangrq1018](https://github.com/yangrq1018) | 4 | | [woshidama323](https://github.com/woshidama323) | 3 | | [crackcomm](https://github.com/crackcomm) | 3 | -| [mshogin](https://github.com/mshogin) | 2 | -| [herenow](https://github.com/herenow) | 2 | -| [tk42](https://github.com/tk42) | 2 | -| [andreygrehov](https://github.com/andreygrehov) | 2 | | [azhang](https://github.com/azhang) | 2 | -| [bretep](https://github.com/bretep) | 2 | -| [Christian-Achilli](https://github.com/Christian-Achilli) | 2 | -| [cornelk](https://github.com/cornelk) | 2 | -| [gam-phon](https://github.com/gam-phon) | 2 | | [if1live](https://github.com/if1live) | 2 | | [lozdog245](https://github.com/lozdog245) | 2 | -| [MarkDzulko](https://github.com/MarkDzulko) | 2 | +| [Asalei](https://github.com/Asalei) | 2 | | [soxipy](https://github.com/soxipy) | 2 | -| [blombard](https://github.com/blombard) | 1 | +| [tk42](https://github.com/tk42) | 2 | +| [herenow](https://github.com/herenow) | 2 | +| [mshogin](https://github.com/mshogin) | 2 | +| [andreygrehov](https://github.com/andreygrehov) | 2 | +| [bretep](https://github.com/bretep) | 2 | +| [Christian-Achilli](https://github.com/Christian-Achilli) | 2 | +| [dsinuela-taurus](https://github.com/dsinuela-taurus) | 2 | +| [cornelk](https://github.com/cornelk) | 2 | +| [gam-phon](https://github.com/gam-phon) | 2 | +| [MarkDzulko](https://github.com/MarkDzulko) | 2 | +| [romanornr](https://github.com/romanornr) | 2 | +| [mortensorensen](https://github.com/mortensorensen) | 1 | | [cavapoo2](https://github.com/cavapoo2) | 1 | -| [CodeLingoTeam](https://github.com/CodeLingoTeam) | 1 | +| [tongxiaofeng](https://github.com/tongxiaofeng) | 1 | +| [tonywangcn](https://github.com/tonywangcn) | 1 | +| [varunbhat](https://github.com/varunbhat) | 1 | +| [idealhack](https://github.com/idealhack) | 1 | +| [hannut91](https://github.com/hannut91) | 1 | +| [vyloy](https://github.com/vyloy) | 1 | +| [arttobe](https://github.com/arttobe) | 1 | +| [shoman4eg](https://github.com/shoman4eg) | 1 | +| [cangqiaoyuzhuo](https://github.com/cangqiaoyuzhuo) | 1 | +| [whilei](https://github.com/whilei) | 1 | +| [yuhangcangqian](https://github.com/yuhangcangqian) | 1 | +| [phieudu241](https://github.com/phieudu241) | 1 | +| [snipesjr](https://github.com/snipesjr) | 1 | +| [snussik](https://github.com/snussik) | 1 | +| [taewdy](https://github.com/taewdy) | 1 | +| [threehonor](https://github.com/threehonor) | 1 | +| [xiiiew](https://github.com/xiiiew) | 1 | +| [antonzhukov](https://github.com/antonzhukov) | 1 | +| [blombard](https://github.com/blombard) | 1 | | [CodeLingoBot](https://github.com/CodeLingoBot) | 1 | +| [CodeLingoTeam](https://github.com/CodeLingoTeam) | 1 | | [Daanikus](https://github.com/Daanikus) | 1 | | [daniel-cohen](https://github.com/daniel-cohen) | 1 | -| [DirectX](https://github.com/DirectX) | 1 | -| [frankzougc](https://github.com/frankzougc) | 1 | -| [idoall](https://github.com/idoall) | 1 | -| [Jimexist](https://github.com/Jimexist) | 1 | -| [lookfirst](https://github.com/lookfirst) | 1 | -| [m1kola](https://github.com/m1kola) | 1 | -| [mattkanwisher](https://github.com/mattkanwisher) | 1 | | [merkeld](https://github.com/merkeld) | 1 | -| [mKurrels](https://github.com/mKurrels) | 1 | -| [starit](https://github.com/starit) | 1 | +| [shanhuhai5739](https://github.com/shanhuhai5739) | 1 | +| [DirectX](https://github.com/DirectX) | 1 | +| [dnldd](https://github.com/dnldd) | 1 | +| [Juneezee](https://github.com/Juneezee) | 1 | +| [fclairamb](https://github.com/fclairamb) | 1 | +| [frankzougc](https://github.com/frankzougc) | 1 | +| [gemscng](https://github.com/gemscng) | 1 | +| [Jdpurohit](https://github.com/Jdpurohit) | 1 | +| [jimexist](https://github.com/jimexist) | 1 | +| [lookfirst](https://github.com/lookfirst) | 1 | | [zeldrinn](https://github.com/zeldrinn) | 1 | +| [mattkanwisher](https://github.com/mattkanwisher) | 1 | +| [mgravitt](https://github.com/mgravitt) | 1 | +| [mKurrels](https://github.com/mKurrels) | 1 | +| [m1kola](https://github.com/m1kola) | 1 | +| [idoall](https://github.com/idoall) | 1 | +| [starit](https://github.com/starit) | 1 | diff --git a/cmd/documentation/documentation.go b/cmd/documentation/documentation.go index efbaf378..a40c10b4 100644 --- a/cmd/documentation/documentation.go +++ b/cmd/documentation/documentation.go @@ -7,10 +7,12 @@ import ( "fmt" "log" "net/http" + "net/url" "os" "path/filepath" "slices" "sort" + "strconv" "strings" "text/template" "time" @@ -36,6 +38,8 @@ const ( // ContributorFile defines contributor file ContributorFile = "CONTRIBUTORS" + + defaultGithubAPIPerPageLimit = 100 ) var ( @@ -168,99 +172,14 @@ func main() { if verbose { fmt.Println("Fetching repository contributor list...") } - contributors, err = GetContributorList(config.GithubRepo, verbose) + contributors, err = GetContributorList(context.TODO(), config.GithubRepo, verbose) if err != nil { log.Fatalf("Documentation Generation Tool - GetContributorList error %s", err) } - // Github API missing contributors + // Github API missing/deleted user contributors contributors = append(contributors, []Contributor{ - { - Login: "andreygrehov", - URL: "https://github.com/andreygrehov", - Contributions: 2, - }, - { - Login: "azhang", - URL: "https://github.com/azhang", - Contributions: 2, - }, - { - Login: "bretep", - URL: "https://github.com/bretep", - Contributions: 2, - }, - { - Login: "Christian-Achilli", - URL: "https://github.com/Christian-Achilli", - Contributions: 2, - }, - { - Login: "cornelk", - URL: "https://github.com/cornelk", - Contributions: 2, - }, - { - Login: "gam-phon", - URL: "https://github.com/gam-phon", - Contributions: 2, - }, - { - Login: "if1live", - URL: "https://github.com/if1live", - Contributions: 2, - }, - { - Login: "lozdog245", - URL: "https://github.com/lozdog245", - Contributions: 2, - }, - { - Login: "MarkDzulko", - URL: "https://github.com/MarkDzulko", - Contributions: 2, - }, - { - Login: "blombard", - URL: "https://github.com/blombard", - Contributions: 1, - }, - { - Login: "cavapoo2", - URL: "https://github.com/cavapoo2", - Contributions: 1, - }, - { - Login: "CodeLingoTeam", - URL: "https://github.com/CodeLingoTeam", - Contributions: 1, - }, - { - Login: "CodeLingoBot", - URL: "https://github.com/CodeLingoBot", - Contributions: 1, - }, - { - Login: "Daanikus", - URL: "https://github.com/Daanikus", - Contributions: 1, - }, - { - Login: "daniel-cohen", - URL: "https://github.com/daniel-cohen", - Contributions: 1, - }, - { - Login: "DirectX", - URL: "https://github.com/DirectX", - Contributions: 1, - }, - { - Login: "frankzougc", - URL: "https://github.com/frankzougc", - Contributions: 1, - }, // idoall's contributors were forked and merged, so his contributions // aren't automatically retrievable { @@ -268,51 +187,11 @@ func main() { URL: "https://github.com/idoall", Contributions: 1, }, - { - Login: "Jimexist", - URL: "https://github.com/Jimexist", - Contributions: 1, - }, - { - Login: "lookfirst", - URL: "https://github.com/lookfirst", - Contributions: 1, - }, - { - Login: "m1kola", - URL: "https://github.com/m1kola", - Contributions: 1, - }, - { - Login: "mattkanwisher", - URL: "https://github.com/mattkanwisher", - Contributions: 1, - }, - { - Login: "merkeld", - URL: "https://github.com/merkeld", - Contributions: 1, - }, - { - Login: "mKurrels", - URL: "https://github.com/mKurrels", - Contributions: 1, - }, - { - Login: "soxipy", - URL: "https://github.com/soxipy", - Contributions: 2, - }, { Login: "starit", URL: "https://github.com/starit", Contributions: 1, }, - { - Login: "zeldrinn", - URL: "https://github.com/zeldrinn", - Contributions: 1, - }, }...) sort.Slice(contributors, func(i, j int) bool { @@ -469,21 +348,30 @@ func GetTemplateFiles() (*template.Template, error) { return tmpl, filepath.Walk(toolDir, walkFn) } -// GetContributorList fetches a list of contributors from the github api -// endpoint -func GetContributorList(repo string, verbose bool) ([]Contributor, error) { - contents, err := common.SendHTTPRequest(context.TODO(), - http.MethodGet, - repo+GithubAPIEndpoint, - nil, - nil, - verbose) - if err != nil { - return nil, err - } +// GetContributorList fetches a list of contributors from the Github API endpoint +func GetContributorList(ctx context.Context, repo string, verbose bool) ([]Contributor, error) { + var contributors []Contributor + vals := url.Values{} + vals.Set("per_page", strconv.Itoa(defaultGithubAPIPerPageLimit)) - var resp []Contributor - return resp, json.Unmarshal(contents, &resp) + for page := 1; ; page++ { + vals.Set("page", strconv.Itoa(page)) + + contents, err := common.SendHTTPRequest(ctx, http.MethodGet, common.EncodeURLValues(repo+GithubAPIEndpoint, vals), nil, nil, verbose) + if err != nil { + return nil, err + } + + var resp []Contributor + if err := json.Unmarshal(contents, &resp); err != nil { + return nil, err + } + + contributors = append(contributors, resp...) + if len(resp) < defaultGithubAPIPerPageLimit { + return contributors, nil + } + } } // GetDocumentationAttributes returns specific attributes for a file template diff --git a/cmd/documentation/documentation_test.go b/cmd/documentation/documentation_test.go new file mode 100644 index 00000000..433c5449 --- /dev/null +++ b/cmd/documentation/documentation_test.go @@ -0,0 +1,15 @@ +package main + +import ( + "testing" + + "github.com/stretchr/testify/require" +) + +func TestGetContributorList(t *testing.T) { + t.Parallel() + + c, err := GetContributorList(t.Context(), DefaultRepo, true) + require.NoError(t, err, "GetContributorList must not error") + require.NotEmpty(t, c, "GetContributorList must not return empty list") +} diff --git a/engine/currency_state_manager.md b/engine/currency_state_manager.md index 908f2ed8..f1ad8170 100644 --- a/engine/currency_state_manager.md +++ b/engine/currency_state_manager.md @@ -27,22 +27,22 @@ Join our slack to discuss all things related to GoCryptoTrader! [GoCryptoTrader + This allows for an internal state check to compliment internal and external strategies. - -## Contribution - -Please feel free to submit any pull requests or suggest any desired features to be added. - -When submitting a PR, please abide by our coding guidelines: - -+ Code must adhere to the official Go [formatting](https://golang.org/doc/effective_go.html#formatting) guidelines (i.e. uses [gofmt](https://golang.org/cmd/gofmt/)). -+ Code must be documented adhering to the official Go [commentary](https://golang.org/doc/effective_go.html#commentary) guidelines. -+ Code must adhere to our [coding style](https://github.com/thrasher-corp/gocryptotrader/blob/master/doc/coding_style.md). -+ Pull requests need to be based on and opened against the `master` branch. -## Donations - - - -If this framework helped you in any way, or you would like to support the developers working on it, please donate Bitcoin to: - +## Contribution + +Please feel free to submit any pull requests or suggest any desired features to be added. + +When submitting a PR, please abide by our coding guidelines: + ++ Code must adhere to the official Go [formatting](https://golang.org/doc/effective_go.html#formatting) guidelines (i.e. uses [gofmt](https://golang.org/cmd/gofmt/)). ++ Code must be documented adhering to the official Go [commentary](https://golang.org/doc/effective_go.html#commentary) guidelines. ++ Code must adhere to our [coding style](https://github.com/thrasher-corp/gocryptotrader/blob/master/doc/coding_style.md). ++ Pull requests need to be based on and opened against the `master` branch. + +## Donations + + + +If this framework helped you in any way, or you would like to support the developers working on it, please donate Bitcoin to: + ***bc1qk0jareu4jytc0cfrhr5wgshsq8282awpavfahc***