Commit Graph

80 Commits

Author SHA1 Message Date
James Loh
b2a3d75490 Caddy: Migrate redirects to placeholders
- We want customers to now copy the example Caddyfile so this still gives people the option but also makes it easier for customers who don't want to redirect themselves
2025-07-15 16:21:12 +10:00
James Loh
8d0d565df9 Caddy: Move to more templated approach
no ref

- The goal here is to be able to provide more functionality to self-hosters through snippets and other segmented config
- Some customers run Admin <-> content domains on separate ones which our current config doesn't support
- Our current config also hardcodes a www redirect which complicates setups when you don't have that domain setup or don't even want it
- Moving to a default template customers will have to copy which includes snippets allows us to update these later on without breaking peoples setups
2025-07-15 15:44:28 +10:00
James Loh
190a350bd5 AP: Proxy requests to shared Pro infra
no ref

- We're opening our AP setup to self-hosters as well now which saves them running the AP setup locally
2025-07-15 14:50:50 +10:00
Chris Raible
a752641576 Added traffic analytics configuration for file based salt store (#26)
ref https://linear.app/ghost/issue/PROD-2300/add-file-based-salt-store-for-self-hosters

In the analytics service, we generate user signatures based on a randomly generated salt per site_uuid. The salts are regenerated every day at midnight UTC so we can't pass a static value, and they need to be persisted throughout the day so the same user (based on IP and user agent) will generate the same signature in any given day.

We use an adapter-like pattern for the persistence of the salts — the default is a MemoryStore intended for local development, and we use Firestore in production. The MemoryStore appears to work well enough, but if the analytics service container is rebooted (i.e. during an update), all the salts from the current day will be lost, and it will skew the "unique visitors" and related metrics.

This PR adds the configuration to use a simple file based store for the salts, and a volume mounted into the Traffic Analytics service for persistent storage. This was the salts are persisted across container reboots, which avoids skewing the unique visitors metrics.

There are no additional setup steps required — all sites can use the same configuration, and the volumes are managed by docker compose.
2025-07-14 19:28:31 -07:00
renovate[bot]
aa46049727 Update ghost/traffic-analytics Docker tag to v1.0.1 (#28)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-07-15 02:26:42 +00:00
James Loh
b76e2a9ef4 Simplify initial setup by not include --profile arg
no ref

- As Chris notes this isn't actually required when doing the initial setup/mentioning the specific container you want to deploy
- This standardises us in just doing the `--rm run` command for all of them to keep everything nice and neat
2025-07-15 11:40:53 +10:00
James Loh
43891c65b5 Simplify TB initial setup
no ref

- By outputting the tokens in the format the `.env` file expects this greatly simplifies the initial setup by just makign it copy -> paste able
- This will mean users don't need to load up the UI and copy/paste tokens at all and can do the whole setup (after they've signed up for TB) from the CLI
2025-07-15 11:40:53 +10:00
Chris Raible
2f310b7ddc Added experimental script to automate fetching Tinybird tokens 2025-07-15 11:40:53 +10:00
Chris Raible
62ff21363f Added check for interactive mode in tinybird-login service 2025-07-15 11:28:49 +10:00
James Loh
e994f882b3 Tweak README with new installation steps 2025-07-14 16:21:08 +10:00
Chris Raible
90c56742bd Updated .env.example 2025-07-14 16:21:08 +10:00
Chris Raible
ee03de0a27 --wip-- [skip ci] 2025-07-14 16:21:08 +10:00
Chris Raible
26944f0a0a Updated Ghost config and Tinybird instructions to replace stats token with JWTs 2025-07-14 16:21:08 +10:00
renovate[bot]
880f6f1d29 Pin dependencies (#25)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-07-14 03:50:33 +00:00
James Loh
31f0c5a7d7 AP: Add docker images to auto-merge config
no ref

- Since we've created v1 now we can auto-merge new updates + pin digests
- This helps us ensure customers are running the specific version we want
2025-07-14 13:42:36 +10:00
James Loh
d06485b4d4 AP: Tag specific version along with migrate
- AP migration should be tagged along with the main AP instance to keep things in line
  - Renovate will now group them together so it bumps them at the same time in the same PR/commit etc to keep things aligned
- Moving to specific version pinning as well to make the renovate diffs clearer
2025-07-14 13:42:36 +10:00
Fabien O'Carroll
615b60fbb8 Bumped ActivityPub image to v1 2025-07-14 13:42:36 +10:00
renovate[bot]
c6ac2b7fd8 Pin ghost/traffic-analytics Docker tag to c93660c (#24)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-07-14 02:43:19 +00:00
James Loh
3b31abe46d TA: Setup auto-merge via Renovate
no ref

- This just makes updates simpler by having Renovate handle them for us via commits
- This means it'll auto bump all minor + match updates to TA images but _not_ major and it'll instead open up a PR
2025-07-14 12:39:35 +10:00
James Loh
63660bc207 TA: Pin to specific version
no ref

- This just makes the renovate difs clearer as to what version its bumping it to and from
2025-07-14 12:39:35 +10:00
Chris Raible
e526b047f3 Bumped traffic analytics to v1 2025-07-14 12:39:35 +10:00
James Loh
6dbf3e4b9a Fix tests not running on renovate branches
no ref

- This prevented renovate auto-merging the digest pin commits
2025-07-14 11:16:22 +10:00
renovate[bot]
19ebc03bdb Pin dependencies 2025-07-10 16:42:09 +10:00
James Loh
91a279fae9 Renovate: Slightly tweak config
ref

- Use best-practices instead of recommended as its the newer default
- Don't rate limit, just slam us
- Use branch merges over PRs where told to
- Pin all deps not just dev ones
- Everything else that was removed is now just included via `config:best-practices`
2025-07-10 16:36:51 +10:00
James Loh
e24893139b Pin to specific versions of packages to make renovate clearer 2025-07-10 16:30:23 +10:00
James Loh
c07d7cf7cf Don't pin AP Migration container as well 2025-07-10 16:26:23 +10:00
James Loh
3d8e896403 Restrict MySQL version to 8.0.x
no ref

- We officially only support MySQL v8, not 8.4 or 9, so can't go above this
2025-07-10 16:23:01 +10:00
James Loh
a081853573 Disable Major docker upgrades
no ref

- At the moment this would try and upgrade us to MySQL v9 which we don't officially support
2025-07-10 16:23:01 +10:00
James Loh
eac2324292 Start setting up initial renovate config 2025-07-10 16:23:01 +10:00
James Loh
afa3bb1ea9 Update AP with additional endpoints and make default proxying easier 2025-07-10 16:04:59 +10:00
Fabien O'Carroll
6e2f94f25d Updating ActivityPub paths
We do not support running on a subdirectory
We need to proxy webfinger requests for AP to work
2025-07-10 16:04:59 +10:00
Chris Raible
6e5400acb5 Updated Tinybird tracker endpoint to use versioned API endpoint (#17)
ref https://linear.app/ghost/issue/PROD-2270/update-the-ghost-docker-repo-to-use-the-apiv1page-hit-endpoint

The Analytics Service is now using a new versioned endpoint at `/api/v1/page_hit` instead of the `/tb/web_analytics` endpoint. This update's Ghost's configuration to use this new endpoint for the tracker requests.

Note: this will require a docker compose pull to get the latest version of the analytics service, which includes the new versioned endpoint.
2025-07-09 20:53:40 -07:00
James Loh
1630a4e60f Slightly tweak wording after feedback
Closes https://linear.app/ghost/issue/ENG-2435
2025-07-10 13:13:19 +10:00
James Loh
cad2d17db5 Ghost: Added some very basic mail setup placeholders
ref https://linear.app/ghost/issue/ENG-2435

- Mail must be setup to have a proper functioning Ghost install
- Bulk mail doesn't require env variables but standard SMTP mail does
- We'll update the docs but prompting users in the example env file is a starting point
2025-07-10 13:11:09 +10:00
James Loh
bb09cd2c2a Start MySQL container earlier on in migration
- The idea here is this can start up and do its thing in the background whilst we're rsync'ing content and saves us waiting longer later on
2025-07-09 17:15:30 +10:00
James Loh
511a50079f Add some additional help commands and a help script
no ref

- This should help users with the 'What next?' question at the end of their migration and gives them something to come back to once done
2025-07-09 17:15:30 +10:00
James Loh
87b0b3a556 Implemented some automatic search for Ghost installations
no ref

- Our docs suggest installation Ghost by default in `/var/www/` so search one level deep for Ghost installations
- If we find any, prompt the user if any of these are the ones they want to migrate otherwise let users prompt for a custom dir
2025-07-09 17:02:35 +10:00
James Loh
c18cc88f7f Make rsync progress quieter
no ref

- rsync will print us out a nice progress screen if we lower the verbosity which is nicer than outputting all of the files we're moving
2025-07-09 17:02:35 +10:00
James Loh
8411a38ccc Silence MySQL password errors
no ref

- Using a passsord on the CLI causes MySQL to print 'mysql: [Warning] Using a password on the command line interface can be insecure.' which doing via environment variables prevents
- This also fixes our other `cut` usages in the event the string has a
  `=` in it not printing the whole string
2025-07-09 17:02:35 +10:00
James Loh
c9b43fb74e Silence stopping Nginx
no ref

- No reason beyond making things quieter
2025-07-09 15:16:43 +10:00
James Loh
7f332ff9ac Start exposing Ghost on localhost and template port
no ref

- The thinking here is this allows users to more easily run multiple Ghost instances on the same server by templating the Ghost port
- Majority wont need this but it also simplifies the message around having to forward traffic if you don't start Caddy
2025-07-09 15:16:43 +10:00
James Loh
7bb2eec30b Don't prompt users to import configuration
no ref

- 99% of users will want to import their configuration by default so just do it and let users know
- We now just print the config we're importing then continue on
2025-07-09 15:16:43 +10:00
James Loh
0a64820e78 Migrate script: Copy and wording updates
no ref
2025-07-09 15:16:43 +10:00
Chris Raible
1585fe040f Moved Tinybird tracker token to the analytics service 2025-07-08 17:34:46 -07:00
James Loh
6890f5c4c6 Improved MySQL migration system
no ref

- Currently we prompt a user for a username to dump their Ghost database with which can be confusing if users can't remember their database details
- The Ghost user _likely_ has the required permissions to dump the database itself so we can first try that without prompting the user and _then_ prompt them if it doesn't work
- We added `--no-tablespaces` to the dump command to try and reduce the likelyhood of hitting users not having the `PROCESS` permission since most hosted DBs don't let this and Ghost's DB setup doesn't require DBs be dumped with that anyway
2025-07-08 15:57:13 +10:00
James Loh
fb8703d862 Update CLAUDE.md to latest version 2025-07-08 14:57:49 +10:00
James Loh
60c018371e Redo MySQL healthcheck to be more generic
no ref

- This follows a more Docker standard approach by using MySQL admin's `ping` tool
- `start_interval` and `start_period` are required just to give MySQL a bit of time to start up and create the initial DBs on first boot
- We need to use `$$` to escape the variable in the compose file so it
  goes through to the healthcheck as a variable to expanded there
2025-07-08 14:25:13 +10:00
James Loh
9563a311be Added in yaml schema to aid IDE completion
no ref
2025-07-08 14:16:24 +10:00
James Loh
f384c7b906 TB: Move configuration to UI instead of env
- TB can now be toggled via the UI and setting it via env variables means the setting can never be toggled on/off properly
- Doing it this way lets users control things more
2025-07-03 17:56:11 +10:00
James Loh
22af0fb447 TB: Remove requirement for UUID setting
Closes https://linear.app/ghost/issue/ENG-2442/
ref e97d6c027c

- Ghost used to require this to set/manage a UUID in TB/clickhouse but we now use Ghost's generated one directly, instead of requiring it here
- Cleaning this up because otherwise if they don't match the UI and the tracking events don't go to the same bucket
2025-07-03 14:38:33 +10:00