Adopt a8c-secrets for encrypted secrets - #4731
Conversation
| # | ||
| lane :configure_secrets do | ||
| unless FastlaneCore::CommandExecutor.which('a8c-secrets') | ||
| UI.user_error!('`a8c-secrets` was not found on your PATH. See https://github.com/Automattic/a8c-secrets for installation and usage instructions.') |
There was a problem hiding this comment.
Earlier iterations had more detailed instructions. Apart from being uglier to look at in code, they seemed like a liability. Better to use the project's README.md as the single source of truth.
| "$(dirname "${BASH_SOURCE[0]}")/shared_setup.sh" | ||
|
|
||
| echo "--- :closed_lock_with_key: Installing Secrets" | ||
| bundle exec fastlane run configure_apply | ||
|
|
There was a problem hiding this comment.
Notice the removal of the decrypt call because now it runs as part of the shared setup script.
| FIREBASE_SECRETS_PATH = $(A8C_SECRETS_DIR)/GoogleService-Info.plist | ||
| SECRETS_PATH = $(A8C_SECRETS_DIR)/pocket_casts_credentials.json | ||
| GOOGLE_SIGN_IN_SECRETS_PATH = $(A8C_SECRETS_DIR)/GoogleSignIn.txt |
There was a problem hiding this comment.
These pointed to ~/.configure/... before. DRYed them here. While we can't rely on a8c-secrets which at this level (and setting up an aggregate target + build phase to run it seems a bit too much) at least we can centralize the place where the path is hardcoded.
There was a problem hiding this comment.
Pull request overview
Migrates the iOS app’s secret management from the legacy mobile-secrets/.configure flow to a8c-secrets, updating local build configuration, Fastlane automation, and Buildkite pipelines to decrypt and consume secrets from the new location.
Changes:
- Introduces a
configure_secretsFastlane lane and wires it intobefore_allto ensure secrets are decrypted for lanes that need them. - Moves Xcode secret path build settings into
config/PocketCasts.base.xcconfig(pointing at~/.a8c-secrets/...) and updates scripts/messages accordingly. - Updates Buildkite pipelines to run a new
install-secrets.shhelper instead ofconfigure_apply, and removes the legacy.configure/.configure-filesartifacts from the repo.
Reviewed changes
Copilot reviewed 26 out of 35 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| scripts/build-phases/generate-credentials.sh | Updates guidance for missing secrets; continues generating credentials/GoogleService plist from decrypted secrets. |
| podcasts/Credentials/replace_secrets.rb | Updates failure guidance to point to the new secrets lane. |
| podcasts.xcodeproj/project.pbxproj | Removes hardcoded .configure secrets paths from build settings. |
| Makefile | Updates dSYM upload to locate Firebase plist via a8c-secrets which. |
| fastlane/Fastfile | Adds configure_secrets lane and runs it from before_all. |
| config/PocketCasts.base.xcconfig | Defines A8C_SECRETS_DIR and related secret path build settings. |
| .gitignore | Removes legacy .configure-files ignore rules. |
| .github/CODEOWNERS | Updates secrets ownership path to .a8c-secrets/. |
| .gitattributes | Treats .a8c-secrets/**/*.age as binary. |
| .configure-files/fastlane_match_pwd.txt.enc | Removes legacy encrypted secret artifact from repo. |
| .configure | Removes legacy mobile-secrets configuration file from repo. |
| .buildkite/release-pipelines/* | Switches pipelines to use the new secrets installation helper. |
| .buildkite/commands/*.sh | Routes shared setup/release steps through install-secrets.sh and removes old configure_apply calls. |
| .buildkite/commands/install-secrets.sh | New helper to install a8c-secrets and run fastlane configure_secrets. |
| .a8c-secrets/repo-id | Adds repo identifier for a8c-secrets. |
| .a8c-secrets/keys.pub | Adds public age recipients (dev/ci). |
| set -euo pipefail | ||
|
|
||
| echo "--- :closed_lock_with_key: Installing Secrets" | ||
| curl -fsSL https://raw.githubusercontent.com/Automattic/a8c-secrets/main/install.sh | bash |
There was a problem hiding this comment.
We should definitively install a8c-secrets inside all our CI agents at some point and reprovision them, so that repos adopting of a8c-secrets don't have to do this curl at all.
I see 2 solutions in the short term:
- Either bite the bullet already and install
a8c-secretsinside all our agents before going further. Technically that shouldn't be that complicated (adding a shared Ansible role in our buildkite-ci repo that downloads and install the right binary based on architecture, then invoke that role in all our agents) - Or accept that we'll temporarily have our scripts download and install it via
curlduring each CI job, but in that case I agree with Copilot that we should also verify the expected checksum.- Sure one could say "eh there's little change of supply-chain attack because this
curl'd URL is hosted by a repository owned by a8c in the first place" - But since this is a tool related to handling secrets, I'd still prefer for us to be extra careful and add more checks than less when it comes to security
- And the argument like "this is temporary, as we'll soon remove this to use the one pre-installed in the agent" is not acceptable IMHO, because we all know how this kind of "this is temporary" cases can go
- Sure one could say "eh there's little change of supply-chain attack because this
Besides, writing all that made me realize that even if we re-provision all our Agents with Ansible to include a8c-secrets pre-installed, this won't cover the case of jobs running inside Mac VMs, unless we update hostmgr to make it mount a8c-secrets as a shared volume inside the VM and add that shared volume into the $PATH (which would IMHO be a better approach than having to rebuild our VM images just to include a8c-secrets binary in them, and would also allow us to update a8c-secrets on our Mac hosts without having to rebuild VMs every time we update it)
| if [ ! -f $SECRETS_PATH ]; then | ||
| echo "error: $SECRETS_PATH not found! Please run \`bundle exec fastlane run configure_apply\`." | ||
| echo "error: $SECRETS_PATH not found! Please run \`bundle exec fastlane configure_secrets\`." |
|
| App Name | Pocket Casts Prototype Build | |
| Build Number | 16480 | |
| Version | PR #4731 | |
| Bundle ID | au.com.shiftyjelly.podcasts.prototype | |
| Commit | 5004748 | |
| Installation URL | 63bj7nueldc20 |
AliSoftware
left a comment
There was a problem hiding this comment.
I'm at the end of my day so won't have the time to properly run the test steps until tomorrow at the earliest, but overall the code logic looks good so far 👍
| set -euo pipefail | ||
|
|
||
| echo "--- :closed_lock_with_key: Installing Secrets" | ||
| curl -fsSL https://raw.githubusercontent.com/Automattic/a8c-secrets/main/install.sh | bash |
There was a problem hiding this comment.
We should definitively install a8c-secrets inside all our CI agents at some point and reprovision them, so that repos adopting of a8c-secrets don't have to do this curl at all.
I see 2 solutions in the short term:
- Either bite the bullet already and install
a8c-secretsinside all our agents before going further. Technically that shouldn't be that complicated (adding a shared Ansible role in our buildkite-ci repo that downloads and install the right binary based on architecture, then invoke that role in all our agents) - Or accept that we'll temporarily have our scripts download and install it via
curlduring each CI job, but in that case I agree with Copilot that we should also verify the expected checksum.- Sure one could say "eh there's little change of supply-chain attack because this
curl'd URL is hosted by a repository owned by a8c in the first place" - But since this is a tool related to handling secrets, I'd still prefer for us to be extra careful and add more checks than less when it comes to security
- And the argument like "this is temporary, as we'll soon remove this to use the one pre-installed in the agent" is not acceptable IMHO, because we all know how this kind of "this is temporary" cases can go
- Sure one could say "eh there's little change of supply-chain attack because this
Besides, writing all that made me realize that even if we re-provision all our Agents with Ansible to include a8c-secrets pre-installed, this won't cover the case of jobs running inside Mac VMs, unless we update hostmgr to make it mount a8c-secrets as a shared volume inside the VM and add that shared volume into the $PATH (which would IMHO be a better approach than having to rebuild our VM images just to include a8c-secrets binary in them, and would also allow us to update a8c-secrets on our Mac hosts without having to rebuild VMs every time we update it)
| # Decrypt the app secrets into the local `a8c-secrets` directory the build reads from. | ||
| # | ||
| lane :configure_secrets do | ||
| unless FastlaneCore::CommandExecutor.which('a8c-secrets') | ||
| if is_ci? | ||
| UI.important('`a8c-secrets` was not found on the PATH, but given CI execution was detected, we assume secrets has already been decrypted. See https://linear.app/a8c/issue/AINFRA-2681.') | ||
| next | ||
| else | ||
| UI.user_error!('`a8c-secrets` was not found on your PATH. See https://github.com/Automattic/a8c-secrets for installation and usage instructions.') | ||
| end | ||
| end | ||
|
|
||
| sh('a8c-secrets', 'decrypt') | ||
| end | ||
|
|
There was a problem hiding this comment.
I wonder if it would be worth it on the long term to create a dedicated Fastlane Action in releases-toolkit that would take care of (1) installing a8c-secrets binary if not present (providing a version and shasum) + (2) doing this validation dance + finally call decrypt…
This way we could DRY the setup across repos.
The drawback is that now we'd be relying on Ruby and Fastlane to do all that (including the install) while (1) some jobs might not require Ruby to run but might still need secrets, so installing and decrypting secrets directly via bash helpers in .buildkite/commands like you did is more universal and doesn't require to install Ruby and Fastlane just for that (2) so we'd still have 2 sources of truth to maintain…
So maybe not such a good idea after all?
There was a problem hiding this comment.
The thought of moving to release-toolkit crossed my mind also but haven't had the time to work on it yet.
That's probably for the best, because the points you raise on adding an unnecessary dependency on Ruby tooling are strong. Add to that the desire to have less of the devs workflow depend on Ruby, and pushing to release-toolkit would be shooting ourselves in the foot and create more migration work for later.
I actually like the suggestion you left below to push the checks to the build phase(s) that need the secrets.
| if [ ! -f $SECRETS_PATH ]; then | ||
| echo "error: $SECRETS_PATH not found! Please run \`bundle exec fastlane run configure_apply\`." | ||
| if [ ! -f "$SECRETS_PATH" ]; then | ||
| echo "error: $SECRETS_PATH not found! Please run \`bundle exec fastlane configure_secrets\`." |
There was a problem hiding this comment.
While I like that we have a configure_secrets lane that wraps the call to a8c-secrets decrypt in order to providing better error messages if the binary is not found before calling it, and in order for it to be automatically called before each lane is executed… the whole idea behind migrating to a Rust-based a8c-secrets was to avoid needing Ruby and for users to be able to run a8c-secrets binary with verbs that were sensible and easy to understand.
I wonder if nudging people to call fastlane configure_secrets (which effectively only calls a8c-secrets decrypt) would thus hide that simplicity we were aiming for with a8c-secrets behind something more obscure, making them become less familiar with the a8c-secrets CLI itself (e.g. "how do I edit a secret? Oh I need to call a8c-secrets command? But I never use it otherwise, I always use fastlane configure_secrets, so I've never heard of a8c-secrets command"…)
Maybe I'm overthinking it, but I think having the error messages printed by Xcode's build phase mention a8c-secrets decrypt directly rathen than fastlane configure_secrets could at least be a good middle ground there?
| # Decrypt the secrets. | ||
| # | ||
| # This is redundant on dev machines most of the time, but it | ||
| # has such a negligible overhead that it's worth running it | ||
| # here to keep the individual lanes cleaner. | ||
| configure_secrets unless lane == :configure_secrets |
There was a problem hiding this comment.
Interesting approach to have a8c-secrets decrypt be guaranteed to be run before every single Fastlane lane to be sure we're always up-to-date…
But that makes me wonder if it's always necessary for all lanes?
For example, while for lanes that compile the app I can see this being needed/useful, for other lanes (like if we were to call configure_code_signing to update certificates etc), I don't think we'd need any of the secrets files managed by a8c-secrets in this repo, so that would make those be decrypted for nothing (and sure this doesn't take long to decrypt them in general, but that can also lead to errors if you don't have the private key configured on your machine, and I could see an AppsInfra engineer who runs match to update certs not necessary having the compilation secrets configured for every repo…)
Also I haven't thought about what happens if we have external contributors? Admittedly, I'm not sure they would be expected to call any fastlane lane to build and test the project, as they would probably instead only do that via Xcode. But still, that could still be another case where having configure_secrets being called before each lane could be a blocker for them if applied to all lanes even the ones not doing any compilation?
There was a problem hiding this comment.
Addressed in c6aa0ca by removing the lane.
kean
left a comment
There was a problem hiding this comment.
I was assigned a a reviewer, but I don't really have much to add to the conversion here. No reason not to make this change.
|
Claude finished @mokagio's task in 2m 0s —— View job Code Review: Adopt a8c-secrets for encrypted secrets
Overall this is a clean, well-structured migration. The secrets path is nicely DRYed into 🔴 Possible missing secret:
|
|
Back to draft because of the ongoing work for the |
Generated by 🚫 Danger |
Prevents EOL normalization from corrupting the encrypted ciphertext and suppresses meaningless text diffs on the encrypted blobs. --- Generated with the help of Claude Code, https://claude.ai/code Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The three remaining `configure` secrets, re-encrypted with `age` so they can be consumed via `a8c-secrets decrypt`. --- Generated with the help of Claude Code, https://claude.ai/code Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Extract the repeated "install secrets" step into `.buildkite/commands/install-secrets.sh`, matching the wcios convention: it installs the `a8c-secrets` binary, puts it on `PATH`, and runs the new `configure_secrets` fastlane lane (guard + `a8c-secrets decrypt`), all in one process. `shared_setup.sh` and the release pipelines call the script instead of inlining the decrypt. Installing the binary here is what lets CI agents run the decrypt at all — the previous inline `a8c-secrets decrypt` assumed a binary the agents don't ship. Drop the redundant `before_all` decrypt: the parent build lane won't have `~/.local/bin` on `PATH`, and secrets are already on disk from install-secrets. Agents still need `A8C_SECRETS_IDENTITY` set in Buildkite for decrypt to succeed. --- Generated with the help of Claude Code, https://claude.ai/code Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Ask the tool for the decrypted file location instead of hardcoding the `a8c-secrets` directory layout, so `upload_dsyms` keeps working if the path convention changes. --- Generated with the help of Claude Code, https://claude.ai/code Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
--- Generated with the help of Claude Code, https://claude.ai/code Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replaces a shell `command -v` probe with fastlane's own cross-platform binary lookup. --- Generated with the help of Claude Code, https://claude.ai/code Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Route developers through the single fastlane lane instead of a raw `a8c-secrets decrypt`, so a first run also gets the binary preflight check. --- Generated with the help of Claude Code, https://claude.ai/code Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Piping upstream's `install.sh` from `main` into `bash` ran an unreviewed script that could change at any time, and then installed whatever the `latest` release happened to be, unverified. For tooling that handles secrets that's a supply-chain risk worth closing. Download the pinned release binary directly and refuse to install it unless its SHA-256 matches the value recorded here, so the tool can only change through a reviewed diff in this repo. Upstream publishes no `.sha256` assets, so the expected hash is recorded inline and has to be bumped together with the version. --- Generated with the help of Claude Code, https://claude.ai/code Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The pinned version and its checksum move to `a8c-ci-toolkit`, so this repo — and every other one adopting `a8c-secrets` — stops carrying its own copy of the download-and-verify logic. The toolkit pin is a PR head rather than a release, since the command isn't shipped yet. It has to go back to a released version before this merges. The architecture guard and the log header go with the inlined code: the toolkit command rejects unsupported platforms itself, and it opens its own Buildkite section — a caller's header immediately followed by that one renders empty. --- Generated with the help of Claude Code, https://claude.ai/code Co-Authored-By: Claude Code Opus 4.8 <noreply@anthropic.com>
b4753b3 to
9db3c0a
Compare
Co-authored-by: Gio Lodi <giovanni.lodi42@gmail.com>
3f4f4dd to
dbe29b8
Compare
dbe29b8 to
c6aa0ca
Compare
The four files in `.configure` are build-time inputs: the `match` password, the Google Service plist, the Google Sign-In key and the credentials JSON. None of the lanes these eight stages invoke read them — the upload lanes and the release-management lanes authenticate purely through Buildkite `env` secrets (`GITHUB_TOKEN`, the ASC API key, `SENTRY_AUTH_TOKEN`, `FIREBASE_APP_DISTRIBUTION_ACCOUNT_KEY`), and the `new_*` lanes only touch git and GlotPress. Olivier raised this on #4857. Decrypting stays harmless today because `before_all` calls `configure_apply` for every lane regardless, so this only removes the redundant explicit step. It becomes load-bearing with #4731, where `before_all` goes away and these stages would otherwise pay for secrets they never read. `install-secrets.sh` is left in place with `shared_setup.sh` as its only caller: it stays the named seam that #4731 swaps for `a8c-secrets`. --- Generated with the help of Claude Code, https://claude.ai/code Co-Authored-By: Claude Code Opus 5 <noreply@anthropic.com>
|
Version |
Twelve call sites each spelled out their own secrets step, so changing how secrets are decrypted meant touching all twelve. This is groundwork for swapping `configure_apply` for `a8c-secrets` (AINFRA-1541): afterwards that swap is a one-file diff. `build.sh` is the only behaviour change — it had no explicit secrets step and leaned on `configure_apply` in fastlane's `before_all`. It now decrypts through `shared_setup.sh` like the other build scripts. Extracted from Automattic#4731 to shrink it to the parts still under discussion. --- Generated with the help of Claude Code, https://claude.ai/code Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

What it does
Replaces
mobile-secrets/.configurewitha8c-secrets. Context: paaHJt-96q-p2How to test
Automattic contributors only
.configureand.configure-files/now appear as new, untracked files in the repo. Delete them.rm -rf .configure .configure-filesa8c-secretsfollowing the instructions on the repository. You can find the dev key our internal secrets store searching fora8c-secrets - pocket-casts-ios - dev private keybundle exec fastlane configure_secrets