fix: bound repository sync concurrency and handle pre-creation suborg lookups

Dockerfile

@@ -11,6 +11,7 @@ LABEL version="1.0" \
 ## to the image to be as small as possible
 COPY  package*.json /opt/safe-settings/
 COPY  index.js /opt/safe-settings/
+COPY  full-sync.js /opt/safe-settings/
 COPY  lib /opt/safe-settings/lib
 
 ## Install the app and dependencies

README.md

@@ -205,6 +205,13 @@ When a repo-level change (a push to `.github/repos/<repo>.yml`, or a `repository
 
 To handle this, after applying a repo-yml change `safe-settings` re-evaluates the repo's suborg membership. If the matched suborg source set changed, it runs the repo through the apply pipeline a second time so newly matched suborg settings are applied and settings from a no-longer-matching suborg can be removed in the same sync.
 
+For a repository that does not exist yet, team and custom-property membership
+lookups are deferred until after creation. A 404 from a membership endpoint is
+treated as an unmatched selector only when the repository itself also returns
+404; other lookup failures remain errors. Name-based `suborgrepos` matching still
+applies before creation. This allows `force_create` validation and apply runs to
+work when earlier changes have already configured team- or property-based suborgs.
+
 **Scope:** Re-evaluation runs only on the repo-yml change paths (`Settings.sync` and the per-repo loop of `Settings.syncSelectedRepos`). Global settings changes (`syncAll`) and suborg-yml changes (`syncSubOrgs`) already iterate all relevant repos and do not need it.
 
 **Loop prevention.** Two guards prevent infinite re-evaluation:

docs/docker-debugging.md

@@ -32,6 +32,14 @@ Build the local image:
 docker build -t safe-settings:local .
 ```
 
+The image also includes the one-shot full-sync entrypoint. To preview a sync
+without applying settings or starting the webhook server:
+
+```bash
+docker run --rm --env-file ./.env --env FULL_SYNC_NOP=true \
+	safe-settings:local npm run full-sync
+```
+
 Run container in foreground with explicit runtime env and port mapping:
 
 ```bash

docs/github-action.md

@@ -10,6 +10,14 @@ Follow the [Create the GitHub App](deploy.md#create-the-github-app) guide to cre
 ## Defining the GitHub Action Workflow
 Running a full-sync with `safe-settings` can be done via `npm run full-sync`. This requires installing Node, such as with [actions/setup-node](https://github.com/actions/setup-node) (see example below). When doing so, the appropriate environment variables must be set (see the [Environment variables](#environment-variables) document for more details).
 
+Installation repositories are processed in batches of up to ten, with each batch
+settling before the next starts. A repository failure does not stop later
+repositories from being processed. Failures are logged with the repository name
+and retained in the sync errors, so a partial failure still produces a failed
+check and a nonzero full-sync exit status. Dry runs report these errors alongside
+planned changes. The same batching applies when scanning installation repositories
+for suborg configuration changes; existing repository restrictions still apply.
+
 
 ### Example GHA Workflow
 The below example uses the GHA "cron" feature to run a full-sync every 4 hours. While not required, this example uses the `.github` repo as the `admin` repo (set via `ADMIN_REPO` env var) and the safe-settings configurations are stored in the `safe-settings/` directory (set via `CONFIG_PATH` and `DEPLOYMENT_CONFIG_FILE`).

lib/settings.js

@@ -1061,11 +1061,11 @@ class Settings {
       })
   }
 
-  logError (msg) {
+  logError (msg, repo = this.repo) {
     this.log.error(msg)
     this.errors.push({
-      owner: this.repo.owner,
-      repo: this.repo.repo,
+      owner: repo.owner,
+      repo: repo.repo,
       msg,
       plugin: this.constructor.name
     })
@@ -1074,7 +1074,7 @@ class Settings {
     // by the syncAll/syncSelectedRepos top-level catch (e.g. invalid
     // disable_plugins entries) would go unnoticed by PR reviewers.
     if (this.nop) {
-      const nopcommand = new NopCommand(this.constructor.name, this.repo, null, msg, 'ERROR')
+      const nopcommand = new NopCommand(this.constructor.name, repo, null, msg, 'ERROR')
       this.appendToResults([nopcommand])
     }
   }
@@ -2114,10 +2114,7 @@ class Settings {
         }
       } catch (e) {
         if (this.nop) {
-          const nopcommand = new NopCommand(this.constructor.name, this.repo, null, `${e}`, 'ERROR')
-          this.log.error(`NOPCOMMAND ${JSON.stringify(nopcommand)}`)
-          this.appendToResults([nopcommand])
-          // throw e
+          this.logError(`${e}`, repo)
         } else {
           throw e
         }
@@ -2586,13 +2583,24 @@ class Settings {
 
   async eachRepositoryRepos (github, log) {
     log.debug('Fetching repositories')
-    return github.paginate('GET /installation/repositories').then(repositories => {
-      return Promise.all(repositories.map(repository => {
-        const { owner, name } = repository
-        return this.checkAndProcessRepo(owner.login, name)
-      })
+    const repositories = await github.paginate('GET /installation/repositories')
+    const CONCURRENCY = 10
+    const results = []
+    for (let i = 0; i < repositories.length; i += CONCURRENCY) {
+      const batch = repositories.slice(i, i + CONCURRENCY)
+      const batchResults = await Promise.allSettled(
+        batch.map(({ owner, name }) => this.checkAndProcessRepo(owner.login, name))
       )
-    })
+      batchResults.forEach((result, index) => {
+        if (result.status === 'fulfilled') {
+          results.push(result.value)
+        } else {
+          const { owner, name } = batch[index]
+          this.logError(`Error processing repository ${owner.login}/${name}: ${result.reason}`, { owner: owner.login, repo: name })
+        }
+      })
+    }
+    return results
   }
 
   async checkAndProcessRepo (owner, name) {
@@ -2910,6 +2918,27 @@ class Settings {
     // actually references teams/properties.
     let repoTeamSlugs
     let repoProperties
+    let repoMissing = false
+
+    const getMembership = async lookup => {
+      if (repoMissing) return []
+      try {
+        return await lookup()
+      } catch (error) {
+        if (error.status !== 404) throw error
+        // Membership endpoints also return 404 for a repo awaiting force_create.
+        // Verify the repo itself is missing, rather than hiding permission errors.
+        try {
+          await this.github.rest.repos.get(repo)
+        } catch (repoError) {
+          if (repoError.status !== 404) throw repoError
+          repoMissing = true
+          this.log.debug(`Suborg membership deferred for missing repository ${repo.owner}/${repo.repo}`)
+          return []
+        }
+        throw error
+      }
+    }
 
     for (const override of overridePaths) {
       const data = await this.loadYaml(override.path)
@@ -2924,14 +2953,14 @@ class Settings {
 
       if (!matched && data.suborgteams) {
         if (repoTeamSlugs === undefined) {
-          repoTeamSlugs = (await this.getReposTeams(repo)).map(team => team.slug)
+          repoTeamSlugs = (await getMembership(() => this.getReposTeams(repo))).map(team => team.slug)
         }
         matched = data.suborgteams.some(teamslug => repoTeamSlugs.includes(teamslug))
       }
 
       if (!matched && data.suborgproperties) {
         if (repoProperties === undefined) {
-          repoProperties = await this.getRepoCustomPropertyValues(repo)
+          repoProperties = await getMembership(() => this.getRepoCustomPropertyValues(repo))
         }
         matched = this.repoMatchesProperties(repoProperties, data.suborgproperties)
       }