Skip to content

Commit b7b2ed5

Browse files
gavinbarronCopilot
andauthored
ci: Route Generation pipeline package restores through CFS feeds (CFSClean) (#1450)
* Route metadata/OpenAPI capture jobs through CFS feeds (CFSClean) Pipeline 79 (Generation) fails CFSClean because the capture-metadata and capture-openapi jobs reach public package feeds: dotnet restore/tool-install -> api.nuget.org and npm install -> registry.npmjs.org. PR #1448 only covered the typewriter/kiota stages and wrote nuget.config to the sources dir, where a later 'checkout: self' wipes it. Add reusable create-cfs-nuget-config.yml (NuGetAuthenticate + nuget.config written to Agent.TempDirectory so checkout cannot wipe it) and create-cfs-npmrc.yml (npmAuthenticate + CFS npm registry applied to the user profile). Wire both into capture-metadata.yml (after checkouts) and the nuget config into capture-openapi's convert_openapi job, and pass --configfile to the hidi 'dotnet tool install' steps. Feed: GraphDeveloperExperiences_Public (upstreams nuget.org and npmjs.org). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: d3f8fec7-b00b-46be-ba39-7e1f3e7f7188 * Fix typewriter/kiota CFS nuget.config ordering (CFSClean) The typewriter/kiota nuget.config was created in the job before build-and-publish-typewriter.yml runs 'checkout: self', which cleans the sources dir and wiped the untracked config -> dotnet build restored from nuget.org (22 api.nuget.org hits in build_and_publish_typewriter). Move the CFS config creation to the reusable create-cfs-nuget-config.yml (writes to Agent.TempDirectory, checkout-proof). For typewriter, create it AFTER 'checkout: self' and restore via -p:RestoreConfigFile. For kiota, pass --configfile to the tool install. Remove the now-redundant inline NuGetAuthenticate/create-config steps from generation-pipeline.yml. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: d3f8fec7-b00b-46be-ba39-7e1f3e7f7188 * Suppress dotnet CLI CDN checks to stop builds.dotnet.microsoft.com egress (CFSClean) The Convert v1.0-graphexplorer job intermittently hit builds.dotnet.microsoft.com via dotnet.exe (not UseDotNet@2 and not generate-open-api.ps1, which only runs hidi locally). It is the dotnet CLI's workload/advertising-manifest + first-run checks piggybacking on the hidi 'dotnet tool install'. Set DOTNET_CLI_WORKLOAD_UPDATE_NOTIFY_DISABLE, DOTNET_SKIP_WORKLOAD_INTEGRITY_CHECK, DOTNET_CLI_TELEMETRY_OPTOUT, DOTNET_NOLOGO as pipeline variables (surfaced as env vars to all steps) to suppress the CDN egress. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: d3f8fec7-b00b-46be-ba39-7e1f3e7f7188 --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: d3f8fec7-b00b-46be-ba39-7e1f3e7f7188
1 parent b1be4eb commit b7b2ed5

7 files changed

Lines changed: 66 additions & 32 deletions

File tree

‎.azure-pipelines/generation-pipeline.yml‎

Lines changed: 9 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -130,6 +130,14 @@ parameters:
130130
displayName: Skip Open API PR Generation.
131131
variables:
132132
buildConfiguration: 'Release'
133+
# dotnet CLI hygiene for 1ES network isolation (CFSClean): stop the CLI from reaching the
134+
# public .NET CDN (builds.dotnet.microsoft.com) for workload/advertising-manifest and
135+
# first-run/telemetry checks that piggyback on any `dotnet` invocation (e.g. the hidi tool
136+
# install). Pipeline variables are surfaced as environment variables to every step.
137+
DOTNET_CLI_WORKLOAD_UPDATE_NOTIFY_DISABLE: 'true'
138+
DOTNET_SKIP_WORKLOAD_INTEGRITY_CHECK: 'true'
139+
DOTNET_CLI_TELEMETRY_OPTOUT: 'true'
140+
DOTNET_NOLOGO: 'true'
133141
cleanMetadataFileBeta: '$(Build.SourcesDirectory)/msgraph-metadata/clean_beta_metadata/cleanMetadataWithDescriptionsbeta.xml'
134142
cleanMetadataFileV1: '$(Build.SourcesDirectory)/msgraph-metadata/clean_v10_metadata/cleanMetadataWithDescriptionsv1.0.xml'
135143
cleanMetadataFileWithAnnotationsV1: '$(Build.SourcesDirectory)/msgraph-metadata/clean_v10_metadata/cleanMetadataWithDescriptionsAndAnnotationsv1.0.xml'
@@ -201,20 +209,6 @@ extends:
201209
targetPath: '$(Build.ArtifactStagingDirectory)'
202210
artifactName: typewriter
203211
steps:
204-
- task: NuGetAuthenticate@1
205-
displayName: 'Authenticate to Azure Artifacts'
206-
207-
- pwsh: |
208-
@"
209-
<?xml version="1.0" encoding="utf-8"?>
210-
<configuration>
211-
<packageSources>
212-
<clear />
213-
<add key="GraphDeveloperExperiences_Public" value="https://microsoftgraph.pkgs.visualstudio.com/0985d294-5762-4bc2-a565-161ef349ca3e/_packaging/GraphDeveloperExperiences_Public/nuget/v3/index.json" />
214-
</packageSources>
215-
</configuration>
216-
"@ | Set-Content -Path "$(Build.SourcesDirectory)/nuget.config" -Encoding UTF8
217-
displayName: 'Create nuget.config (central feed)'
218212
- template: /.azure-pipelines/generation-templates/build-and-publish-typewriter.yml@self
219213
- stage: stage_build_and_publish_kiota
220214
dependsOn: [] # remove the implicit dependency to any previous stage
@@ -226,20 +220,7 @@ extends:
226220
targetPath: '$(Build.ArtifactStagingDirectory)'
227221
artifactName: kiota
228222
steps:
229-
- task: NuGetAuthenticate@1
230-
displayName: 'Authenticate to Azure Artifacts'
231-
232-
- pwsh: |
233-
@"
234-
<?xml version="1.0" encoding="utf-8"?>
235-
<configuration>
236-
<packageSources>
237-
<clear />
238-
<add key="GraphDeveloperExperiences_Public" value="https://microsoftgraph.pkgs.visualstudio.com/0985d294-5762-4bc2-a565-161ef349ca3e/_packaging/GraphDeveloperExperiences_Public/nuget/v3/index.json" />
239-
</packageSources>
240-
</configuration>
241-
"@ | Set-Content -Path "$(Build.SourcesDirectory)/nuget.config" -Encoding UTF8
242-
displayName: 'Create nuget.config (central feed)'
223+
- template: /.azure-pipelines/generation-templates/create-cfs-nuget-config.yml@self
243224
- template: /.azure-pipelines/generation-templates/build-and-publish-kiota.yml@self
244225
# Downloads the latest public beta metadata. If there are changes, we checkin
245226
# the public metadata into microsoftgraph/msgraph-metadata, and then run the
Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
steps:
22
- pwsh: |
33
dotnet tool install Microsoft.OpenApi.Kiota `
4-
--tool-path "$(Build.ArtifactStagingDirectory)"
4+
--tool-path "$(Build.ArtifactStagingDirectory)" `
5+
--configfile "$(Agent.TempDirectory)/nuget.config"
56
displayName: 'Install Kiota via dotnet tool'

‎.azure-pipelines/generation-templates/build-and-publish-typewriter.yml‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,11 @@ steps:
66
submodules: recursive
77
persistCredentials: true
88

9-
- pwsh: dotnet build $(Build.SourcesDirectory)/src/Typewriter/Typewriter.csproj --configuration $(buildConfiguration)
9+
# Create the CFS nuget.config AFTER checkout (a checkout clean would wipe a config written
10+
# into the sources dir) and restore against it, so dotnet build does not reach nuget.org.
11+
- template: /.azure-pipelines/generation-templates/create-cfs-nuget-config.yml@self
12+
13+
- pwsh: dotnet build $(Build.SourcesDirectory)/src/Typewriter/Typewriter.csproj --configuration $(buildConfiguration) -p:RestoreConfigFile="$(Agent.TempDirectory)/nuget.config"
1014
displayName: 'Build Typewriter'
1115

1216
- task: CopyFiles@2

‎.azure-pipelines/generation-templates/capture-metadata.yml‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,11 @@ steps:
4848
- template: /.azure-pipelines/generation-templates/checkout-metadata.yml@self
4949
- template: /.azure-pipelines/generation-templates/set-user-config.yml@self
5050

51+
# Route NuGet and npm through the CFS central feed for 1ES network isolation (CFSClean).
52+
# Placed after all checkouts so the generated config files are not wiped by a checkout clean.
53+
- template: /.azure-pipelines/generation-templates/create-cfs-nuget-config.yml@self
54+
- template: /.azure-pipelines/generation-templates/create-cfs-npmrc.yml@self
55+
5156
# required for TypeSpec
5257
- task: UseNode@1
5358
inputs:
@@ -153,7 +158,7 @@ steps:
153158
parameters:
154159
version: '9.x'
155160

156-
- pwsh: dotnet tool install --global Microsoft.OpenApi.Hidi --version 1.*
161+
- pwsh: dotnet tool install --global Microsoft.OpenApi.Hidi --version 1.* --configfile "$(Agent.TempDirectory)/nuget.config"
157162
displayName: 'Install hidi tool'
158163

159164
# verify that generated metadata is parsable as an Edm model

‎.azure-pipelines/generation-templates/capture-openapi.yml‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -61,6 +61,9 @@ jobs:
6161
persistCredentials: true
6262

6363
- template: /.azure-pipelines/generation-templates/checkout-metadata.yml@self
64+
# Route NuGet through the CFS central feed for 1ES network isolation (CFSClean).
65+
# Placed after all checkouts so the generated config is not wiped by a checkout clean.
66+
- template: /.azure-pipelines/generation-templates/create-cfs-nuget-config.yml@self
6467
# required for the hidi to run
6568
- template: /.azure-pipelines/generation-templates/use-dotnet-sdk.yml@self
6669
parameters:
@@ -71,7 +74,7 @@ jobs:
7174
parameters:
7275
version: '9.x'
7376

74-
- pwsh: dotnet tool install --global Microsoft.OpenApi.Hidi --version 1.*
77+
- pwsh: dotnet tool install --global Microsoft.OpenApi.Hidi --version 1.* --configfile "$(Agent.TempDirectory)/nuget.config"
7578
displayName: install hidi
7679

7780
- pwsh: |
Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
# Routes npm through the CFS central feed (GraphDeveloperExperiences_Public, which upstreams
2+
# npmjs.org) instead of hitting registry.npmjs.org directly, for 1ES network isolation (CFSClean).
3+
# The authenticated .npmrc is copied to the user profile so both global (`npm install -g`) and
4+
# project (`npm ci`) commands pick up the registry + credentials without per-step configuration.
5+
steps:
6+
- pwsh: |
7+
@"
8+
registry=https://microsoftgraph.pkgs.visualstudio.com/0985d294-5762-4bc2-a565-161ef349ca3e/_packaging/GraphDeveloperExperiences_Public/npm/registry/
9+
always-auth=true
10+
"@ | Set-Content -Path "$(Agent.TempDirectory)/.npmrc" -Encoding UTF8
11+
displayName: 'Create .npmrc (CFS central feed)'
12+
13+
- task: npmAuthenticate@0
14+
displayName: 'Authenticate npm to CFS feed'
15+
inputs:
16+
workingFile: '$(Agent.TempDirectory)/.npmrc'
17+
18+
- pwsh: |
19+
Copy-Item -Path "$(Agent.TempDirectory)/.npmrc" -Destination (Join-Path $HOME '.npmrc') -Force
20+
displayName: 'Apply CFS .npmrc to user profile'
Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
# Routes dotnet/NuGet restore and tool installs through the CFS central package feed
2+
# (GraphDeveloperExperiences_Public) instead of public nuget.org, for 1ES network isolation
3+
# (CFSClean). The config is written to $(Agent.TempDirectory) so it survives any subsequent
4+
# `checkout` step (which cleans the sources directory and would wipe a config placed there).
5+
# Consumers must pass `--configfile "$(Agent.TempDirectory)/nuget.config"` to dotnet commands.
6+
steps:
7+
- task: NuGetAuthenticate@1
8+
displayName: 'Authenticate to Azure Artifacts (CFS feed)'
9+
10+
- pwsh: |
11+
@"
12+
<?xml version="1.0" encoding="utf-8"?>
13+
<configuration>
14+
<packageSources>
15+
<clear />
16+
<add key="GraphDeveloperExperiences_Public" value="https://microsoftgraph.pkgs.visualstudio.com/0985d294-5762-4bc2-a565-161ef349ca3e/_packaging/GraphDeveloperExperiences_Public/nuget/v3/index.json" />
17+
</packageSources>
18+
</configuration>
19+
"@ | Set-Content -Path "$(Agent.TempDirectory)/nuget.config" -Encoding UTF8
20+
displayName: 'Create nuget.config (CFS central feed)'

0 commit comments

Comments
 (0)