add go mod require & replace support #868

willie-yao · 2025-11-21T22:05:26Z

What this PR does / why we need it:
Continuation of #847

Which issue(s) this PR fixes (optional, using fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when the PR gets merged):
Fixes #

Special notes for your reviewer:

willie-yao · 2025-12-04T23:54:07Z

targets/windows/handle_zip.go

 	// This must happen after build deps are installed so Go is available
 	if err := spec.Preprocess(client, sOpt, worker, opts...); err != nil {
-		return llb.Scratch()
+		return dalec.ErrorState(worker, err)


This change exposes the error of the state to the user as this is in an error check

Copilot

Pull request overview

This PR adds support for go mod require and replace directives in the gomod generator, allowing users to programmatically modify go.mod files during the build process. This is a continuation of #847 and enables important use cases like replacing dependencies with forks, working with multi-module repositories, and pinning specific dependency versions.

Key Changes:

New GomodEdits, GomodReplace, and GomodRequire types for declarative go.mod manipulation
Preprocessing logic that generates patches from gomod edits before building
Internal SourceLLB type for programmatically generated patch sources
Validation and extensive test coverage for the new functionality

Reviewed changes

Copilot reviewed 20 out of 20 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
website/docs/sources.md	Documents the new gomod replace/require directives (contains format errors that need correction)
spec.go	Defines GomodEdits, GomodReplace, and GomodRequire types with marshaling/unmarshaling logic
preprocess.go	Implements preprocessing logic to generate patches from gomod edits
source_llb.go	Introduces internal-only SourceLLB type for programmatically generated sources
source_generated.go	Generated code for SourceLLB integration
source.go	Integrates LLB field into Source type
load.go	Adds validation for gomod edits
generator_gomod_test.go	Unit tests for unmarshaling and validation of gomod directives
spec_test.go	Validation tests for SourceGenerator gomod edits
test/source_test.go	Unit tests for gomod require/replace with debug/patched-sources target
test/linux_target_test.go	Integration tests for gomod edits across different scenarios
test/fixtures/simple-go-require.yml	Example fixture demonstrating correct usage
targets/windows/handle_zip.go	Adds Preprocess call before building
targets/linux/rpm/distro/pkg.go	Adds Preprocess call before building
targets/linux/deb/distro/pkg.go	Adds Preprocess call before building
targets/linux/distro_handler.go	Standard import reordering
frontend/gateway.go	Standard import reordering
frontend/debug/handle_sources.go	Adds Preprocess call before getting sources
docs/spec.schema.json	Updates JSON schema with new gomod edit types
docker-bake.hcl	Removes trailing blank line

Copilot · 2025-12-08T19:00:26Z

website/docs/sources.md

+              # String format: "old@version=new@version"
+              - "github.com/old/[email protected]=github.com/new/[email protected]"


The documentation shows an incorrect string format for the replace directive. The string format uses : as the separator, not =.

According to the code implementation in spec.go, the string format is parsed by splitting on : (line 244), so it should be:

- "github.com/old/[email protected]:github.com/new/[email protected]"

The = is only used internally when constructing the go mod edit argument, not in the YAML format.

Suggested change

# String format: "old@version=new@version"

- "github.com/old/[email protected]=github.com/new/[email protected]"

# String format: "old@version:new@version"

- "github.com/old/[email protected]:github.com/new/[email protected]"

Copilot · 2025-12-08T19:00:26Z

website/docs/sources.md

+              # String format: "module@version"
+              - "github.com/some/[email protected]"


The documentation shows an incorrect string format for the require directive. The string format should use : as the separator and must include the module path before the :.

According to the code implementation in spec.go, the string format is parsed by splitting on : into module and version (line 335-340), so it should be:

- "github.com/some/pkg:github.com/some/[email protected]"

This matches the pattern module:module@version where the version field must contain @version.

Copilot · 2025-12-08T19:00:27Z

website/docs/sources.md

+              - "github.com/some/[email protected]"
+              # Struct format
+              - module: github.com/another/pkg
+                version: v2.0.0


The documentation shows an incorrect struct format for the require directive. The version field must include the full module@version format, not just the version number.

According to the code validation in spec.go (lines 360-362), the version field is validated to contain @, so it should be:

- module: github.com/another/pkg version: github.com/another/[email protected]

This matches the pattern where version contains the full module path with @version.

Suggested change

version: v2.0.0

version: github.com/another/pkg@v2.0.0

Copilot · 2025-12-08T19:00:27Z

website/docs/sources.md

+              - "example.com/internal/[email protected]=../../shared"
+            require:
+              - "github.com/new/[email protected]"


The documentation shows incorrect string formats in the multi-module example. Both the replace and require directives use the wrong format:

Replace (line 501): Should use : instead of =:

- "example.com/internal/[email protected]:../../shared"

Require (line 503): Should include the module path before ::

- "github.com/new/dependency:github.com/new/[email protected]"

These formats must match the parsing logic in spec.go which splits on : for both directives.

Suggested change

- "example.com/internal/[email protected]=../../shared"

require:

- "github.com/new/[email protected]"

- "example.com/internal/[email protected]:../../shared"

require:

- "github.com/new/dependency:github.com/new/dependency@v1.5.0"

spec.go

cpuguy83 · 2025-12-10T18:12:39Z

spec.go

+	Replace []GomodReplace `yaml:"replace,omitempty" json:"replace,omitempty"`
+	// Require applies go.mod require directives before downloading module dependencies.
+	// Each entry can be either a string "module:version" or a struct with Module and Version fields.
+	Require []GomodRequire `yaml:"require,omitempty" json:"require,omitempty"`


So this injects totally new requires?

I think so. Is that a concern or should we be doing this a different way? @DannyBrito can maybe also clarify since he wrote the original struct.

I guess the main thing is, do we have a use-case for this right now?

sorry I missed this but tbh I don't really understand the question here; I reused initial implementation of the GomodEdits https://github.com/project-dalec/dalec/pull/769/changes#diff-f23a639898f3d395e15d7331eeab9d8d6217fe161c2851610171da61d9f92dd5R255.

Basically, do we need to support adding new requires in this feature.

we don't need to inject new requires since they'll get cleaned up by go mod tidy

we should only use require to update existing requires. require: old:new, not require: new

scripts/gomod-patch.sh

test/fixtures/simple-go-require.yml

cpuguy83 · 2026-01-05T22:34:40Z

@willie-yao This one needs a rebase to get CI fixes and I think one merge conflict?

Signed-off-by: Danny Brito <[email protected]>

Signed-off-by: William Yao <[email protected]> use go 1.18 in go.mod Signed-off-by: William Yao <[email protected]> Fix Bionic go path Signed-off-by: William Yao <[email protected]> Reviews Signed-off-by: William Yao <[email protected]> set source maps Signed-off-by: William Yao <[email protected]>

willie-yao · 2026-01-06T22:38:45Z

@cpuguy83 The test failures seem to be still related to runners not having enough memory. Otherwise CI looks good?

Signed-off-by: William Yao <[email protected]>

jeremyrickard assigned willie-yao Nov 24, 2025

willie-yao force-pushed the go-replace branch 4 times, most recently from 8f760ff to 7b08746 Compare December 4, 2025 21:31

willie-yao commented Dec 4, 2025

View reviewed changes

willie-yao force-pushed the go-replace branch from 72465dc to 981a7eb Compare December 4, 2025 23:55

willie-yao temporarily deployed to deploy-docs December 5, 2025 15:36 — with GitHub Actions Inactive

willie-yao force-pushed the go-replace branch from d64502c to d509f3a Compare December 8, 2025 17:51

willie-yao temporarily deployed to deploy-docs December 8, 2025 17:58 — with GitHub Actions Inactive

willie-yao marked this pull request as ready for review December 8, 2025 18:55

Copilot AI review requested due to automatic review settings December 8, 2025 18:55

willie-yao marked this pull request as draft December 8, 2025 18:56

Copilot started reviewing on behalf of willie-yao December 8, 2025 18:56 View session

Copilot AI reviewed Dec 8, 2025

View reviewed changes

willie-yao force-pushed the go-replace branch 2 times, most recently from a7c7424 to c561d30 Compare December 10, 2025 17:55

willie-yao temporarily deployed to deploy-docs December 10, 2025 18:16 — with GitHub Actions Inactive

cpuguy83 requested changes Dec 10, 2025

View reviewed changes

willie-yao temporarily deployed to deploy-docs December 15, 2025 18:23 — with GitHub Actions Inactive

willie-yao temporarily deployed to deploy-docs December 15, 2025 18:30 — with GitHub Actions Inactive

willie-yao temporarily deployed to deploy-docs December 15, 2025 21:55 — with GitHub Actions Inactive

willie-yao force-pushed the go-replace branch from 83e0da0 to 28b9ea2 Compare December 16, 2025 00:52

willie-yao temporarily deployed to deploy-docs December 16, 2025 00:52 — with GitHub Actions Inactive

willie-yao temporarily deployed to deploy-docs December 16, 2025 22:06 — with GitHub Actions Inactive

willie-yao force-pushed the go-replace branch from 07cb9e0 to fd89736 Compare December 17, 2025 02:27

willie-yao temporarily deployed to deploy-docs December 17, 2025 02:27 — with GitHub Actions Inactive

willie-yao temporarily deployed to deploy-docs December 17, 2025 02:54 — with GitHub Actions Inactive

willie-yao temporarily deployed to deploy-docs December 17, 2025 03:03 — with GitHub Actions Inactive

willie-yao temporarily deployed to deploy-docs December 29, 2025 15:29 — with GitHub Actions Inactive

willie-yao force-pushed the go-replace branch from cf8b522 to 4958b52 Compare December 31, 2025 19:39

willie-yao temporarily deployed to deploy-docs December 31, 2025 19:39 — with GitHub Actions Inactive

willie-yao changed the title ~~wip: add go mod require & replace support~~ add go mod require & replace support Jan 5, 2026

willie-yao marked this pull request as ready for review January 5, 2026 19:12

DannyBrito added 3 commits January 5, 2026 14:44

add go require & replace

48b31ef

Signed-off-by: Danny Brito <[email protected]>

add temp fixture & target

a917486

Signed-off-by: Danny Brito <[email protected]>

add tests and validation

bf36e95

Signed-off-by: Danny Brito <[email protected]>

willie-yao force-pushed the go-replace branch from 4958b52 to bcbe383 Compare January 5, 2026 22:44

willie-yao temporarily deployed to deploy-docs January 5, 2026 22:44 — with GitHub Actions Inactive

willie-yao force-pushed the go-replace branch from bcbe383 to 15753a6 Compare January 5, 2026 23:18

willie-yao temporarily deployed to deploy-docs January 5, 2026 23:18 — with GitHub Actions Inactive

willie-yao force-pushed the go-replace branch from 15753a6 to ec7f960 Compare January 5, 2026 23:25

willie-yao temporarily deployed to deploy-docs January 5, 2026 23:25 — with GitHub Actions Inactive

cpuguy83 added this to the v0.20 milestone Jan 6, 2026

willie-yao temporarily deployed to deploy-docs January 7, 2026 02:51 — with GitHub Actions Inactive

willie-yao force-pushed the go-replace branch from 8c4f560 to 399cf15 Compare January 7, 2026 02:56

willie-yao temporarily deployed to deploy-docs January 7, 2026 02:56 — with GitHub Actions Inactive

Remove require directive

b0d33bb

Signed-off-by: William Yao <[email protected]>

willie-yao force-pushed the go-replace branch from 399cf15 to b0d33bb Compare January 7, 2026 03:00

willie-yao deployed to deploy-docs January 7, 2026 03:00 — with GitHub Actions Active

		# String format: "old@version=new@version"
		- "github.com/old/[email protected]=github.com/new/[email protected]"

		# String format: "module@version"
		- "github.com/some/[email protected]"

add go mod require & replace support #868

Are you sure you want to change the base?

add go mod require & replace support #868

Conversation

willie-yao commented Nov 21, 2025

Uh oh!

willie-yao Dec 4, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Dec 8, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 8, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 8, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 8, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cpuguy83 Dec 10, 2025

Choose a reason for hiding this comment

Uh oh!

willie-yao Dec 17, 2025

Choose a reason for hiding this comment

Uh oh!

cpuguy83 Jan 5, 2026

Choose a reason for hiding this comment

Uh oh!

DannyBrito Jan 5, 2026

Choose a reason for hiding this comment

Uh oh!

cpuguy83 Jan 5, 2026

Choose a reason for hiding this comment

Uh oh!

sozercan Jan 6, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

cpuguy83 commented Jan 5, 2026

Uh oh!

willie-yao commented Jan 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants