Skip to content

test(cache): optimize chunk size and scale down cache_handle_test object sizes#4816

Merged
geertj merged 1 commit into
GoogleCloudPlatform:masterfrom
geertj:test-cache-optimize
Jun 30, 2026
Merged

test(cache): optimize chunk size and scale down cache_handle_test object sizes#4816
geertj merged 1 commit into
GoogleCloudPlatform:masterfrom
geertj:test-cache-optimize

Conversation

@geertj

@geertj geertj commented Jun 27, 2026

Copy link
Copy Markdown
Collaborator

Refactored downloader.ReadChunkSize from a package-level global constant to a Job struct-level field (initialized from DefaultReadChunkSize constant) to allow tests to override it.

In cache_handle_test.go:

  • Scaled down TestObjectSize from 16MiB to 1200KiB.
  • Scaled down CacheMaxSize to 5MiB and ReadContentSize to 32KiB.
  • SetupTest now overrides the ReadChunkSize directly on the local fileDownloadJob instance to 1MiB. This allows transition and cancellation tests to pass on the smaller object size without global side- effects.
  • Fixed Test_shouldReadFromCache_WithJobStateIsCompleted to dynamically compute requiredOffset based on object size instead of assuming >= 9MiB.
  • Fixed Test_RandomRead_CacheForRangeReadFalseButCacheHit to download 1MiB instead of 2MiB.

This speeds up package test execution time from ~12.6s down to ~2.1s (a ~6x speedup).

TAG=agy
CONV=8c7b5ab6-91ae-4a84-911e-1788917d5ab8

…ect sizes

Refactored downloader.ReadChunkSize from a package-level global constant to a
Job struct-level field (initialized from DefaultReadChunkSize constant) to
allow tests to override it.

In cache_handle_test.go:
- Scaled down TestObjectSize from 16MiB to 1200KiB.
- Scaled down CacheMaxSize to 5MiB and ReadContentSize to 32KiB.
- SetupTest now overrides the ReadChunkSize directly on the local
  fileDownloadJob instance to 1MiB. This allows transition and
  cancellation tests to pass on the smaller object size without global side-
  effects.
- Fixed Test_shouldReadFromCache_WithJobStateIsCompleted to dynamically compute
  requiredOffset based on object size instead of assuming >= 9MiB.
- Fixed Test_RandomRead_CacheForRangeReadFalseButCacheHit to download 1MiB
  instead of 2MiB.

This speeds up package test execution time from ~12.6s down to ~2.1s (a ~6x
speedup).

TAG=agy
CONV=8c7b5ab6-91ae-4a84-911e-1788917d5ab8
@geertj geertj requested a review from a team as a code owner June 27, 2026 22:58
@geertj geertj requested a review from kislaykishore June 27, 2026 22:58
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request focuses on optimizing the test suite performance by introducing configuration flexibility for chunk sizes. By moving the read chunk size from a global constant to a struct-level field, the tests can now operate on smaller, more efficient data sets without side effects. These changes collectively improve the developer experience by significantly reducing the time required to run package tests.

Highlights

  • Refactoring ReadChunkSize: Converted downloader.ReadChunkSize from a package-level global constant to a configurable field within the Job struct, allowing for test-specific overrides.
  • Test Optimization: Scaled down test object sizes and cache parameters in cache_handle_test.go, resulting in a ~6x reduction in package test execution time.
  • Dynamic Test Logic: Updated multiple test cases to dynamically calculate offsets and sizes based on the configured ReadChunkSize rather than relying on hardcoded values.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the file downloader to use an instance-specific ReadChunkSize on the Job struct instead of a global constant, allowing for more flexible configuration and testing. The corresponding tests have been updated to use smaller test sizes and reference the dynamic chunk size. The feedback suggests adding a documentation comment to the newly exported ReadChunkSize field to clarify its purpose and warn against concurrent modifications.

Comment thread internal/cache/file/downloader/job.go
@geertj geertj requested a review from charith87 June 29, 2026 19:16
@geertj geertj merged commit 76f6c2b into GoogleCloudPlatform:master Jun 30, 2026
13 of 15 checks passed
raj-prince pushed a commit that referenced this pull request Jul 1, 2026
…ect sizes (#4816)

Refactored downloader.ReadChunkSize from a package-level global constant to a
Job struct-level field (initialized from DefaultReadChunkSize constant) to
allow tests to override it.

In cache_handle_test.go:
- Scaled down TestObjectSize from 16MiB to 1200KiB.
- Scaled down CacheMaxSize to 5MiB and ReadContentSize to 32KiB.
- SetupTest now overrides the ReadChunkSize directly on the local
  fileDownloadJob instance to 1MiB. This allows transition and
  cancellation tests to pass on the smaller object size without global side-
  effects.
- Fixed Test_shouldReadFromCache_WithJobStateIsCompleted to dynamically compute
  requiredOffset based on object size instead of assuming >= 9MiB.
- Fixed Test_RandomRead_CacheForRangeReadFalseButCacheHit to download 1MiB
  instead of 2MiB.

This speeds up package test execution time from ~12.6s down to ~2.1s (a ~6x
speedup).

TAG=agy
CONV=8c7b5ab6-91ae-4a84-911e-1788917d5ab8
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants