Skip to content

perf: Optimize folder UpdateTime timestamp by using int64 instead of time.Time #4859

Open
vedantdas-source wants to merge 4 commits into
masterfrom
folder-time-mem
Open

perf: Optimize folder UpdateTime timestamp by using int64 instead of time.Time #4859
vedantdas-source wants to merge 4 commits into
masterfrom
folder-time-mem

Conversation

@vedantdas-source

Copy link
Copy Markdown
Collaborator

Description

By default,Folder.UpdateTime was stored as a time.Time value, which occupies 24 bytes. By converting this field to a Unix nanosecond int64 representation which uses 8 bytes, we can safely reduce the structural footprint of every Folder object .

Changes Made

  • Changed Folder.UpdateTime from time.Time to int64.
  • Added a zero-value check !t.IsZero() in GCSFolder to ensure that uninitialized folder timestamps default to 0 instead of a negative value.
  • Updated the mock bucket generation and FUSE assertions to compare the equivalent UnixNano() values.

Benchmarking results

Using time.Time

goos: linux
goarch: amd64
pkg: github.com/googlecloudplatform/gcsfuse/v3/internal/storage/gcs
cpu: Intel(R) Xeon(R) CPU @ 2.60GHz
BenchmarkGCSFolder-48           12818931                94.74 ns/op      96 B/op          2 allocs/op

Using int64

goos: linux
goarch: amd64
pkg: github.com/googlecloudplatform/gcsfuse/v3/internal/storage/gcs
cpu: Intel(R) Xeon(R) CPU @ 2.60GHz
BenchmarkGCSFolder-48           14288154                87.04 ns/op      72 B/op          2 allocs/op

All existing tests pass successfully.

Link to the issue in case of a bug fix.

Testing details

  1. Manual - NA
  2. Unit tests - Added a BenchmarkGCSFolder benchmark in folder_test.go to compare memory and CPU performance.
  3. Integration tests - NA

Any backward incompatible change? If so, please explain.

@vedantdas-source vedantdas-source requested a review from a team as a code owner July 9, 2026 10:36
@github-actions github-actions Bot added the remind-reviewers Auto remind reviewers in attention set for review post 24hrs of inactivity on PR. label Jul 9, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the Folder struct's UpdateTime field from time.Time to int64 (Unix nanoseconds) across the GCS and fake storage implementations, and adds a benchmark test for GCSFolder. The review feedback highlights a critical concurrency issue in RenameFolder where the mutex lock b.mu is not acquired, potentially causing race conditions. Additionally, the reviewer recommends replacing instances of time.Now() with the mock clock b.clock.Now() in the fake bucket implementation to maintain deterministic test behavior.

Comment thread internal/storage/fake/bucket.go
Comment thread internal/storage/fake/bucket.go Outdated
Comment thread internal/storage/fake/bucket.go Outdated
@codecov

codecov Bot commented Jul 9, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 83.70%. Comparing base (b61c988) to head (c598bc3).

Additional details and impacted files
@@             Coverage Diff             @@
##           master    #4859       +/-   ##
===========================================
+ Coverage        0   83.70%   +83.70%     
===========================================
  Files           0      170      +170     
  Lines           0    20954    +20954     
===========================================
+ Hits            0    17539    +17539     
- Misses          0     2760     +2760     
- Partials        0      655      +655     
Flag Coverage Δ
unittests 83.70% <100.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Comment thread internal/storage/gcs/folder.go

@raj-prince raj-prince left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

Comment thread internal/storage/fake/bucket.go
Comment thread internal/storage/fake/bucket.go
@raj-prince raj-prince added the execute-integration-tests Run only integration tests label Jul 9, 2026
@meet2mky meet2mky added the execute-integration-tests-on-zb To run E2E tests on zonal bucket. label Jul 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

execute-integration-tests Run only integration tests execute-integration-tests-on-zb To run E2E tests on zonal bucket. remind-reviewers Auto remind reviewers in attention set for review post 24hrs of inactivity on PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants