feat(metro-file-map): Lazily stat files and populate symlinks for Node crawled file trees

[kitten]

Summary

Note

This is part of the same pick/patch-set from an experiment on the Expo repo,
like #1676, #1677, and #1687, for metro-file-map (conflicts are expected, since changes were pulled out into separate PRs for clarity)

After #1677 the highest impact on initial crawling performance with the Node crawler are the individual lstat calls and readlink calls. The former happens for every file that's discovered and can hence get quite large for monorepos (or assets that aren't excluded with blocklists). The latter, readlinks, are expensive if we're in an isolated installation (pnpm & bun), which can contain a much larger amount of symlinks than was likely anticipated by the current code paths.

This is the last (and most minimal iteration) of a few attempts to reduce the cost.

The idea is to skip the lstat call for new files that aren't in the previous FS snapshot (entry doesn't exist). To make this safe, these files are also excluded from getDifference, i.e. when both the previous and new entries have 0 | null mtime values. In both paths 0 | null indicates that the file is skipped for lstat.

The mtime is later added via an fs.promises.lstat call in getOrComputeSha1 i.e. when the file is actually accessed. If the file isn't accessed the mtime value remains at null.

This cascades to require/allow us to lazily evaluate symlinks for mtime values of null. Their values aren't read until accessed in #resolveSymlinkTargetToNormalPath in TreeFS. This does introduce an update on read, but lazily populates the symlink value. This trades off the async readlink call on all symlinks for a sync readlink call on a subset of symlinks.

We've observed this to be an overall win, but this could be influenced further by this commit: expo/expo@af67435 (which could also be upstreamed but changes the getContent signature. This change can be avoided by awaiting first, but seems overall like a better change than awaiting getContent unconditionally before plugins)

Changelog: [Internal] Lazily stat Node crawled files and lazily populate symlink targets

Test plan

• Unit tests were added to demonstrate this
• This can further be tested E2E in the linked Expo PR, although not in isolation (if necessary I can create a new, isolated branch with a pnpm patch for these changes)