Add deterministic Baseline Profile generation and startup benchmarks #294

[masoudkarimi]

Summary

Adds deterministic Baseline Profile generation and startup performance benchmarking for the Android application.

• Adds a dedicated baselineprofile module.
• Generates and packages Baseline and Startup Profiles.
• Adds live and fixture backend flavors.
• Uses a deterministic fixture API during profile generation and benchmarking.
• Prevents profile generation from depending on the rate-limited production API.
• Excludes fixture-only implementation classes from generated profile rules.
• Adds cold-start benchmarks with and without Baseline Profile compilation.
• Reports the market screen as fully drawn after its content has loaded.
• Measures both Time To Initial Display (TTID) and Time To Full Display (TTFD).
• Handles notification permission during profile generation and benchmarking.
• Adds a targeted suppression for the AGP 9 Instantiatable Lint false positive.

Motivation

Baseline Profile generation executes critical user journeys multiple times. Using the production API made generation unreliable because of its rate limit.

The fixture backend provides consistent paging data while keeping the production repository, database, domain, ViewModel, and Compose UI paths active.

Validation

• Generated profiles on an API 33+ connected device.
• Verified fixture-only classes are excluded from profile rules.
• Built the production release APK.
• Built the fixture benchmark APK.
• Ran startup benchmarks on a physical device.
• Added fully-drawn reporting for TTFD measurements.

Commands used:

./gradlew :app:generateBaselineProfile
./gradlew :app:assembleLiveRelease
./gradlew :baselineprofile:assembleFixtureBenchmarkRelease
./gradlew :baselineprofile:connectedFixtureBenchmarkReleaseAndroidTest

[Kaaveh]

@claude

[claude]

Claude finished @Kaaveh's task in 2m 24s —— View job

---

Code Review

• Gather context and understand the PR
• Read key changed files
• Analyze implementation for correctness, best practices, and issues
• Post comprehensive review feedback

---

Summary

This is a well-structured PR that adds deterministic Baseline Profile generation and startup benchmarks. The core approach — isolating the fixture API in a separate product flavor, filtering fixture-only classes from the profile rules, and reporting TTFD via ReportDrawnWhen — is solid. A few issues worth addressing before merging:

---

Issues

🔴 Hardcoded package name in BaselineProfileGenerator

BaselineProfileGenerator.kt:58 hardcodes "ir.composenews" as the package name for rule.collect(...):

rule.collect(
    packageName = "ir.composenews",  // hardcoded
    ...
)

StartupBenchmarks.kt correctly reads targetAppId from instrumentation arguments (line 50–53). The generator should do the same, or at minimum use the same mechanism — otherwise profile collection would silently target the wrong app if the applicationId ever changes or gets a suffix per variant.

Fix this →

🔴 Leftover commented-out include in settings.gradle.kts

settings.gradle.kts:57 has a stale commented-out include directly above the real one:

//include(":baselineprofile")   // ← dead code
...
include(":baselineprofile")

Additionally, the real include(":baselineprofile") appears at the bottom of the file after the check(JavaVersion...) block, breaking the convention followed by all other modules.

Fix this →

🟡 Missing API-level guard on notification permission in BaselineProfileGenerator

StartupBenchmarks.kt:57 correctly gates the POST_NOTIFICATIONS grant behind a Build.VERSION.SDK_INT >= TIRAMISU check. BaselineProfileGenerator.kt:53–59 calls grantRuntimePermission unconditionally. While profile generation requires API 33+ on non-rooted devices in practice, an explicit guard keeps the code consistent and future-safe.

Fix this →

🟡 Java version inconsistency in baselineprofile/build.gradle.kts

baselineprofile/build.gradle.kts:18–19 sets sourceCompatibility/targetCompatibility = JavaVersion.VERSION_11, while the app module and the rest of the project use VERSION_17. There's no reason for the benchmark module to target a lower JVM version.

Fix this →

🟡 ReportDrawnWhen edge case for empty market lists in production

MarketListScreen.kt:

ReportDrawnWhen {
    lazyPagingItems.itemCount > 0 &&
        lazyPagingItems.loadState.refresh is LoadState.NotLoading
}

If the market list ever returns zero items (network error state where items are empty, or a genuine empty response), the itemCount > 0 gate will never trigger and TTFD will never be reported in production. The empty-error case would be better handled by also triggering on an error load state, so reportFullyDrawn() is always eventually called:

ReportDrawnWhen {
    lazyPagingItems.loadState.refresh !is LoadState.Loading
}

That said, for the benchmarks themselves this is not a problem since the fixture API always returns 60 items.

🟡 CI no longer builds or verifies debug APKs

build.yml replaced ./gradlew assemble (which built all variants of all modules) with only assembleLiveRelease + assembleFixtureBenchmarkRelease. This means:

• Debug builds are never assembled in CI
• Module-level compile errors in non-release paths could slip through

Consider adding :app:assembleLiveDebug alongside the release build, or restoring a per-module assembleDebug pass.

---

Minor observations

• app/build.gradle.kts:57 comment says "Mock MarketApi" — should be "fixture" to match the PR's consistent terminology.
• filterPredicate string matching in BaselineProfileGenerator.kt:70–72 ("FixtureMarketsApi", "BackendModulesKt") will silently fail to exclude classes if either is renamed. A Kotlin-level @KeepForProfiles annotation or a separate Proguard keep file would be more robust long-term, though the current approach is fine for now.
• The managed device block (lines 31–37 of baselineprofile/build.gradle.kts) is commented out. Leaving a note about why (e.g., "requires device farm — enable for CI with GMD") would help future maintainers.

---

What's well done

• The fixture flavor approach cleanly isolates the deterministic API without touching any real production code paths (repository, paging source, domain, ViewModel, Compose UI all remain active).
• filterPredicate excluding FixtureMarketsApi and BackendModulesKt from the profile rules is exactly right.
• ReportDrawnWhen in PagedMarketListScreen is the correct way to signal TTFD for paged content.
• targetAppId from instrumentation arguments in StartupBenchmarks is the right pattern for variant-aware benchmarks.
• Reading targetAppId dynamically in androidComponents.onVariants (build.gradle.kts:62–68) avoids hardcoding.

[masoudkarimi]

@Kaaveh
Run claude again. I fixed the mentioned issues.

[Kaaveh]

@claude let's review the new changes again

[claude]

Claude finished @Kaaveh's task in 1m 43s —— View job

---

Code Review (Updated)

• Gather context from previous review
• Examine the latest diff to see what changed
• Check if previously identified issues were addressed
• Review any new changes
• Post comprehensive review feedback

---

All previously raised issues have been addressed. Here's the status on each:

Previously Identified Issues — All Fixed ✅

🔴 Hardcoded package name → Fixed

BaselineProfileGenerator.kt now declares a targetPackage property that reads from instrumentation arguments, matching StartupBenchmarks.kt:

private val targetPackage: String
    get() = InstrumentationRegistry.getArguments().getString("targetAppId")
        ?: error("targetAppId instrumentation argument not set")

The rule.collect(packageName = targetPackage, ...) call is now variant-aware. ✅

🔴 Stale commented-out include in settings.gradle.kts → Fixed

The duplicate //include(":baselineprofile") is gone, and the real include(":baselineprofile") is now placed before the check(JavaVersion...) block. ✅

🟡 Missing API-level guard on notification permission → Fixed

The grantRuntimePermission call in BaselineProfileGenerator is now inside:

if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.TIRAMISU) { ... }

Consistent with StartupBenchmarks. ✅

🟡 Java version inconsistency → Fixed

baselineprofile/build.gradle.kts now uses VERSION_17 for both sourceCompatibility and targetCompatibility. ✅

🟡 ReportDrawnWhen edge case for empty lists → Fixed

Updated to:

ReportDrawnWhen {
    lazyPagingItems.loadState.refresh !is LoadState.Loading
}

This correctly fires on error and empty states, so reportFullyDrawn() is always eventually called in production. ✅

🟡 CI missing debug build → Fixed

build.yml now explicitly assembles :app:assembleLiveDebug alongside the release and benchmark APKs. ✅

Minor: "Mock MarketApi" comment → Fixed

app/build.gradle.kts now says "Use a separate build variant to provide the fixture MarketApi implementation." — consistent terminology. ✅

---

New Implementation — Looks Good

ComposeNewsApplication.kt cleanly uses + backendModules to merge flavor-specific Koin modules:

modules(
    listOf(
        dispatcherModule,
        localDatasourceModule,
        ...
    ) + backendModules,
)

No production code paths are affected by the flavor split. ✅

FixtureMarketsApi returns 60 deterministic entries with correct paging math (offset = (page-1) * perPage), ensuring the LazyColumn is always scrollable during profile generation. ✅

baselineprofile/build.gradle.kts uses androidComponents.onVariants to inject targetAppId from the actual built APK artifacts loader — no hardcoding anywhere in the module config. ✅

---

One Remaining Minor Observation

The filterPredicate in BaselineProfileGenerator.kt uses string matching on class names:

filterPredicate = { profileRule ->
    !profileRule.contains("FixtureMarketsApi") &&
        !profileRule.contains("BackendModulesKt")
}

This is fine for now, but will silently stop working if either class is renamed. Worth keeping in mind for future refactors. No action needed for this PR.

---

This PR is in good shape. All blocking and non-blocking issues from the previous review have been resolved.

[masoudkarimi]

Hi @Kaaveh, we have two possible approaches for keeping the Baseline Profiles up to date:

1. Keep profile generation manual

We can keep the current CI setup and regenerate the profiles manually whenever a change affects startup, navigation, or another critical user journey.

The developer making the change should run:

./gradlew :app:generateBaselineProfile

If the generated profile files change, they should review and commit them.

When needed, performance benchmarks can be run separately on a physical device:

./gradlew :baselineprofile:connectedFixtureBenchmarkReleaseAndroidTest

2. Automate profile generation in GitHub Actions

We can add a CI job that:

1. Starts a Gradle-managed Android emulator.
2. Generates the Baseline Profiles.
3. Compares them with the committed profile files.
4. Fails the pipeline if they differ, asking the developer to regenerate and commit them.

The disadvantage is the additional CI time and cost. GitHub-hosted jobs run on fresh VMs, and Android system images are not covered by our existing Gradle cache. Unless we add a separate cache, use a custom runner image, or use a persistent self-hosted runner, the job will need to download the Android system image again.

It must then create and boot the emulator, build the app, and execute the profile generator. Because of this overhead, running profile generation for every PR may not be worthwhile.

I suggest keeping profile generation manual for now:

• Keep :baselineprofile:assembleFixtureBenchmarkRelease in every PR to ensure the profile generator and benchmark code compile.
• Regenerate and commit the profiles manually when startup, navigation, or critical user journeys change.
• Run performance benchmarks separately on a physical device when needed, since emulator benchmark results are not reliable.

[Kaaveh]

Yes, I agree with your suggestion.