[AMORO-4048] Saving cleanup opeartion process info in table_process

[zhangwl9]

Why are the changes needed?

Close #4048.

Brief change log

This PR enhances the table cleanup scheduling logic by ensuring that all cleanup operations for tables are reliably tracked and persisted in the table_process table. The main implementation steps are:
2. Before executing a scheduled cleanup, a new TableProcessMeta record is created and persisted to represent the cleanup process, including metadata such as process ID, table ID, type, stage, and timestamps.
3. After the cleanup operation completes (successfully or with failure), the process record is updated with the final status, end time, and any failure message.

How was this patch tested?

• Add some test cases that check the changes thoroughly including negative and positive cases if possible

• Add screenshots for manual tests if appropriate

• Run test locally before making a pull request

Documentation

• Does this pull request introduce a new feature? (yes / no)
• If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

[codecov-commenter]

Codecov Report

❌ Patch coverage is 94.59459% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 28.84%. Comparing base (fda105e) to head (5779284).
⚠️ Report is 24 commits behind head on master.

Files with missing lines	Patch %	Lines
...amoro/server/scheduler/PeriodicTableScheduler.java	94.59%	3 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##             master    #4077      +/-   ##
============================================
+ Coverage     22.39%   28.84%   +6.44%     
- Complexity     2552     3958    +1406     
============================================
  Files           458      656     +198     
  Lines         42116    52462   +10346     
  Branches       5917     6644     +727     
============================================
+ Hits           9433    15133    +5700     
- Misses        31871    36229    +4358     
- Partials        812     1100     +288     

Flag	Coverage Δ
core	28.84% <94.59%> (?)
trino	?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[Copilot]

Pull request overview

This PR addresses [AMORO-4048] by persisting cleanup executor run information into the table_process table, similar to how optimizing executions are tracked.

Changes:

• Add persistence logic in PeriodicTableScheduler to insert/update table_process records for cleanup operations (RUNNING → SUCCESS/FAILED).
• Update inline scheduler tests to validate cleanup process persistence and expose testing hooks.
• Add a new test covering cleanup process status transitions and stored failure messages.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 6 comments.

File	Description
amoro-ams/src/main/java/org/apache/amoro/server/scheduler/PeriodicTableScheduler.java	Adds creation and completion persistence for cleanup processes in table_process.
amoro-ams/src/test/java/org/apache/amoro/server/scheduler/inline/PeriodicTableSchedulerTestBase.java	Exposes test-only wrappers to call the new persistence methods.
amoro-ams/src/test/java/org/apache/amoro/server/scheduler/inline/TestPeriodicTableSchedulerCleanup.java	Adds a persistence-focused test validating cleanup process lifecycle in table_process.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 5 comments.

Comments suppressed due to low confidence (1)

amoro-ams/src/main/java/org/apache/amoro/server/scheduler/PeriodicTableScheduler.java:167

• This changes cleanup-time persistence semantics in a way that can produce incorrect state: (1) persistUpdatingCleanupTime(tableRuntime) is called on success, but persistCleanupResult(...) later also updates lastCleanTime (see PersistenceHelper.persistAndSetCompleted), causing duplicate/competing updates. (2) On failures, persistCleanupResult still updates lastCleanTime via persistAndSetCompleted, which can incorrectly mark a failed cleanup as recently completed and potentially suppress subsequent retries/scheduling. A concrete fix is to update lastCleanTime only on success (and do it in exactly one place), and keep process status persistence independent of lastCleanTime updates.

  private void executeTask(TableRuntime tableRuntime) {
    TableProcessMeta cleanProcessMeta = null;
    CleanupOperation cleanupOperation = null;
    Throwable executionError = null;

    try {
      if (isExecutable(tableRuntime)) {
        cleanupOperation = getCleanupOperation();
        cleanProcessMeta = createCleanupProcessInfo(tableRuntime, cleanupOperation);

        execute(tableRuntime);
        // Different tables take different amounts of time to execute the end of execute(),
        // so you need to perform the update operation separately for each table.
        persistUpdatingCleanupTime(tableRuntime);
      }
    } catch (Throwable t) {
      executionError = t;
      logger.error(
          "Failed to execute cleanup operation {} for table {}",
          cleanupOperation,
          tableRuntime.getTableIdentifier(),
          t);
    } finally {
      persistCleanupResult(tableRuntime, cleanupOperation, cleanProcessMeta, executionError);

      scheduledTables.remove(tableRuntime.getTableIdentifier());
      scheduleIfNecessary(tableRuntime, getNextExecutingTime(tableRuntime));
    }
  }

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[zhangwl9]

@xxubai could you help me review the code when you are free? Thank you very much