[DSIP-107][Scheduler] Add schedule missed fire policy by liang-wenjie · Pull Request #18464 · apache/dolphinscheduler

liang-wenjie · 2026-08-04T09:01:26Z

Was this PR generated or assisted by AI?

YES. AI assisted with reviewing the previous implementation, refining domain naming and compatibility behavior, adding focused tests, and preparing this pull request. The changes were reviewed and validated by the contributor.

Purpose of the pull request

This pull request adds a schedule-level missed fire policy for Cron schedules as an independent part of DSIP #18454.

It is split from the closed PR #18458 so that missed-fire handling can be reviewed separately from the fixed-interval trigger design. Fixed-interval scheduling is intentionally out of scope for this pull request.

The policy uses scheduler-domain terminology instead of Quartz-specific names. DolphinScheduler currently uses Quartz 2.3.2 and explicitly calls withMisfireHandlingInstructionIgnoreMisfires() when building Cron triggers, so the default is FIRE_ALL_MISSED to preserve the existing behavior.

Related to #18454.
Supersedes the missed-fire policy portion of #18458.

Brief change log

Add ScheduleMissedFirePolicy with SKIP_MISSED, FIRE_ONCE_NOW, and FIRE_ALL_MISSED.
Persist missedFirePolicy in schedule API models and the missed_fire_policy database column.
Add CronScheduleBuilderFactory with separate implementations for all three policies.
Default missing or legacy policy values to FIRE_ALL_MISSED to preserve the current IgnoreMisfires behavior.
Add schedule form options and English/Chinese locale text in the UI.
Add unit tests for factory selection, all three Quartz mappings, and the null/default behavior.

Verify this pull request

This change added tests and can be verified as follows:

Added CronScheduleBuilderFactoryTest covering all policies and the default behavior.
Maven Spotless checks passed for the affected DAO, Quartz scheduler, and API modules.
License headers were added to both new upgrade SQL files.
UI Prettier checks passed.
UI ESLint checks passed.
vue-tsc --noEmit passed.
Java unit tests could not be executed locally because the local environment provides a JRE without javac; they are expected to run in CI.

Pull Request Notice

This pull request does not introduce an incompatible change. Existing schedules and requests that omit the new field retain the current Quartz IgnoreMisfires behavior through the FIRE_ALL_MISSED default.

ruanwenjun · 2026-08-04T09:45:19Z

    private Date endTime;
    private String crontab;
    private String timezoneId;
+    private ScheduleMissedFirePolicy missedFirePolicy = ScheduleMissedFirePolicy.FIRE_ONCE_NOW;


Right now, the default misfire policy is FIRE_ALL_MISSED?

The previous implementation did not set an explicit policy, so Quartz used SMART_POLICY. In Quartz 2.3.2, CronTriggerImpl.updateAfterMisfire() explicitly translates SMART_POLICY to MISFIRE_INSTRUCTION_FIRE_ONCE_NOW. Therefore, I kept FIRE_ONCE_NOW as the default to preserve the current behavior rather than changing existing schedules to FIRE_ALL_MISSED.

Which version are you using? ds use withMisfireHandlingInstructionIgnoreMisfires to set the policy.

You are right. DolphinScheduler uses Quartz 2.3.2, and the current dev implementation explicitly calls withMisfireHandlingInstructionIgnoreMisfires(), which maps to MISFIRE_INSTRUCTION_IGNORE_MISFIRE_POLICY. I had incorrectly reasoned from Quartz SMART_POLICY instead of checking the existing builder call. Updated in 2868f09: API/database/UI defaults and null fallback now use FIRE_ALL_MISSED, and the default factory test verifies the existing IgnoreMisfires behavior.

ruanwenjun · 2026-08-04T09:47:29Z

+
+import org.quartz.CronScheduleBuilder;
+
+final class QuartzScheduleMissedFirePolicyApplier {


Don't add this util, it's better to create CronScheduleBuilderFactory and add three different implementation.

Updated in c73327e. I replaced the util with CronScheduleBuilderFactory and added three separate implementations: SkipMissedCronScheduleBuilderFactory, FireOnceNowCronScheduleBuilderFactory, and FireAllMissedCronScheduleBuilderFactory. The focused test now verifies both factory selection and the Quartz misfire instruction for each policy.

ruanwenjun · 2026-08-04T09:51:20Z

    start_time: '开始时间',
    end_time: '结束时间',
    crontab: 'Crontab',
+    missed_fire_policy: '错过触发策略',


Suggested change

missed_fire_policy: '错过触发策略',

missed_fire_policy: '定时错过策略',

Updated to 定时错过策略 in c73327e. Thanks for the suggestion.

liang-wenjie · 2026-08-06T01:23:50Z

The latest commit 2868f09 addresses the review feedback and the previous CI issues. The Backend, Frontend, Test, API-Test, E2E, Docs, and CodeQL workflows currently show action_required with no jobs, so they appear to be waiting for a maintainer to approve the fork workflows. When convenient, could a maintainer please approve the workflow runs and continue the review? The PR also still needs a milestone and a valid type label for the milestone-label check. Thank you.

ruanwenjun

Should also update incompatible.md

ruanwenjun · 2026-08-06T01:55:57Z

+    private ScheduleMissedFirePolicy defaultMissedFirePolicy(ScheduleMissedFirePolicy missedFirePolicy) {
+        return missedFirePolicy == null ? ScheduleMissedFirePolicy.FIRE_ALL_MISSED : missedFirePolicy;
+    }


Is this needed? Your already set a initialize value at scheduleParam.

Removed the redundant fallback helper. SchedulerServiceImpl now uses ScheduleParam#getMissedFirePolicy directly, since ScheduleParam already initializes the default value.

ruanwenjun · 2026-08-06T02:00:40Z

+    public CronScheduleBuilder createCronScheduleBuilder(String cronExpression) {
+        return CronScheduleBuilder.cronSchedule(cronExpression)
+                .withMisfireHandlingInstructionFireAndProceed();
+    }


Suggested change

public CronScheduleBuilder createCronScheduleBuilder(String cronExpression) {

return CronScheduleBuilder.cronSchedule(cronExpression)

.withMisfireHandlingInstructionFireAndProceed();

}

public CronScheduleBuilder createCronScheduleBuilder(Schedule schedule) {

return CronScheduleBuilder.cronSchedule(cronExpression)

.withMisfireHandlingInstructionFireAndProceed()

.inTimeZone(DateUtils.getTimezone(schedule.getTimezoneId()));

}

Updated CronScheduleBuilderFactory to accept the complete Schedule. All three implementations now create the cron schedule from Schedule#getCrontab and apply the schedule timezone internally. The factory tests also verify the configured timezone for every policy.

liang-wenjie · 2026-08-06T11:30:42Z

Addressed the latest review feedback in 060743f:

removed the redundant SchedulerServiceImpl default-policy helper;
changed CronScheduleBuilderFactory and all implementations to accept Schedule and configure the timezone internally;
added timezone assertions to CronScheduleBuilderFactoryTest;
documented the 3.5.0 missed_fire_policy schema change in the English and Chinese incompatible guides;
moved the upgrade DDL from the incorrect 3.3.2_schema post files into the current 3.5.0_schema MySQL/PostgreSQL scripts, which addresses the historical-version schema-check failures.

Local verification:

spotless:check for the changed backend modules: passed
CronScheduleBuilderFactoryTest: 4 tests passed
git diff --check: passed

The previous dead-link failure is unrelated to this PR content: lychee v0.24.0 rejects the existing repository setting include_fragments = false (it expects a string or table). The milestone-label check still requires a maintainer-added milestone/type label.

ruanwenjun

LGTM

liang-wenjie · 2026-08-07T02:01:56Z

Thank you for the approval. I checked the remaining failed checks on commit 060743f:

Every Unit-Test matrix job failed before Maven started because Maven Central returned HTTP 429 while mvnw was downloading apache-maven-3.8.4-bin.zip. This is an external rate-limit failure, not a test failure.
milestone-label-check ran before milestone 3.5.0 was assigned. The PR now has milestone 3.5.0 and valid labels, so that check should pass when rerun.
dead-link still fails while parsing the repository existing lychee.toml with lychee v0.24.0 (include_fragments = false is rejected); it does not reach or check this PR documentation links.

I do not have repository admin permission to rerun these jobs. Could a maintainer please rerun the failed Test and Mergeable checks (and handle/rerun dead-link as appropriate)? The remaining Backend/API-Test/E2E/CodeQL workflows are still in progress.

SbloodyS

The selected missed-fire policy is never submitted and updates reset it

The new selector writes its value to timingForm.missedFirePolicy, but use-modal.ts#getTimingData only serializes startTime, endTime, crontab, and timezoneId into the schedule request.

Consequently:

Creating a schedule with SKIP_MISSED or FIRE_ONCE_NOW always stores the default FIRE_ALL_MISSED; the selector currently has no effect.
Updating a schedule with a non-default policy omits the field, after which ScheduleParam supplies FIRE_ALL_MISSED and SchedulerServiceImpl#updateSchedule overwrites the existing policy.

Please include missedFirePolicy in the schedule JSON constructed by getTimingData.

For backward compatibility, the backend should also distinguish creation defaults from an omitted update field: an omitted field during creation may use FIRE_ALL_MISSED, while an omitted field during update should preserve the currently stored policy. Please add create/update regression tests covering all three policies and an update request from a legacy client that omits the field.

…y on update Address review feedback (SbloodyS): - Frontend: include missedFirePolicy in the schedule create/update payload so the selected policy is actually persisted to the backend. - Backend: distinguish an omitted JSON field from an explicit value. A new missedFirePolicySet marker tracks field presence, because Jackson cannot tell omission from an explicit null. - create: omitted or explicit null falls back to FIRE_ALL_MISSED - update: when the client omits the field (e.g. an older client), the existing stored policy is preserved instead of being overwritten - Add unit tests covering create/update semantics and JSON presence detection. Co-Authored-By: WorkBuddy <workbuddy@tencent.com>

liang-wenjie · 2026-08-07T06:53:48Z

Thanks for the review, @SbloodyS. I've addressed both points:

Frontend serialization — missedFirePolicy is now included in the schedule create/update payload in use-modal.ts (getTimingData), so the selected policy is actually persisted. The timing form already defaults to FIRE_ALL_MISSED and the modal binds/loads it correctly on edit.
Backward-compatible update — Jackson cannot distinguish an omitted JSON field from an explicit null, so I added a missedFirePolicySet presence marker in ScheduleParam. The setter flips it only when missedFirePolicy is present in the JSON:
- Create: omitted or explicit null falls back to FIRE_ALL_MISSED (safe default for new schedules).
- Update: the stored policy is overwritten only when the field is present and non-null; when an older client omits it, the existing policy is preserved.
Added unit tests covering create (all 3 policies + default fallback) and update (all 3 policies + preserve-on-omit), plus a test asserting the presence marker distinguishes omitted vs explicit values.

Let me know if you'd prefer a different default or behavior.

SbloodyS · 2026-08-07T07:20:08Z

You should check failed tests. @liang-wenjie

…re-policy-v2' into 2dev/feat/add-schedule-missed-fire-policy-v2

liang-wenjie · 2026-08-08T03:46:50Z

Thanks, @SbloodyS. I checked and fixed the failed schedule API tests in c804e50cf5 (test(api): fix schedule insert mock). The branch has also been updated with dev; current head is cd5a311164.

The relevant checks are now green:

Unit-Test (dolphinscheduler-api | Java 8) — passed
Unit-Test (dolphinscheduler-api | Java 11) — passed
API-Test / Backend / Frontend / E2E / schema checks — passed

The only remaining failed check is dead-link. It fails before checking any links because the workflow installs lychee v0.24.0, while the repository's existing .github/workflows/lychee.toml has include_fragments = false; v0.24.0 expects a string or table for this option. This is unrelated to the files changed by this PR.

Could you please re-review the missed-fire-policy changes when convenient? The requested frontend serialization, backward-compatible update behavior, and create/update regression coverage are all included in the current branch. Thank you.

SbloodyS · 2026-08-10T01:55:20Z

Thanks for addressing the previous frontend serialization and legacy-update compatibility issues. Those paths now look correct.

Reject invalid policy values instead of silently applying another policy

JSONUtils enables READ_UNKNOWN_ENUM_VALUES_AS_NULL, so an invalid request such as:

{
  "missedFirePolicy": "FIRE_ONCE_NWO"
}

invokes ScheduleParam#setMissedFirePolicy(null) and marks the field as present.
The current service behavior then silently handles it as follows:

Create: insertSchedule replaces the invalid value with FIRE_ALL_MISSED.
Update: updateSchedule ignores it and preserves the previous policy.
Both requests return successfully even though the requested policy was not applied. The create case is particularly risky because FIRE_ALL_MISSED may trigger multiple unexpected workflow executions after scheduler downtime.

Please reject an explicitly provided null or unrecognized policy with a request-validation error. Only an omitted field should use the create default or preserve the existing update value. Please add regression tests for explicit null and an unknown enum value, in addition to the existing omitted-field tests.

@liang-wenjie

liang-wenjie · 2026-08-12T01:43:55Z

Thanks for the detailed review, @SbloodyS. I have addressed the latest invalid-policy feedback in \1831d55e5\ (\ix(api): reject invalid missed fire policy\).

What changed:

Explicit \
ull\ and unrecognized \missedFirePolicy\ values are now rejected with \REQUEST_PARAMS_NOT_VALID_ERROR\ for both create and update.
Omitted \missedFirePolicy\ keeps the previous compatible behavior: create uses the default \FIRE_ALL_MISSED\, and update preserves the existing stored policy.
Added regression coverage for explicit null and unknown enum values on both insert and update, while keeping the existing omitted-field tests.

Local verification:

\SchedulerServiceTest\: 14 tests passed
\spotless:check -pl dolphinscheduler-api\: passed

Could you please re-review and clear the previous changes-requested review when convenient? Thank you.

SbloodyS

LGTM

boring-cyborg · 2026-08-12T02:13:07Z

Awesome work, congrats on your first merged pull request!

feat(schedule): add missed fire policy

89cd1ed

liang-wenjie requested review from SbloodyS, caishunfeng and ruanwenjun as code owners August 4, 2026 09:01

github-actions Bot added UI ui and front end related backend test labels Aug 4, 2026

github-actions Bot assigned liang-wenjie Aug 4, 2026

ruanwenjun reviewed Aug 4, 2026

View reviewed changes

liang-wenjie and others added 2 commits August 5, 2026 09:21

refactor(schedule): apply review suggestions

c73327e

fix(schedule): preserve existing misfire behavior

2868f09

liang-wenjie requested a review from ruanwenjun August 5, 2026 03:56

ruanwenjun reviewed Aug 6, 2026

View reviewed changes

fix: address missed fire policy review feedback

060743f

github-actions Bot added the document label Aug 6, 2026

ruanwenjun previously approved these changes Aug 7, 2026

View reviewed changes

ruanwenjun added this to the 3.5.0 milestone Aug 7, 2026

ruanwenjun added the feature new feature label Aug 7, 2026

SbloodyS requested changes Aug 7, 2026

View reviewed changes

SbloodyS added the first time contributor First-time contributor label Aug 7, 2026

liang-wenjie dismissed ruanwenjun’s stale review via dcf1d0d August 7, 2026 06:53

Merge branch 'dev' into 2dev/feat/add-schedule-missed-fire-policy-v2

f253dc4

liang-wenjie added 2 commits August 7, 2026 16:36

test(api): fix schedule insert mock

c804e50

Merge remote-tracking branch 'origin/2dev/feat/add-schedule-missed-fi…

cd5a311

…re-policy-v2' into 2dev/feat/add-schedule-missed-fire-policy-v2

fix(api): reject invalid missed fire policy

1831d55

Merge branch 'dev' into 2dev/feat/add-schedule-missed-fire-policy-v2

bf2a739

SbloodyS approved these changes Aug 12, 2026

View reviewed changes

SbloodyS merged commit d661eea into apache:dev Aug 12, 2026
118 of 119 checks passed

SbloodyS added the DSIP label Aug 12, 2026

SbloodyS changed the title ~~[DSIP-18454][Scheduler] Add schedule missed fire policy~~ [DSIP-107][Scheduler] Add schedule missed fire policy Aug 12, 2026


		import org.quartz.CronScheduleBuilder;

		final class QuartzScheduleMissedFirePolicyApplier {

	missed_fire_policy: '错过触发策略',
	missed_fire_policy: '定时错过策略',

Uh oh!

Conversation

liang-wenjie commented Aug 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Was this PR generated or assisted by AI?

Purpose of the pull request

Brief change log

Verify this pull request

Pull Request Notice

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ruanwenjun Aug 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

liang-wenjie commented Aug 6, 2026

Uh oh!

ruanwenjun left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

liang-wenjie commented Aug 6, 2026

Uh oh!

ruanwenjun left a comment

Choose a reason for hiding this comment

Uh oh!

liang-wenjie commented Aug 7, 2026

Uh oh!

SbloodyS left a comment

Choose a reason for hiding this comment

Uh oh!

liang-wenjie commented Aug 7, 2026

Uh oh!

SbloodyS commented Aug 7, 2026

Uh oh!

liang-wenjie commented Aug 8, 2026

Uh oh!

SbloodyS commented Aug 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Reject invalid policy values instead of silently applying another policy

Uh oh!

liang-wenjie commented Aug 12, 2026

Uh oh!

SbloodyS left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

boring-cyborg Bot commented Aug 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

liang-wenjie commented Aug 4, 2026 •

edited

Loading

ruanwenjun Aug 5, 2026 •

edited

Loading

SbloodyS commented Aug 10, 2026 •

edited

Loading