Skip to content

Commit 437516a

Browse files
authored
Merge pull request #334 from durable-workflow/sdk-python/218-checkpointed-batch
Add bounded checkpointed batch example
2 parents edcd90f + 0a34792 commit 437516a

4 files changed

Lines changed: 407 additions & 0 deletions

File tree

‎README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -99,6 +99,7 @@ for the complete cross-SDK contract.
9999
- [Python SDK guide](https://durable-workflow.com/docs/2.0/polyglot/python/)
100100
- [Complete SDK reference](docs/sdk-reference.md)
101101
- [Runnable examples](examples/)
102+
- [Checkpointed batch source example](examples/CHECKPOINTED_BATCH.md) (unreleased accessor; synthetic provider)
102103
- [Separate workflow and activity worker processes](docs/guides/separate-workers.md)
103104
- [Symmetric SDK playground](https://github.com/durable-workflow/sample-app#symmetric-sdk-playground)
104105

‎examples/CHECKPOINTED_BATCH.md‎

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
# Resume a bounded batch
2+
3+
This source example uses the unreleased acknowledged-checkpoint API from
4+
[SDK333](https://github.com/durable-workflow/sdk-python/pull/333). It requires a
5+
Server advertising `activity_heartbeat_resume` v1. Its source tests do not prove
6+
published-package compatibility, real process loss or Server persistence.
7+
8+
On Linux, from a source checkout with its development environment, point it at
9+
an isolated development Server. Supply a **new** directory for each invocation.
10+
The synthetic provider receipts remain there for inspection; existing
11+
directories are refused.
12+
13+
```bash
14+
export SERVER_URL=http://127.0.0.1:8080
15+
export WORKFLOW_TOKEN=your-development-token
16+
python -m examples.checkpointed_batch ./receipts-before before-commit
17+
python -m examples.checkpointed_batch ./receipts-commit after-commit
18+
python -m examples.checkpointed_batch ./receipts-checkpoint after-checkpoint
19+
```
20+
21+
The importable `BatchActivity.execute` handler carries only serializable provider
22+
configuration, so it also passes the SDK's cooperative callback preflight.
23+
The activity processes three items, with a maximum of sixteen bounded items.
24+
Its two attempts, one-second retry delay, ten-second heartbeat timeout,
25+
thirty-second start-to-close timeout and sixty-second schedule-to-close timeout
26+
stay fixed. Recovery does not extend those budgets. The example's worker wait
27+
is bounded separately.
28+
29+
| First-attempt failure | Replacement action |
30+
| --- | --- |
31+
| Before provider commit | Commit item zero, then acknowledge progress |
32+
| After commit, before acknowledgement | Reconcile the stable provider key, then acknowledge progress |
33+
| After acknowledgement | Resume at the received next-item cursor |
34+
| Acknowledgement response lost | The next claim may contain the checkpoint or none; reconcile any repeated provider operation |
35+
36+
`last_heartbeat_details(result_type=Progress)` reads the immutable received
37+
snapshot. Unsupported claims refuse instead of inventing empty state. A
38+
supported empty checkpoint starts from zero. The typed cursor binds the full
39+
business input and must be within the item range.
40+
41+
Provider receipts are separate durable state. A key uses the business ID and
42+
item index, never an attempt ID. Each receipt also binds the entire batch
43+
fingerprint, so appending, truncating or reordering input under a repeated
44+
business ID refuses even without a checkpoint. Revisited keys require the exact
45+
receipt. Partial or conflicting receipts are preserved and require
46+
reconciliation. File and directory entries are synchronized before commit or
47+
reconciliation returns. This is a private synthetic provider, not an actual
48+
payment or other external service.
49+
50+
The ordering is provider commit/reconcile, then `await ctx.heartbeat(...)`, then
51+
continue. A failed heartbeat stops the attempt; an uncertain response does not
52+
mean the checkpoint failed to commit. The SDK snapshot does not advance in
53+
place. A checkpoint does not atomically commit an external business effect.
54+
Use the real provider's idempotency, operation lookup and fencing contract,
55+
retain the business ID across restarts of that operation, and choose a new ID
56+
for a new business operation. The example does not promise exactly-once effects.
57+
58+
The source tests use synthetic files and acknowledgement callbacks without a
59+
Server. They cover all three failure boundaries, both lost-response outcomes,
60+
changed input without a checkpoint, malformed cursors and partial receipts:
61+
62+
```bash
63+
pytest -q tests/test_checkpointed_batch_example.py
64+
```

‎examples/checkpointed_batch.py‎

Lines changed: 168 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,168 @@
1+
"""Bounded source example using the unreleased checkpoint API. See CHECKPOINTED_BATCH.md."""
2+
from __future__ import annotations
3+
4+
import argparse
5+
import asyncio
6+
import hashlib
7+
import json
8+
import os
9+
from collections.abc import Awaitable, Callable, Generator
10+
from dataclasses import asdict, dataclass
11+
from pathlib import Path
12+
from typing import Any
13+
from uuid import uuid4
14+
15+
from durable_workflow import ActivityRetryPolicy, Client, Worker, activity, workflow
16+
from durable_workflow.errors import ActivityHeartbeatResumeUnavailable, NonRetryableError
17+
18+
FAILURES = ("before-commit", "after-commit", "after-checkpoint")
19+
ACTIVITY = "examples.checkpointed-batch"
20+
21+
22+
@dataclass(frozen=True)
23+
class Batch:
24+
business_id: str
25+
items: list[str]
26+
failure: str
27+
28+
def snapshot(self) -> tuple[str, tuple[str, ...], str]:
29+
if not isinstance(self.items, list) or not 1 <= len(self.items) <= 16 or self.failure not in FAILURES:
30+
raise NonRetryableError("use a business ID and one to sixteen bounded items")
31+
try:
32+
valid = all(isinstance(value, str) and 1 <= len(value.encode()) <= 128
33+
for value in [self.business_id, *self.items])
34+
except UnicodeEncodeError:
35+
valid = False
36+
if not valid:
37+
raise NonRetryableError("use a business ID and one to sixteen bounded items")
38+
items = tuple(self.items)
39+
encoded = json.dumps([self.business_id, items], ensure_ascii=False, separators=(",", ":")).encode()
40+
return self.business_id, items, hashlib.sha256(encoded).hexdigest()
41+
42+
43+
@dataclass(frozen=True)
44+
class Progress:
45+
batch_sha256: str
46+
next_item: int
47+
48+
49+
@dataclass(frozen=True)
50+
class ReceiptStore:
51+
"""Synthetic durable provider state, independent of Server checkpoints."""
52+
53+
root: Path
54+
55+
@classmethod
56+
def create(cls, root: Path) -> ReceiptStore:
57+
root.mkdir(mode=0o700)
58+
return cls(root)
59+
60+
def sync_directory(self) -> None:
61+
descriptor = os.open(self.root, os.O_RDONLY | os.O_DIRECTORY)
62+
try:
63+
os.fsync(descriptor)
64+
finally:
65+
os.close(descriptor)
66+
67+
def commit_or_reconcile(self, business_id: str, fingerprint: str, index: int, item: str) -> bool:
68+
key = json.dumps([business_id, index], ensure_ascii=False, separators=(",", ":")).encode()
69+
path = self.root / (hashlib.sha256(key).hexdigest() + ".json")
70+
receipt = json.dumps({"key": [business_id, index], "batch_sha256": fingerprint, "item": item},
71+
ensure_ascii=False, sort_keys=True, separators=(",", ":")).encode()
72+
try:
73+
with path.open("xb") as stream:
74+
stream.write(receipt)
75+
stream.flush()
76+
os.fsync(stream.fileno())
77+
created = True
78+
except FileExistsError:
79+
with path.open("rb") as stream:
80+
if stream.read(4097) != receipt:
81+
raise NonRetryableError(
82+
"provider receipt conflicts or is partial; reconcile before retrying",
83+
) from None
84+
os.fsync(stream.fileno())
85+
created = False
86+
self.sync_directory()
87+
return created
88+
89+
90+
async def process_batch(
91+
store: ReceiptStore, batch: Batch, received: Progress | None, attempt: int,
92+
acknowledge: Callable[[Progress], Awaitable[None]],
93+
) -> dict[str, Any]:
94+
business_id, items, fingerprint = batch.snapshot()
95+
if type(attempt) is not int or attempt < 1:
96+
raise NonRetryableError("the activity attempt must be positive")
97+
if received is not None and (
98+
received.batch_sha256 != fingerprint or type(received.next_item) is not int
99+
or not 0 <= received.next_item <= len(items)
100+
):
101+
raise NonRetryableError("checkpoint belongs to different input or is out of range")
102+
start = received.next_item if received is not None else 0
103+
def fail(point: str, index: int) -> None:
104+
if attempt == 1 and index == 0 and batch.failure == point:
105+
raise RuntimeError("planned first-attempt failure at " + point)
106+
107+
for index in range(start, len(items)):
108+
fail("before-commit", index)
109+
await asyncio.to_thread(store.commit_or_reconcile, business_id, fingerprint, index, items[index])
110+
fail("after-commit", index)
111+
await acknowledge(Progress(fingerprint, index + 1))
112+
fail("after-checkpoint", index)
113+
return {"business_id": business_id, "completed": len(items)}
114+
115+
116+
@workflow.defn(name=ACTIVITY)
117+
class BatchWorkflow:
118+
def run(self, ctx: workflow.WorkflowContext, batch: dict[str, Any]) -> Generator[Any, Any, Any]:
119+
return (yield ctx.schedule_activity(
120+
ACTIVITY, [batch], retry_policy=ActivityRetryPolicy(max_attempts=2, backoff_seconds=[1]),
121+
heartbeat_timeout=10, start_to_close_timeout=30, schedule_to_close_timeout=60,
122+
))
123+
124+
125+
@dataclass(frozen=True)
126+
class BatchActivity:
127+
"""Importable handler carrying only the provider's serializable configuration."""
128+
129+
store: ReceiptStore
130+
131+
@activity.defn(name=ACTIVITY)
132+
async def execute(self, body: dict[str, Any]) -> dict[str, Any]:
133+
ctx = activity.context()
134+
try:
135+
received = ctx.last_heartbeat_details(result_type=Progress)
136+
except ActivityHeartbeatResumeUnavailable:
137+
raise NonRetryableError("this example requires activity_heartbeat_resume v1") from None
138+
139+
async def acknowledge(progress: Progress) -> None:
140+
await ctx.heartbeat(asdict(progress))
141+
142+
return await process_batch(self.store, Batch(**body), received, ctx.info.attempt_number, acknowledge)
143+
144+
145+
async def run(root: Path, failure: str) -> None:
146+
store = await asyncio.to_thread(ReceiptStore.create, root)
147+
queue = "checkpointed-batch-" + uuid4().hex
148+
async with Client(os.environ.get("SERVER_URL", "http://127.0.0.1:8080"),
149+
token=os.environ.get("WORKFLOW_TOKEN"), namespace="default", timeout=10) as client:
150+
worker = Worker(client, task_queue=queue, workflows=[BatchWorkflow], activities=[BatchActivity(store).execute])
151+
handle = await client.start_workflow(
152+
workflow_type=ACTIVITY, task_queue=queue, workflow_id=queue,
153+
input=[asdict(Batch("demo-" + uuid4().hex, ["first", "second", "third"], failure))],
154+
)
155+
await worker.run_until(workflow_id=handle.workflow_id, timeout=70)
156+
print(json.dumps(await handle.result(timeout=3)))
157+
158+
159+
def main() -> None:
160+
parser = argparse.ArgumentParser(description=__doc__)
161+
parser.add_argument("receipts", type=Path, help="a new private synthetic-provider directory")
162+
parser.add_argument("failure", choices=FAILURES)
163+
args = parser.parse_args()
164+
asyncio.run(run(args.receipts, args.failure))
165+
166+
167+
if __name__ == "__main__":
168+
main()

0 commit comments

Comments
 (0)