perf: serialize dataset push payloads once per chunk

[vdusek]

Pushing to a dataset serialized every item in its own asyncio.to_thread hop (through Crawlee's json_dumps), so a push paid one thread round-trip per item. That helper also pretty-prints with indent=2, and each payload was UTF-8 encoded twice — once for the per-item size check, once again while chunking.

_check_and_serialize and _chunk_by_size are replaced by a single blocking _serialize_chunk(items, offset) that fills one size-bounded JSON array and returns the next offset. push_data drives it with one asyncio.to_thread per chunk and pushes each chunk before serializing the next, so payloads are still streamed rather than all materialized up front.

Serialization is now local and compact (json.dumps(..., separators=(',', ':'))) rather than Crawlee's json_dumps, whose indent=2 is deliberate for the on-disk files people actually read. Nothing reads the wire format. Dropping the indent also lets CPython take the C encoder fast path, which it only does when indent is None.

Measured on 20k realistic items against a mocked API client:

	time	payload
before	1.081s	10.78 MB
after	0.075s	9.34 MB

The payload saving is shape-dependent — larger for small or deeply nested items — and compounds with the brotli work in #1052.

Two spurious API calls disappear as a side effect: the old chunker emitted an empty [] chunk whenever the first item of a chunk landed within 2 bytes of the limit, and it pushed [] rather than nothing when PPE charging drove the push limit down to 0.

Stored items are unaffected. The only observable difference is that raw request bodies are no longer pretty-printed.

✍️ Drafted by Claude Code

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.97%. Comparing base (cad8d7e) to head (0df0c12).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1081      +/-   ##
==========================================
+ Coverage   91.83%   91.97%   +0.13%     
==========================================
  Files          51       51              
  Lines        3235     3229       -6     
==========================================
- Hits         2971     2970       -1     
+ Misses        264      259       -5     

Flag	Coverage Δ
e2e	35.33% <9.52%> (+0.03%)	⬆️
integration	56.95% <85.71%> (-0.08%)	⬇️
unit	84.20% <100.00%> (+0.99%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.