Describe the bug
If ckan site configuration does not define ckanext.datapusher_plus.max_content_length, it should use the default declared in
|
- key: ckanext.datapusher_plus.max_content_length |
|
editable: true |
|
default: 1256000000000 |
|
description: | |
|
Maximum content length in bytes |
Instead datapusher+ uses the value defined in
|
MAX_CONTENT_LENGTH = tk.asint( |
|
tk.config.get("ckanext.datapusher_plus.max_content_length", "5000000") |
|
) |
which results in an error shown with wrong value.
With default values, the code passes this check as conf.PRREVIEW_ROWS is 0.
|
# Check size before download |
|
if cl: |
|
try: |
|
if int(cl) > max_content_length and conf.PREVIEW_ROWS > 0: |
|
raise utils.JobError( |
|
f"Resource too large to download: {DataSize(int(cl)):.2MB} " |
|
f"> max ({DataSize(int(max_content_length)):.2MB})." |
|
) |
|
except ValueError: |
|
pass |
But if fails this check as not 0 equals true.
|
if length > max_content_length and not conf.PREVIEW_ROWS: |
|
raise utils.JobError( |
|
f"Resource too large to process: {length} > max ({max_content_length})." |
|
) |
To Reproduce
Steps to reproduce the behavior:
- Do not define site configuration for datapusher+
- Upload file larger than 5Mb and smaller than 1GB
Expected behavior
File should be handled by datapusher+ without errors.
Screenshots

Describe the bug
If ckan site configuration does not define
ckanext.datapusher_plus.max_content_length, it should use the default declared indatapusher-plus/ckanext/datapusher_plus/config_declaration.yaml
Lines 12 to 16 in 474ce97
Instead datapusher+ uses the value defined in
datapusher-plus/ckanext/datapusher_plus/config.py
Lines 86 to 88 in 474ce97
which results in an error shown with wrong value.
With default values, the code passes this check as
conf.PRREVIEW_ROWSis 0.datapusher-plus/ckanext/datapusher_plus/jobs/stages/download.py
Lines 277 to 286 in 474ce97
But if fails this check as
not 0equals true.datapusher-plus/ckanext/datapusher_plus/jobs/stages/download.py
Lines 423 to 426 in 474ce97
To Reproduce
Steps to reproduce the behavior:
Expected behavior
File should be handled by datapusher+ without errors.
Screenshots