Skip to content

Wrong default in max content length if its not defined in site configuration #337

Description

@Zharktas

Describe the bug

If ckan site configuration does not define ckanext.datapusher_plus.max_content_length, it should use the default declared in

- key: ckanext.datapusher_plus.max_content_length
editable: true
default: 1256000000000
description: |
Maximum content length in bytes

Instead datapusher+ uses the value defined in

MAX_CONTENT_LENGTH = tk.asint(
tk.config.get("ckanext.datapusher_plus.max_content_length", "5000000")
)

which results in an error shown with wrong value.

With default values, the code passes this check as conf.PRREVIEW_ROWS is 0.

# Check size before download
if cl:
try:
if int(cl) > max_content_length and conf.PREVIEW_ROWS > 0:
raise utils.JobError(
f"Resource too large to download: {DataSize(int(cl)):.2MB} "
f"> max ({DataSize(int(max_content_length)):.2MB})."
)
except ValueError:
pass

But if fails this check as not 0 equals true.

if length > max_content_length and not conf.PREVIEW_ROWS:
raise utils.JobError(
f"Resource too large to process: {length} > max ({max_content_length})."
)

To Reproduce
Steps to reproduce the behavior:

  1. Do not define site configuration for datapusher+
  2. Upload file larger than 5Mb and smaller than 1GB

Expected behavior
File should be handled by datapusher+ without errors.

Screenshots

Image

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions