You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
S3: "unsupported extension" warning for file names containing dots when default_extension is set #19690
S3Source.get_fields uses pathlib.Path(...).suffix, which is purely lexical and returns everything after the last dot. For S3 files whose names contain dots — e.g. events.account.update-2026-05-27-<hash>.gz — stripping the .gz compression suffix leaves .update-2026-05-27-<hash> as the apparent format extension. That is not a real format, so the inferrer emits an "unsupported extension" warning.
Because the leftover value is non-empty, the default_extension fallback on the path_spec is never reached — so even with default_extension: json set, every such file warns and is not resolved.
To Reproduce
Configure an S3 path_spec with default_extension: json.
Point it at files whose names contain dots before the compression suffix, e.g. s3://.../events.account.update-2026-05-27-11-<hash>.gz.
Observe an "unsupported extension" warning for every such file; default_extension is not applied.
Expected behavior
When a file name has no recognised format extension (csv, tsv, json, parquet, avro), the configured default_extension should be used so the file is ingested, instead of warning. Files with a recognised extension (including compressed *.json.gz, *.parquet.gz) should behave as before.
Activity