Skip to content

rows_from_file() splits a single-column CSV file on a letter, or raises csv.Error #907

Open

Description

@L4XB

When rows_from_file() detects the format of a single-column CSV file, it either fails or splits the column on a letter. That affects sqlite-utils memory file.csv and any Python call that leaves format= unset.

$ printf 'name\nAnn\nBob\n' > names.csv
$ sqlite-utils memory names.csv "select * from names"
...
_csv.Error: Could not determine delimiter

$ printf 'url\nhttps://example.com/a\nhttps://example.org/b\n' > urls.csv
$ sqlite-utils memory urls.csv "select * from urls"
[{"ur": "https://examp", "": "e.com/a"},
 {"ur": "https://examp", "": "e.org/b"}]

(sqlite-utils 4.2.1)

The detection calls csv.Sniffer().sniff() on the first 2048 bytes without a delimiters= list. The sniffer then treats any character as a candidate, so a file with a single column either has no consistent candidate (csv.Error) or has a letter that happens to repeat the same number of times on every line (l above).

A fix could sniff only for the delimiters a CSV file is realistically written with (,, ;, tab, |) and read the file as one column when none of them is found. insert --sniff calls the sniffer the same way, but that is an explicit request to detect the dialect, so I would leave it unchanged.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions