Skip to content

Better DateTime parsing, options to promote NULL int columns to float#4

Open
behrenhoff wants to merge 2 commits into
jbn:masterfrom
behrenhoff:master
Open

Better DateTime parsing, options to promote NULL int columns to float#4
behrenhoff wants to merge 2 commits into
jbn:masterfrom
behrenhoff:master

Conversation

@behrenhoff

@behrenhoff behrenhoff commented Nov 9, 2018

Copy link
Copy Markdown

When dealing with data types where no 1:1 mapping to np.type is possible, the code needs to have more possibilites to modify column properties.

In particular, this PR addresses

  • DateTime columns (exported as ISO 8601 string, column index put into parse_csv's parse_dates argument)
  • Dealing with "Integer" & "Long Integer" columns which may be NULL: promotion rules to np.float_ (either all int columns or the ones not marked "NOT NULL")
  • There were no tests at all. This commit introduces a few basic tests for column parsing.

When dealing with data types where no 1:1 mapping to np.type is
possible, the code needs to have more possibilites to modify column
properties.

In particular, this commit addresses
 - DateTime columns (exported as ISO 8601 string, column index put into
 parse_csv's parse_dates argument)
 - Integer & Long Integer columns which may be NULL and promotion rules
 into np.float_ (either all int columns or the ones not markes "NOT
 NULL")
 - There were no tests at all. This commit introduces a few basic tests
 for column parsing.
@DeflateAwning

Copy link
Copy Markdown

I have reviewed these changes, and approve of them being integrated to the upstream repo

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants