Repository navigation
Planner cannot handle default values in scalar fields #91
Description
Activity
This is possibly fixed by #96; separate investigation pending.
I can confirm that the now merged #96 remedies the problem described in OP.
For
class Model(BaseModel): x: int | None = None y: int | None = None data = [ {"z": 1}, {"z": 2, "x": 3}, ]
the planner now produces
bindings = [ { "x": null }, { "x": 3 } ] model_dump = [ { "x": null, "y": null }, { "x": 3, "y": null } ]
An better example would be
class Model(BaseModel): x: int y: int = 42 z: int = Field(default_factory=lambda: 0) data = [ {"x": 1}, {"x": 2}, {"x": 3}, ]
Previously this also produced a
polars.exceptions.ColumnNotFoundError; with #96 however, responsibiltiy for default resolution is passed to Pydantic, so the above producesbindings = [ { "x": 1 }, { "x": 2 }, { "x": 3 } ] model_dump = [ { "x": 1, "y": 42, "z": 0 }, { "x": 2, "y": 42, "z": 0 }, { "x": 3, "y": 42, "z": 0 } ]
This is possibly fixed by #96; separate investigation pending.
I can confirm that the now merged #96 remedies the problem described in OP.
For
class Model(BaseModel):
x: int | None = None
y: int | None = Nonedata = [
{"z": 1},
{"z": 2, "x": 3},
]the planner now produces
bindings = [
{
"x": null
},
{
"x": 3
}
]model_dump = [
{
"x": null,
"y": null
},
{
"x": 3,
"y": null
}
]There is an interesting aspect about this worth mentioning: Note that the data here is asymmetrical, the second row in the data has
xthat's missing entirely from the first row.graphtyloads the data into apl.LazyFramewith default settings applied; so for asymmetrical data, the frame will be filled withNone/null.This behavior is perfectly reasonable and library users can pass their own pre-processed
pl.LazyFrametographty.ModelMaterializerandgraphty.LazyFramePlannerif they wish to do so. Just note that for asymmetrical data, the frame will by default haveNone/nullvalues implicitly.An better example would be
class Model(BaseModel):
x: int
y: int = 42
z: int = Field(default_factory=lambda: 0)data = [
{"x": 1},
{"x": 2},
{"x": 3},
]Previously this also produced a
polars.exceptions.ColumnNotFoundError; with #96 however, responsibiltiy for default resolution is passed to Pydantic, so the above producesbindings = [
{
"x": 1
},
{
"x": 2
},
{
"x": 3
}
]model_dump = [
{
"x": 1,
"y": 42,
"z": 0
},
{
"x": 2,
"y": 42,
"z": 0
},
{
"x": 3,
"y": 42,
"z": 0
}
]There should be tests for default value resolution in
graphty; opened #98.And closing this as resolved.
The planner cannot handle default values in scalar fields, e.g for the following model and data
the planner tries to look up
xandyin the dataframe and crashes withpolars.exceptions.ColumnNotFoundError.Note that this concerns default values in general, i.e. also default factories.
Both the ungrouped and grouped code path for scalar handling are likely affected.