Skip to content

fix(sql): fill []*T in Select and skip unexported struct fields - #4435

Open
MrBeldum wants to merge 3 commits into
gofr-dev:developmentfrom
MrBeldum:fix/sql-select-pointer-slice
Open

MrBeldum wants to merge 3 commits into
gofr-dev:developmentfrom
MrBeldum:fix/sql-select-pointer-slice

Conversation

@MrBeldum

@MrBeldum MrBeldum commented Oct 5, 2026

Copy link
Copy Markdown

Fixes #4426.

1. []*T destinations came back as N nil pointers. selectSlice only routed rows through rowsToStruct when the element kind was Struct. For *T it called rows.Scan with a **T, dropped the error, and appended a nil pointer per row. Now a *struct element allocates a new T per row, fills it via rowsToStruct, and appends the pointer.

2. Unexported fields panicked. rowsToStruct called v.Field(i).Addr().Interface() on every field matching a column, which panics for unexported fields (reflect.Value.Interface: cannot return value obtained from unexported field or method). Unexported fields and fields tagged db:"-" are now skipped, the same as encoding/json and sqlx. Their columns are scanned into a throwaway value.

3. Scan errors are logged (error scanning row: ...) instead of being discarded with _ =, in both the scalar-slice and struct paths.

Tests in pkg/gofr/datasource/sql/db_test.go:

  • TestDB_SelectMultiRowIntoSliceOfPointers: []*user gets filled pointers, tag mapping included.
  • TestDB_SelectSkipsUnexportedAndIgnoredFields: no panic; id (unexported) and Secret (db:"-") stay zero.
  • TestDB_SelectLogsScanErrors: a non-numeric value scanned into int is logged for both slice and struct destinations.

Without the change, the first two tests fail. go test, go vet, and staticcheck pass for ./pkg/gofr/datasource/sql/.

Select into a []*T destination appended one nil pointer per row: the
element kind was not Struct, so rows were scanned into a **T, the error
was dropped, and nothing was filled in. Allocate a T per row and fill it
through rowsToStruct instead.

rowsToStruct also panicked on an unexported field whose name matched a
column, since Addr().Interface() is not allowed there. Skip unexported
fields and fields tagged db:"-", like encoding/json and sqlx do.

Scan errors are now logged instead of silently discarded.

Fixes gofr-dev#4426

@NitinKumar004 NitinKumar004 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for picking up #4426, @MrBeldum. I checked it at b92f22b6 with sqlmock and against a real postgres:16, using the same app built from development and from this branch.

What I verified

  • []*T into a row struct now works. On Postgres, []*user goes from [nil nil] on development to filled structs, with tag mapping.
  • Unexported fields no longer panic, and scan errors are now logged instead of dropped. On Postgres a NULL name scanned into a string field now logs error scanning row: sql: Scan error on column index 1 ..., where development was silent.
  • gofmt, go vet and go test -race -count=3 ./pkg/gofr/datasource/sql/ are clean. golangci-lint v2.12.2 reports 0 new issues.
  • Mutations: removing the []*T case, the IsExported skip, or either of the two new log calls fails the new tests.

Blocking: []*time.Time, []*sql.NullString and other []*Scanner destinations now come back as zero values

The new []*T case catches every pointer to a struct. But time.Time and sql.Scanner types such as sql.NullString, sql.NullTime or decimal.Decimal are structs that scan a single column themselves. On development they went through rows.Scan, which handles **time.Time and **sql.NullString fine. Now they go through rowsToStruct instead. That finds no exported field matching the column, scans the column into the throwaway value, and appends a zero value, without logging anything.

Real postgres:16, table users(id int, name text, created_at timestamptz) with rows (1,'alice','2026-10-06') and (2,NULL,'2026-10-07'):

destination development this PR
[]*time.Time (SELECT created_at) 2026-10-06, 2026-10-07 0001-01-01, 0001-01-01
[]*sql.NullString (SELECT name) "alice"/valid, nil ""/invalid, ""/invalid
[]*int64 7 8 7 8

The same shows up with sqlmock. A small helper fixes it, and it also fixes []time.Time / []sql.NullString, which were already zero-valued on development because they hit the Struct case:

// isRowStruct reports whether Select maps a row's columns onto t's fields. Struct types that scan a
// single column themselves, time.Time and sql.Scanner implementations such as sql.NullString, are
// scanned directly instead.
func isRowStruct(t reflect.Type) bool {
	return t.Kind() == reflect.Struct && t != reflect.TypeFor[time.Time]() &&
		!reflect.PointerTo(t).Implements(reflect.TypeFor[sql.Scanner]())
}
		case isRowStruct(elemType):
			...
		case elemType.Kind() == reflect.Pointer && isRowStruct(elemType.Elem()):
			...

With that applied on top of this branch, the whole sql package passes. The same probe then gives:

[]*time.Time      -> [2026-10-06T12:00:00Z 2026-10-06T12:00:00Z]
[]*sql.NullString -> [alice/valid nil]
[]time.Time       -> [2026-10-06T12:00:00Z 2026-10-06T12:00:00Z]
[]sql.NullString  -> [{alice true} { false}]
[]*int64          -> [7 8]

Could you add those destinations as rows in a test next to TestDB_SelectMultiRowIntoSliceOfPointers? Then the regression stays pinned.

Smaller notes

  • nit: the tag == "-" check doesn't change behaviour today. Without it, a db:"-" field maps to a column literally named -, so it is skipped anyway, and removing the check leaves every test green. The intent is clearer with it, so I'd keep it. But the Secret row in TestDB_SelectSkipsUnexportedAndIgnoredFields can only catch a regression if the column it uses is named -.
  • Worth a line in the PR description: scan errors now log at ERROR once per row. Code that maps a nullable column into a plain string will start logging where it was silent before. I think that's the right call, since the row was being silently truncated at the first failing column, but it's a visible change.

Requesting changes for the []*time.Time / []*Scanner regression. The rest looks good.

[]*T previously routed every pointer-to-struct through rowsToStruct, so
[]*time.Time and []*sql.NullString (and value forms) came back as zeros.
Introduce isRowStruct so single-column Scanner/time types use rows.Scan.

Add regression rows next to TestDB_SelectMultiRowIntoSliceOfPointers.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

sql: Select into []*T returns nil pointers, and a struct with an unexported field matching a column panics

2 participants