Write Parquet NUMERIC with precision up to 76 and name the array column in overflow messages

[huiliang-liu]

JIRA: PTT-1957

Summary

Writing a NUMERIC value with more than 38 digits to Parquet loses digits or fails. In a field report, numeric[] values with more than 18 decimal places came back rounded (0.1234567890123456789 → 0.123456789012345679, and a value with 61 decimal places → 0), and declaring a precision above 38 was rejected. The customer needs these values as decimal256 on the Parquet side.

This PR has two commits.

1. Write Parquet NUMERIC columns with precision up to 76

• PXF used to reject any NUMERIC(p,s) with p > 38 because values went through HiveDecimal. Parquet's DECIMAL has no such limit, and Arrow readers (PyArrow) read precision up to 76 as decimal256.
• The limit is now 76 (ParquetFileAccessor.MAX_DECIMAL_PRECISION). For precision 39–76, values are parsed with BigDecimal (DecimalUtilities.parseDecimalStringWithBigDecimal) and written as a FIXED_LEN_BYTE_ARRAY sized for the precision (32 bytes at 76).
• pxf.parquet.write.decimal.overflow keeps its error / round (half-up) / ignore (NULL) behavior for these columns.
• Unchanged:
  • Precision ≤ 38 still goes through HiveDecimal, so those files are byte-for-byte the same.
  • NUMERIC without a typmod still maps to DECIMAL(38,18). Spark, Hive and Trino only read precision ≤ 38, so wider columns are opt-in by declaring the precision.
  • ORC writes.
• The read path already decoded wide decimals with BigDecimal. No change was needed there.
• Docs (hdfs_parquet, cfg_server) and CHANGELOG are updated.

2. Name the array column in Parquet numeric overflow messages (from #89)

• For a NUMERIC[] element that overflowed, the error (or the ignore-mode warning) named the table's first column instead of the array column. The column name is now passed down from fillGroup.
• Tests cover how a NUMERIC[] column's scale is chosen on write. NUMERIC[] → DECIMAL(38,18). (38,20) and (38,37) keep their scale. (76,61) and TEXT[] keep every digit. (77,61) is rejected.
• Supersedes Name the array column in Parquet numeric overflow messages #89. Its automation case that expected NUMERIC(45,25)[] to be rejected now checks a (76,61) round trip instead.

Testing

• pxf-hdfs unit tests: 539 passed. New tests are in ParquetWriteTest and DecimalUtilitiesTest.
• Local WHPG 7 cluster with file:parquet:
  • numeric(76,61) and numeric(76,61)[] write and read back exactly, including 61-decimal-place and 15-integer-digit values, negatives and NULLs.
  • PyArrow 21 reads the PXF file as decimal256(76, 61) with the correct values.
  • A decimal256(76, 61) file written by PyArrow reads through PXF exactly.
• Automation:
  • New parquetWriteNumericWithMaxPrecision (numeric_max_precision).
  • Updated parquetWriteNumericArrayScale (numeric_array_scale).
  • Expected message in numeric_with_large_precision changed from 38 to 76.
  • numeric_max_precision and numeric_array_scale pass under pxf_regress locally. numeric_with_large_precision was not re-run.
  • mvn test-compile passes. The full HDFS automation suite was not run.

[huiliang-liu]

The CI is green: https://github.com/EnterpriseDB/whpg-extensions-packaging/actions/runs/38698498687

[adam8157]

One small thing: PG allows NaN in a constrained NUMERIC(p,s) column. With ignore, the ≤ 38 path stores it as NULL (HiveDecimal.create("NaN") returns null), but the new path calls new BigDecimal("NaN") and fails with a raw NumberFormatException. The error path already behaves the same way, so this isn't new, but it might be worth handling NaN explicitly in parseDecimalStringWithBigDecimal.