Skip to content

Latest commit

 

History

History
74 lines (59 loc) · 4.32 KB

BACKLOG.md

File metadata and controls

74 lines (59 loc) · 4.32 KB

Research promotion gates

The active question is causal reliability of spectral inverse models. Its first experiment is a controlled synthetic problem with a public spectral shape. Passing software tests or reproducing that experiment does not clear any of the physical-validation gates below. A failed hidden-direction correction remains a negative result; it does not justify adding more interpretability machinery until the simple controls are understood.

The first fixed study reproduced H1 in five of five primary seeds and rejected H2 in all five. GLS dominated the hidden correction under the primary geometry. The network also missed the fixed oracle-effect explanation tolerance. Preserve these results; do not tune the existing protocol after inspecting its test set.

1. Move from controlled geometry to a physical simulation grid

Bounded execution completed, broad promotion remains open. The physical-grid study separately trained on 128 published equilibrium states and held out 32 calibration and 32 test states. Its augmentation gate passed 4/5 seeds; exact training-support grid inference was more accurate. Both failed uncertainty coverage on 116 external photochemical states. The instrument/dilution interventions remain explicitly approximate, and discrete-grid intervals are not a continuous retrieval posterior.

Use a licensed, versioned atmosphere grid with hashed assets and independently varying target and nuisance parameters. Declare the physical intervention, bin response and observation-noise assumptions before fitting. Hold out physical parameter combinations, not only random noise realizations of the same spectra.

Repeat the paired audit with matched-prior and deployment-prior references where tractable, an independent-nuisance training control, and explicit ambiguous cases. Promotion requires evidence that the intervention preserves the intended target while changing a defensible nuisance. A synthetic template amplitude cannot be renamed as abundance without this step.

2. Audit an independently trained retrieval

Freeze an external retrieval model, its training distribution and its preprocessing. Specify a falsifiable prediction about nuisance response before inspecting held-out interventions. Compare the intervention effect with ordinary prior dependence and approximation error. Retain failures and test whether a simple estimator or changed training distribution accounts for the effect before attributing it to a hidden mechanism. Claim only what the independently evaluated model and data support.

3. Extend uncertainty beyond one validation residual scale

Finite-grid posterior intervals, a nuisance envelope, validation-only worst-group calibration and width-based abstention are now tested. Their external failures leave this promotion gate open. The next diagnostic must predict errors on fresh physical states better than a simple residual baseline; the exposed 116-state challenge cannot become its untouched test set.

The completed adequacy diagnosis did not justify the more complex withheld-wavelength score over chi-square. Retain that negative result; model-family adequacy and fresh physical validation remain open, even though an oracle containing the correct family performs well.

Add posterior or conditional uncertainty estimates only with explicit assumptions and held-out calibration tests. Separate prior-predictive computational calibration from robustness under a changed prior or observation model. For correlated noise, validate the covariance and whitening against a known reference before interpreting coverage; never repair coverage by fitting an uncertainty scale on test residuals.

Deferred legacy scope

Full tierra and petitRADTRANS integration, molecule-specific opacity comparison, joint stellar-atmosphere retrieval and NEO characterization are deferred. Their interface contracts or placeholder modules are not implemented capabilities. They should return to active scope only when needed for a specific promotion gate and accompanied by independent validation.

The existing synthetic nested-sampling example and conditional WASP-39b check remain supporting inference tests. They do not satisfy the promotion gates above.