probe_eddy_current: filter noisy calibration points

[nefelim4ag]

After the: https://klipper.discourse.group/t/make-btt-eddy-great-again/25139

I wanted to implement a way to give the user feedback if the calibration is poor.
But it seems that "bad" is too hard to define for everyone.

Instead of a sophisticated and probably not really required slope analysis.
There is now a pure analytical implementation, we have samples, we have stddev, we can try to calculate standard error, and if the standard error for the difference between samples is too large, bail out earlier.

Goals of this change:

• Add feedback if calibration or part of calibration is bad.
• Define some objective metrics to compare against.
• If the sensor is too high, bad, or anything, this would force one to use the adequate z_offset or move the sensor.

A fast local test would look like:

G28
G90
G0 Z0.005
PROBE_EDDY_CURRENT_CALIBRATE CHIP=btt_eddy
TESTZ Z=-0.005
ACCEPT

As my sensor has a pretty low noise level, with the default test and patched max Z (https://github.com/Klipper3d/klipper/blob/master/klippy/extras/probe_eddy_current.py#L95),
bails out at:

// Frequency too noisy at step 18.050
// probe_eddy_current: MAD=3.867Hz ~ 0.0002mm in 16597 queries

And normal calibration up to 4mm

// probe_eddy_current: MAD=11.988Hz ~ 0.000226mm in 3727 queries

It is questionable if this sort of SNR metric is correct, mad_mm, but this is what I've come up with for now.
I moved the sensor up by 1 mm:

probe_eddy_current: MAD=11.775Hz ~ 0.0004mm in 3727 queries

Ah, well, it is as expected, larger distance - larger noise (in mm).
So, I think it is an okay metric, at least it is correlated with what I would expect.

Thanks.

---

My sensor output distribution at a standstill looks like below. Because it is not a normal distribution, and does not even seem to be any sort of distribution to me. So, I chose the conservative approach with Median absolute deviation/and Medians, which I hope so, would work okay with such a distribution.
An alternative (I guess) would be to calculate buckets and do something with the frequency of samples ... but I guess it's not worth it.

# ~9 LSB bits is jumping around
$ sort 1mm.log | uniq -c
     18 3154502.213 # raw 35282509 
    137 3154512.048 # raw 35282620
    446 3154518.396 # raw 35282691
    386 3154528.23 
    303 3154534.578
     63 3154544.413
      3 3154550.761 # raw 35283053 
$ sort 2mm.log | uniq -c
     22 3094345.808
    178 3094349.653
    399 3094361.633
    785 3094365.567
     81 3094377.548
     35 3094381.392
# ~7 LSB bits noisy
$ sort 3mm.log | uniq -c
    666 3058982.76  # raw 34214143
    356 3058985.174
    486 3058998.406
    100 3059000.82  # raw 34214345
$ sort 4mm.log | uniq -c
      1 3037164.956
    448 3037178.993
    715 3037180.513
    148 3037194.55
    116 3037196.159
$ sort 5mm.log| uniq -c
     11 3023109.645
    361 3023110.628
      2 3023111.612
     27 3023125.201
    187 3023126.096

[nefelim4ag]

I did some checks, and I think there is no reason to replace the freq_avg and shift calibration points.
Let them work the same way as other logic (probe, scan) does.
All filter math would work close to what I wanted it to with MAD over the frequency average value.
And still give the valuable SNR data.

Probably it may make sense to show it in some positional points to describe the underlying curve.

---

// probe_eddy_current: MAD=18.780Hz ~ 0.000354mm in 2485 queries
// z_offset: 0.250 MAD=46.058Hz ~ 0.000367mm
// z_offset: 1.010 MAD=23.532Hz ~ 0.000302mm
// z_offset: 1.770 MAD=19.859Hz ~ 0.000389mm
// z_offset: 2.530 MAD=12.360Hz ~ 0.000356mm
// z_offset: 3.290 MAD=14.105Hz ~ 0.000582mm
// z_offset: 4.050 MAD=7.871Hz ~ 0.000460mm
...
// probe_eddy_current: MAD=19.875Hz ~ 0.000351mm in 2481 queries
// z_offset: 0.250 MAD=46.126Hz ~ 0.000364mm
// z_offset: 1.010 MAD=24.188Hz ~ 0.000310mm
// z_offset: 1.770 MAD=20.595Hz ~ 0.000400mm
// z_offset: 2.530 MAD=21.247Hz ~ 0.000614mm
// z_offset: 3.290 MAD=9.702Hz ~ 0.000396mm
// z_offset: 4.050 MAD=5.865Hz ~ 0.000342mm

Part of the run-to-run variation is contributed by the oscillating bed; the Z input shaper helps with it, but does not cancel it completely. On the other hand, it does reproduce the run of the probe/bed mesh, so it is probably acceptable.
Increasing the pause after the last move time makes values more stable.

---

Also, I have to point out that my calibration range is with the high slope, so my SNR would be fine overall.

[nefelim4ag]

Hmmm, I decided to do more testing:

diff --git a/klippy/extras/probe_eddy_current.py b/klippy/extras/probe_eddy_current.py
index 0cad3af4f..42c9f0f17 100644
--- a/klippy/extras/probe_eddy_current.py
+++ b/klippy/extras/probe_eddy_current.py
@@ -98,7 +98,7 @@ class EddyCalibration:
         toolhead.dwell(1.)
         self.drift_comp.note_z_calibration_start()
         # Move to each 40um position
-        max_z = 4.0
+        max_z = 10.0
         samp_dist = 0.040
         req_zpos = [i*samp_dist for i in range(int(max_z / samp_dist) + 1)]
         sparse_max_z = self.sparse_z
@@ -118,9 +118,9 @@ class EddyCalibration:
             next_pos[2] += zpos
             move(next_pos, move_speed)
             # Note sample timing
-            start_query_time = toolhead.get_last_move_time() + 0.050
-            end_query_time = start_query_time + 0.100
-            toolhead.dwell(0.200)
+            start_query_time = toolhead.get_last_move_time() + 0.100
+            end_query_time = start_query_time + 0.400
+            toolhead.dwell(end_query_time - start_query_time + 0.1)
             # Find Z position based on actual commanded stepper position
             toolhead.flush_step_generation()
             kin_spos = {s.get_name(): s.get_commanded_position()
@@ -223,7 +223,7 @@ class EddyCalibration:
             "probe_eddy_current: MAD=%.3fHz ~ %.6fmm in %d queries\n" % (
                 avg_mad, avg_mad_mm, total))
         skip = 5
-        for i in range(skip, len(filtered), (len(filtered) - skip)//5):
+        for i in range(skip, len(filtered), (len(filtered) - skip)//20):
             pos, _ = filtered[i]
             mad = total_mad[i]
             mad_mm = total_mad_mm[i]

So, to avoid a zone where my bed is oscillating +100ms from the move, and 400ms to collect the data.
And I can say that according to this test, noise and SNR are weird.
I was expecting something that SNR would monotonically degrade over distance, but it seems to be slightly more complicated.
Assuming my sensor is 2.5mm above the nozzle.
It seems that the first 2.5 + 5 mm = 7.5mm of the working distance, and so half of the coil diameter (15mm) has uniform noise.
Let's average the data to ~260um.
And after that, SNR degrades over time, but also not strongly monotonically.

// probe_eddy_current: MAD=8.032Hz ~ 0.000580mm in 25096 queries
// z_offset: 0.250 MAD=32.293Hz ~ 0.000248mm
// z_offset: 0.730 MAD=28.639Hz ~ 0.000302mm
// z_offset: 1.210 MAD=18.339Hz ~ 0.000257mm
// z_offset: 1.690 MAD=12.386Hz ~ 0.000226mm
// z_offset: 2.170 MAD=11.145Hz ~ 0.000262mm
// z_offset: 2.650 MAD=12.125Hz ~ 0.000362mm
// z_offset: 3.130 MAD=2.873Hz ~ 0.000108mm
// z_offset: 3.610 MAD=5.411Hz ~ 0.000254mm
// z_offset: 4.090 MAD=4.534Hz ~ 0.000264mm
// z_offset: 4.570 MAD=1.113Hz ~ 0.000080mm
// z_offset: 5.050 MAD=1.994Hz ~ 0.000173mm
// z_offset: 5.530 MAD=6.003Hz ~ 0.000631mm
// z_offset: 6.010 MAD=7.868Hz ~ 0.000990mm
// z_offset: 6.490 MAD=4.053Hz ~ 0.000608mm
// z_offset: 6.970 MAD=2.501Hz ~ 0.000442mm
// z_offset: 7.450 MAD=5.163Hz ~ 0.001088mm
// z_offset: 7.930 MAD=7.228Hz ~ 0.001772mm
// z_offset: 8.410 MAD=2.553Hz ~ 0.000732mm
// z_offset: 8.890 MAD=3.178Hz ~ 0.001055mm
// z_offset: 9.370 MAD=1.696Hz ~ 0.000648mm
// z_offset: 9.850 MAD=7.860Hz ~ 0.003493mm

[nefelim4ag]

I've added the total frequency range over calibrated 4mm.

// probe_eddy_current: MAD=21.189Hz ~ 0.000378mm in 2524 queries
// Total frequency range: 225184.672
...

I'm not sure if it is required to make suggestions here, so just data for comparison:

FreakyDude: freq range 33319.3
SnakeKVC3mm: freq range 31301.6
dicord-lan: freq range 31941.5
discord-TheAbbys: freq range 26407.6
discourse-22601: freq range 54397.6
discourse-25025: freq range 26236.7
discourse-25174: freq range 55244.0
discord-sht36v3-mhzpower: freq range 131720.1
supergun03mm: freq range 213008.4 # BTT Eddy without case distance to the bed
supergun2mm: freq range 123531.7 # BTT Eddy without case distance to the bed
# And this is for the carto
2.5mm: freq range 209715.2
2.5mm_new: freq range 217647.4
3.5mm: freq range 127718.6
30c: freq range 125855.6
40c: freq range 126489.2
50c: freq range 126534.2
60c: freq range 127326.9
70c: freq range 127190.6
IDRIVE17: freq range 222539.5
IDRIVE18: freq range 223052.5
IDRIVE19: freq range 223971.0
IDRIVE20: freq range 225998.3
IDRIVE21: freq range 228149.5
pos_center: freq range 211142.6
pos_right_back_edge: freq range 231844.3
pos_right_front_edge: freq range 216378.9

[ifreislich]

Some samples. BTT Eddy Coil, coil height from the surface to the closest PCB side. Results are without the patch in #7119 (comment)

Original (coil starting height 3.6mm)

13:27:58 z_offset: 3.295 MAD=18.904Hz ~ 0.003063mm
13:27:58 z_offset: 2.535 MAD=22.962Hz ~ 0.002788mm
13:27:58 z_offset: 1.775 MAD=24.133Hz ~ 0.002112mm
13:27:58 z_offset: 1.015 MAD=34.785Hz ~ 0.002162mm
13:27:58 z_offset: 0.255 MAD=43.069Hz ~ 0.001843mm
13:27:57 Total frequency range: 45248.228Hz
13:27:57 probe_eddy_current: MAD=23.636Hz ~ 0.002093mm in 2526 queries```

Retake:
```14:01:24 z_offset: 4.055 MAD=0.604Hz ~ 0.000145mm
14:01:24 z_offset: 3.295 MAD=11.335Hz ~ 0.002013mm
14:01:24 z_offset: 2.535 MAD=22.481Hz ~ 0.002728mm
14:01:24 z_offset: 1.775 MAD=15.657Hz ~ 0.001380mm
14:01:23 z_offset: 1.015 MAD=43.249Hz ~ 0.002751mm
14:01:23 z_offset: 0.255 MAD=44.496Hz ~ 0.001941mm
14:01:23 Total frequency range: 45457.973Hz
14:01:22 probe_eddy_current: MAD=24.900Hz ~ 0.002192mm in 2525 queries```

Coil height 1.6mm
```13:38:06 z_offset: 5.100 MAD=10.725Hz ~ 0.000884mm
13:38:06 z_offset: 4.340 MAD=57.142Hz ~ 0.003257mm
13:38:06 z_offset: 3.580 MAD=28.667Hz ~ 0.000977mm
13:38:06 z_offset: 2.820 MAD=67.735Hz ~ 0.001504mm
13:38:06 z_offset: 2.060 MAD=34.793Hz ~ 0.000402mm
13:38:05 z_offset: 1.300 MAD=79.612Hz ~ 0.000475mm
13:38:05 Total frequency range: 202836.013Hz
13:38:05 probe_eddy_current: MAD=44.265Hz ~ 0.001335mm in 2524 queries```

Coil height 1.95mm
```14:17:33 z_offset: 5.850 MAD=23.476Hz ~ 0.002540mm
14:17:33 z_offset: 5.090 MAD=26.736Hz ~ 0.002080mm
14:17:33 z_offset: 4.330 MAD=49.837Hz ~ 0.002780mm
14:17:33 z_offset: 3.570 MAD=51.035Hz ~ 0.001822mm
14:17:33 z_offset: 2.810 MAD=53.075Hz ~ 0.001131mm
14:17:33 z_offset: 2.050 MAD=39.050Hz ~ 0.000454mm
14:17:33 Total frequency range: 130195.910Hz
14:17:32 probe_eddy_current: MAD=39.499Hz ~ 0.001599mm in 2526 queries```

Coil height 2.3mm
```14:13:37 z_offset: 5.950 MAD=13.048Hz ~ 0.001663mm
14:13:37 z_offset: 5.190 MAD=19.437Hz ~ 0.001682mm
14:13:36 z_offset: 4.430 MAD=26.695Hz ~ 0.001543mm
14:13:36 z_offset: 3.670 MAD=63.472Hz ~ 0.002345mm
14:13:36 z_offset: 2.910 MAD=33.956Hz ~ 0.000761mm
14:13:36 z_offset: 2.150 MAD=49.206Hz ~ 0.000626mm
14:13:36 Total frequency range: 122728.293Hz
14:13:36 probe_eddy_current: MAD=40.982Hz ~ 0.001621mm in 2525 queries```

[ifreislich]

With the patch #7119 (comment). Z=0, coil height +1.6mm.

// probe_eddy_current: MAD=23.710Hz ~ 0.002391mm in 23199 queries
// Total frequency range: 213546.442Hz
// z_offset: 0.250 MAD=38.417Hz ~ 0.000373mm
// z_offset: 0.690 MAD=27.895Hz ~ 0.000276mm
// z_offset: 1.130 MAD=24.708Hz ~ 0.000351mm
// z_offset: 1.570 MAD=48.595Hz ~ 0.000939mm
// z_offset: 2.010 MAD=48.841Hz ~ 0.001312mm
// z_offset: 2.450 MAD=49.414Hz ~ 0.001727mm
// z_offset: 2.890 MAD=80.027Hz ~ 0.003617mm
// z_offset: 3.330 MAD=37.873Hz ~ 0.002208mm
// z_offset: 3.770 MAD=33.633Hz ~ 0.002531mm
// z_offset: 4.210 MAD=12.007Hz ~ 0.001134mm
// z_offset: 4.650 MAD=12.744Hz ~ 0.001643mm
// z_offset: 5.090 MAD=9.068Hz ~ 0.001220mm
// z_offset: 5.530 MAD=13.261Hz ~ 0.002410mm
// z_offset: 5.970 MAD=19.465Hz ~ 0.003943mm
// z_offset: 6.410 MAD=5.852Hz ~ 0.001537mm
// z_offset: 6.850 MAD=1.668Hz ~ 0.000518mm
// z_offset: 7.290 MAD=10.932Hz ~ 0.003690mm
// z_offset: 7.730 MAD=17.071Hz ~ 0.006511mm
// z_offset: 8.170 MAD=2.662Hz ~ 0.001441mm
// z_offset: 8.610 MAD=18.049Hz ~ 0.011148mm
// z_offset: 9.050 MAD=14.167Hz ~ 0.008371mm