[ENH] BIDS Metadata reporting

[TheChymera]

Very early draft of reporting as discussed with @satra and @yarikoptic .
Already works with 000108 and its respective report can be produced with pytest -vvs tests/test_bids_utils.py

Missing:

• Tests with actual assertions
• Docstrings
• OME-ZARR support, pending Updating BIDS validator and schema to contemporary upstream equivalent #1050
• Value sorting

[TheChymera]

Sample output for DANDI:000108:

Here is the subject to session summary:
	-`MITU01`	121 (20210612h12m11s58, 20220317h17m41s16, 20210926h21m32s13, ...)
Here is the session to subject summary:
	-`20210612h12m11s58`	1 (MITU01)
	-`20220317h17m41s16`	1 (MITU01)
	-`20210926h21m32s13`	1 (MITU01)
	-`20210707h22m03s23`	1 (MITU01)
	-`20210615h23m54s27`	1 (MITU01)
	-`20210820h19m29s49`	1 (MITU01)
	-`20220326h16m14s39`	1 (MITU01)
	-`20210919h18m20s55`	1 (MITU01)
	-`20211213h11m20s42`	1 (MITU01)
	-`20210917h20m34s56`	1 (MITU01)
	-`20211119h19m23s36`	1 (MITU01)
	-`20220326h07m16s28`	1 (MITU01)
	-`20210704h00m54s58`	1 (MITU01)
	-`20211105h16m12s44`	1 (MITU01)
	-`20211020h18m48s11`	1 (MITU01)
	-`20210718h20m15s50`	1 (MITU01)
	-`20210829h21m53s32`	1 (MITU01)
	-`20220118h15m27s25`	1 (MITU01)
	-`20210922h18m10s45`	1 (MITU01)
	-`20220312h12m46s45`	1 (MITU01)
	-`20210719h09m41s20`	1 (MITU01)
	-`20220113h11m23s59`	1 (MITU01)
	-`20220103h10m16s20`	1 (MITU01)
	-`20211016h18m33s42`	1 (MITU01)
	-`20210721h22m29s00`	1 (MITU01)
	-`20210809h18m30s07`	1 (MITU01)
	-`20211221h14m59s10`	1 (MITU01)
	-`20210813h22m03s27`	1 (MITU01)
	-`20210911h19m39s42`	1 (MITU01)
	-`20210906h17m50s41`	1 (MITU01)
	-`20220311h10m19s37`	1 (MITU01)
	-`20210904h19m09s43`	1 (MITU01)
	-`20211019h18m49s16`	1 (MITU01)
	-`20211226h18m00s52`	1 (MITU01)
	-`20211001h11m49s01`	1 (MITU01)
	-`20210707h09m25s26`	1 (MITU01)
	-`20211111h11m08s18`	1 (MITU01)
	-`20210723h17m40s25`	1 (MITU01)
	-`20211210h15m09s34`	1 (MITU01)
	-`20220108h14m30s00`	1 (MITU01)
	-`20220315h17m32s36`	1 (MITU01)
	-`20220318h15m33s47`	1 (MITU01)
	-`20211025h19m35s28`	1 (MITU01)
	-`20211012h17m50s50`	1 (MITU01)
	-`20211024h19m12s35`	1 (MITU01)
	-`20220102h19m54s44`	1 (MITU01)
	-`20211104h16m15s36`	1 (MITU01)
	-`20210701h13m53s49`	1 (MITU01)
	-`20220118h20m49s07`	1 (MITU01)
	-`20210811h08m54s42`	1 (MITU01)
	-`20210702h09m39s16`	1 (MITU01)
	-`20210703h01m05s04`	1 (MITU01)
	-`20210704h21m07s40`	1 (MITU01)
	-`20210521h17m17s06`	1 (MITU01)
	-`20220325h18m27s22`	1 (MITU01)
	-`20211114h20m33s12`	1 (MITU01)
	-`20210717h14m15s57`	1 (MITU01)
	-`20211214h18m26s20`	1 (MITU01)
	-`20220316h10m52s23`	1 (MITU01)
	-`20210910h16m27s07`	1 (MITU01)
	-`20220326h13m51s50`	1 (MITU01)
	-`20211007h18m22s29`	1 (MITU01)
	-`20211107h14m57s54`	1 (MITU01)
	-`20210907h18m00s34`	1 (MITU01)
	-`20211230h17m22s57`	1 (MITU01)
	-`20210708h07m45s44`	1 (MITU01)
	-`20211005h21m12s47`	1 (MITU01)
	-`20210816h18m00s11`	1 (MITU01)
	-`20220104h20m30s15`	1 (MITU01)
	-`20210827h18m23s51`	1 (MITU01)
	-`20220317h10m43s39`	1 (MITU01)
	-`20210704h12m16s34`	1 (MITU01)
	-`20211017h18m55s49`	1 (MITU01)
	-`20220322h10m15s10`	1 (MITU01)
	-`20211212h13m47s11`	1 (MITU01)
	-`20220103h19m31s41`	1 (MITU01)
	-`20211102h18m41s09`	1 (MITU01)
	-`20220316h16m47s38`	1 (MITU01)
	-`20210815h19m09s40`	1 (MITU01)
	-`20211026h18m06s00`	1 (MITU01)
	-`20210615h18m19s52`	1 (MITU01)
	-`20211109h17m54s46`	1 (MITU01)
	-`20220104h13m37s33`	1 (MITU01)
	-`20210718h00m37s58`	1 (MITU01)
	-`20220113h16m46s11`	1 (MITU01)
	-`20210524h18m29s31`	1 (MITU01)
	-`20220310h10m08s20`	1 (MITU01)
	-`20210902h20m33s45`	1 (MITU01)
	-`20211021h20m11s22`	1 (MITU01)
	-`20210614h22m41s59`	1 (MITU01)
	-`20210626h17m19s58`	1 (MITU01)
	-`20210611h15m51s46`	1 (MITU01)
	-`20220104h19m06s08`	1 (MITU01)
	-`20210817h18m20s17`	1 (MITU01)
	-`20211227h20m09s43`	1 (MITU01)
	-`20210702h17m19s06`	1 (MITU01)
	-`20210703h16m32s10`	1 (MITU01)
	-`20211123h17m20s20`	1 (MITU01)
	-`20211112h12m13s25`	1 (MITU01)
	-`20210708h18m31s21`	1 (MITU01)
	-`20220309h18m20s33`	1 (MITU01)
	-`20210822h18m19s32`	1 (MITU01)
	-`20211006h20m36s26`	1 (MITU01)
	-`20210829h00m40s41`	1 (MITU01)
	-`20210722h16m33s43`	1 (MITU01)
	-`20210706h23m08s14`	1 (MITU01)
	-`20210709h05m43s00`	1 (MITU01)
	-`20211002h21m40s58`	1 (MITU01)
	-`20210901h20m38s50`	1 (MITU01)
	-`20210903h14m43s46`	1 (MITU01)
	-`20220311h18m03s49`	1 (MITU01)
	-`20210614h15m30s08`	1 (MITU01)
	-`20210720h20m19s32`	1 (MITU01)
	-`20211113h13m49s50`	1 (MITU01)
	-`20210812h19m21s32`	1 (MITU01)
	-`20210825h18m14s24`	1 (MITU01)
	-`20220321h14m52s06`	1 (MITU01)
	-`20210915h15m45s27`	1 (MITU01)
	-`20211008h20m03s57`	1 (MITU01)
	-`20220108h22m32s52`	1 (MITU01)
	-`20220310h18m01s48`	1 (MITU01)
Here is the subject to sample summary:
	-`MITU01`	112 (110, 94, 11, ...)
Here is the subject to stain summary:
	-`MITU01`	7 (NPY, CR, YO, ...)

[codecov]

Codecov Report

Merging #1061 (1ae0a3b) into master (e4dc875) will increase coverage by 0.06%.
The diff coverage is 98.21%.

@@            Coverage Diff             @@
##           master    #1061      +/-   ##
==========================================
+ Coverage   88.35%   88.42%   +0.06%     
==========================================
  Files          72       73       +1     
  Lines        9250     9304      +54     
==========================================
+ Hits         8173     8227      +54     
  Misses       1077     1077              

Flag	Coverage Δ
unittests	88.42% <98.21%> (+0.06%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
dandi/bids_utils.py	96.00% <98.00%> (+3.69%)	⬆️
dandi/bids_validator_xs.py	91.83% <100.00%> (+0.05%)	⬆️
dandi/tests/test_bids_utils.py	100.00% <100.00%> (ø)
dandi/tests/fixtures.py	96.87% <0.00%> (-1.18%)	⬇️
dandi/_version.py	44.21% <0.00%> (ø)
dandi/dandiapi.py	88.02% <0.00%> (+0.13%)	⬆️
dandi/support/pyout.py	95.00% <0.00%> (+5.00%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e4dc875...1ae0a3b. Read the comment docs.

[TheChymera]

apparently this has another subject, I need to check.

[TheChymera]

Subject discussion continued here: dandisets/000108#10

[yarikoptic]

@TheChymera - what are the plans for this PR?

[TheChymera]

@yarikoptic this is old code which @satra said he needed. It's a separate function without a CLI binding, so it would need a re-design from scratch if we still want this. But in that case we should discuss the actual behaviour we want.

[satra]

we are doing this using notebooks to generate a dashboard for 26 and 108 (both bids datasets). perhaps we can generalize some of that. i think this PR was opened in relation to the validation work in CLI. the dashboard generation could live somewhere else for now, perhaps as a bids summary generator in bids-specification.

the other related component is to do appropriate diffs between two states at a higher level (subjects, data modalities, etc., and not just so many assets added/deleted).

let's perhaps close this and see if this can be done in bids.