Use doc comments to generate the component docs

[MOZGIII]

There's been a bit of a discussion for this before, but I decided to create an issue that we can track the progress on this idea with.

So, rust has this doc-comments macro (and a syntax sugar /// for it), and a rustdoc utility that can generate the human-readable documentation from them.

It would be kind of cool to be able to use the same comments we have in the code as the documentation:

• we get to have the documentation driven from the actual source code, thus it's less likely to diverge;
• the code will be properly documented, in addition to docs being available to the website;
• we'll have a single source of truth for the documentation of a component;
• it'll be much more natural to add the docs together with the code, "while you're at it".

There are multiple approaches we could take to derive the website docs from the doc-comments:

• rustdoc generates the HTML documentation pages; we could parse the generated HTML and fetch the data out of it;
• rustdoc could also generate raw Markdown documentation (via -w / --output-format flags) - that option has been deprecated though;
• rustdoc is (re)gaining the ability to generate JSON via (--output-format json) - it's currently a WIP, but we could use that once it's implemented and goes out of nightly (or we could use it in nightly).
• we can consume the doc-comment macros directly - see https://rustc-dev-guide.rust-lang.org/rustdoc.html;
• we could add a plugin to rustdoc to produce the output in the whatever form we need - that option has been deprecated though;
• we can parse the raw rust code manually and fetch the ///s from it.

So, that's all I have to pose the problem, let's discuss!

[bruceg]

I agree with this in general, for the same reasons you outline, though there are many specifics to iron out. We have also discussed the possibility of using a procedural macro that would output the documentation files which would be controlled with attributes much like serde does. I don't think that ever made it into an issue, though.

[lukesteensen]

My primary hesitation here is that there are two distinct audiences here, end users and contributors. It seems to me that doc comments should be aimed at contributors and not at end users.

[jszwedko]

Closing since we took a different approach with #12223