testing `cg_gcc` is difficult

[workingjubilee]

cc @GuillaumeGomez @antoyo

The backend currently only supports x86-64, but some contributors primarily use aarch64 machines, but have cause to work on the codegen backends. They instead have to force cg_gcc's code through emulators, which is often an inadequate solution. It would be better if it simply supported all of our tier 1 architectures.

Meta

rustc --version --verbose:

rustc 1.79.0-nightly (ccfcd950b 2024-04-15)
binary: rustc
commit-hash: ccfcd950b333fed046275dd8d54fe736ca498aa7
commit-date: 2024-04-15
host: x86_64-unknown-linux-gnu
release: 1.79.0-nightly
LLVM version: 18.1.3

[workingjubilee]

Hopefully this is just tweaking a build flag?

[antoyo]

It supports other architectures, but we decided to only enable x86-64 because GCC requires to build a toolchain per target, so this is going to take some CI resources to build them.

Can you describe your use case a bit more?
If this is for local development, there might be a simple solution.

[antoyo]

There might be a bug to fix for aarch64 in cg_gcc, though.

[workingjubilee]

@antoyo The Rust Foundation provides both aarch64 and x86_64 virtual desktops for the purpose of working on the compiler. So if someone starts work on a contribution using an aarch64 desktop, finds out they need to modify cg_gcc, they have to either

• learn about emulators! some will regard this as fun, at least?
• switch to an x86_64 desktop! this is... easy, kinda? but can be a pain in the ass because not everything can be easily moved via git. build directories are real and now they have to throw that out and wait however long it takes to rebuild what they were working on.

It's just these little tripping hazards which cause unnecessary stumbles for contributors. They are technically possible to overcome but it can be annoying af, yanno? I think it makes sense to support both architectures since we do have both x86_64-unknown-linux-gnu and aarch64-unknown-linux-gnu at tier 1, and building gcc is relatively easy on those platforms, and it is much easier to e.g. emulate Linux than it is to emulate another architecture. And by "emulate" that probably more specifically means "run the kernel inside a hypervisor", realistically, but that version is still comparatively seamless.

[workingjubilee]

Someone primarily using an aarch64-apple-darwin machine to contribute (and we do have a few of those) and doesn't currently have access to the virtual desktops would be stuck with Rosetta2 JITting the entire compiler, probably? And then libgccjit generates x86-64 code to compile test binaries? Which then Rosetta2 JITs? And that can be fast but... that's a bit of a lot that "ability to directly use native binaries, even if using a foreign object encoding" bypasses (e.g. they could also have Asahi or Parallels on the machine).

Any recommendations for unblocking people running into this issue should probably be PRed to the dev guide! As should... probably... more general build/test/etc. instructions for both codegen backends?

[GuillaumeGomez]

One important thing to note is that currently the GCC backend is broken in the compiler (something about statics). I'm currently (although slowly) looking into it.

There are multiple issues we need to fix to make it easier for everyone to contribute on GCC backend:

• Allow to pick which backend to use for x.py commands.
• Have more pre-compiled targets for libgccjit.so (you can build it yourself, but we need to update the guide as you mentioned).
• Fix the backend with latest rustc updates.

[workingjubilee]

Oof! That sounds "exciting".

Allow to pick which backend to use for x.py commands.

Ah, something like --cg-backend=gcc, I guess?

We'd probably want to allow setting something like --cg-bootstrap=llvm ("use LLVM to build the compiler itself") and --cg-backend=gcc ("use cg_gcc to build any other binaries") individually? Do we have an issue for that open yet?

Have more pre-compiled targets for libgccjit.so (you can build it yourself, but we need to update the guide as you mentioned).

Ah, having build instructions there would be great! Or even a pointer to build instructions from the guide into a README in the repo somewhere, if they're still likely to be updated often in tandem with cg_gcc.

[GuillaumeGomez]

Oof! That sounds "exciting".

It is. 😉

Allow to pick which backend to use for x.py commands.

Ah, something like --cg-backend=gcc, I guess?

We'd probably want to allow setting something like --cg-bootstrap=llvm ("use LLVM to build the compiler itself") and --cg-backend=gcc ("use cg_gcc to build any other binaries") individually? Do we have an issue for that open yet?

Exactly. It's in my todo list once the GCC backend is fixed (no use to add a new build system feature if it cannot be tested nor used haha).

Have more pre-compiled targets for libgccjit.so (you can build it yourself, but we need to update the guide as you mentioned).

Ah, having build instructions there would be great! Or even a pointer to build instructions from the guide into a README in the repo somewhere, if they're still likely to be updated often in tandem with cg_gcc.

There are already in the rustc_codegen_gcc repository here. It's just that I don't know how well it would integrate with the current build system. In our own build system, it's very easy to use though. Again, more plumbing to be done to make everything as smooth as it should be which I did already... if you are on x86_64 target. 😅

[antoyo]

Would pre-built libgccjit be helpful?
We already have cross-compilers from x86-64 to m68k and arm in this repo.
We could have some (cross-)compilers from aarch64 if needed.

[workingjubilee]

I think pre-built libgccjit would be good, but I also think sufficient build/cross-build tools/instructions available that it's easy enough to build libgccjit on one's own should be fine? It seems reasonable to expect compiler contributors to put up with inconveniences occasionally, but a lot of the sorts of persons who will be contributing to the compiler are the sort of person who will want to do something "properly" so it won't actually occur to them to simply give up and switch to x86-64, they'll look at how to get cg_gcc working "the right way" on their system first. So even instructions to give up would be helpful. ^^;