Debug build of libLLVM.so is enormous

[Aaron1011]

This isn't an issue with Rust per se - however, it has implications for people building the compiler.

The libLLVM.so shipped with the Rust compiler (libLLVM-9-rust-1.41.0-nightly.so at the time of writing) is currently 80MB. While this seems incredibly large for a shared library, it's still only slightly larger than librustc_driver (63MB).

This is unfortunate, but things become much worse when llvm.release-debuginfo=true is set in config.toml. This makes libLLVM-9-rust-1.41.0-dev-de1a7dbf6.so take up 2.1 GB of space.

To make matters worse, we currently copy libLLVM.so to a target directory when building the compiler:

rust/src/bootstrap/dist.rs

Line 2167 in ae1b871

builder.install(&llvm_dylib_path, &dst_libdir, 0o644);

With #67077, we may have to copy it into even more places. This represents a non-trivial amount of disk space.

I don't think there's much we can directly do about the size of libLLVM.so, other than making sure upstream is aware of this. However, we might want to consider symlinking libLLVM.so, rather than copying it. I know this works for shared libraries on Linux, but I'm not sure about other platforms.

[tesuji]

Also in ~/.rustup/toolchains:

> find -name '*libLLVM*' -exec ls -lai {} \+
129111035 -rw-r--r-- 1 lzutao lzutao 81161488 Dec  3 09:33 ./beta-x86_64-unknown-linux-gnu/lib/libLLVM-9-rust-1.40.0-beta.so
129111071 -rw-r--r-- 1 lzutao lzutao 81161488 Dec  3 09:33 ./beta-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libLLVM-9-rust-1.40.0-beta.so
126485192 -rw-r--r-- 1 lzutao lzutao 83195704 Dec  5 12:54 ./master/lib/libLLVM-9-rust-1.41.0-nightly.so
126485193 -rw-r--r-- 1 lzutao lzutao 83195704 Dec  5 12:54 ./master/lib/rustlib/x86_64-unknown-linux-gnu/lib/libLLVM-9-rust-1.41.0-nightly.so
126755208 -rw-r--r-- 1 lzutao lzutao 83195704 Dec  7 04:09 ./nightly-x86_64-unknown-linux-gnu/lib/libLLVM-9-rust-1.41.0-nightly.so
126755209 -rw-r--r-- 1 lzutao lzutao 83195704 Dec  7 04:09 ./nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libLLVM-9-rust-1.41.0-nightly.so
> find -name '*libLLVM*' -exec sha256sum {} \+
b837248c30682a65177aa00ea138f33d7ad2596fd7cf158d41cb5d858ebe2df7  ./nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libLLVM-9-rust-1.41.0-nightly.so
b837248c30682a65177aa00ea138f33d7ad2596fd7cf158d41cb5d858ebe2df7  ./nightly-x86_64-unknown-linux-gnu/lib/libLLVM-9-rust-1.41.0-nightly.so
4fdc606fff426a248fc6b814aabd2ff9087f13893b9bfb6ed0724ad89c39995a  ./beta-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libLLVM-9-rust-1.40.0-beta.so
4fdc606fff426a248fc6b814aabd2ff9087f13893b9bfb6ed0724ad89c39995a  ./beta-x86_64-unknown-linux-gnu/lib/libLLVM-9-rust-1.40.0-beta.so
b837248c30682a65177aa00ea138f33d7ad2596fd7cf158d41cb5d858ebe2df7  ./master/lib/rustlib/x86_64-unknown-linux-gnu/lib/libLLVM-9-rust-1.41.0-nightly.so
b837248c30682a65177aa00ea138f33d7ad2596fd7cf158d41cb5d858ebe2df7  ./master/lib/libLLVM-9-rust-1.41.0-nightly.so

I think these libLLVM files in the same toolchain could be symlink too.

[nagisa]

I don't think there's much we can directly do about the size of libLLVM.so, other than making sure upstream is aware of this.

It is a well known fact to people working with LLVM that a debug build is both large in its outputs and very stressful on the toolchain. This might improve when we get good support for separated debuginfo files, but the toolchain is not quite there yet.

[nagisa]

(This also likely does not affect windows as much as it does affect other platforms)

[jyn514]

Bootstrap is now using hard links for llvm whenever possible:

rust/src/bootstrap/lib.rs

Lines 1520 to 1524 in 77f4f82

	if let Ok(()) = fs::hard_link(&src, dst) {
	// Attempt to "easy copy" by creating a hard link
	// (symlinks don't work on windows), but if that fails
	// just fall back to a slow `copy` operation.
	} else {

I'm going to close this since upstream is aware of the issue, I don't see much we can do on our end.