buffer for stdout is heap-allocated

[abgros]

While playing around with global allocators, I noticed that this code fails.

use std::alloc::{GlobalAlloc, Layout};

struct StupidAlloc;

unsafe impl GlobalAlloc for StupidAlloc {
	unsafe fn alloc(&self, _layout: Layout) -> *mut u8 {
		0 as _
	}

	unsafe fn dealloc(&self, _ptr: *mut u8, _layout: Layout) {}
}

#[global_allocator]
static GLOBAL: StupidAlloc = StupidAlloc;

fn main() {
	println!("Hello world!");
}

A message is printed: memory allocation of 1024 bytes failed, and then the program terminates. Removing the line with println! fixes this, so I assume that it is trying to allocate 1 KiB and then crashing when it can't.

However, there's no need to allocate memory here, because the string to be printed is static. So although this isn't strictly a bug, it certainly might be considered a performance issue in a IO-heavy program.

[lolbinarycat]

The issue is not with println!, as this happens if you write directly to Stdout.

I'm pretty sure these 1024 bytes are a lazily-initialized buffer, so this will only happen the first time, not each time println! is called.

Still, it could be argued that this buffer should be in static memory, instead of being allocated on the heap.

Also note that the global allocator may be used in the pre-main initialization code.

use std::alloc::{GlobalAlloc, Layout};
use std::io::Write;

struct StupidAlloc;

unsafe impl GlobalAlloc for StupidAlloc {
	unsafe fn alloc(&self, _layout: Layout) -> *mut u8 {
		0 as _
	}

	unsafe fn dealloc(&self, _ptr: *mut u8, _layout: Layout) {}
}

#[global_allocator]
static GLOBAL: StupidAlloc = StupidAlloc;

fn main() {
	std::io::stdout().write(b"hello");
}

[moxian]

This is not neccessarily related to io, and looks more related to threading, somehow

The number in memory allocation of XXX bytes failed has changed over the years with no corresponding changes to io
Most recently it went 56->1024 in nightly-2024-10-25 / f61306d / #123550

But previously

10 -> 56 nightly-2024-04-11

converted 2024-04-10 to 8b2459c1f21187f9792d99310171a15e64feb9cf
converted 2024-04-11 to aa067fb984d36462548bb785da221bfaf38253f0
  commit[0] 2024-04-09: Auto merge of #123683 - pietroalbini:pa-cve-2024-24576-nightly, r=pietroalbini
  commit[1] 2024-04-10: Auto merge of #122918 - jieyouxu:port-backtrace-line-tables-only, r=workingjubilee
  commit[2] 2024-04-10: Auto merge of #123708 - matthiaskrgr:rollup-uf9w1e9, r=matthiaskrgr
  commit[3] 2024-04-10: Auto merge of #122812 - dtolnay:mode, r=workingjubilee
  commit[4] 2024-04-10: Auto merge of #122393 - a1phyr:specialize_read_buf_exact, r=joboet
  commit[5] 2024-04-10: Auto merge of #123725 - GuillaumeGomez:rollup-gk2bbrg, r=GuillaumeGomez
  commit[6] 2024-04-10: Auto merge of #123185 - scottmcm:more-typed-copy, r=compiler-errors
  commit[7] 2024-04-10: Auto merge of #121346 - m-ou-se:temp-lifetime-if-else-match, r=compiler-errors
  commit[8] 2024-04-10: Auto merge of #123745 - weihanglo:update-cargo, r=weihanglo

• note Windows: set main thread name without re-encoding #123534

5->10 in nightly-2022-06-05

converted 2022-06-04 to a6b8c6954829669a5c4fa320c3e6132edf04fcfc
converted 2022-06-05 to 4e725bad73747a4c93a3ac53106e4b4006edc665
  commit[0] 2022-06-03: Auto merge of #95833 - notriddle:notriddle/human-readable-signals, r=yaahc
  commit[1] 2022-06-03: Auto merge of #97670 - spastorino:simplify-universal-impl-trait-lowering2, r=Dylan-DPC
  commit[2] 2022-06-04: Auto merge of #97717 - matthiaskrgr:rollup-lalaii2, r=matthiaskrgr
  commit[3] 2022-06-04: Auto merge of #97604 - nnethercote:inline-bridge-Buffer-methods, r=eddyb
  commit[4] 2022-06-04: Auto merge of #97690 - nikic:update-llvm-4, r=cuviper
  commit[5] 2022-06-04: Auto merge of #97729 - Dylan-DPC:rollup-dv43xo9, r=Dylan-DPC
  commit[6] 2022-06-04: Auto merge of #97137 - Kobzol:ci-llvm-pgo-pid, r=Mark-Simulacrum
  commit[7] 2022-06-04: Auto merge of #97529 - Urgau:bootstrap-check-cfg-features, r=Mark-Simulacrum
  commit[8] 2022-06-04: Auto merge of #97191 - wesleywiser:main_thread_name, r=ChrisDenton

• note Call the OS function to set the main thread's name on program init #97191

4->5 in nightly-2021-09-30

converted 2021-09-29 to 8f8092cc32ec171becef8ceacec7dbb06c5d7d7e
converted 2021-09-30 to 11491938f80988c7261a1179cf71a25c379c8783
  commit[0] 2021-09-28: Auto merge of #89048 - oli-obk:in_tracing_we_trust, r=jackh726
  commit[1] 2021-09-28: Auto merge of #7733 - flip1995:rustup, r=flip1995
  commit[2] 2021-09-29: Auto merge of #88950 - Nadrieril:deconstruct-pat, r=oli-obk
  commit[3] 2021-09-29: Auto merge of #89328 - flip1995:clippyup, r=Manishearth
  commit[4] 2021-09-29: Auto merge of #89331 - GuillaumeGomez:rollup-b10unye, r=GuillaumeGomez
  commit[5] 2021-09-29: Auto merge of #89011 - bjorn3:restructure_rt, r=dtolnay

• note Restructure std::rt #89011

6 ->4 in nightly-2021-01-25

converted 2021-01-24 to 4d0dd02ee07bddad9136f95c9f7846ebf3eb3fc5
converted 2021-01-25 to 1d0d76f8dd4f5f6ecbeab575b87edaf1c9f56bb8
  commit[0] 2021-01-23: Auto merge of #80579 - RalfJung:no-fallible-promotion, r=oli-obk
  commit[1] 2021-01-23: Auto merge of #81304 - jonas-schievink:rollup-d9kuugm, r=jonas-schievink
  commit[2] 2021-01-24: Auto merge of #79811 - Aaron1011:expn-data-disambig, r=petrochenkov
  commit[3] 2021-01-24: Auto merge of #80594 - bjorn3:abi_refactor3, r=petrochenkov
  commit[4] 2021-01-24: Auto merge of #80919 - cjgillot:defkey-span, r=oli-obk
  commit[5] 2021-01-24: Auto merge of #80838 - nagisa:nagisa/stack-probe-type, r=cuviper
  commit[6] 2021-01-24: Auto merge of #81250 - sivadeilra:remove_xp_compat, r=joshtriplett,m-ou-se
  commit[7] 2021-01-24: Auto merge of #81337 - petrochenkov:kvcollect, r=Aaron1011

(Although perhaps a better and more useful way to find this out would be by grabbing the backtrace from inside the first alloc call instead of bisecting...)

@rustbot label: +T-libs +A-allocators +A-runtime

[joboet]

Is there any particular reason you need this? Otherwise I doubt that this has any significant performance impact, it's really just one extra very well-predicted check per io::stdout() call. The advantage is that we can reuse the public LineWriter and don't have to recreate its logic in Stdout.

[joboet]

This is not neccessarily related to io, and looks more related to threading, somehow

The number in memory allocation of XXX bytes failed has changed over the years with no corresponding changes to io
Most recently it went 56->1024 in nightly-2024-10-25 / f61306d / #123550

But previously

10 -> 56 nightly-2024-04-11

converted 2024-04-10 to 8b2459c1f21187f9792d99310171a15e64feb9cf
converted 2024-04-11 to aa067fb984d36462548bb785da221bfaf38253f0
  commit[0] 2024-04-09: Auto merge of #123683 - pietroalbini:pa-cve-2024-24576-nightly, r=pietroalbini
  commit[1] 2024-04-10: Auto merge of #122918 - jieyouxu:port-backtrace-line-tables-only, r=workingjubilee
  commit[2] 2024-04-10: Auto merge of #123708 - matthiaskrgr:rollup-uf9w1e9, r=matthiaskrgr
  commit[3] 2024-04-10: Auto merge of #122812 - dtolnay:mode, r=workingjubilee
  commit[4] 2024-04-10: Auto merge of #122393 - a1phyr:specialize_read_buf_exact, r=joboet
  commit[5] 2024-04-10: Auto merge of #123725 - GuillaumeGomez:rollup-gk2bbrg, r=GuillaumeGomez
  commit[6] 2024-04-10: Auto merge of #123185 - scottmcm:more-typed-copy, r=compiler-errors
  commit[7] 2024-04-10: Auto merge of #121346 - m-ou-se:temp-lifetime-if-else-match, r=compiler-errors
  commit[8] 2024-04-10: Auto merge of #123745 - weihanglo:update-cargo, r=weihanglo

• note Windows: set main thread name without re-encoding #123534

5->10 in nightly-2022-06-05

converted 2022-06-04 to a6b8c6954829669a5c4fa320c3e6132edf04fcfc
converted 2022-06-05 to 4e725bad73747a4c93a3ac53106e4b4006edc665
  commit[0] 2022-06-03: Auto merge of #95833 - notriddle:notriddle/human-readable-signals, r=yaahc
  commit[1] 2022-06-03: Auto merge of #97670 - spastorino:simplify-universal-impl-trait-lowering2, r=Dylan-DPC
  commit[2] 2022-06-04: Auto merge of #97717 - matthiaskrgr:rollup-lalaii2, r=matthiaskrgr
  commit[3] 2022-06-04: Auto merge of #97604 - nnethercote:inline-bridge-Buffer-methods, r=eddyb
  commit[4] 2022-06-04: Auto merge of #97690 - nikic:update-llvm-4, r=cuviper
  commit[5] 2022-06-04: Auto merge of #97729 - Dylan-DPC:rollup-dv43xo9, r=Dylan-DPC
  commit[6] 2022-06-04: Auto merge of #97137 - Kobzol:ci-llvm-pgo-pid, r=Mark-Simulacrum
  commit[7] 2022-06-04: Auto merge of #97529 - Urgau:bootstrap-check-cfg-features, r=Mark-Simulacrum
  commit[8] 2022-06-04: Auto merge of #97191 - wesleywiser:main_thread_name, r=ChrisDenton

• note Call the OS function to set the main thread's name on program init #97191

4->5 in nightly-2021-09-30

converted 2021-09-29 to 8f8092cc32ec171becef8ceacec7dbb06c5d7d7e
converted 2021-09-30 to 11491938f80988c7261a1179cf71a25c379c8783
  commit[0] 2021-09-28: Auto merge of #89048 - oli-obk:in_tracing_we_trust, r=jackh726
  commit[1] 2021-09-28: Auto merge of #7733 - flip1995:rustup, r=flip1995
  commit[2] 2021-09-29: Auto merge of #88950 - Nadrieril:deconstruct-pat, r=oli-obk
  commit[3] 2021-09-29: Auto merge of #89328 - flip1995:clippyup, r=Manishearth
  commit[4] 2021-09-29: Auto merge of #89331 - GuillaumeGomez:rollup-b10unye, r=GuillaumeGomez
  commit[5] 2021-09-29: Auto merge of #89011 - bjorn3:restructure_rt, r=dtolnay

• note Restructure std::rt #89011

6 ->4 in nightly-2021-01-25

converted 2021-01-24 to 4d0dd02ee07bddad9136f95c9f7846ebf3eb3fc5
converted 2021-01-25 to 1d0d76f8dd4f5f6ecbeab575b87edaf1c9f56bb8
  commit[0] 2021-01-23: Auto merge of #80579 - RalfJung:no-fallible-promotion, r=oli-obk
  commit[1] 2021-01-23: Auto merge of #81304 - jonas-schievink:rollup-d9kuugm, r=jonas-schievink
  commit[2] 2021-01-24: Auto merge of #79811 - Aaron1011:expn-data-disambig, r=petrochenkov
  commit[3] 2021-01-24: Auto merge of #80594 - bjorn3:abi_refactor3, r=petrochenkov
  commit[4] 2021-01-24: Auto merge of #80919 - cjgillot:defkey-span, r=oli-obk
  commit[5] 2021-01-24: Auto merge of #80838 - nagisa:nagisa/stack-probe-type, r=cuviper
  commit[6] 2021-01-24: Auto merge of #81250 - sivadeilra:remove_xp_compat, r=joshtriplett,m-ou-se
  commit[7] 2021-01-24: Auto merge of #81337 - petrochenkov:kvcollect, r=Aaron1011

(Although perhaps a better and more useful way to find this out would be by grabbing the backtrace from inside the first alloc call instead of bisecting...)

@rustbot label: +T-libs +A-allocators +A-runtime

We've gotten rid of some pre-main allocations, so the execution now actually reaches main and only stops with the buffer allocation. Before those changes, the allocation failure just occurred earlier. So no, this isn't about threading.

[ptrca]

Side note: By default, C (glibc) also uses heap-allocated buffers for stdin/stdout for standard functions like puts, printf, fgets, etc.

[workingjubilee]

This issue does not contain evidence of a problem, it contains a hypothesis. Closing.