Build failure: Missing zephyr/offsets.h due to race condition in platformio-build.py

[Hygdrasil]

Describe the bug
Compiling an cleaned application with zephyr on multiple threads results sometimes in a race condition where some of the pregenerated zephyr headers are used before they are generated.

To Reproduce
I've encountered an interesting issue while working with Zephyr that I'd love some help with. When compiling an application with Zephyr on multiple threads, I occasionally run into a race condition where some of the pregenerated Zephyr headers are being used before they're actually generated.

Expected behavior
Unfortunately, I wasn't able to create a minimal reproducible example, but I can share the error logs and my proposed solution below.

Impact
annoyance

Logs and console output

11:42:39  Compiling .pio/build/custom_zephyr_board/kernel/zephyr/kernel/main_weak.c.o
11:42:39  Indexing .pio/build/custom_zephyr_board/zephyr/liboffsets.a
11:42:39  Compiling .pio/build/custom_zephyr_board/kernel/zephyr/kernel/banner.c.o
11:42:39  Compiling .pio/build/custom_zephyr_board/kernel/zephyr/kernel/busy_wait.c.o
11:42:39  Compiling .pio/build/custom_zephyr_board/kernel/zephyr/kernel/device.c.o
11:42:39  Compiling .pio/build/custom_zephyr_board/kernel/zephyr/kernel/errno.c.o
11:42:39  Compiling .pio/build/custom_zephyr_board/kernel/zephyr/kernel/fatal.c.o
11:42:39  Compiling .pio/build/custom_zephyr_board/kernel/zephyr/kernel/init.c.o
11:42:39  Compiling .pio/build/custom_zephyr_board/kernel/zephyr/kernel/kheap.c.o
11:42:39  Compiling .pio/build/custom_zephyr_board/kernel/zephyr/kernel/mem_slab.c.o
11:42:39  Compiling .pio/build/custom_zephyr_board/kernel/zephyr/kernel/float.c.o
11:42:39  In file included from .piocore/packages/framework-zephyr/kernel/init.c:17:
11:42:39  ~/test_project/.pio/build/custom_zephyr_board_workspace/.piocore/packages/framework-zephyr/kernel/include/offsets_short.h:10:10: fatal error: zephyr/offsets.h: No such file or directory
11:42:39     10 | #include <zephyr/offsets.h>

Environment (please complete the following information):

• OS: Linux,
• Toolchain
• Version: 4.0
  Additional context
  I've found that this can be fixed by applying the following diff:

index ef5e644e..bee72bc8 100644
--- a/scripts/platformio/platformio-build.py
+++ b/scripts/platformio/platformio-build.py
@@ -2027,11 +2027,10 @@ for target, target_config in target_configs.items():
         "config": target_config,
     }
 
-    if any(
-        d.get("id", "").startswith("zephyr_generated_headers")
-        for d in target_config.get("dependencies", [])
-    ):
-        env.Depends(lib[0].sources, offset_header_file)
+    # prevent build of zephyr before offset_header is generated, as it's required for compilation of some of the libraries
+    # this is a workaround for the issue when some of the libraries are built before the offset header file is generated, which leads to build failure due to missing offsets.h file
+    # Note: ideally, the dependency on the offset header file should be added only to those libraries that require it
+    env.Depends(lib[0].sources, offset_header_file)
 
 # Offsets library compiled separately as it's used later for custom dependencies
 offsets_lib = build_library(env, target_configs["offsets"], PROJECT_SRC_DIR)

This workaround resolves the issue, though I think it would be even better to fix the logic in the conditional statement to only add the dependency where it's actually needed.

What do you think? I'd be happy to discuss alternative approaches or provide any additional information that might be helpful!

Thanks for taking the time to look into this! 😊