Repository navigation
SVStringBreak fails parsing when locale is set - #105
Merged
Merged
Conversation
Previously the function 'SVStringBreak' used the C function 'strxfrm' to extract the substrings for variable name and layer number. This function considers current locale settings for string processing and it's purpose relates to comparing strings in a given locale. It's usage here is relying on undefined behaviour, because the return value is not checked and that value gives the necessary target buffer size minus one. That target size in turn can be larger than the length of the substrings extracted here, depending on the C implementation and locale. The function was replaced with a call to 'strncpy' and the platform specific distinctions removed.
for more information, see https://pre-commit.ci
analytophile
approved these changes
Aug 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Previously the function
SVStringBreakused the C functionstrxfrmto extract the substrings for variable name and layer number. This function considers current locale settings for string processing and it's purpose relates to comparing strings in a given locale. It's usage here is relying on undefined behaviour, because the return value is not checked and that value gives the necessary target buffer size minus one. That target size in turn can be larger than the length of the substrings extracted here, depending on the C implementation and locale.The function was replaced with a call to
strncpyand the platform specific distinctions removed.The issue appeared for me only when calling the Raven BMI bindings from within an IPython session. Raven itself does not initialize any locales, but in the IPython session the locale gets intialized and set to the environment settings. Than parsing of variables like
SOIL[0]would fail and Raven would exit withERROR : StringToSVtype: Unrecognized State Variable (2).I was able to reproduce the issue within Raven itself by setting a locale in Raven before the first call to
strxfrm, e.g. adding the linein
SVStringBreak.Because of the undefined behavior being implementation specific I do not know if it will show the same symptoms on other platforms, I tested on a Debian Linux system.
I also noticed there were different code paths for different platforms with
__APPLE__and__CYGWIN__having their target buffer sizes increased in thestrxfrmcall. My guess is this might relate to this issue.