Skip to content

feat: support back-references in byte mode #602

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Omkar-Wagholikar wants to merge 5 commits into
base: main
Choose a base branch
from
+253 −34
Open

feat: support back-references in byte mode #602

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
6211418
feat: support back-references in byte mode
Omkar-Wagholikar Oct 14, 2026
17a83d8
fast_regex: improve fancy-bytes coverage and rename test
Omkar-Wagholikar Oct 15, 2026
24fd689
Merge branch 'main' of https://github.com/uutils/sed into fix/543-byt…
Omkar-Wagholikar Oct 15, 2026
3e172c6
Merge branch 'main' into fix/543-byte-mode-backrefs
codspeedbot Oct 18, 2026
b046ee3
processor: share compiled REs via Rc instead of cloning them per line
codspeedbot Oct 18, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions Cargo.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ clap = { version = "4.4", features = ["wrap_help", "cargo"] }
clap_complete = "4.5"
clap_mangen = "0.3"
divan = { package = "codspeed-divan-compat", version = "5.0.0" }
fancy-regex = "0.18.0"
fancy-regex = "0.19.0"
hex = "0.4"
libc = "0.2.153"
memchr = "2.7.4"
Expand All @@ -55,7 +55,7 @@ phf = "0.14.0"
phf_codegen = "0.14.0"
predicates = "3.1.3"
rand = { version = "0.10.0" }
regex = "1.10.4"
regex = { version = "1.10.4", features = ["perf-dfa-full"] }

@sylvestre sylvestre Oct 14, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

perf-dfa-full is unrelated to back-references, please split that into another pr.
it also impacts the binary size, please give numbers there

@Omkar-Wagholikar Omkar-Wagholikar Oct 15, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't exactly unrelated, the fancy-regex 0.19 dropped regex-automata/dfa (0.18 enabled it), so the bump essentially disables regex's full DFA.

Without adding the perf-dfa-full flag, a 1M+line long s/[0-9]+.[0-9]+…/IP/ run goes from ~0.75s to ~10s.

Binary size (release, x86_64 Linux): main 3,214,528 B; this PR without the flag 3,128,960 B; with the flag 3,302,656 B (+88 KB vs main which is ~3% bigger).

sha2 = { version = "0.11.0", default-features = false, features = ["alloc"] }
tempfile = "3.10.1"
terminal_size = "0.4.2"
Expand Down
4 changes: 2 additions & 2 deletions fuzz/Cargo.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

16 changes: 8 additions & 8 deletions src/sed/command.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ pub struct ProcessingContext {
/// Whether sed operates on bytes or UTF-8 characters
pub character_mode: CharacterMode,
/// Previously compiled RE, saved for reuse when specifying an empty RE
pub saved_regex: Option<Regex>,
pub saved_regex: Option<Rc<Regex>>,
/// Modification of input processing action
// This is required to avoid doubly borrowing the reader in the 'N'
// command.
Expand Down Expand Up @@ -106,12 +106,12 @@ pub struct ByteSpace {
#[derive(Debug)]
/// Types of address specifications that precede commands
pub enum Address {
Re(Option<Regex>), // Line that matches (optional) regex
Line(usize), // Specific line
RelLine(usize), // Relative line
Last, // Last line
StepMatch(usize), // Lines matching specified step from first
StepEnd(usize), // Range ending at specified step from first
Re(Option<Rc<Regex>>), // Line that matches (optional) regex
Line(usize), // Specific line
RelLine(usize), // Relative line
Last, // Last line
StepMatch(usize), // Lines matching specified step from first
StepEnd(usize), // Range ending at specified step from first
}

#[derive(Debug)]
Expand Down Expand Up @@ -226,7 +226,7 @@ impl ReplacementTemplate {
#[derive(Debug, Default)]
/// Substitution command
pub struct Substitution {
pub regex: Option<Regex>, // Regular expression
pub regex: Option<Rc<Regex>>, // Regular expression
pub replacement: ReplacementTemplate, // Specified broken-down replacement
pub occurrence: usize, // Which occurrence to substitute
pub print_flag: bool, // True if 'p' flag
Expand Down
9 changes: 5 additions & 4 deletions src/sed/compiler.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -488,9 +488,9 @@ fn compile_address(
line.advance();
}

Ok(Address::Re(compile_regex(
lines, line, &re, context, icase, false,
)?))
Ok(Address::Re(
compile_regex(lines, line, &re, context, icase, false)?.map(Rc::new),
))
}
'$' => {
line.advance();
Expand Down Expand Up @@ -883,7 +883,8 @@ fn compile_subst_command(
context,
subst.ignore_case,
subst.multiline,
)?;
)?
.map(Rc::new);

// Catch invalid group references at compile time, if possible.
if let Some(regex) = &subst.regex
Expand Down
Loading
Loading