Repository navigation
Vectorise followed_rule() for mtp = FALSE - #192
Open
peterbruun wants to merge 1 commit into
Open
peterbruun wants to merge 1 commit into
peterbruun wants to merge 1 commit into
Conversation
followed_rule() compared the natural and shifted treatment one observation at a time with mapply(all.equal), which dominates the density ratio estimation time for large data with deterministic (mtp = FALSE) interventions. The comparison is now vectorised for numeric, factor and character treatments, with the same semantics as the element-wise all.equal() (relative tolerance sqrt(.Machine$double.eps), absolute when |x| <= tolerance; NA equals NA; factors must share class and levels). Other types fall back to the previous element-wise all.equal(). Adds tests comparing the new implementation with the previous one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Thank you for developing and maintaining lmtp. It is a great and very impressive package, and it has made it possible for us to estimate longitudinal causal effects in a large clinical cohort in a principled way. This PR is a small contribution back.
Summary
With
mtp = FALSE,followed_rule()decides for every observation whether theobserved treatment equals the shifted treatment. It does this one observation at a
time with
mapply(function(x, y) isTRUE(all.equal(x, y)), ...). Becausefollowed_rule()is called for every time point and every fold, this element-wiseall.equal()becomes the main cost of the density ratio estimation for large datawith deterministic (dynamic) interventions.
This PR replaces it with a vectorised comparison that has the same semantics as the
element-wise
all.equal():sqrt(.Machine$double.eps)(an absolute difference when
|x| <= toleranceorxis not finite), as inall.equal.numeric();NAequalsNA, andNAnever equals a value.Other column types (e.g. logical) fall back to the previous element-wise
all.equal(), so their behaviour is unchanged. The row-wiseapply(followed, 1, prod)is replaced byrowSums(), which returns the same 0/1numeric vector.
Tests
tests/testthat/test-followed_rule.Rcompares the new implementation with theprevious one (copied into the test file as the reference) using
expect_identical()for:
the tolerance, values near zero, and
NA;Inf,-Inf,NaNandNA;NA;mtp = TRUE;Benchmark
One call of
followed_rule(), binary treatment, 10% of observations not following,R 4.6.2. Results were identical in every case.
followed_rule()is called once per time point and fold, so for long follow-upthese savings add up. For example, 28 time points × 5 folds × several strategies
means hundreds of calls.
We noticed this in a target trial emulation with about 10,000 patients, 28 daily
time points and deterministic threshold strategies (
mtp = FALSE). There,followed_rule()dominated the fitting time. We have used the vectorised version(patched in with
assignInNamespace()) for all analyses, after checking that itmatched the original on the real data for every strategy and time point.
The NEWS entry is under 1.5.5 / General.
This PR was prepared with the help of Claude Code.