Skip to content

Fix expressed known isoforms missing from discovered transcripts and trust read ends with --fl_data - #443

Open
andrewprzh wants to merge 12 commits into
masterfrom
fl_data_known_isoforms
Open

andrewprzh wants to merge 12 commits into
masterfrom
fl_data_known_isoforms

Conversation

@andrewprzh

@andrewprzh andrewprzh commented Nov 2, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

A PacBio FLNC run (-d pacbio_ccs --fl_data, poly-A tails trimmed) lost about 18% of the annotated transcripts with at least 100 reads from transcript_models.gtf, including whole genes. There were three causes:

  • Annotated isoforms with identical intron chains, or chains a few bases apart at one splice site (NAGNAG acceptors), collided in the intron graph and overwrote each other.
  • The fl_pacbio preset (pacbio_ccs + --fl_data) set fl_only. That turned off building known isoforms from read assignments, the only rescue for known isoforms without a full-length path.
  • With trimmed tails no read end was trusted. Terminal exons that other reads splice into therefore got no full-length path.

Changes

  • b18714a: IntronGraph.collapse_vertex_set never merges two annotated introns. A graph path maps to every known isoform that threads to it. _reference_isoform_for_path chooses among them using read assignments.
  • f44b264: fl_only is removed, so construct_nonfl_isoforms runs for every preset.
  • 7341e9e: with --fl_data, a read 3' end without a detected tail is treated as a poly-A site and checked against the assigned isoforms. Explicit --polya_trimmed modes take precedence.
  • 9356fa2: with --fl_data, a read 5' end at a predicted TSS is kept even inside an exon that other reads splice into.
  • a4b2a33: in detect_similar_isoforms, a weaker near-identical novel model no longer removes a better-supported one.
  • 44ad280: with --fl_data, 5' ends are trusted except near the acceptor of an annotated or well-supported intron entering the exon, where reads with an unaligned short first exon start (IntronGraph.is_trusted_five_prime).
  • a258838: adds the CI test Mouse.PB_simulated.FL.reduced_db (1.2M FLNC-like PacBio reads simulated with badread: full length, oriented, no tails; -d pacbio --fl_data, reduced annotation). Also adds misc/simulate_fl_reads_badread.py, docs and the changelog entry.
  • d1bfac4: fixes from code review.
    • A poly-A site set after assignment (--fl_data, every --polya_trimmed mode, external tails without a side) now makes the assigner classify the read again, as for a tail found in the read sequence (LongReadAssigner.reassign_with_polya_site). Isoforms the site contradicts are dropped when others agree with it. The read is matched again (match_inconsistent) when the site contradicts all of them or the read was inconsistent. The steps that depend on the assignment run again: the inconsistent-MAPQ filter, exon correction and the antisense check. cd4f6d2 replaces this reclassification.
    • A trusted 5' end inside an exon is no longer threaded to a vertex past the intron entering (leaving) that exon (IntronGraph.five_prime_vertex_limit). As in is_trusted_five_prime, only annotated or well-supported introns count; rare novel introns are alignment noise.
    • The log notes that the poly-A percentage includes sites set by --fl_data or --polya_trimmed.
  • 7a86112: re-baselines Mouse and Human ONT TSS (transcript metrics) and the new FL test (transcript and quantification metrics). Performance baselines are unchanged.
  • 770e0c4: re-baselines the SIRVs, ONT polyA and Mouse ONT/PB simulation tests (accuracy only), see below.
  • 904d653: shortens the --fl_data and poly-A site text in docs/cmd.md.
  • cd4f6d2: the read strand is predicted before assignment, so the poly-A site set by --fl_data or --polya_trimmed goes in before the read profiles are built, exactly as a tail found in the sequence. This replaces the reclassification after assignment from d1bfac4.
    • ReadStrandPredictor (alignment/read_strand.py) tries these signals in order:

      1. a tail detected at one read end;
      2. for spliced reads, annotated introns (exact match) plus canonical sites of the other introns, all of which must agree;
      3. for unspliced reads, the annotated transcripts overlapping the read. Transcript spans are used, not gene records, because a gene can hold transcripts on both strands;
      4. --stranded forward/reverse as the last resort.
    • The predicted strand only places the site. read_assignment.strand still comes from the assignment. When the strand is undecided or the assignment gives the other one, move_polya_site moves the site and assign_again rebuilds the profiles and assigns the read once more. This happens for 1.8% of FL reads and 0.9% of genic reads on one ONT chromosome. --polya_trimmed all uses the mapped strand and never assigns twice.

    • LongReadAssigner.reassign_with_polya_site and its helpers are removed.

    • A tag: read without a TS tag now gets its strand from the predictor. Before, the side came straight from --stranded, which overrode a tail found in the sequence.

    • Cost, measured with per-read timers on the FL dataset (1.2M genic reads):

      per-read processing extra over assignment
      d1bfac4 (reclassification) 707 µs 172 µs (95% of reads reclassified, plus redone exon correction)
      cd4f6d2 419 µs 22 µs (15 µs strand prediction, 6 µs site move and re-assignments)
    • Re-baselines the FL test's quantification sections. See the Results section.

Results

New FL test, same options on both sides (gffcompare recall/precision, %):

master this branch
full 89.1 / 98.7 93.2 / 98.7
known 93.1 / 99.4 98.0 / 99.5
novel 66.1 / 91.8 67.5 / 93.1

Against the d1bfac4 baselines, cd4f6d2 keeps every accuracy metric of SIRVs.Set4.R10.opts2, Mouse ONT TSS and Human ONT TSS within tolerance. These configs also use --fl_data. On the FL test, transcript metrics are within tolerance: full recall 93.1→93.2, novel recall 67.3→67.5. Quantification improved enough to leave the tolerance and was re-baselined. For known isoforms, good/fair matches went 41/155→43/158 and missed (overlap) 572→540. For novel isoforms, missed (full) went 2223→2193. For genes, false reports went 23→22. The set of known transcripts missing from the FL run is unchanged: 48 transcripts in the same categories.

Re-baselined in 7a86112 (recall/precision, %):

Test Change
Mouse ONT TSS full 88.3→89.3; known 97.0→98.0; novel 56.4/95.2→57.3/95.9
Human ONT TSS full 84.9→87.1; known 95.0→96.6; novel 50.4/94.7→53.0/96.2

Re-baselined in 770e0c4, from the CI runs on 7a86112. Only sections with a metric outside the 1% tolerance were rewritten. None of these tests uses --fl_data or --polya_trimmed, so the drift comes from b18714a, f44b264 and a4b2a33. Transcript metrics are recall/precision, %; td50 and td10 move the same way unless listed.

Test Change
SIRVs Set4 real R10 and opts1 full 76.8/100→78.3/98.2; known 97.7→100; novel precision 100→91.7
SIRVs Set4 real R9 full 72.5→75.4; known 97.7→100; novel 30.8/88.9→34.6/90.0; precision at td50: full 98.0→96.2, novel 88.9→80.0
SIRVs simulated R10 / no_trunc full recall 68.1→71.0 / 66.7→69.6; known +2.3; novel 19.2/83.3→23.1/85.7; full precision at td10 about −1.5
SIRVs simulated R10 real_exp full recall 59.4→60.9; known 95.3→97.7
Human ONT polyA_1 full recall 71.5→72.7; known 75.4→77.0; novel 36.6/94.7→37.2/96.3
Mouse ONT polyA full recall 59.7→60.2; known 85.5→86.3; novel 19.3/41.7→19.5/42.1
Mouse ONT R9 reduced_db full 58.6/91.2→59.3/90.8; known 64.7/93.1→65.4/92.6; novel 21.8/65.3→22.4/66.2; novel quantification: false 2103→2217, missed 14886→14627
Mouse ONT R9 trunc reduced_db full recall 92.2→93.7; known 96.9→97.9; novel 63.2/87.8→67.9/88.9; novel precision at td10 44.6→43.0; novel quantification: corr_full 0.834→0.864, missed 2906→2300
Mouse ONT R10 reduced_db novel quantification: missed 5405→5199
Mouse ONT R10 no_gtf novel quantification: corr_full 0.5315→0.5399
Mouse PB reduced_db novel quantification: false 495→472, missed 2387→2153

The one extra false novel in SIRVs Set4 real R10 and opts1 is a two-exon fragment of SIRV708. It comes from internal priming at an A-rich exon junction, with a 206-read tail peak. On master it was removed only because a 3-read model that contains it exactly came first in storage order. With a4b2a33 that model's 6-read sibling absorbs it first, so it no longer removes the fragment. This depends on the artificial SIRV708 design and is accepted.

Performance baselines are unchanged. Mouse ONT R10 reduced_db (CPU 33119 s vs 20479 s baseline) and Mouse PB reduced_db (CPU 7209 s vs 5039 s) still exceed CPU and wall time, but master's own runs show the same: 32750 s for Mouse ONT R10 today and 7506 s for Mouse PB on 10-30. RSS is within tolerance.

Known limitation, left for separate work: a read whose short last exon is soft-clipped ends at the preceding donor and gets an injected tail there with --fl_data. A position-based guard for this was tested and rejected because it cost novel recall.

Testing

  • New unit tests in test_known_isoform_collisions.py, test_nonfl_known_isoforms.py and test_read_strand.py, with additions to test_external_polya.py, test_intron_graph_refine.py and test_intron_path_threading.py. The poly-A site tests check that a read marked before assignment, or assigned again after its site moved, gets the same assignment as when its tail is found in the sequence.
  • The branch is rebased on master dfb2a40. The commit SHAs above are the rebased ones. The CI runs below ran on the same commits before the rebase. The full unit test suite passes after the rebase: 2002 passed, 4 skipped.
  • cd4f6d2: run_pipeline.py was run locally on the four configs that use --fl_data (Mouse.PB_simulated.FL.reduced_db, SIRVs.Set4.R10.opts2, Mouse.ONT_simulated.TSS, Human.ONT_simulated.TSS), with the results above. The FL run's CPU and wall time went over the baseline while the machine load was about 40. Performance baselines are unchanged. No CI runs have been dispatched for this commit yet.
  • Local runs of the CI configs above, plus regional repros: the Fcgr3 NAGNAG pair, now reported with both isoforms, and Slc39a7, now present with --fl_data.
  • CI runs on a258838, dispatched on this branch: Human.ONT_simulated.polyA_1, Mouse.ONT_simulated.polyA, SIRVs.Set4.real, Mouse.ONT_simulated.R10.no_gtf, Human.PB_simulated.uniform_cov (passed).
  • CI runs on 7a86112: Unit tests, Short runs, Mouse.ONT_simulated.TSS and Human.ONT_simulated.TSS passed. The other ten workflows (SIRVs.Set4.options, SIRVs.Set4.real, SIRVs.Set4.simulated_R10, Human.ONT_simulated.polyA_1, Mouse.ONT_simulated.polyA, Mouse.ONT_simulated.R10.no_gtf, Mouse.ONT_simulated.R10.reduced_db, Mouse.ONT_simulated.R9.reduced_db, Mouse.ONT_simulated.R9.trunc.reduced_db, Mouse.PB_simulated.reduced_db) failed only on the drift re-baselined in 770e0c4, plus the performance overrun shared with master. Against the new baselines, every accuracy check of these runs is within tolerance. They have not been re-run on 770e0c4.
  • Not run on this branch: SIRVs.Set4.simulated_R9, SIRVs.Set4.simulated_perfect, SIRVs.simulated_perfect_polya, SIRVs.Set4.real.noGTF and SIRVs.Set4.R10.ubam.
  • The new Mouse.PB_simulated.FL.reduced_db workflow can't be dispatched until it is on master. Its config was run locally with run_pipeline.py on the code of 7a86112, and all accuracy checks passed. CPU and wall time were within the baseline in an earlier run with the same results; the last run exceeded them while other heavy jobs shared the machine. The workflow file differs from Mouse.PB_simulated.reduced_db.yml only in its name, cron day and run name.

🤖 Generated with Claude Code

@andrewprzh

Copy link
Copy Markdown
Collaborator Author

Review follow-up: 9682e09 and 1a27fdd

A code review of caae97c found three problems. 9682e09 fixes all of them.

  1. A poly-A site set after assignment did not reclassify the read. With --fl_data the site is set only after the read is assigned, because its position comes from the corrected exons and its side from the read strand. Before this fix the site only added verification events. So two isoforms with the same intron chain and different 3' ends stayed ambiguous (or became inconsistent_ambiguous). A read ending 100 bp before the isoform end also stayed a full splice match while its type said inconsistent.
    Now LongReadAssigner.reassign_with_polya_site follows the path a tail found in the read sequence takes in match_consistent:
    • each isoform match on the read strand is checked with PolyAVerifier.verify_read_ends;
    • matches the site contradicts are dropped when others agree with it;
    • if it contradicts all of them, or the read was already inconsistent, the read goes through match_inconsistent again.
      Afterwards the inconsistent-MAPQ filter, the aligned-tail mark, exon correction and the antisense check run again. gene_assignment_type is recomputed too (ReadAssignment.set_assignment). Events the alignment processor adds after assignment (aligned_polya_tail, antisense) are left out of the classification. In unit tests the result equals the assignment the same read gets with the tail detected at that position.
  2. The same applies to every --polya_trimmed mode, and to external tails without a side. Each of them now reclassifies the read in the same way.
  3. A trusted 5' end inside an exon could be threaded to a vertex past the intron entering the exon. For example, in SIRVs a start at 10561 was threaded to a read-start vertex at 8999. Now such an end takes only a vertex inside its exon (IntronGraph.five_prime_vertex_limit, with the same bound as is_start_internal/is_end_internal). If there is none, the read gets no start (end) vertex, as on master. Only annotated or well-supported introns bound the exon, the same filter as in is_trusted_five_prime. Otherwise a 1-read noise intron that ends at SIRV108's TSS removes SIRV108 from SIRVs opts2. Trusted ends inside such an exon also no longer add to the vertex of reads starting outside it.

Smaller items: type hints for the new code, and a log note when --fl_data or --polya_trimmed is set: the poly-A percentage then counts the sites these options set, not only tails found in the sequences.

Local validation, caae97c → 1a27fdd (gffcompare recall/precision, %)

Run Change
Mouse.PB_simulated.FL.reduced_db full 93.0/98.8 → 93.1/98.8; known 97.9/99.6 → 98.0/99.6; novel 67.4/93.1 → 67.3/93.1; missed known transcripts in quantification 5595 → 5514
Same FL reads with --polya_trimmed all full 93.8/98.9 → 93.9/98.9; known 97.6/99.6 → 97.8/99.5; novel unchanged
SIRVs.Set4.R10.opts2 full 85.5/98.3 → 88.4/96.8; novel 61.5/94.1 → 69.2/90.0: SIRV108 and SIRV608 found; SIRV503 fragment (also reported on master)
Mouse / Human ONT TSS unchanged within 0.1
Mouse PB simulation, chr1+17, --fl_data, full annotation known isoforms with ≥20 reads missing from models: 11 → 9

CPU time on the FL test: about +5% in a run on a quiet machine. That is within the baseline, which is unchanged.

CI

  • 1a27fdd updates the baselines of the four CI tests that use --fl_data and moved:

    • Mouse ONT TSS and Human ONT TSS: transcript metrics, all up, coming from the earlier commits of this branch;
    • the FL test: transcript and quantification metrics.

    SIRVs.Set4.R10.opts2 stays within tolerance. No CI test uses --polya_trimmed.

  • Dispatched on 1a27fdd: Mouse.ONT_simulated.TSS, Human.ONT_simulated.TSS, SIRVs.Set4.options (opts1–3).

  • The workflows that failed on caae97c (Human ONT polyA_1, Mouse ONT polyA, Mouse ONT R10 no_gtf, SIRVs Set4 real R10/R9) don't use --fl_data or --polya_trimmed, so 9682e09 doesn't change them. All of their failures are baseline drift, listed in the description. They are not re-baselined yet.

@andrewprzh

Copy link
Copy Markdown
Collaborator Author

CI on 1a27fdd:

  • Unit tests, Short runs, Mouse.ONT_simulated.TSS and Human.ONT_simulated.TSS: passed, the TSS tests against the updated baselines.

  • SIRVs.Set4.options: opts2 (--fl_data) and opts3 passed. opts1 fails only on baseline drift from earlier commits of this branch, and doesn't use --fl_data:

    • known recall 98.7 → 100: SIRV604 is no longer lost to the intron-chain collision;
    • novel precision 100 → 91.7: one more false novel, transcript74.SIRV7.nnic, a contained two-exon fragment of SIRV708 that comes from 6fb89b7. It is the same transcript as in SIRVs Set4 real R10.

    Novel recall is unchanged. opts1 is not re-baselined yet.

@andrewprzh

Copy link
Copy Markdown
Collaborator Author

5c9fc3b re-baselines the accuracy metrics that moved out of tolerance in the CI runs on 1a27fdd. The updated tests are SIRVs real R10/R9/opts1, SIRVs simulated R10 (all three configs), Human and Mouse ONT polyA, and Mouse ONT R9/R9 trunc/R10/no_gtf and Mouse PB simulations. The per-test changes are in the description.

  • The extra false novel in SIRVs real R10 and opts1 is a SIRV708 internal-priming fragment. On master it was hidden only by storage order in detect_similar_isoforms. This is accepted as a SIRV-specific case.
  • Performance baselines are unchanged. Mouse ONT R10 reduced_db and Mouse PB reduced_db still exceed CPU and wall time, and master's own runs show the same numbers.
  • The workflows have not been re-run on 5c9fc3b.

🤖 Generated with Claude Code

@andrewprzh
andrewprzh force-pushed the fl_data_known_isoforms branch from 8f159fa to cd4f6d2 Compare November 18, 2026 23:06

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant