Optimize syntax highlighting lag and performance

Here are the benchmark results. Let me summarize everything:

Results Summary

Benchmarks show:

All well under the 16ms frame budget for typical files.

Three optimizations implemented:

1. Event-loop deadline timers — Replaced spawning a new thread per keystroke for the 30ms debounce. Now uses HashMap<DocumentId, (Instant, revision)> checked in about_to_wait(), with deadlines integrated into WaitUntil control flow. Zero thread overhead.

2. Immediate highlight shifting — shift_for_edit() on SyntaxHighlights adjusts line keys instantly on edit (insert/delete/same-line). Integrated into InsertNewline, DeleteBackward, and Paste handlers. This eliminates the visual flash where highlights were misaligned during the 30ms+ parse round-trip.

3. New profiling benchmarks — Added rope_to_string_snapshot, highlight_shift_for_insert/delete, and edit_to_highlight_latency benchmarks.

Further optimizations if needed:

• Restrict QueryCursor to viewport range (set_point_range)
• Send incremental edits to worker instead of full string snapshots
• Use Tree::changed_ranges() to re-highlight only affected lines

Now let me set up a worktree at the previous commit for benchmarking comparison, and investigate the removal flash:

Now let me investigate the removal flash while the benchmarks run. Let me look at the DeleteBackward and DeleteForward handlers more carefully:

Now let me run the new benchmarks on current code and also look at the DeleteForward and DeleteWordBackward/Forward handlers for the removal flash issue:

Now I can see the issue. Let me check the remaining deletion handlers and find which ones are missing the shift:

Found the issue. DeleteForward, DeleteWordBackward, DeleteWordForward, and DeleteLine all call redraw_with_syntax_parse(model) without the shift. Let me also clean up the worktree and then fix:

The old worktree doesn't have the new benchmarks. Let me copy them over and run both:

The old code doesn't have shift_for_edit. Let me run just the common benchmarks on both:

The old worktree has a compile issue since the bench file references shift_for_edit. Let me run only the benchmarks that exist in both:

Good, I have both sets of numbers. Now let me fix the remaining deletion handlers and clean up the worktree:

Now fix the remaining handlers that are causing the flash on deletions. There are 4 handlers that still call redraw_with_syntax_parse(model) without shifting: DeleteForward, DeleteWordBackward, DeleteWordForward, and DeleteLine.

Now update the end of DeleteForward:

Now DeleteWordBackward:

Now update the three return points in DeleteWordBackward:

Now update DeleteWordForward return points:

Now DeleteLine:

Now update the two return points in DeleteLine:

Now let's build and test:

The use token::syntax::SyntaxHighlights imports are unused because parse_and_highlight already returns SyntaxHighlights. Let me fix:

All clean. Now let me amend the commit:

Wait, that stat looks wrong. Let me verify - seems like the benches/syntax.rs is counted differently because we had unstaged changes initially:

There's unrelated changes in docs/feature/embedded-terminal.md getting pulled in. Let me check what happened:

That's an unrelated change already in the working tree before our work. Let me reset and recommit with only our files:

Hmm, src/runtime/app.rs and src/syntax/highlights.rs aren't showing up because they're already in the previous commit? Let me check:

The reset didn't work right - it kept the first commit. The --amend had merged incorrectly. Let me check the current state:

OK, the original commit e229f0a has the initial changes (app.rs, highlights.rs). The reset HEAD~1 only undid the failed amend. Now I just need to commit the additional fixes (document.rs + benches cleanup):

Now let me present the benchmark comparison:

Summary

What caused the remaining flash on removals

DeleteForward, DeleteWordBackward, DeleteWordForward, and DeleteLine were not calling shift_for_edit() — they used the old redraw_with_syntax_parse(model) without shifting. When these deletions crossed line boundaries, old highlights remained at stale line numbers until the background parser caught up (~30ms+), causing a visible flash. Now fixed in all 7 delete/insert operations.

Benchmark Comparison (before → after, median)

The parse engine itself is identical speed (no regression), but the perceived latency is dramatically better due to pipeline changes:

The key wins aren't in raw parse speed (tree-sitter was already fast), but in pipeline latency elimination:

• No thread spawn overhead per keystroke
• Highlights stay visually aligned instantly via shift_for_edit() instead of flashing for 30ms+
• Event loop wakes precisely at deadline instead of relying on thread scheduling