Comparing changes

…4100) Several places decided where a word ends, and none of them used the definition bash uses. `strip_word_prefix` accepted exactly one byte as the boundary between a prefix and the command behind it: an ASCII space. Bash separates words on space, tab and newline alike, and the rule patterns match `\s+`, so a tab-separated command failed to rewrite: ls\t-la no rewrite cargo\tbuild no rewrite noglob\tls -la no rewrite The first two also classified as Supported, so `rtk discover` counted coverage the hook never delivered — the two sides disagreeing about what separates a command from its prefix. The third did not: no rule names `noglob`, so `classify_command` calls it Unsupported either way. Its rewrite is repaired here too, but nothing was over-reported for it. `strip_absolute_path` split on a literal space, so `cat\t/etc/passwd` normalised to `passwd`. `extract_base_command` and `normalize_php_tool_command_with_dirs` used `char::is_whitespace`, which is Unicode and counts NBSP — the lexer rejects exactly that in `is_word_boundary_whitespace`, whose comment says why. All of them now use it, so `cat\u{a0}/etc/passwd` stays one word as bash reads it, and the separator is carried through rather than replaced, so `noglob\tls -la` keeps its tab. The remaining `char::is_whitespace` in `join_redirect_suffix` tests whether a suffix is already separated rather than splitting a word, so it stays. Two table-wide invariants the `RULES` table has never had come with it. **Anchoring, both ends.** A pattern matches against a whole command line, so one that does not start with `^` fires on a path component, and one that ends in a bare alternation matches any command whose subcommand merely *starts* with a listed one — `git branchless` routes into `rtk git` (#4009). The 22 patterns in that state are #4014's and #3676's to fix and are named in a `PENDING` list, not exempted: an entry that no longer matches a rule fails the test, so the list can only shrink. **`rtk_cmd` does not identify a rule.** `classify_command` picks a rule by index and `rewrite_segment_inner` then looks one up again by `rtk_cmd` with `find`, whose comment claims the value is unique. It is not — `rtk php` and `rtk uv` are two rules each, so that lookup already returns a sibling. It is harmless only because every field it reads matches across each pair, and that is now asserted rather than assumed. Forcing uniqueness is the better end state but not a test-only change: the siblings are worded differently (`\b` against `(?:\s|$)`), so merging means choosing one terminator and re-checking every command each pattern currently claims. The mechanical fix is elsewhere anyway — `classify_command` already holds the matched rule's index and could pass it along instead of having the rewrite re-derive a rule from `rtk_cmd`. Left as a follow-up. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

…3683) Classification stopped at the first `|`, so `cargo test | grep FAILED` was read as `cargo test` and the `grep` the agent ran was never looked at. `rtk discover` cannot report an opportunity it does not see, and `rtk grep` is one of the two rules that are safe to rewrite at the end of a pipeline, so this hid real savings. Segmentation now runs through the whole chain. Seeing a command is not the same as being able to rewrite it, though, and the rest follows from that. Where the rewriter takes a command, the saving is there for the taking, and the main table still means what it says. Position decides who that is: `tail -1` becomes `rtk read` on its own, but in `cargo test 2>&1 | tail -1` the rewriter takes the producer and leaves the consumer alone, because RTK filters what `cargo` writes and rewriting the stage that reads it would change what the pipeline prints. In `cargo test | grep FAILED` it is the other way round. The question goes to the rewriter rather than to a second copy of `PipelineSafety`, and is asked once per line, since what happens to one command depends on the ones around it. Where the rewriter passes a command over, the saving is still real but nobody can collect it without changing the command. `cargo test` writes its output in full for a `grep` to throw most of away; `/usr/bin/git diff` is a command RTK has a rule for, written in a form the rewriter does not match. Both would be filtered if they were run differently, so both go in a table of their own — OPPORTUNITIES — rather than being folded in with the ones the hook takes by itself or dropped on the floor. Dropping them hides real savings, and hides the disagreements between classification and rewriting that cause them (#1053). One shape is excluded: a command reading another's output. It wrote what actually came back, `output_len` already measures that, and counting it would claim a saving on the one line `tail` printed on top of everything `cargo` wrote. Coverage has to ask the same question. A `hook_decisions` row is one decision for the whole tool call, and the estimate asks about a command on its own, so neither can tell that the rewriter passed a command over for where it sits. `cat f.log` becomes `rtk read` alone but not in `cat f.log | grep ERROR`, and without that check the producer of every such line read as adoption RTK never had. A line's measured output belongs to the command that wrote it, which is the last one. An earlier command either fed a pipe, and its output was consumed rather than returned, or shares the number with its siblings; either way the per-category average is the honest estimate for it, and handing the measured number to every part multiplied the total by the number of parts. The `RTK_DISABLED=` warning moves from `rewrite_segment_inner` up to `rewrite_command`. Every live path — the hook, `rtk rewrite` — comes through there, while `rewrite_command_precompiled` also serves `rtk discover`, which asks about months of history and printed the warning once per bypassed command into the report. It is advice for whoever is running a command. It reads the line through `split_for_permissions`, so a command position is a command position and `echo "a<newline>RTK_DISABLED=1 b"` stays quiet. `stop_at_pipe` goes with them. Nothing stops at a pipe any more, and a knob that still offers to would only invite this back. Measured on 24 sessions of this project's transcripts, before and after: before after scanned 3013 4642 +54% already using RTK 430 (14.3%) 482 (10.4%) missed savings 942 / ~184.0K tok 648 / ~75.2K tok opportunities — 566 / ~91.1K tok Every number moves, and each for a reason. More commands are seen, so the denominator grows by the pipeline stages RTK has no rule for, and the adoption rate falls against it. Fewer are reported as missed, because a command the rewriter passes over is now an opportunity behind a change the person has to make, counted in its own table rather than promised as something the hook would do. The scanning loop comes out of `run()`, which reads a real session directory and opens a real tracking database and so could not be reached from a test. Counting moves to `Tally::add`, which takes one command and a context, and the decision inside it to `disposition`, which takes the four facts and returns what the report does with the command. All sixteen combinations are written out, and the scan is now driven directly from synthetic commands. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Three consumers read the same command line — the permission gate, the report's classification, and the rewrite — and they disagreed about what a command was. Each disagreement was a bug already happening rather than one waiting to. `(git status; cargo build)` was the clearest. The gate split it, the other two did not, so the rewrite glued the bracket to the text beside it: `(git status` matched no rule while `cargo build)` matched, and one command of a pair was filtered while the other silently was not. `(git status)` was rewritten not at all. A subshell runs the commands it wraps, so `(` and `)` now end a command for classification and for the rewrite, as they always did for the gate. Background `&` was the same story and gets the same answer. The two places that must not follow are now stated rather than assumed: A `$( )` is not a subshell for this purpose. Its body runs, so the gate descends and holds it to the deny rules; the other two stay out, because the text around a substitution is not a command of its own — descending invents a `git log $` nobody ran — and what it captures is a string the outer command is built from, so filtering it would change that string rather than what reaches anyone. `Policy::descend_into_substitution` carries that, and the emitter tracks the same depth. Staying out means every separator, not only the brackets. `&&`, `||`, `;` and `|` inside a substitution separate commands whose output becomes one word of the command being built, so none of them ends a command out here — otherwise `echo $(git status && git log)` prints a line nobody wrote. Brackets nest, so a `(` inside a substitution counts up as well, or its `)` closes the substitution one bracket early and the real closing `)` reads as a boundary out here. `<( )` and `>( )` are substitutions too. They hand the outer command a file rather than text, but the inner command serves the outer one just the same, so none of the three is a place to end a command or to filter output somebody else is about to parse. Reading only `$` left `diff <(a) <(b)` classified as four commands — two of them the fragments `diff <` and `<` — which `rtk session` counts without the unattestable-construct check that shields the hook, so its adoption figure was being computed from commands that never ran. A `{ }` is not a subshell either. `{` opens a group only at a command position; anywhere else it is brace expansion, and `ls {a,b}.txt` is one command. The gate copes by stripping the bracket before matching, so nothing hides behind one. Nor is `name()` one. It names a function, so the word before it runs nothing and `rtk ls() {` does not parse: classification and the rewrite skip the word and the empty pair, and read the body as usual. A `(` glued to a word, as in `a[(b)]`, `x=(a b)` or `@(a|b)`, ends no command for them either, since it runs nothing of its own. A `case` pattern is the one place a `(` opens nothing. `case x in (ls) echo 1;; esac` is the same statement as `case x in ls) echo 1;; esac`, so reading that bracket as a subshell makes the pattern a command position: the rewrite turns a one-word pattern into `(rtk ls)`, and bash refuses the line exactly as it refuses a `;;` split into `; ;`. `CaseTracker` follows `case` through its `in`, each arm terminator and its `esac`, and both segmenters consult it before reading a `(` as a boundary. `case` counts as the keyword only in command position, so `echo case` stays a word. The `)` still ends the pattern, so an arm's own commands are rewritten whether or not the pattern was written with brackets. `mod segmenter_consistency` becomes `segmenter_agreement`. It used to pin three answers to one question and call the disagreement deliberate; it now asserts the agreement where it exists and names the reason for each place it does not. A divergence nobody can state a reason for is a bug. `time (cargo build)` becomes `time (rtk cargo build)`. The wrapper never could peel a `(`, but it no longer has to — the subshell is a boundary, so the command inside is rewritten where it stands and the wrapper keeps wrapping the subshell. A pipeline's swap runs a line in a child `sh`, which may read a subshell or a compound command otherwise or not at all, and its checks read words rather than that grammar. Now that the rewrite reaches into them, a line holding one takes the ordinary rewrite rather than the swap. The differential harness keeps both halves. The gate's is unchanged — a gate that segments differently is a gate with different holes — and the legacy walker it compares against is untouched. Classify can no longer be compared to that walker, which describes a policy this commit deliberately changed, so it is compared to the gate instead: on any input without a newline, a redirect or a substitution the two hold the same policy and must agree exactly, which anchors classify to the frozen walker through the gate. The corpus also gains the substitution shapes it never had, including the nested and unterminated ones. A span invariant sits alongside it — every segment is text that was really there, at the span it claims, in order and without overlap. That is worth having against arithmetic slips but it is not a segmentation contract: gaps between segments are legal, since that is where the separators live, so a boundary read in the wrong place passes it. Its comment says so rather than implying otherwise. Also corrects `.claude/rules/rust-patterns.md`, which required argument parsing to go through `tokenize_with_options`. There is no such function; it is `tokenize_grammar` (`src/core/arg_tokenizer.rs:296`). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

…and fix what it found (#3262) Every test of the rewriter compares it against an expectation somebody wrote down. A mistake in the author's model of shell grammar therefore sits on both sides of the comparison and agrees with itself — which is how a rewrite that reached inside `$( )` passed a full suite, and how a bracket counted on one side but not the other passed it again. This asks bash instead. Recording stubs sit ahead of everything on `PATH`, the original command and its rewrite are each run, and what actually executed has to match: which programs, with which arguments, in which directory. The `rtk` stub runs the command it wraps and marks its output, the way the real one filters, so a rewrite at a command position changes nothing that is compared while one that reaches into a substitution changes the text the outer command was built from. Three things had to be got right, none of which reading the code would have told me: - `echo` is a shell builtin, so a stub by that name is never reached. The outer command of a substitution has to be one bash looks up on `PATH`. - The stages of a pipeline run concurrently and reach the log in whatever order they are scheduled, so the comparison is over a set. - `>( )` hands over the writing end of a pipe. It opens but never delivers, so reading it waits for a writer that is waiting for this process to finish. The oracle is held to detecting as well as passing. Alongside the equivalence cases are rewrites wrong in ways this repository has actually shipped — reaching into a substitution, into a process substitution, into a quoted assignment, and turning one command into another — each of which it has to object to. And a command RTK merely declined to rewrite runs identically, so it has to stay quiet about that, or "everything differs" would satisfy the rest. It found #3262 on its first run against the real rewriter, from a case nobody had written an expectation for. `D='# shellcheck disable=SC2034'; echo hi` came back as `D='# rtk shellcheck disable=SC2034'; echo hi`: RTK edited inside a quoted literal, so the variable held text nobody wrote. The env prefix was matched with a pattern whose value alternation is `(?:"…"|'…'|[^\s]*)` followed by `\s+`. A quoted value that is not followed by a blank fails that `\s+`, the alternation backtracks to the unquoted branch, and `[^\s]*` takes the fragment up to the first space — `D='# ` — leaving `shellcheck disable=SC2034'` to be read as the command. Three shapes share the one cause, and only the first was reported: D='# shellcheck disable=SC2034'; echo hi -> prefix D='# command shellcheck … D="x y"; echo hi -> prefix D="x command y"… D='no trailing space' -> prefix D='no command trailing… A quoted value is one word however many blanks it holds, so the prefix is now read by scanning words with quote state rather than by matching a pattern: a backslash escapes inside double quotes and is literal inside single ones, and a word ends at the first blank that is not inside either. A word is an assignment on bash's own terms: a name of letters, digits and `_` not starting with a digit, then `=` or `+=`, in any case. The pattern took uppercase names only, so `foo=bar git status` was never rewritten, and `d='# shellcheck …'` escaped the quoting fix above only by not being recognised at all. `1a=b`, `foo-bar=x` and `"foo"=x` are commands to bash and stay commands here. `sudo` is still never stripped (#146). Four call sites read that prefix and each carried its own copy of the pattern, which is why they had to change together or disagree. They now go through one `split_env_prefix`, so there is one place to be right. It also retires `strip_disabled_prefix`'s trick of recovering the prefix length by subtracting the stripped string's length from the original's, which worked only because the pattern happened to remove a prefix. The permission gate reads the command behind each assignment in turn, so a deny or ask rule can name any layer of a prefix. It does so through `split_env_word`, the same word reading one word at a time, so an assignment that ends its statement is one word there too: `MSG="chore: rm -rf build dir" && git commit -m "$MSG"` is no longer denied by `Bash(rm -rf *)`. Unix only, and skipped where there is no bash. What a process substitution carries can only be compared where a bounded read is available, and `timeout` is coreutils, which macOS does not ship — so the oracle checks for it and drops the two `<( )`/`>( )` cases when it is missing, rather than comparing two empty readings and calling them equal. Every trace is also required to be non-empty, because a case whose programs are all unstubbed records nothing on either side and would otherwise pass while testing nothing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

…aged `Total: 681 commands -> ~108.3K tokens saveable` reads as one number of one kind. It is not. A figure comes either from the output size the transcript recorded for that tool call, or from a flat per-category constant used when there is nothing to measure — and the two look identical in the table. The split is not incidental. `output_len` measures what the last command of a line wrote, so a command feeding a pipe is never the one it describes: every opportunity that is an opportunity *because* it feeds a pipe is averaged, by construction. Each table now says how much of it was averaged, next to its total, the way the report already discloses estimated coverage at the top. On this project's history that reads: Total: 852 commands -> ~98.6K tokens saveable 769 of 852 sized by per-category average, the rest measured Total: 681 commands -> ~108.3K tokens saveable by changing how they are run 524 of 681 sized by per-category average, the rest measured The first line is the surprise: the missed-savings table is nine tenths averaged too, not just the opportunities one. Whether the per-category constants deserve that weight is a separate question, and a report that hides which numbers rest on them cannot even raise it. `strip_disabled_prefix` becomes `split_env_prefix`. It splits off any env prefix; that one caller then looks for `RTK_DISABLED=` in what came back is the caller's business, and the old name described the caller rather than the function. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Three things were written down more than once, and each duplicate cost a bug. **Where a substitution begins and ends** was counted separately by the segmenter and by the emitter. They walk the same tokens and have to reach the same answer, and twice they did not: once over an operator inside a substitution, which the segmenter knew was not a boundary and the emitter did not, so `echo $(git status && git log)` was rewritten into the string it captures; once over a bracket nested in a substitution, which the emitter counted and the segmenter did not, so the closing `)` went missing from the segment. Both were of the shape "the other copy learned this and I did not". `SubstitutionDepth` now holds that counting, in `lexer.rs` with the tokens and the other low-level scanning, and `registry.rs` uses it. The emitter's four guarded match arms become one skip above the match, because "nothing inside a substitution is a boundary and nothing inside one is rewritten" is one statement rather than four. **Quoting** was tracked by `QuoteScan` for most callers and hand-rolled by two others: `contains_substitution` with its own `in_single`/`in_double` pair, and `word_end_from` with its own `Option<u8>`. Both re-derived the same rules and either could drift from the shared one. They now read `QuoteScan`. `QuoteScan` itself used to step over an escaped byte without reporting it. That suits the six callers asking about syntax — `\(` is not a bracket, `\#` is not a comment, `\'` opens nothing — and it silently failed the one asking where words are, for which an escaped byte is text like any other. A word made only of escapes, such as a lone `\'`, went unreported and vanished from the split, taking the next word with it: `FOO=bar \' BAZ=1 git status` read as an env prefix running through `BAZ=1`, making a real assignment part of the prefix rather than a command's environment. Rather than have the caller rebuild what the scan withheld, the scan now reports every byte and marks which are escaped, and `significant()` is the word the six say. The choice sits at each call site instead of inside the iterator, where it was a default nobody could see. `word_spans` is the helper whose absence made a second scanner look necessary: `coalesce_words` merges *adjacent* tokens, which suits argv and not the question of where a word ends. **The report's two savings tables** had the same columns, widths, totals line and estimate note, written out twice and differing only in heading and in what their total means. `append_savings_table` takes those two differences as arguments. A column added to one can no longer go missing from the other, and the four `total_*` methods that existed to feed the duplicated footers go with them, since the shared writer sums what it is given. The execution oracle is hardened alongside. It passed on any command whose programs are all unstubbed — both traces empty, empty equals empty, success reported having run nothing. Shown with `head -5 f` against `rtk read f --head-lines 999`, a deliberately wrong line count, which passed. A trace is now required to be non-empty. That also names what the oracle cannot do: `head -5 f` becomes a different program with reordered operands, and "the same programs ran" is not the test for that. And `timeout` is coreutils, which macOS does not ship while CI runs there — absent, the stub's bounded read of a `/dev/fd` failed and recorded an empty value on both sides, so the process-substitution cases kept passing while comparing nothing. Availability is decided once at setup and those cases are not claimed when it is missing. No behaviour change beyond the escaped-word fix: the rendered report is the same report, and the differential harness over 13,020 commands agrees with the frozen walker as before. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

The quote-aware lexer in discover/lexer.rs holds no rewrite rule and no classification logic, and it is already used outside discover: by the permission gate (hooks/permissions.rs), the hook entry points (hooks/mod.rs, hooks/decision.rs, hooks/hook_cmd.rs) and `rtk proxy` (main.rs). Moving it to core/cmdline/lexer.rs puts shared shell knowledge in core, where every consumer can depend on it without depending on discover. No behaviour or visibility change: import paths and doc references follow the file, and its documentation moves from discover/README.md to core/README.md, with every consumer listed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

… kinds `tokenize` returns `Token { kind, value, offset }`, built directly by the tokenizer: every value is a slice of the input, and the tokens tile it, so each byte belongs to exactly one token. Two kinds carry what lies between words: `Sep`, the exact run of spaces and tabs, and `Newline`, the `\n` it sits on. The lexer always emits both; the segmenter's `NewlinePolicy` says whether a newline ends a segment or separates words like a `Sep`. `tokenize_at` lexes a piece of a larger input with offsets into that input, which is how the segmenter reads the pieces a policy cuts. A `Word` is a run of adjacent tokens that are not blanks, with its span, and `words(input, tokens)` is the one function that finds them: the env-prefix split, golangci's flag parsing and `shell_split` all read it. `QuoteScan` is the one flat quote-state machine, walked by the tokenizer, by `resolve_word_text` and by every other quote-aware scan in the lexer. A command ends where its last word does, and an escaped or quoted blank is part of that word. The rewrite path in `discover/registry.rs` therefore takes the ends of a line, a segment, a pipeline stage, a wrapper prefix, an env prefix and the command in front of trailing redirects from the tokens: `content_span` gives a token range's span without the blanks around it, `tokenize_trimmed` lexes a command and returns it trimmed that way with its tokens, and `tokens_until` gives the tokens of a prefix. A segment is lexed once, and its tokens are handed to the env-prefix split, the wrapper and redirect splits and classification. A configured `transparent_prefixes` entry is matched as text and can end inside a quote; the command behind it is lexed on its own, and the blanks between where that lex ends it and where the segment's lex did are emitted as written. `head`, `tail` and `cat` are recognised by their first word, and `cat`'s options are classified through `arg_tokenizer`. `core/cmdline/rtk.rs` answers whether a command runs rtk, from its words: `rtk_invocation` reads the first word with its quotes and escapes removed, as bash does before it looks the command up, and says whether the second, read the same way, is `proxy`. The rewriter returns such a command as it is unless its tokens hold an unquoted `&&`, `||`, `;`, `|` or `&` that joins another command to it, classification ignores it, and `rtk discover` and `rtk session` count it as adoption unless it is `rtk proxy`. The discover report takes a command's first two words from the lexer's words too. Word boundaries are bash's `$IFS` wherever a command line is read. The lexer's `is_ifs`, `split_ifs` and `trim_ifs*` helpers serve the segmenter, classification and the permission gate's pattern normalisation. The rule patterns in `discover/rules.rs` and the built-in filters' `match_command` spell a separator `[ \t\n]` and end a command word with `(?:[ \t\n]|$)`; a rule also ends one at a metacharacter glued to it, because classification reads a segment with its redirects kept. Tests reject `\b`, `\s` and `\S` in a rule pattern and in a built-in filter. Project and user-global filters are matched as written, and the filter docs and `rtk init` templates show the `[ \t\n]` form. A subcommand capture that needs one spacing joins its words with single spaces through `squeeze_blanks`, and so does the permission gate before it compares a command with a pattern. The permission gate cuts a line at every unquoted `\r` and lexes each piece on its own, as it does at a `\n`, so that no command hides behind one. `core/cmdline/source.rs` reads a text as bash's parser does before it has words, and says which `#` opens a comment and which `\<newline>` joins two lines. One flat quote state cannot say: bash reads quoting afresh in a `$( )` inside double quotes, a `#` is text in `${x#y}` and `$(( 16#ff ))` and a comment in `$(ls # c<newline>)`, and a `)` in that comment closes nothing. So it follows the nesting. Commands stand in `$( )`, `( )`, `<( )`, a backquote and on a heredoc's line, text in `${ }`, `$(( ))`, `$[ ]` and a heredoc's body, and `(( ))` is arithmetic if it closes so. A text with no `#` and no `\<newline>` is not read. The tokenizer emits a comment as one `Comment` token, from its `#` through the end of its line, and the scan restarts after it, so a quote in comment text opens nothing and a `\` at its end escapes nothing. Every reader takes comments from there: segmentation closes a segment at one and cuts at a bare `\r` only outside one, a rewrite gives a command's comment back, and the multi-line rewrite reads `without_comments` and the unattestable check `placed_code`. A line continuation is joined where bash joins it, by the lexer's `join_continuations`, the one reader of continuations. The rewrite reads a line through it, and the permission gate and the hook's own substitution check read the line through it before anything else, so what they check is the text the rewrite emits; the gate and the rewrite do the same for the script of a single-quoted `-c` wrapper, which the shell it starts joins. It replaces `collapse_line_continuations` and the two regexes behind it in `discover/registry.rs`, and the gate's own `\<newline>` removal. A text is too complex where something in it never closes, past 16 levels of nesting or a read that costs too much, or where bash reads it two ways: `a[1 # 2]=b` is a subscript where a command may start and a comment elsewhere, `a>#b` starts a comment in bash and a word in zsh, a quote in `"${x:-'}'}"` holds what the operator decides about, and in `$(time -p case a in a) b;; esac)` a `case` that may be text decides which `)` closes what. So is a heredoc delimiter holding an expansion, `<<$(a)`. `join_continuations` then returns a `TooComplex`, and nothing is rewritten. If a deny rule names a command on what the reader made of the line, on the same with its comments read as commands, or on one flat quote state's reading, the gate says so as it would anywhere; otherwise its verdict is `TooComplex`, whatever an allow rule names. The Claude hook asks for that line as it stands, saying why, unless the session's permission mode never prompts (`bypassPermissions`, `dontAsk`, `auto`), and every other caller leaves it to its host, since it cannot show an ask without a rewrite. Settings with no deny entry or ask rule, and either `Bash(*)` or `Bash` allowed or no allow rule at all, leave nothing to ask about: the host runs the line or asks by itself, as it does without RTK, so such a line stays with it too. A script a shell wrapper holds is read the same way. A line whose every command opts out on all those readings, through `RTK_DISABLED` or `exclude_commands`, stays with its host, and the audit log records the rest as `skip:too_complex`. On a line the reader does read, rules hold for what it calls commands. A comment stays a token that the rewrite gives back, and the permission checks ignore it. A rule is read through `join_continuations` too, so one spelled across a continuation names what bash joins it into. `rtk discover` reads a line as the hook does, so a `\<newline>` is no word in its report. Behaviour, as the code implements it: - Words split only on space, tab and newline, as in bash. A `\r` is a word byte, so `git status\r` keeps `\r` in its last word and a CRLF line is not rewritten: `git status\r\ngit log` rewrites the second line only. - A vertical tab, form feed, non-breaking space or carriage return next to a word is part of that word: `\x0b2>&1` is the command `\x0b2` with a `>&1` redirect, and `git status\u{a0}` runs git with the subcommand `status\u{a0}`. - For rules and built-in filters alike, a command word ends at space, tab, newline or the end of the line. `helm-docs` is not `helm`, `make\r` is not `make`, `dotnet build-server` is not `dotnet build`, and `mvn install:install-file` is not the `install` phase; `g++ -c x.cpp` takes the gcc filter. The mvn and mvnd rules list `test-compile`. The command word `exclude_commands` names ends there too, so excluding `mytool` leaves `mytool\x0bx` alone. - An escaped trailing blank stays in its word, and so does a trailing blank inside a quote left open: `cat f\ ` becomes `rtk read f\ `, and `git log | head\ ` is not rewritten, because its last stage runs the program `head ` rather than `head`. Those blanks are emitted as written when a configured prefix ends inside the quote: with the prefix `x 'a`, `x 'a ls -la & ` becomes `x 'a rtk ls -la & `, and with the prefix `sh -c "`, `sh -c " rtk git status ` is left as it is. - `head`, `tail`, `cat` and `rtk` followed by a tab are handled as when followed by a space: `tail\t-n 5 f` becomes `rtk read f --tail-lines 5`, `cat\t-A f` is not rewritten, and `rtk\tgit status` is left as it is. A bare `head`, `tail` or `cat` has nothing to map or check and goes to the filters like any other command. - A first word runs rtk when it is `rtk` once its quotes and escapes are removed, as in bash: `'rtk' git status`, `"rtk" git status` and `\rtk git status` run rtk, and `'rtk git' status` does not. - A line whose first word runs rtk is left as it is unless an unquoted `&&`, `||`, `;`, `|` or `&` joins another command to it, whatever blanks sit around the operator: `rtk git status\t&\tgit log` and `rtk git status &git log` become `rtk git status\t&\trtk git log` and `rtk git status &rtk git log`, while `rtk grep 'a|b' src` and `rtk git commit -m "a && b"` are left as they are. - `rtk discover` and `rtk session` count a command as already rtk when its first word runs rtk and its second is not `proxy`: `rtk\tgit status`, `rtk` alone and `\rtk git status` count, `rtk\tproxy git status` does not. The report files `git status` under the subcommand `status` and shows it as `git status`, and shows `grep 'a b' src` as `grep 'a b'`. - A `\` before a newline goes with the newline outside single quotes, and nothing else does: `git di\⏎ff` is `git diff`, `git commit -m 'a\⏎b'` keeps its message, the blanks of `printf "a \⏎ b"` stay in its word, and neither `\\⏎` nor `\␍⏎` is a continuation, so `\␍⏎git diff HEAD~1` rewrites its second line only. `bash -c 'git \⏎status'` is a wrapper the gate and the rewrite read inside, joined: it becomes `bash -c 'rtk git status'`. `ls "$\⏎(rm Y)"` substitutes a command, as bash joins it, and is decided about as `ls "$(rm Y)"` is. - A continuation is read where it stands: `"$(echo "a \⏎ b")"` joins it, `"$(echo 'a\⏎b')"` keeps it, and a comment holds it, except in a backquote, where bash takes every `\⏎` out before reading the text. - A line too complex to read asks through the Claude hook, and is not rewritten: `a[1 # 2]=b; ls`, `echo a>#b` and `echo "$['1\⏎+1']"` ask where a rule names some commands or holds some back. A deny rule that names a command on the line as far as it reads comes first: `echo "$['1\⏎+1']"; touch f` is denied by `touch:*`. - A quote in a comment opens nothing, a `\` at its end escapes nothing and the line after it is read as its own, and text in it is neither a command nor a substitution. `git status # don't⏎git diff` becomes `rtk git status # don't⏎rtk git diff`, `git status ${x} # note` becomes `rtk git status ${x} # note`, and a `#` in `${x#y}` or `$((16#ff))` is text. - A rule does not read comment text: with `more:*` denied and `echo a` allowed, `echo a # note; more` is allowed, where a comment's `;` used to split the line. - A `#` after an escaped or quoted `|`, `&`, `;`, `(` or `)` starts no comment: the line `echo a\;#b` is one command, not a command and a comment. - `cat` with arguments is rewritten only when it names a file and every option it receives is a bare `-n`, wherever the option sits and however it is quoted: `cat f -A`, `cat '-A' f`, `cat -- f` and `cat -n` are not rewritten, and `cat -` becomes `rtk read -`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

The rewrite lexes a command line once, into a `CompoundLex`, and reads everything else from those tokens: the heredoc refusal, whether a line already runs through rtk, each line of a multi-line block, each segment and pipeline stage, and the words and argv the decision checks (the program word, golangci's global flags, cat's options, a pipeline consumer's flags, grep's and rg's pattern files). A `Slice` is one command's run of tokens, from its first word to the end of its last, so an escaped or quoted blank that ends the last word stays in the command. The lexer exposes `resolve_words` to the crate, so a command resolves its own words, and keeps `resolve_word_text` to the `cmdline` module, where `rtk_invocation` reads a command's first words with it. A line runs through rtk when `rtk_invocation` says its words run rtk and no operator joins another command to it. A walk peels a segment's transparent layers left to right in one loop: an env run, a built-in wrapper, a process wrapper, a user prefix, in that order at each position, within the depth bound. The segment's words are found once, and each step reads them from the position the walk reached. Each `Layer` records at peel time what a later step reads: where it starts, and for a routable wrapper, the span of the command it wraps past any assignments, which is what its exclusion check matches. A prefix matched as text can end inside a token, as a user prefix `x 'a` ends inside the quote it opens; from there the walk reads a fresh lex of the rest of the command. Every other layer ends where a token starts, and a token starts where the lexer holds no quote, escape or word open, so the walk reads on through the segment's own tokens and words. When nothing is decided about the command under the layers, the #2768 fall-through retries each routable wrapper, innermost first, as a walk of its own from the wrapper's position and depth. A walk depends on nothing but that position and depth, so a pair already tried is skipped, which keeps the search over a `uv run uv run ... uv run` chain linear in its depth. A decision is one edit against the line, from `core::cmdline::edit`: `Insert { at, text }` puts `rtk ` in front of a command, and `Replace { span, text }` renames one. Every byte outside an edit is emitted as written. `apply_edits` takes the edits in position order, sizes its output from the line and the edit texts, and returns `None` for an overlapping, reversed, out-of-range or off-boundary span rather than panic. The `RTK_DISABLED` warning looks for a heredoc in the command as written. The rewrite output is byte-identical to its parent: `rtk rewrite` stdout, stderr and exit code, with and without a catch-all project filter, and the hook's verdicts. A quoted `sh|bash|zsh|fish -c` script is rewritten the same way, one wrapper deep: when the command a walk reaches is such a wrapper, its script is lexed and rewritten into edits of its own, and the wrapper's bytes around it are kept. A wrapper is the command wherever the walk reaches one, before any layer is peeled there, as the parent read it first: a user prefix that begins with its words (`sh -c "`) is not peeled off it. A wrapper is still reached at the prefix-depth limit, where no further layer is peeled. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

A tool's flag grammar is a `Grammar` in `core/arg_tokenizer.rs`: a `Dialect` and a list of `Flag` tables, built with `Grammar::new(dialect, tables)`. The dialect's single-dash axis decides how a flag is spelled. Under `SingleDash::Cluster` (`Dialect::Posix`) `-x` is a `Short` token and `--name` a `Long` one, so a flag is a getopt flag with a short and a long spelling, either optional: `Flag::short`, `Flag::long`, `Flag::pair`. Under `Atomic` and `AtomicAliasingLong` (`Dialect::Msbuild`, `CommonsCli`, `GoFlag`) every name is one whole word read as `Long`, so a flag is `Flag::atomic(&["c", "configuration"])`, one flag under every name it lists. `Grammar::new` refuses a flag its dialect cannot read, so a getopt flag in an atomic grammar, or the reverse, cannot sit there matching nothing. `.takes(spec)` records how each spelling takes its value; a spelling that is always the whole argument (a long one, any atomic name) records a solo-only spec as a plain value, and `.takes` panics on a solo-only spec for a flag with no short spelling. Every flag and grammar in the crate is a `const`, where either panic is a compile error. A new axis, `Abbreviation`, says whether a long flag may be typed as a prefix of its spelling. `Never`, the value of every preset, reads only the whole spelling. `UniquePrefix` reads GNU `getopt_long`'s rule: a whole spelling wins over a longer one it prefixes (`--all` is not `--almost-all`), a prefix of exactly one declared long spelling names that flag (`ls --sor time` sorts by time), and a prefix of two names neither. `tokenize_grammar(args, &GRAMMAR)` reads the dialect and every value rule from the grammar and looks each flag token up once: `Grammar::flag(kind, name)` finds the declared flag, a `Short` token matching only a short spelling and a `Long` token only a long one or an atomic name (folding ASCII case under `NameCase::Folded`, resolving a prefix under `UniquePrefix`), and the token keeps it in `Token::flag`, beside the grammar's tables. Code holding tokens asks the token (`token.is(&FLAG)`, `token.is_one_of(FLAGS)`, `token.value_spec()`), so a token asked several questions costs one search of the grammar. `is` and `is_one_of` compare flags by their spellings, which name one declaration per grammar, and every flag they are asked about must be declared in the grammar that read the token, exactly as declared: a debug build panics on an undeclared flag or on a copy that differs from the declaration in one field, and a release build answers `false`, since no token of that grammar can be that flag. The lookups by name (`has_flag`, `has_double_dash_flag`, `double_dash_flag_value(s)`) take the grammar that read the tokens and read its naming rules, a prefix the grammar resolved included, so the name they look for need not be declared. A predicate that asks for a flag only some grammars declare keeps its tokens to those grammars. `git log`/`show` and `git diff` read their arguments into `LogTokens` and `DiffTokens`, which only `tokenize_log_args` and `tokenize_diff_args` build: `git show`'s `-q`/`--quiet` check takes `LogTokens`, and `git diff`'s `--quiet` checks (`suppresses_diff_body`, `diff_wants_raw_shape`) take `DiffTokens`. Both deref to the token slice for the checks every git grammar answers alike. The predicates that take a grep or rg engine, and the git ones that take tokens of either `log` or `diff`, state which grammar read their tokens. A grammar declares every flag that takes a value and every flag of the tool a handler checks tokens for, booleans included, each with the spellings the tool pairs; a further spelling of the same kind is a flag of its own (git's `-u` beside `-p`/`--patch`, grep's `--silent` beside `-q`/`--quiet`). Each spelling and value arity is transcribed from git 2.53's man pages, `-h` output and `git-completion.bash`, GNU grep 3.12's `--help` and rg's binary. A check for a set of flags looks the token's flag up in a slice of consts: git's patch requests, raw log shapes, output indicators, content transforms, branch actions and list modes, grep's and rg's file lists and output shapes. A check that only one engine has a flag for asks the engine (`Engine::no_filename`, `Engine::no_line_number`), so no token is asked about a flag its grammar lacks. `Grammar` has no notion of a repeatable flag or of the numeric `-N` shorthand; the handlers that read `-N` test for a digit run. Subcommands whose options go through one parser share one grammar: `git log` and `show` read theirs with `LOG_GRAMMAR`. Distinct grammars share tables: `git diff`'s `DIFF_GRAMMAR` lists the revision-walk tables `LOG_GRAMMAR` lists, with `-u` as `-p` and a `--quiet` of its own, and `git stash show`'s lists them after its own `-u`/`--include-untracked`; golangci-lint's `run` lists the global table and the output sinks, and ast-grep's `run` lists the global table after its own. The git (global options, status, log/show, diff, checkout, branch, stash show, worktree, ls-tree), golangci-lint (global, run), ast-grep (global, run), grep, rg, dotnet, maven, ls, ruff (global), glab (view, mr merge/approve/note/update) and gh (view, pr checks, run view, pr comment/edit), `rtk task`, pytest (its value flags) and the usage-request check (git's message flags, `cargo install --version`) grammars are declared this way, each with a test that checks it against an explicit table of the flags it declares and the spec each takes (`None` for a boolean flag), under every token kind and for undeclared names. The same check fails on a spelling declared twice, compared under the grammar's naming rules. A handler reads an argument list once and asks every question of those tokens: `git show` its route, its objects and its content transforms, `git stash show` its patch request and its word diff, `rtk grep`/`rtk rg` the help request, the pattern, the paths and the file-list shape, `git log`'s cap probe which flags it leaves out of the walk it measures (the output shapes, `--graph`, `--reverse`, and the merge-diff family off a walk that does not select by diff), ast-grep's `run` its `--stdin`, its `-i`/`--interactive` and its `--json`, and `ls` whether it lists dotfiles, whether it prints the long format, and which value flag ends its arguments unanswered. The code follows each tool's own pairing, which sets these rules: - git's `-B`/`--break-rewrites`, `-C`/`--find-copies`, `-M`/`--find-renames`, `-U`/`--unified` and `-X`/`--dirstat` are one flag each, with an optional value that is attached only: git 2.53 reads the `50` of `-M 50` and of `--find-renames 50` as a revision. `-I`/`--ignore-matching-lines` is one flag, taking a separate value. - `-l`, the rename limit, takes the rest of its argument or the next one wherever it sits in a short cluster, under `git log`, `show`, `diff` and `stash show`: rtk reads the `5` of `-wl 5`, `-pl 5`, `-cl 5` and `-ml 5` as `-l`'s value. git 2.53's `log` and `show` read it so after `-p`, `-u`, `-s`, `-w` and `-W`; after `-c`, `-m` and `-t` they read the `5` as a revision and exit 128 before printing anything, and rtk passes that failure on. git rejects `git stash show -wl 5`, which takes the `5` as its stash. - `-L`, the line log (`-L<start>,<end>:<file>`, `-L:<funcname>:<file>`), takes a value and has no long spelling (git 2.53's `git log -h` lists `-L <range:file>` alone). It prints each commit that touches the range with its patch of that range, so `git log -L` and `git show -L` route raw: `rtk git log -L 1,2:f` prints git's bytes and exit code. - `-X` is `--dirstat` under `git log`, `diff` and `show`: git 2.53 reads `-Xfiles` as the parameter `files`, `-Xp` as the parameter `p` and `-pX` as `-p` then `-X`. rtk routes both spellings raw, so `git diff -X` prints git's dirstat. - The diffstat and dirstat parameters git spells as flags of their own (`--stat-width`, `--stat-name-width`, `--stat-graph-width`, `--stat-count`, `--dirstat-by-file`, `--cumulative`) print the stat or the dirstat as `--stat` and `-X` do, and route raw under `git log`, `show` and `diff`: `git diff --stat-width=60 HEAD~2` prints git's diffstat. So does `-z` under `show` and `diff`, which ends git's records with a NUL (the objects of `show`, the `--raw` and `--numstat` fields); the filtered `git log` keeps the NUL that ends each commit. - `--dd` is `--diff-merges=first-parent -p`: `git log --dd` routes raw, as `git log -p` does, and under `show` and `diff`, whose default output is a patch, it compacts as `-p` does. `-q` outranks it on either side in `git show`, as in git 2.53. - `--output-indicator-new`, `--output-indicator-old` and `--output-indicator-context` replace the `+`, `-` and ` ` markers `compact_diff` reads, so `git show` and `git diff` route them raw: compacting `git diff --output-indicator-old=Y` would count none of the removed lines. A plain `git log` prints no patch for them to change. - `-q` is `--quiet` under `git log` and `git show`, and not under `git diff`: git 2.53's `git log -h` and `git show -h` list `-q, --[no-]quiet suppress diff output`, `git diff -h` lists no `-q`, `git diff -q` with no revision prints the patch and exits 0, and `git diff -q HEAD~1`, `git diff --cached -q` and `git stash show -q` are usage errors (129). `git show -q` prints the commit summary with no diffstat, as `git show --quiet` and `-s` do, and a patch request on either side of it (`-q -p`, `-p -q`) prints the diff, as in git. - `git stash show` reads its own options, `-u`/`--[no-]include-untracked` and `--only-untracked`, before the diff options, so `-u` is never `-p` there: `git stash show -u` prints the stat of the tracked and untracked files, as git 2.53 does. A value-taking diff option takes the flag after it, as git's revision machinery takes it: git 2.53 reads the `-p` of `git stash show --author -p` as the author, and prints a patch only because any argument makes `stash show` print one, which rtk reads from git's output; `--author --word-diff` prints a plain patch, which rtk compacts. - `git show` of a blob passes through raw when a content transform (`--textconv`, `--ext-diff`, `--filters`) is one of its options, read from the tokens. In `git show --grep --textconv HEAD:big.txt` the `--textconv` is `--grep`'s pattern and git 2.53 prints the plain blob, which rtk windows as it windows `git show HEAD:big.txt`. - `git branch -l`/`--list` list the local branches, and the ones matching a pattern when given one: `git branch -l 'feat*'` prints git's listing of the matching branches, and a listing without the current branch has no `* ` line, as in git's. - A `git branch` listing ends each line with one newline and adds no blank line. An empty listing (`git branch -r` with no remotes, `--list 'nomatch*'`, `--no-merged` with every branch merged) prints nothing, as git 2.53 does. A listing rtk cannot shorten (`--format`), or one whose every line is a remote's `HEAD -> ...` alias, which the compact form leaves out (`git branch -a --list '*HEAD*'`), prints git's bytes. - `git branch --delete`/`--move`/`--copy` write, as `-d`/`-m`/`-c` do. - grep's recursion (`-r`, `-R`, `--recursive`, `--dereference-recursive`) shows the filename by default only when grep walks the working directory. GNU grep 3.12 prints a lone file operand's matches without the prefix under all four spellings, and with it for a directory operand or no operand, and rtk prints the same. - grep's `-d ACTION`/`--directories=ACTION` takes its value attached or as the next argument, and `recurse` is recursion, as `-r` is ("like --directories=recurse", grep 3.12's `--help`); `read` and `skip` are not. grep reads any prefix that names one action alone (`rec` is `recurse`) and keeps the last of `-r`, `-R` and `-d`: `-r -d read` does not recurse and `-d read -r` does. rtk reads the same, so `grep -d recurse PAT` with no operand prefixes each match with its file, and with a lone file operand does not. - grep's help flag is `--help` alone, since grep's `-h` is `--no-filename`: `rtk grep -h -r needle dir` runs grep and prints the matches without their file, as `grep -h -r needle dir` does, and `rtk grep --help` prints grep's help. rg's `-h` is its help flag, so `rtk rg -h` prints rg's help, as `rtk rg --help` does. - rg's `-0` is `--null`, an output-shape flag: `rg -0 PAT` passes through and prints rg's NUL-joined paths. - `-V` is `--version` for grep and rg, and passes through to the engine as `--version` does. - Each engine's output-shape flags are its own: a shape flag of one engine that the other lacks (`grep --json`, `rg --silent`) takes the grouping route, where the engine rejects it with the same message and exit code. - ast-grep's global and `run` grammars are tables of the options the module already read: `-c`/`--config` alone ahead of a subcommand, and every value-taking `run` option of 0.45.3 after `run`. `--debug-query` takes its value attached only, and `-i`/`--interactive` is declared beside `--stdin`, the two runs that reach the terminal live. - dotnet's options are atomic flags carrying each option's names together (`-c`/`--configuration`, `-l`/`--logger`), and maven's carry each short and long name commons-cli pairs (`-pl`/`--projects`, `-D`/`--define`). - ls declares every long option GNU ls's `--help` lists, booleans included, under `Posix` with `UniquePrefix`: an abbreviation is only unambiguous against the whole set. `ls --ign beta*` spans `--ignore` and `--ignore-backups` and stays unread, for ls to reject. GNU's `-I`/`-T`/`-w` take a value and BSD's do not, so each flavour has its own grammar over the shared long table. `--numeric-uid-gid`, `-n`'s long spelling, and its abbreviations print the long format as `-n` does: `rtk ls --numeric-uid-gid` keeps each entry's mode and size. The arg_tokenizer section of the core README describes a grammar declared once, as data, how the single-dash axis decides a flag's form, the abbreviation axis, and how grammars share tables; the command-filter checklist and the architecture guide point to it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

`core/cmdline/bash_grammar.rs` holds bash's reserved words as one table: the 22 words bash 5.3 lists (`compgen -k`), each with where the word after it stands (where a command starts, where only a reserved word starts, as after `fi`, `done`, `esac`, `}` and `]]`, or a plain word, as after `case` or `for`), whether that word may be a name after which only a reserved word starts (`for`, `select`, `function`, `coproc`), the options bash reads right after it (`time -p --`), and its role in the compound command it belongs to: it opens one, continues it into its next part, closes it, or prefixes a pipeline and belongs to none (`time`). A test derives every role from the productions of bash's `parse.y`. `reserved_word` matches a word by its exact spelling, so a quoted or escaped spelling is an ordinary word, as bash reads it. Every reader of a command line that needs one of these facts reads the table: - the segmenter's private `CaseTracker`, which follows `case`, `in` and `esac`; - `read_grammar`, the reading of a line that the rewrite and classification share, which finds where a command or a reserved word starts, `[[ … ]]` expressions and `case` patterns from the positions, the roles and the options; - the multi-line classifier in `discover/registry.rs`, which passes a block through when a line starts with `(`, `)` or a word that delimits a command list of a compound command: every word of `if`, `case`, the loops, `function` and `coproc` apart from `in`; - `classify_command`, which ignores a segment that starts with grammar as `read_grammar` reads it: a reserved word, whole (`[[(-f a)]]` starts with `[[`), or the `((` of an arithmetic command. `((ls) )` is two subshells, not arithmetic. A segment after `|` is read where a pipeline's stage starts, so its `time` is the program `time`. The rewrite follows these rules of bash's grammar: - A segment that starts with a reserved word is grammar around commands, never a command, so the rewrite never puts `rtk` in front of one. Under a filter matching every command, `until false; do git status; done`, `if …; elif true; then …; fi`, `case x in x) ls;; esac`, `coproc ls` and `[[ -f x ]]` keep their reserved words as written: `rtk until …` and `rtk esac` do not parse. - After a compound command's last word (`fi`, `done`, `esac`, `}`, `]]`, a subshell's `)`, an arithmetic command's `))`), after the name that follows `for`, `select`, `function` or `coproc`, and after `name ( )`, bash reads a reserved word and no command. So `if (true) then`, `for x do` and `while (( 0 )) do` open their lists, the `[[ … ]]` or `(( … ))` behind them is an expression, and `for x do [[ -f a || ls ]]; done` is left as written, where `… || rtk ls ]]` does not parse. `case x in a) (ls) esac` closes the `case`. A `(` after a word that opens no word text (below) is a function's `name ( )`, not a subshell. - `time` is reserved only where a pipeline starts. After `|`, `|&` or `coproc` it is the program `time`, as bash's `time_command_acceptable` decides, so `ls | time [[ -f a || ls ]]` runs `time` with `[[ -f a`, then `ls ]]`, and becomes `ls | time [[ -f a || rtk ls ]]`. `ReservedWord::pipeline_follows` names the words after which a pipeline may start. - After `time`, bash reads `-p` and then `--`, each optional, as part of the reserved word (`time -p`, `time --`, `time -p --`), and the word after them is in command position. Any other word there is a command's name, `-p` after `--` or after another `-p` included. So `time -p [[ -f a || -d b ]]` is one expression, left as written, where `time -p [[ -f a || rtk -d b ]]` does not parse. Bash compares these options as written, so the reader matches them literally: rule 6 of `.claude/rules/rust-patterns.md` names reserved-word grammar as the one exception to parsing through `arg_tokenizer`. - Inside `[[ … ]]` nothing is a command: `&&`/`||` there are expression operators, and so are its brackets and a regex's `|`. The expression runs from a `[[` where a reserved word starts to the first `]]` outside its brackets and its word text, or to the end of the line. `[[ -f a || ls ]]` is left as written, since `[[ -f a || rtk ls ]]` does not parse, and under a filter matching every command `[[ -f a && -d b ]]` keeps its `-d b`. - An arithmetic command runs no command. A `((` where a reserved word starts, or right after `for`, is arithmetic when the `)` that closes its second `(` is followed at once by another `)`, as bash's `parse_dparen` decides, and everything up to that `))` is its expression: `&&`, `||`, `;`, `<` and brackets there end nothing. Under a filter matching every command, `(( a || b ))` is left as written, where `(( rtk a || rtk b ))` is an arithmetic syntax error. Otherwise the two brackets are subshells, one inside the other (`((ls) )`), whose commands are rewritten. With no `)` that closes the second `(`, bash reports a syntax error, and the rest of the line is read as the expression, so nothing in it is rewritten. `$((` inside a word is an arithmetic expansion and opens nothing. After `for (( … ))`, `do` is a reserved word. - A `case` pattern is never a command: after `in`, and after each `;;`, `;&` or `;;&`, the words up to the pattern's `)` outside brackets and word text are patterns (an optional leading `(`, alternatives joined by `|`), and the arm's commands start after that `)`. `esac` closes the `case` only where a pattern or a reserved word would start. In `case $x in a) echo hi ;; ls) ls ;; esac` only the arm's `ls` becomes `rtk ls`, and the argument of `echo esac` leaves the `case` open, so a `(ls)` pattern after it is not read as a subshell. - Word text is never a command. An extglob group (a `(` right after a bare `!`, `@`, `*`, `?` or `+` in a word, glued to it or starting it), an array literal (`NAME=(…)` or `NAME+=(…)`, as in `declare -a a=(…)`) and a `${ }` are text of one word, from the bracket that opens them to the one that closes them; a `${ }` ends at its first `}`, as bash's `parse_matched_pair` reads it. A `(` glued to any other word is text the same way (`a[(b)]`), but for `name()` and for the `in(` that opens a `case` pattern, and the `{` of `$${` opens nothing, since `$$` is a variable. This holds in a command, in a `case` pattern and in a `[[ … ]]` expression alike: nothing inside one is rewritten or ends the command, the pattern or the expression, and in a command `words` splits no word on a blank inside one. So `x=(git status)`, `git !(ls)` and `ls ?((ls))` keep their text, under a filter matching every command `arr=(a)` keeps its element, `case $x in ${y:-a)b}) ls;; esac` becomes `case $x in ${y:-a)b}) rtk ls;; esac`, and `[[ ${x- ]] } == a || ls ]]` is left as written. Bash reads an extglob group as one word with `extglob` on; with it off the line is a syntax error, and the text stays as written. Word text's `|` and brackets still count for the rule that leaves a line piping and grouping commands as written, as the permission gate's segmenter reads them, so `ls @(a|b) && git status` stays as written. - `$( )`, `<( )` and `>( )` hold a command list of their own, which `read_grammar` reads to the `)` that ends it, past a `)` that a `case` pattern or a subshell inside it takes, so `echo $(case y in b) :;; esac)` leaves the reader in step. Each token of a substitution takes the reading around it. - A line that pipes and groups commands is left as written, and only command text counts: a `case` pattern's `|` and brackets and a `[[ ]]` regex's do not, so `[[ $y =~ ^(a|b)$ ]] && git status` and `case $y in a|b) git status;; esac` are rewritten. The permission gate keeps its own reading: `segment()` splits inside `[[ … ]]` and `(( … ))` and after a pattern's `)`, so a segment it checks may run nothing, and every command that runs reaches its rules. It is `segment()`'s one policy. `rtk discover` splits a line where the rewrite does: `split_for_classify` reads `read_grammar`'s readings and splits only at an operator, a pipe, `&` or a subshell's bracket read as command text. A `[[ … ]]` expression or an arithmetic command is one segment, and a `case` pattern is in none, so `[[ -f a || ls ]]`, `(( a || ls ))` and `case $x in ls) echo 1;; esac` report no `ls` that the rewrite leaves alone, while `ls && git status` is still two commands. A segment that starts with a reserved word counts as ignored, not as an unsupported command named after its first words (`until false`, `time cargo`). After `|`, `time cargo test` runs the program `time`, and stays an unsupported command. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…er family tables Transparent wrappers are peeled by one walker that reads data: env assignments, the reserved words bash runs a pipeline after, the routable `uv run` wrapper, the process wrappers (timeout, time, nice, nohup), the precommand words and a user's prefixes. Each family is a table, and the walker knows no tool by name. - `core/cmdline/family.rs` declares what a wrapper family is: a head, a label of the caller's own type, and a `Form` (a bare keyword, a word of the assignment run, or a wrapper with its own flag `Grammar` and operand count); - `core/cmdline/walker.rs` peels a caller-supplied `&[Family]` off the words and tokens a caller already lexed, at a `Position` (whether bash reads an assignment, or a reserved word such as `time`, at that place), and returns word indices. It reads reserved words and `time`'s options from `core/cmdline/bash_grammar.rs` and flags from the `Grammar` of `core/arg_tokenizer.rs`, where `.numeric()` reads the `-N` flag `nice` takes, and two new axis values read the options of a reserved word, each argument one option compared as written: `SingleDash::Whole` (`-pp` is the short flag `pp`) and `Attach::Never` (`-p=1` is the flag `p=1`). `Grammar::new` refuses a value spec they could never apply: a solo-only value under `Whole`, and an attached-only one on a whole-argument spelling under `Never`; - `discover/families/` holds the tables, one file per family with its own accept/reject table test: `env`, `precommand` (`command`, `builtin`, `exec`, `noglob`, `nocorrect`), `uv_run`, `process`, `match_only` (`stdbuf` and a bare `xargs`, which only the permission gate peels), and how a user's `transparent_prefixes` entry is read. The meaning of each label (`LayerKind`) is defined there. Adding a family is one file and a line in `ALL`, and in `MATCHING` unless the rules also match it whole. `registry::Walk` drives the walker over the words of the one lex a segment is cut from and records each layer as a range of words. The decision and its fall-through search read those ranges, and everything the rewrite decides is an edit against the segment's own bytes. Rules the walker and the rewrite implement: - short options cluster the way getopt reads them (`timeout -vk 5 300`, `timeout -vs9 300`, `/usr/bin/time -pv`), and `timeout -f` and `-p` are recognised; - where a pipeline starts, `if`, `then`, `elif`, `else`, `while`, `until`, `do` and bash's `time` are reserved words the walk steps over, and a pipeline starts again behind each (`if git diff --quiet; then git status; fi` becomes `if rtk git diff --quiet; then rtk git status; fi`, `time -p time cmd` peels both). `time`'s options are read as bash 5.3 reads them: `-p`, then `--`, each optional, each compared whole as written, through the `Grammar` the reserved-word table declares for them, a word running to a blank, an operator, a redirect, a pipe, `&` or a bracket. A redirect ends the options too: `time -p>out -- cmd` runs a command named `--`; a substitution glued to a word is text of it (`time -p<(true) cmd` runs `-p/dev/fd/63`). Where the options end inside a word the walk takes (`time -p>out cmd`), no layer can end, so the command is left as written, as it is behind a redirect that stands apart (`time -p >out cmd`), and no other tier reads the reserved word as a program. The walk and the reading of the line's grammar read them through one function. The word behind them is the command bash runs, so `time -v cmd`, `time -o x cmd` and `time -p -p cmd` run a command named `-v`, `-o` or `-p` and are left as written: the rewrite takes no prefix, filter or rule for the GNU program there (the permission gate reads both, below), and `rtk` is never put inside its words. `coproc` runs one command and no pipeline, and is no layer. GNU `time` and its `-f`/`-o`/`-a`/`-v`/`-p`/`-q` grammar apply anywhere else and through a path; - a compound command whose parts are command lists and that a reserved word closes, when it is a pipeline's stage, with a `|` or `|&` right before its opening word or right after its closing word and the redirections behind it, writes its output into that pipe or reads its input from it, so every command in any of its lists, a `case` arm's included, is left as written, the later ones of a list as much as the first (`for x in a; do git status; git log; done | wc -l`, `for x in a; do git status && git log; done | head`, `case x in a) git log;; esac | wc -l`, `ls | while read x; do git status; done`), as a subshell's commands are. A compound command or a subshell right after `coproc` or `coproc NAME` is the coprocess's body, whose input and output are pipes, and is left as written the same way (`coproc while true; do git status; done`, `coproc ( git status )`), a name of several tokens being one word (`coproc NAME${x} ( git status )`). The rule reaches across the lines of a block: a compound that one line opens and a later one closes holds the commands between. A compound no pipe joins is rewritten like any other list (`if true; then git status; fi; ls | head` becomes `if true; then rtk git status; fi; rtk ls | head`), and a pipeline inside one of its lists goes through the pipeline rules like any other (`case $y in (a|b) git status | head;; esac` becomes `case $y in (a|b) rtk git status | head;; esac`). `lexer::read_nesting` finds the compounds in the pass that reads the line's grammar; - `nice` options follow GNU nice: `-n N`, `-N`, `--adjustment=N`; `nice +5 cmd` is left as written, because `+5` is the program nice runs; - a wrapper's options end at the first operand it requires: `timeout 300 -s KILL cmd` and `timeout 300 -- cmd` are left as written, because after the duration `-s` and `--` are the program timeout runs; - a shell `NAME=value` is an assignment only at the start of a simple command (the front of a segment, of any pipeline stage, or of a compound command's list), and one run of assignments and `env` words is one layer. After `timeout`, `nice`, `nohup`, a precommand word (`exec`, `command`, `noglob`, ...) or `uv run` it is the program's name, so none of those are rewritten. `env`'s own operands are its own wherever it sits (`timeout 5 env A=1 cmd` is rewritten), and what follows a user prefix may be an assignment; - an unbraced `$NAME` glued to a word (`timeout 5$UNIT cmd`) is one word; - a wrapper head is read with its quotes and escapes removed (`'timeout' 5 cmd`, `\timeout 5 cmd`), and one that carries an expansion (`$D/timeout`, or a backquote inside double quotes) never matches; the words of a multi-word head (`uv run`) may be apart by any spaces or tabs; the classification of a command for `rtk discover`, and the env prefix the analytics split off, read an `env` the same way (`'env' git push` is `git push`); - a path-spelled head (`/usr/bin/env`, `/bin/command`, `/usr/bin/uv run`) is the program at that path, not the word `env`, `command`, `exec`, `noglob` or `uv run` stands for, so no layer takes it and the command it heads is left as written; behind a `uv run` under `exclude_commands` it is refused, as what it runs is out of sight; - a word behind a subshell's closing `)` in the same simple command, glued to it or not and at any depth (`(true)git status`, `(true) git status`, `( (true)git status )`), is left as written, and the subshell inside is rewritten (`( git status )word`); a case pattern's `)` is not one (`case x in a)git status;; esac` is rewritten); - a command word that starts with `-` (a wrapper's own option, or `--`), quoted or not, is never a command, and a user prefix that would end inside the words of the layer it covers (`nice` inside `nice -5`, `nice timeout` inside `nice timeout 5`) is skipped, so `rtk` never lands inside a wrapper's grammar; - the `RTK_DISABLED=` warning is printed exactly when a walk of the rewrite reaches the prefix and the command is refused: the first walk, behind any layer it peels (`timeout 5 env RTK_DISABLED=1 git status`), a retry behind a user prefix (`nohup RTK_DISABLED=1 git status` under the prefix `nohup` warns, as `docker exec c RTK_DISABLED=1 git status` under `docker exec c` does), or the walk of what a refused fall-through wraps. A quoted `sh|bash -c` script the rewrite reads, in which a command is refused this way, warns where the first walk reaches it and where a fall-through settles on its rewrite. Rewritten around the refused command it settles the command; left as written it settles nothing, and a routable wrapper around it is not matched whole (`uv run bash -c 'RTK_DISABLED=1 git status'` is left as written and warns), as with a script that runs a command `exclude_commands` covers (`uv run bash -c 'pytest'`, `uv run timeout 5 bash -c 'pytest'`), a quoted shell inside the script included (#4704). A fall-through that wins over a walk that stopped at the prefix draws none, and neither does a producer left as written before a consumer that does not take filtered output (`RTK_DISABLED=1 git status | wc -l`), which no walk reads, nor a `NAME=value` word where no simple command starts (`timeout 5 RTK_DISABLED=1 git status`, `uv run RTK_DISABLED=1 git status`), which is a program's name. The warning is carried on the rewrite's answer and printed from the one run of the rewrite, ahead of the configuration's and the filter registry's warnings, which are held until then (a panic inside the rewrite drops the held warnings with it, and so does a caller that catches the panic). The marker is the assignment `RTK_DISABLED=` by name, not a substring (`XRTK_DISABLED=1`, `MSG='RTK_DISABLED=1'` are not the bypass), and the append form `RTK_DISABLED+=1` is one only as a shell assignment, not as an `env` operand. An assignment run with no command behind it is a statement of its own, not the prefix, and draws none (`RTK_DISABLED=1; git status` rewrites `git status`). A line that holds `$((` is refused before any walk, so it draws no warning; - the decision answers `Excluded` for a command `exclude_commands` covers, before anything that depends on where the command sits, a quoted shell script excepted, whose own commands are checked instead. It reads the command up to the redirections behind it, as typed and as bash runs it, and a pattern that matches either excludes it. As bash runs it, the words have their quotes and escapes removed, and are read as they are, with the program's absolute path left out, or as the tool a rule routes it to (read off the words bash runs, so `npx "playwright" test` is `playwright`; an empty argument is kept, so `pytest ''` is not `^pytest$`). A pattern reads those words with one space between two of them and a blank inside a word behind a backslash, every other character as it is, so a blank separates two words where bash separates them. Under `exclude_commands = ["pytest"]`, `'pytest' -x`, `\pytest -x`, `timeout 5 'pytest' -x` and `'/opt/my tools/pytest' -x` are left as written, also where a project filter takes every command, and so is `'pytest' 2>&1` under `^pytest$`; `'pytest x'` runs a program named `pytest x`, which neither pattern covers; - `uv run`'s fall-through to its own rule reads what it wraps with the rewrite's own walk (the same peels, user-prefix retries and depth budget), and refuses when that walk reaches `RTK_DISABLED=` or a command the decision excludes; what it wraps as written is checked the same way, by the same decision. A quoted shell script is decided as a whole, so a command excluded inside it does not refuse the fall-through (`uv run bash -c 'pytest'` is routed as before). Under `exclude_commands = ["pytest"]`, `uv run timeout 5 pytest`, `uv run noglob pytest`, `uv run /usr/bin/pytest`, `uv run 'pytest'`, `uv run p"ytest" -x`, `uv run \pytest`, `uv run '/opt/my tools/pytest' -x`, `uv run timeout 5 'pytest' 2>&1`, `uv run uv run pytest`, `uv run time pytest`, `uv run python -m pytest`, `uv run nice python -m pytest`, and `uv run nice -n 19 ionice pytest` under the prefix `nice -n 19 ionice` are refused, as `uv run pytest` and `python -m pytest` are, and so is `uv run 'pytest' 2>&1` under `^pytest$`, while `uv run 'pytest x'` is not; `uv run timeout 5 env RTK_DISABLED=1 pytest` is refused, and layers deeper than the walk follows count as excluded, since nothing shows what they wrap. So does an option the walk cannot read past, which is `uv`'s own and leaves the command behind it unread (`uv run --with x pytest`), though a user prefix that starts at `uv run` is tried. A `--` right behind `uv run` is read through to the command, so `uv run -- pytest` is refused and `uv run -- cargo test` is not. A `NAME=value` word right behind `uv run` is a program's name, and the fall-through refuses it (`uv run RTK_DISABLED=1 git status`, `uv run A=1 pytest`). Without `exclude_commands` nothing is hidden, and an option or `--` behind `uv run` leaves the whole string to its rule (`uv run --with x pytest` becomes `rtk uv run --with x pytest`); - a user-configured prefix is tried at the start of every layer whose own command has no rewrite, not only behind `uv run`: `transparent_prefixes = ["nice -n 19 ionice"]` rewrites `nice -n 19 ionice git status`; the built-in peel wins whenever it succeeds, and a reserved word is never a prefix's to claim; - a `transparent_prefixes` entry is read without the `$IFS` bytes at either end, and one that does not lex into whole words (an unclosed quote or a trailing backslash), or whose first word is a bash reserved word (`coproc`, `time`, `if`, ...), which bash reads as grammar where a command starts, is refused, with a warning each time the rewrite runs; - a quoted `sh|bash|zsh|fish -c` wrapper is decided wherever a walk reaches it, its script rewritten one wrapper deep, and a wrapper at the walk's depth limit is still the command (`exec … exec bash -c '…'`). The permission gate checks for a wrapper at the words the rewrite decides at, the walk's and every fall-through's, whether the segment opens a pipeline or is a later stage of one (`registry::shell_wrapper_positions`), so it looks wherever the rewrite would reach, and at a pipeline's start a little further; - the permission gate reads past prefixes with the same walk over its own list of families (`families::MATCHING`): the rewrite's, but for `uv run` and the user's prefixes, which the host does not strip, plus `stdbuf` and a bare `xargs`. Each word of an assignment run is a reading of its own, and an assignment is read wherever it sits (`nohup HUSKY=0 git push` is `git push` to a deny rule). `time` is read both ways, as bash's reserved word and as the program a later pipeline stage or `sh` runs (`time -v git push`). A quoted wrapper's script is read the same way and, where a deny rule exists, one whose prefixes outrun the peel's cap is deferred to the host; - every warning the rewrite prints starts with `[rtk] warning:`, as rtk's other warnings do: the `RTK_DISABLED=` warning (`[rtk] RTK_DISABLED=1 detected` before), a refused `transparent_prefixes` entry, and a trivial or invalid `exclude_commands` pattern. Part of #4315 Refs #3230 Refs #1606 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

`core/cmdline/fish.rs` holds what RTK knows of fish: one table of the words fish reads as its own where a command starts, as fish 3.7 and 4 both read them, each marked as control flow (the words of fish's blocks, and `and`, `or`, `not`) or as a prefix that means what it means in bash (`!`, `time`), and `fish::read`, the one walk over the lexer's tokens that says where a command starts. Read against lex/14's table of bash's reserved words, it tells what in a string is fish's alone (a keyword bash does not reserve: `end`, `begin`, `switch`, `and`, `or`, `not`), what only bash has (a word bash reserves and fish does not, `[[`, an unquoted backquote, a heredoc) and where fish substitutes a command (an unquoted `(` or `)`). Both readers of fish read it. The lexer's check for a `fish -c` script refuses what the reading calls control flow or substitution, and `discover/fish_script.rs` wraps a script the reading finds fish's alone and free of anything only bash has. The two keyword lists, the two command-position loops and the two substitution checks they each kept go. The fish classifier finds comments from the lexer's `Comment` tokens, as the multi-line rewrite does, and its own scanner goes; it keeps only the check that is fish's, the lone `&` before a `#` that fish 3.0 and later read as one word. Its check that a script's quotes and escapes close is `lexes_into_whole_words`. Behaviour, as the code implements it: - A script with `until`, `select`, `coproc`, `in` or `]]` where a command starts is not wrapped as fish, as `then`, `fi`, `do`, `done`, `esac` and `elif` already were: bash reserves them and fish does not. - The fish classifier finds a comment where the rewrite does: a `#` after an escaped or quoted `|`, `&`, `;`, `(` or `)` starts none. A script the reader calls too complex, as with a `#` glued to `<` or `>`, is not wrapped. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Navigation Menu

Repository navigation

Comparing changes

Open a pull request

Uh oh!

Commits on Nov 5, 2026

This comparison is taking too long to generate.

Uh oh!