editor: show control characters as notation (#1636)

Keep controls in editor text and render them as safe, visible notation.
This lets the cursor, selection, wrapping, and offsets handle controls
consistently without sending them to the terminal. Strip terminal escape
sequences from pasted text so pasted controls cannot change terminal
state.

Alternative to #1632 and aligned with how vim does it.

We accidentally in pre 0.6 and explicit in 0.6.x rejects edit text that
contains a C0 control other than tab, carriage return, and line feed,
DEL, or a C1 control: `setText()`, `insertText()`, `replaceText()`,
Textarea `initialValue`, and `Input.value` throw `NativeError` with
`InvalidArgument`. The reason is that the editor places its cursor by
display cell, and a control took no cells, so the cursor could neither
move past it nor delete it. In 0.5.14, which accepted controls, the
cursor in `"a\fb"` could not reach column 0, Backspace stopped at
`"a\f"`, and rendering `"A\x07B\x1bC\fD"` wrote raw BEL and ESC bytes to
the terminal while `C` and `D` vanished.

This is an alternative to #1632, which drops these characters. Instead,
editors keep them and show them the way Vim and Neovim do. Measured with
Neovim 0.12.5 and Vim 9.2, both show `a^Lb^@c^[d<85>e^?f`: a C0 control
or DEL as caret notation in 2 cells, and a C1 control as hex in 4 cells.
The cursor stops on each one, `x` deletes one, and writing the file back
gives identical bytes.

Editors now do the same. A per-buffer `ControlDisplay` setting is passed
with the tab width to the native width functions, so cursor movement,
deletion, wrapping, selection, history, and drawing share one rule:

- C0 controls other than tab, LF, and CR, and DEL, show as `^@`..`^_`
and `^?` in 2 cells. C1 controls show as `<80>`..`<9f>` in 4 cells. Tab,
LF, and CR keep their current behavior.
- Cells still never hold a raw control: drawing writes the notation's
printable glyphs.
- Each control is its own grapheme, so it no longer joins the next
character, and a soft wrap never splits its notation.
- `getText()`, undo and redo, and selection copy return the original
bytes.
- Read-only text keeps hiding controls with zero width, so
`TextRenderable`, the console, and other displayed text do not change.
Only `EditBuffer` and its placeholder use notation.

Text input changes with it:

- Paste keeps controls, as Vim does, and still removes escape sequences.
The stripper is now one shared implementation, so Bun and Node give the
same result, and core no longer depends on `strip-ansi`. A lone ESC and
a truncated sequence are removed, so a paste never inserts ESC. Key
sequences with a control stay ignored, so Escape does not insert `^[`.
- `Input` puts its cursor at the native end of the value instead of the
string length.
- Extmarks convert display offsets through native `offsetToPosition`
instead of JavaScript string widths, and Right stops at the native end
of the buffer before a virtual extmark. `deleteLine` adjusts extmarks by
native line offsets.
- `EditBuffer.measureText(text)` and `ot_edit_buffer_measure_text`
return the display offset at the end of a text as the buffer would lay
it out, without storing it. JavaScript width functions give a control
and a tab no cells, so an app that computes editor offsets in JavaScript
needs this: in `"\f@fi"` the `@` is at offset 2, not 0.

Along the way, these bugs that already exist in are fixed:

- A standalone zero-width cluster (zero-width space, word joiner, BOM)
blocked Left and Backspace in the Unicode width methods.
- After `"日\u{fffd}"`, Backspace deleted the wide character. In the
no_zwj method, one emoji step had the wrong size.
- U+FFFD inside a joining sequence, such as `"क्\u{FFFD}\u{301}ष"`, made
layout and cursor movement disagree: Right got stuck, and Backspace
deleted the wrong cells.
- An extmark ending at a line end after a tab lost its last cell. Right
could step into a virtual extmark after CJK text. `deleteLine` moved
extmarks by UTF-16 indexes. `Input` placed its cursor in the middle
after CJK text.

Backspace no longer measures the whole chunk for each step. The native
"backspace 500 chars" benchmark takes 0.21 ms instead of 25.3 ms on
v0.6.0; other benchmarks match v0.6.0 within noise.

Most of the diff passes the new setting to the width functions: of the
added lines, most only add `.hidden`, `.notation`, or `control_display`,
and others of them in existing tests. Net source is +86 lines and net
tests +95.

Not included: a distinct style for the notation (like Vim's SpecialKey),
a literal-insert key, `^M` for a lone CR, controls as word boundaries,
and clipping wide units cell by cell at the left edge of a horizontally
scrolled view.


Fixes #1642

bun.lock

@@ -17,7 +17,6 @@
         "diff": "9.0.0",
         "marked": "17.0.1",
         "string-width": "7.2.0",
-        "strip-ansi": "7.1.2",
       },
       "devDependencies": {
         "@opentui/keymap": "workspace:*",

packages/core/package.json

@@ -73,8 +73,7 @@
     "bun-ffi-structs": "0.3.1",
     "diff": "9.0.0",
     "marked": "17.0.1",
-    "string-width": "7.2.0",
-    "strip-ansi": "7.1.2"
+    "string-width": "7.2.0"
   },
   "peerDependencies": {
     "web-tree-sitter": "0.25.10"

packages/core/scripts/native-abi-pointers.ts

@@ -142,6 +142,7 @@ export const nativePointerPolicies = {
   ot_edit_buffer_history: { 0: context, 1: buffer, 3: buffer, 5: buffer },
   ot_edit_buffer_get_position: { 0: context, 1: buffer, 5: buffer },
   ot_edit_buffer_get_range: { 0: context, 1: buffer, 7: empty, 9: buffer },
+  ot_edit_buffer_measure_text: { 0: context, 1: buffer, 2: empty, 4: buffer },
   ot_edit_buffer_set_defaults: { 0: context, 1: buffer, 3: buffer },
   ot_syntax_style_register: { 0: context, 1: buffer, 2: empty, 4: buffer, 5: buffer },
   ot_syntax_style_resolve: { 0: context, 1: buffer, 2: empty, 4: buffer },

packages/core/src/edit-buffer.test.ts

@@ -99,21 +99,29 @@ describe("EditBuffer", () => {
       expect(cursor.row).toBe(2)
     })
 
-    it("preserves cursor text boundaries across tab width changes and history", () => {
-      buffer.setText("a\tb")
-      buffer.setCursor(0, 4)
-      buffer.insertText("x")
+    // Tabs and control notation before the cursor change width differently.
+    it.each([
+      ["a\tb", 4, 11, 10, "a\tby"],
+      ["\u0007\tb", 2, 3, 2, "\u0007y\tb"],
+      ["a\t\u0007", 5, 12, 11, "a\t\u0007y"],
+    ] as const)(
+      "preserves cursor text boundaries in %j at %d across tab width changes and history",
+      (text, col, after, undone, typed) => {
+        buffer.setText(text)
+        buffer.setCursor(0, col)
+        buffer.insertText("x")
 
-      buffer.setTabWidth(8)
-      expect(buffer.getTabWidth()).toBe(8)
-      expect(buffer.getCursorPosition()).toEqual({ row: 0, col: 11, offset: 11 })
+        buffer.setTabWidth(8)
+        expect(buffer.getTabWidth()).toBe(8)
+        expect(buffer.getCursorPosition()).toEqual({ row: 0, col: after, offset: after })
 
-      expect(buffer.undo()).toBe("cursor:0:4:4")
-      expect(buffer.getCursorPosition()).toEqual({ row: 0, col: 10, offset: 10 })
+        expect(buffer.undo()).toBe(`cursor:0:${col}:${col}`)
+        expect(buffer.getCursorPosition()).toEqual({ row: 0, col: undone, offset: undone })
 
-      buffer.insertText("y")
-      expect(buffer.getText()).toBe("a\tby")
-    })
+        buffer.insertText("y")
+        expect(buffer.getText()).toBe(typed)
+      },
+    )
 
     it("remaps the cursor when tab width changes after an insert", () => {
       buffer.setText("a\tb")
@@ -429,6 +437,41 @@ describe("EditBuffer", () => {
       expect(buffer.positionToOffset(0, 0)).toBe(0)
       expect(buffer.getLineStartOffset(0)).toBe(0)
     })
+
+    // JavaScript string widths give controls, tabs, and line breaks no cells. measureText() must give the offset at
+    // which the same text ends after setText(), and must not store it.
+    it.each([
+      ["", 2, 0, 0],
+      ["a\fb\u0000c\u007f", 2, 9, 9],
+      ["\u0085\u009f", 2, 8, 8],
+      ["a\tb", 8, 10, 10],
+      ["日本\u0301", 2, 4, 4],
+      ["👩‍👧 🇩🇪", 2, 5, 7],
+      ["a\r\nb\rc\n", 2, 6, 6],
+    ] as const)("measures %j with tab width %d as unicode %d and wcwidth %d", (text, tabWidth, unicode, wcwidth) => {
+      for (const [widthMethod, expected] of [
+        ["unicode", unicode],
+        ["wcwidth", wcwidth],
+      ] as const) {
+        const editor = EditBuffer.create(widthMethod, resourceContext)
+        editor.setTabWidth(tabWidth)
+        editor.setText("kept")
+        expect(editor.measureText(text)).toBe(expected)
+        expect(editor.getText()).toBe("kept")
+        editor.setText(text)
+        expect(editor.offsetToPosition(expected)).not.toBeNull()
+        expect(editor.offsetToPosition(expected + 1)).toBeNull()
+        editor.destroy()
+      }
+    })
+
+    it("maps a string index to an offset and back", () => {
+      buffer.setText("\f@file")
+      const index = buffer.getText().indexOf("@")
+      const offset = buffer.measureText(buffer.getText().slice(0, index))
+      expect(offset).toBe(2)
+      expect(buffer.getTextRange(0, offset).length).toBe(index)
+    })
   })
 
   describe("getEOL navigation", () => {

packages/core/src/edit-buffer.ts

@@ -265,6 +265,19 @@ export class EditBuffer extends EventEmitter {
     return this.position(NativeEditPositionQuery.LineStart, row)?.offset ?? 0
   }
 
+  /**
+   * Returns the global display offset at the end of `text` as this buffer would lay it out, without storing it: its
+   * cells under this buffer's width method, tab width, and control notation, plus one per line break.
+   */
+  public measureText(text: string): number {
+    this.guard()
+    return this.lib.contextEditBufferMeasureText(
+      this.native.handle.context,
+      this.native.handle,
+      this.lib.encoder.encode(text),
+    )
+  }
+
   public getTextRange(startOffset: number, endOffset: number): string {
     this.guard()
     if (startOffset >= endOffset) return ""

packages/core/src/lib/extmarks-multiwidth.test.ts

@@ -87,6 +87,24 @@ describe("ExtmarksController - Multi-width Graphemes", () => {
       expect(highlights[0].end).toBe(25)
     })
 
+    // A tab or control takes cells that a string width does not count. A highlight that ends at the end of the first
+    // line must stay on that line. An end past the text covers the rest of it.
+    it.each<[string, number, number, number[][][]]>([
+      ["a\tb\ncd", 0, 4, [[[0, 4]], []]],
+      ["a\fb\ncd", 0, 4, [[[0, 4]], []]],
+      ["a\u0085b\ncd", 0, 6, [[[0, 6]], []]],
+      ["a\fb\ncd", 3, 100, [[[3, 4]], [[0, 2]]]],
+    ])("highlights %j from %d to %d", async (text, start, end, expected) => {
+      await setup(text)
+      const style = SyntaxStyle.create(currentRenderer.nativeScene)
+      textarea.syntaxStyle = style
+      extmarks.create({ start, end, styleId: style.registerStyle("test", { fg: RGBA.fromValues(1, 0, 0, 1) }) })
+      const lines = [0, 1].map((line) =>
+        textarea.getLineHighlights(line).map((highlight) => [highlight.start, highlight.end]),
+      )
+      expect(lines).toEqual(expected)
+    })
+
     it("should correctly highlight text BEFORE multi-width characters", async () => {
       await setup("hello 前后端分离")
 
@@ -269,6 +287,19 @@ describe("ExtmarksController - Multi-width Graphemes", () => {
       // Cursor should be at display-width offset 5 (after "前后 ")
       expect(cursorPos).toBe(5)
     })
+
+    // Wide text takes more cells than string units, so the end of the buffer is past the string length.
+    it.each(["\u0085\u0085\u0085", "日日日日"])("skips a virtual extmark at the end after %j", async (prefix) => {
+      await setup(`${prefix}@ab`)
+      textarea.focus()
+      const end = textarea.editBuffer.getEOL().offset
+      extmarks.create({ start: end - 3, end, virtual: true })
+      textarea.cursorOffset = end - 3
+      currentMockInput.pressArrow("right")
+      expect(textarea.cursorOffset).toBe(end)
+      currentMockInput.pressArrow("right")
+      expect(textarea.cursorOffset).toBe(end)
+    })
   })
 
   describe("Visual vs Byte Offset Issues", () => {

packages/core/src/lib/extmarks.test.ts

@@ -1465,58 +1465,25 @@ Press ESC to return to main menu.`
   })
 
   describe("deleteLine() Operations", () => {
-    it("should adjust extmark positions after deleteLine before extmark", async () => {
-      await setup("Line1\nLine2\nLine3")
-
-      const id = extmarks.create({
-        start: 12,
-        end: 17,
-      })
-
+    // Offsets count display cells. Wide text and controls take more cells than string units, and the last line goes
+    // with the line break before it.
+    it.each<[string, number, number, number, [number, number] | null, string]>([
+      ["Line1\nLine2\nLine3", 3, 12, 17, [6, 11], "Line2\nLine3"],
+      ["Line1\nLine2\nLine3", 8, 6, 11, null, "Line1\nLine3"],
+      ["Line1\nLine2\nLine3", 8, 0, 5, [0, 5], "Line1\nLine3"],
+      ["日日日\nxyz", 0, 7, 10, [0, 3], "xyz"],
+      ["\f\f\f\nxyz", 0, 7, 10, [0, 3], "xyz"],
+      ["ab\ncd", 4, 0, 3, [0, 2], "ab"],
+      ["\f\f", 2, 0, 6, [0, 2], ""],
+    ])("deletes the line of %j at %d and moves [%d, %d) to %j", async (text, cursor, start, end, expected, rest) => {
+      await setup(text)
+      const id = extmarks.create({ start, end })
       textarea.focus()
-      textarea.cursorOffset = 3
-
+      textarea.cursorOffset = cursor
       textarea.deleteLine()
-
       const extmark = extmarks.get(id)
-      expect(extmark?.start).toBe(6)
-      expect(extmark?.end).toBe(11)
-      expect(textarea.plainText).toBe("Line2\nLine3")
-    })
-
-    it("should remove extmark when deleteLine on line containing it", async () => {
-      await setup("Line1\nLine2\nLine3")
-
-      const id = extmarks.create({
-        start: 6,
-        end: 11,
-      })
-
-      textarea.focus()
-      textarea.cursorOffset = 8
-
-      textarea.deleteLine()
-
-      expect(extmarks.get(id)).toBeNull()
-      expect(textarea.plainText).toBe("Line1\nLine3")
-    })
-
-    it("should not adjust extmark when deleteLine after", async () => {
-      await setup("Line1\nLine2\nLine3")
-
-      const id = extmarks.create({
-        start: 0,
-        end: 5,
-      })
-
-      textarea.focus()
-      textarea.cursorOffset = 8
-
-      textarea.deleteLine()
-
-      const extmark = extmarks.get(id)
-      expect(extmark?.start).toBe(0)
-      expect(extmark?.end).toBe(5)
+      expect(extmark && [extmark.start, extmark.end]).toEqual(expected)
+      expect(textarea.plainText).toBe(rest)
     })
   })
 

packages/core/src/lib/extmarks.ts

@@ -1,6 +1,5 @@
 import type { EditBuffer } from "../edit-buffer.js"
 import type { EditorView } from "../editor-view.js"
-import { stringWidth } from "../platform/runtime.js"
 import { ExtmarksHistory, type ExtmarksSnapshot } from "./extmarks-history.js"
 
 export interface Extmark {
@@ -143,9 +142,8 @@ export class ExtmarksController {
       }
 
       const targetOffset = currentOffset + 1
-      const textLength = this.editBuffer.getText().length
-
-      if (targetOffset > textLength) {
+      // A target past the end of the buffer has no position.
+      if (this.editBuffer.offsetToPosition(targetOffset) === null) {
         this.originalMoveCursorRight()
         return
       }
@@ -366,30 +364,17 @@ export class ExtmarksController {
           return
         }
 
-        const text = this.editBuffer.getText()
-        const currentOffset = this.editorView.getVisualCursor().offset
-
-        let lineStart = 0
-        for (let i = currentOffset - 1; i >= 0; i--) {
-          if (text[i] === "\n") {
-            lineStart = i + 1
-            break
-          }
-        }
-
-        let lineEnd = text.length
-        for (let i = currentOffset; i < text.length; i++) {
-          if (text[i] === "\n") {
-            lineEnd = i + 1
-            break
-          }
-        }
-
-        const deleteLength = lineEnd - lineStart
+        // Native deletes the line with the line break after it, or with the one before it on the last line.
+        const row = this.editBuffer.getCursorPosition().row
+        const lineStart = this.editBuffer.getLineStartOffset(row)
+        const [start, end] =
+          row + 1 < this.editBuffer.getLineCount()
+            ? [lineStart, this.editBuffer.getLineStartOffset(row + 1)]
+            : [Math.max(0, lineStart - 1), this.editBuffer.getEOL().offset]
 
         this.originalDeleteLine()
         this.saveSnapshot()
-        this.adjustExtmarksAfterDeletion(lineStart, deleteLength)
+        this.adjustExtmarksAfterDeletion(start, end - start)
       })
   }
 
@@ -610,20 +595,14 @@ export class ExtmarksController {
 
   private updateHighlights(): void {
     this.editBuffer.clearAllHighlights()
-    let text: string | undefined
 
     for (const extmark of this.extmarks.values()) {
       if (extmark.styleId !== undefined) {
-        text ??= this.editBuffer.getText()
         // extmark.start/end are display-width offsets including newlines (from cursor operations)
         // addHighlightByCharRange expects display-width offsets excluding newlines
-        // So we need to subtract the number of newlines before each position
-        const startWithoutNewlines = this.offsetExcludingNewlines(extmark.start, text)
-        const endWithoutNewlines = this.offsetExcludingNewlines(extmark.end, text)
-
         this.editBuffer.addHighlightByCharRange({
-          start: startWithoutNewlines,
-          end: endWithoutNewlines,
+          start: this.offsetExcludingNewlines(extmark.start),
+          end: this.offsetExcludingNewlines(extmark.end),
           styleId: extmark.styleId,
           priority: extmark.priority ?? 0,
           hlRef: extmark.id & 0xffff,
@@ -632,46 +611,10 @@ export class ExtmarksController {
     }
   }
 
-  private offsetExcludingNewlines(offset: number, text: string): number {
-    // offset is a display-width offset from the start of the buffer (includes newlines)
-    // We need to convert to display-width excluding newlines
-    // This means: subtract 1 for each newline encountered before this offset
-    let displayWidthSoFar = 0
-    let newlineCount = 0
-
-    // Walk through the text and calculate display widths
-    let i = 0
-    while (i < text.length && displayWidthSoFar < offset) {
-      if (text[i] === "\n") {
-        displayWidthSoFar++ // newline counts as width 1 in cursor offset
-        newlineCount++
-        i++
-      } else {
-        // Find the next newline or end of string
-        let j = i
-        while (j < text.length && text[j] !== "\n") {
-          j++
-        }
-        const chunk = text.substring(i, j)
-        const chunkWidth = stringWidth(chunk)
-
-        if (displayWidthSoFar + chunkWidth < offset) {
-          // Entire chunk fits before offset
-          displayWidthSoFar += chunkWidth
-          i = j
-        } else {
-          // Offset is within this chunk - need to find exact position
-          // Walk character by character
-          for (let k = i; k < j && displayWidthSoFar < offset; k++) {
-            const charWidth = stringWidth(text[k])
-            displayWidthSoFar += charWidth
-          }
-          break
-        }
-      }
-    }
-
-    return offset - newlineCount
+  // Each newline before the offset adds one, and native layout owns the cell widths of the text between them. An
+  // offset past the end stays past the end, where native highlights clamp it.
+  private offsetExcludingNewlines(offset: number): number {
+    return offset - (this.editBuffer.offsetToPosition(offset)?.row ?? 0)
   }
 
   public create(options: ExtmarkOptions): number {

packages/core/src/lib/paste.ts

@@ -1,5 +1,3 @@
-import { stripANSI } from "../platform/runtime.js"
-
 export type PasteKind = "text" | "binary" | "unknown"
 
 export interface PasteMetadata {
@@ -9,10 +7,16 @@ export interface PasteMetadata {
 
 const PASTE_TEXT_DECODER = new TextDecoder()
 
+// ECMA-48 sequences in ESC and C1 form: a control sequence through its final byte, a control string (OSC, DCS, SOS,
+// PM, APC) through BEL or ST, and any other escape through its final byte. A sequence also ends at the first
+// character that cannot continue it, so a lone ESC and a cut-off sequence go too. Bun and Node share this policy.
+const ANSI_SEQUENCES =
+  /(?:\x1b\[|\x9b)[\x30-\x3f]*[\x20-\x2f]*[\x40-\x7e]?|(?:\x1b[P\]X^_]|[\x90\x98\x9d-\x9f])[^\x07\x1b\x9c]*(?:\x07|\x1b\\|\x9c)?|\x1b[\x20-\x2f]*[\x30-\x7e]?/g
+
 export function decodePasteBytes(bytes: Uint8Array): string {
   return PASTE_TEXT_DECODER.decode(bytes)
 }
 
 export function stripAnsiSequences(text: string): string {
-  return stripANSI(text)
+  return text.replace(ANSI_SEQUENCES, "")
 }

packages/core/src/native-abi.generated.ts

@@ -1,7 +1,7 @@
 // Generated from packages/native/include/opentui.h and scripts/native-abi-pointers.ts.
 // Run `bun run generate:abi` in packages/core. Do not edit.
 // Inspect audit input: bun scripts/native-abi.ts --audit
-// ABI audit SHA-256: fb48f040c5837712cbfeb69068ee1761b689e5c834f28d950bab1cf4f674f5e0
+// ABI audit SHA-256: ad314fd725ad78ca9c5755b0e60785042b4a3c3d350ead726d4b3b21848b8b47
 
 export const nativeSymbols = {
   ot_scene_set_hooks: { args: ["ptr", "buffer", "buffer"], returns: "i32" },
@@ -93,6 +93,7 @@ export const nativeSymbols = {
     args: ["ptr", "buffer", "u32", "u32", "u32", "u32", "u32", "ptr", "u32", "buffer"],
     returns: "i32",
   },
+  ot_edit_buffer_measure_text: { args: ["ptr", "buffer", "ptr", "u32", "buffer"], returns: "i32" },
   ot_edit_buffer_set_defaults: { args: ["ptr", "buffer", "u32", "buffer"], returns: "i32" },
   ot_syntax_style_register: { args: ["ptr", "buffer", "ptr", "u32", "buffer", "buffer"], returns: "i32" },
   ot_syntax_style_resolve: { args: ["ptr", "buffer", "ptr", "u32", "buffer"], returns: "i32" },

packages/core/src/platform/runtime.ts

@@ -4,7 +4,6 @@ import { dirname, isAbsolute, resolve } from "node:path"
 import { fileURLToPath } from "node:url"
 
 import stringWidthLib from "string-width"
-import stripAnsiLib from "strip-ansi"
 
 import { resolveAssetRootPath } from "./assets.js"
 
@@ -16,7 +15,6 @@ export interface WriteFileOptions {
 interface BunLike {
   sleep(msOrDate: number | Date): Promise<void>
   stringWidth(text: string): number
-  stripANSI(text: string): string
   write(destination: string | URL, data: string | ArrayBufferView, options?: WriteFileOptions): Promise<number>
 }
 
@@ -38,7 +36,6 @@ const bun = (globalThis as GlobalWithBun).Bun
 
 export const sleep: (msOrDate: number | Date) => Promise<void> = bun?.sleep ?? standardSleep
 export const stringWidth: (text: string) => number = bun?.stringWidth ?? stringWidthLib
-export const stripANSI: (text: string) => string = bun?.stripANSI ?? stripAnsiLib
 export const writeFile: (
   destination: string | URL,
   data: string | ArrayBufferView,

packages/core/src/renderables/EditBufferRenderable.test.ts

@@ -6,6 +6,7 @@ import { TextareaRenderable } from "./Textarea.js"
 import { createTestRenderer, type TestRenderer } from "../testing/test-renderer.js"
 import { StyledText } from "../lib/styled-text.js"
 import { RGBA } from "../lib/RGBA.js"
+import { RecordingWriteStream } from "../testing/test-streams.js"
 
 describe("EditBufferRenderable", () => {
   let renderer: TestRenderer
@@ -32,6 +33,42 @@ describe("EditBufferRenderable", () => {
     expect(renderer.currentRenderBuffer.getSpanLines()[0].spans[0]).toMatchObject({ text: "kept", fg: color })
   })
 
+  test("Textarea keeps controls and writes them to the terminal as notation", async () => {
+    renderer.destroy()
+    const stdout = new RecordingWriteStream(40, 2)
+    const setup = await createTestRenderer({
+      width: 40,
+      height: 2,
+      stdout: stdout as unknown as NodeJS.WriteStream,
+      bufferedOutput: "stdout",
+    })
+    ;({ renderer, renderOnce } = setup)
+    await renderer.setupTerminal()
+    // The 0.5 editor wrote BEL and ESC to the terminal and lost the characters after them.
+    const text = "A\x07B\x1bC\fD\u0085E"
+    const editor = new TextareaRenderable(renderer, { initialValue: text, placeholder: "p\x00q", width: 40, height: 2 })
+    renderer.root.add(editor)
+    // Remove only the complete CSI and OSC sequences that the renderer writes, so a raw ESC from a cell remains.
+    const terminalSequences = /\x1b\[[\x30-\x3f]*[\x20-\x2f]*[\x40-\x7e]|\x1b\][\x20-\x7e]*(?:\x07|\x1b\\)/g
+    const frame = async () => {
+      stdout.clear()
+      await renderOnce()
+      const output = stdout.text().replace(terminalSequences, "")
+      expect(output).not.toMatch(/[\x00-\x1f\x7f-\x9f]/)
+      return output
+    }
+    expect(await frame()).toContain("A^GB^[C^LD<85>E")
+    editor.focus()
+    editor.gotoBufferEnd()
+    setup.mockInput.pressBackspace()
+    setup.mockInput.pressBackspace()
+    expect(editor.plainText).toBe("A\x07B\x1bC\fD")
+    editor.replaceText("")
+    expect(await frame()).toContain("p^@q")
+    editor.undo()
+    expect(editor.plainText).toBe("A\x07B\x1bC\fD")
+  })
+
   test("brands textarea and input instances", async () => {
     const textarea = new TextareaRenderable(renderer, { width: 20, height: 3 })
     const input = new InputRenderable(renderer, { width: 20 })

packages/core/src/renderables/Input.test.ts

@@ -602,6 +602,22 @@ describe("InputRenderable", () => {
       expect(input.cursorOffset).toBe("world".length) // Cursor should move to end
     })
 
+    // Controls take notation cells, so the cursor offset is a cell count, not the string length. maxLength counts the
+    // string length of the value, controls included.
+    it.each([
+      ["constructor", "a\fb\u0085c", "a\fb\u0085c!"],
+      ["value", "a\fb\u0085c", "a\fb\u0085c!"],
+      ["constructor", "日本", "日本!"],
+      ["value", "\u0000日本", "\u0000日本!"],
+      ["value", "a\fb\u0085c\u007f", "a\fb\u0085c\u007f"],
+    ] as const)("%s keeps %j and puts the cursor after it", (path, text, expected) => {
+      const { input } = createInputRenderable({ width: 20, maxLength: 6, value: path === "constructor" ? text : "" })
+      if (path === "value") input.value = text
+      input.insertText("!")
+      expect(input.value).toBe(expected)
+      input.destroy()
+    })
+
     it("should handle empty value setting", () => {
       const { input } = createInputRenderable({
         value: "not empty",

packages/core/src/renderables/Input.ts

@@ -91,7 +91,7 @@ export class InputRenderable extends TextareaRenderable {
 
       // Set cursor to end of initial value
       if (initialValue) {
-        this.cursorOffset = initialValue.length
+        this.cursorOffset = this.editBuffer.getEOL().offset
       }
     } catch (error) {
       this.rollbackConstruction(error)
@@ -133,7 +133,7 @@ export class InputRenderable extends TextareaRenderable {
     if (currentValue !== newValue) {
       this.runMutation(() => {
         this.setText(newValue)
-        this.cursorOffset = newValue.length
+        this.cursorOffset = this.editBuffer.getEOL().offset
         this.emit(InputRenderableEvents.INPUT, newValue)
       })
     }

packages/core/src/renderables/Textarea.ts

@@ -14,9 +14,8 @@ import {
 import { StyledText, fg } from "../lib/styled-text.js"
 import type { ExtmarksController } from "../lib/extmarks.js"
 
-// Native editing rejects these, so typed and pasted text drops them. Paste keeps tab, CR, and LF.
+// A key sequence with a control character is a key that has no binding, not text. Paste keeps controls.
 const unsupportedKeyCharacters = /[\x00-\x1f\x7f-\x9f]/
-const unsupportedPasteCharacters = /[\x00-\x08\x0b\x0c\x0e-\x1f\x7f-\x9f]/g
 
 function clonePlaceholder(value: StyledText | string | null): StyledText | string | null {
   if (value === null || typeof value === "string") return value
@@ -283,7 +282,7 @@ export class TextareaRenderable extends EditBufferRenderable {
   }
 
   public handlePaste(event: PasteEvent): void {
-    this.insertText(stripAnsiSequences(decodePasteBytes(event.bytes)).replace(unsupportedPasteCharacters, ""))
+    this.insertText(stripAnsiSequences(decodePasteBytes(event.bytes)))
   }
 
   public handleKeyPress(key: KeyEvent): boolean {

packages/core/src/renderables/__tests__/Textarea.paste.test.ts

@@ -26,17 +26,20 @@ describe("Textarea - Paste Tests", () => {
     currentRenderer.destroy()
   })
 
-  // Native editing rejects C0 controls other than tab, CR, and LF, DEL, and C1, so a paste drops them instead of
-  // losing the whole paste. Input also drops CR and LF.
+  // A paste keeps control characters, as in Vim, but drops terminal escape sequences, a lone ESC, and a sequence that
+  // the paste cuts off, on Bun and Node alike. Input also drops CR and LF.
   it.each([
     ["", "x", "x"],
     [" World", "x World", "x World"],
     ["\nLine 2\nLine 3", "x\nLine 2\nLine 3", "xLine 2Line 3"],
     [" 🌟世界👍", "x 🌟世界👍", "x 🌟世界👍"],
-    ["a \x1b[31mred\x1b[0m", "xa red", "xa red"],
-    ["a\fb\u0000c", "xabc", "xabc"],
+    ["a \x1b[31mred\x1b[0m\x1b[2 q\x1b[31", "xa red", "xa red"],
+    ["a\fb\u0000c", "xa\fb\u0000c", "xa\fb\u0000c"],
+    ["\x1b]0;title\x07y\u009d0;t\u009cz\u0090q\u009c\u0098s\u009c\u009ep\u009c\u009fa\x07w", "xyzw", "xyzw"],
     ["\x1b", "x", "x"],
-    ["p\u0085q\u007f", "xpq", "xpq"],
+    ["a\x1bP1$r\x1b\\b\x1bXs\x1b\\c\x1b(Bd\x9b1me\x1b^p\x1b\\\x1b_a\x1b\\\x1b\x07", "xabcde\x07", "xabcde\x07"],
+    ["\x1b]8;;http://x\x1b\\link\x1b]8;;\x1b\\\x1b]0;cut", "xlink", "xlink"],
+    ["p\u0085q\u007f", "xp\u0085q\u007f", "xp\u0085q\u007f"],
     ["\ty\r\nz", "x\ty\nz", "x\tyz"],
   ])("pastes %j into Textarea and Input", async (pasted, textareaText, inputText) => {
     const errors = spyOn(console, "error")

packages/core/src/zig.ts

@@ -5034,6 +5034,22 @@ export class FFIRenderLib {
     )
   }
 
+  public contextEditBufferMeasureText(
+    context: NativeContextHandle,
+    editBuffer: ContextEditBufferHandle,
+    bytes: Uint8Array,
+  ): number {
+    const handle = encodeContextHandle(context, editBuffer)
+    const input = sessionBytes(bytes, "Edit buffer measure length")
+    const output = new Uint32Array(1)
+    const pointer = this.nativeContextPointer(context, "ot_edit_buffer_measure_text")
+    nativeResult(
+      "ot_edit_buffer_measure_text",
+      this.opentui.symbols.ot_edit_buffer_measure_text(pointer, handle, viewOrNull(input), input.length, output),
+    )
+    return output[0]
+  }
+
   public contextEditBufferSetDefaults(
     context: NativeContextHandle,
     editBuffer: ContextEditBufferHandle,

packages/native/include/opentui.h

@@ -1200,8 +1200,9 @@ ot_status ot_syntax_style_destroy(ot_context *, const ot_handle *style);
 /* NULL clears the binding. A non-NULL style must belong to this Context. */
 ot_status ot_edit_buffer_set_syntax_style(ot_context *, const ot_handle *edit_buffer, const ot_handle *style);
 /* Copy UTF-8 bytes into owned storage; bytes may be NULL only for zero length.
- * C0/C1/DEL controls reject except tab, CR, and LF. Queries normalize line endings
- * to LF. preserve_history is 0 (clear history) or 1 (create an undo point).
+ * Editors keep C0, DEL, and C1 controls and show them as notation: ^@..^_ and ^?
+ * in two cells, <80>..<9f> in four. Tab, CR, and LF keep their meaning. Queries
+ * normalize line endings to LF. preserve_history is 0 (clear history) or 1 (create an undo point).
  * Replacement resets the cursor to the origin; insertion uses the primary cursor.
  * No input memory remains borrowed after return. */
 ot_status ot_edit_buffer_set_text(ot_context *, const ot_handle *edit_buffer,
@@ -1275,6 +1276,12 @@ ot_status ot_edit_buffer_get_position(ot_context *, const ot_handle *, uint32_t
 ot_status ot_edit_buffer_get_range(ot_context *, const ot_handle *, uint32_t by_coords,
     uint32_t start_row, uint32_t start_col, uint32_t end_row, uint32_t end_col,
     uint8_t *bytes, uint32_t capacity, uint32_t *out_count);
+/* Lays UTF-8 bytes out as set_text would, without storing them, and reports the offset at
+ * their end: display cells under the buffer's width method, tab width, and control notation,
+ * plus one per line break (LF, CR, or CRLF). bytes may be NULL only for zero length; the
+ * byte limit and UTF-8 validation of set_text apply. Failures preserve out_width. */
+ot_status ot_edit_buffer_measure_text(ot_context *, const ot_handle *edit_buffer,
+    const uint8_t *bytes, uint32_t byte_count, uint32_t *out_width);
 
 #define OT_EDITOR_STYLE_FOREGROUND UINT32_C(1)
 #define OT_EDITOR_STYLE_BACKGROUND UINT32_C(2)

packages/native/src/bench/text-chunk-render-clusters_bench.zig

@@ -125,6 +125,7 @@ fn benchGetRenderClusters(
             arena_alloc,
             &registry,
             4, // tab width
+            .hidden,
             .unicode,
         );
         stats.record(timer.read());
@@ -196,6 +197,7 @@ fn computeBenchName(allocator: std.mem.Allocator, size: usize, text_type: TextTy
         temp_alloc,
         &registry,
         4, // tab width
+        .hidden,
         .unicode,
     );
 

packages/native/src/bench/utf8_bench.zig

@@ -312,7 +312,7 @@ fn benchFindChunkLayoutInfo(
         var stats: BenchStats = .{};
         for (0..iterations) |_| {
             const timer = bench_utils.BenchTimer.start(io);
-            const word_classes = try utf8.findChunkLayoutInfo(alloc, text, 4, true, .unicode, &wrap_breaks);
+            const word_classes = try utf8.findChunkLayoutInfo(alloc, text, 4, .hidden, true, .unicode, &wrap_breaks);
             stats.record(timer.read());
 
             if (wrap_breaks.items.len != 12_800 or word_classes.first != .ascii_word or word_classes.last != .other) {
@@ -355,7 +355,7 @@ fn benchFindWrapPosByWidth(
             var stats: BenchStats = .{};
             for (0..iterations) |_| {
                 const timer = bench_utils.BenchTimer.start(io);
-                _ = utf8.findWrapPosByWidth(text, 40, 4, true, .unicode);
+                _ = utf8.findWrapPosByWidth(text, 40, 4, .hidden, true, .unicode);
                 stats.record(timer.read());
             }
 
@@ -382,7 +382,7 @@ fn benchFindWrapPosByWidth(
             var stats: BenchStats = .{};
             for (0..iterations) |_| {
                 const timer = bench_utils.BenchTimer.start(io);
-                _ = utf8.findWrapPosByWidth(text, 120, 4, true, .unicode);
+                _ = utf8.findWrapPosByWidth(text, 120, 4, .hidden, true, .unicode);
                 stats.record(timer.read());
             }
 
@@ -409,7 +409,7 @@ fn benchFindWrapPosByWidth(
             var stats: BenchStats = .{};
             for (0..iterations) |_| {
                 const timer = bench_utils.BenchTimer.start(io);
-                _ = utf8.findWrapPosByWidth(text, 80, 4, false, .unicode);
+                _ = utf8.findWrapPosByWidth(text, 80, 4, .hidden, false, .unicode);
                 stats.record(timer.read());
             }
 
@@ -436,7 +436,7 @@ fn benchFindWrapPosByWidth(
             var stats: BenchStats = .{};
             for (0..iterations) |_| {
                 const timer = bench_utils.BenchTimer.start(io);
-                _ = utf8.findWrapPosByWidth(text, 80, 4, false, .unicode);
+                _ = utf8.findWrapPosByWidth(text, 80, 4, .hidden, false, .unicode);
                 stats.record(timer.read());
             }
 
@@ -476,7 +476,7 @@ fn benchFindPosByWidth(
             var stats: BenchStats = .{};
             for (0..iterations) |_| {
                 const timer = bench_utils.BenchTimer.start(io);
-                _ = utf8.findPosByWidth(text, 500, 4, true, true, .unicode);
+                _ = utf8.findPosByWidth(text, 500, 4, .hidden, true, true, .unicode);
                 stats.record(timer.read());
             }
 
@@ -503,7 +503,7 @@ fn benchFindPosByWidth(
             var stats: BenchStats = .{};
             for (0..iterations) |_| {
                 const timer = bench_utils.BenchTimer.start(io);
-                _ = utf8.findPosByWidth(text, 90000, 4, true, true, .unicode);
+                _ = utf8.findPosByWidth(text, 90000, 4, .hidden, true, true, .unicode);
                 stats.record(timer.read());
             }
 
@@ -530,7 +530,7 @@ fn benchFindPosByWidth(
             var stats: BenchStats = .{};
             for (0..iterations) |_| {
                 const timer = bench_utils.BenchTimer.start(io);
-                _ = utf8.findPosByWidth(text, 5000, 4, false, true, .unicode);
+                _ = utf8.findPosByWidth(text, 5000, 4, .hidden, false, true, .unicode);
                 stats.record(timer.read());
             }
 
@@ -570,7 +570,7 @@ fn benchCalculateTextWidth(
             var stats: BenchStats = .{};
             for (0..iterations) |_| {
                 const timer = bench_utils.BenchTimer.start(io);
-                _ = utf8.calculateTextWidth(text, 4, true, .unicode);
+                _ = utf8.calculateTextWidth(text, 4, .hidden, true, .unicode);
                 stats.record(timer.read());
             }
 
@@ -597,7 +597,7 @@ fn benchCalculateTextWidth(
             var stats: BenchStats = .{};
             for (0..iterations) |_| {
                 const timer = bench_utils.BenchTimer.start(io);
-                _ = utf8.calculateTextWidth(text, 4, true, .unicode);
+                _ = utf8.calculateTextWidth(text, 4, .hidden, true, .unicode);
                 stats.record(timer.read());
             }
 
@@ -624,7 +624,7 @@ fn benchCalculateTextWidth(
             var stats: BenchStats = .{};
             for (0..iterations) |_| {
                 const timer = bench_utils.BenchTimer.start(io);
-                _ = utf8.calculateTextWidth(text, 4, true, .unicode);
+                _ = utf8.calculateTextWidth(text, 4, .hidden, true, .unicode);
                 stats.record(timer.read());
             }
 
@@ -660,7 +660,7 @@ fn benchCalculateTextWidth(
             var stats: BenchStats = .{};
             for (0..iterations) |_| {
                 const timer = bench_utils.BenchTimer.start(io);
-                _ = utf8.calculateTextWidth(text.items, 4, false, .unicode);
+                _ = utf8.calculateTextWidth(text.items, 4, .hidden, false, .unicode);
                 stats.record(timer.read());
             }
 
@@ -687,7 +687,7 @@ fn benchCalculateTextWidth(
             var stats: BenchStats = .{};
             for (0..iterations) |_| {
                 const timer = bench_utils.BenchTimer.start(io);
-                _ = utf8.calculateTextWidth(text, 4, false, .unicode);
+                _ = utf8.calculateTextWidth(text, 4, .hidden, false, .unicode);
                 stats.record(timer.read());
             }
 
@@ -714,7 +714,7 @@ fn benchCalculateTextWidth(
             var stats: BenchStats = .{};
             for (0..iterations) |_| {
                 const timer = bench_utils.BenchTimer.start(io);
-                _ = utf8.calculateTextWidth(text, 4, false, .unicode);
+                _ = utf8.calculateTextWidth(text, 4, .hidden, false, .unicode);
                 stats.record(timer.read());
             }
 

packages/native/src/buffer.zig

@@ -529,7 +529,7 @@ const TextRow = struct {
             const width = if (cluster != null and target.width_method != .wcwidth)
                 cluster.?.width_cols
             else
-                utf8.getWidthAt(bytes, 0, tab_width, target.width_method);
+                utf8.getWidthAt(bytes, 0, tab_width, .hidden, target.width_method);
             if (width == 0) continue;
             const glyph_x = self.x + @as(i64, @intCast(self.advance_cells));
             self.advance_cells += width;
@@ -1828,7 +1828,7 @@ pub const OptimizedBuffer = struct {
         defer scratch_allocator.free(input);
         var clusters: std.ArrayListUnmanaged(utf8.RenderClusterInfo) = .empty;
         defer clusters.deinit(scratch_allocator);
-        try utf8.findRenderClusterInfo(scratch_allocator, input, TextRow.tab_width, utf8.isAsciiOnly(input), self.width_method, &clusters);
+        try utf8.findRenderClusterInfo(scratch_allocator, input, TextRow.tab_width, .hidden, utf8.isAsciiOnly(input), self.width_method, &clusters);
 
         const PreparedRun = struct { x: u32, char: u32, count: u32 };
         var runs: std.ArrayListUnmanaged(PreparedRun) = .empty;
@@ -1974,7 +1974,7 @@ pub const OptimizedBuffer = struct {
 
         var clusters: std.ArrayListUnmanaged(utf8.RenderClusterInfo) = .empty;
         defer clusters.deinit(self.allocator);
-        try utf8.findRenderClusterInfo(self.allocator, text, TextRow.tab_width, is_ascii_only, self.width_method, &clusters);
+        try utf8.findRenderClusterInfo(self.allocator, text, TextRow.tab_width, .hidden, is_ascii_only, self.width_method, &clusters);
 
         var layout: TextRow = .{ .target = self, .text = text, .clusters = clusters.items, .x = x, .y = @intCast(y) };
         while (layout.next()) |glyph| {
@@ -2285,7 +2285,7 @@ pub const OptimizedBuffer = struct {
         else
             0;
         const indicator_width = if (indicator_length != 0)
-            utf8.getGraphemeWidthAt(indicator_bytes[0..indicator_length], 0, text_buffer.tabWidth(), text_buffer.widthMethod())
+            utf8.getGraphemeWidthAt(indicator_bytes[0..indicator_length], 0, text_buffer.tabWidth(), text_buffer.controlDisplay(), text_buffer.widthMethod())
         else
             0;
         const text_defaults = text_buffer.defaults();
@@ -2386,7 +2386,7 @@ pub const OptimizedBuffer = struct {
             for (vline.chunks.items) |vchunk| {
                 const chunk = vchunk.chunk;
                 const chunk_bytes = chunk.getBytes(text_buffer.memRegistry());
-                const render_clusters = chunk.getRenderClusters(text_buffer.getAllocator(), text_buffer.memRegistry(), text_buffer.tabWidth(), text_buffer.widthMethod()) catch |err| {
+                const render_clusters = chunk.getRenderClusters(text_buffer.getAllocator(), text_buffer.memRegistry(), text_buffer.tabWidth(), text_buffer.controlDisplay(), text_buffer.widthMethod()) catch |err| {
                     if (checked) return err;
                     continue;
                 };
@@ -2429,7 +2429,7 @@ pub const OptimizedBuffer = struct {
                         // Sparse metadata omits printable ASCII and zero-width control characters.
                         // Printable ASCII is one cell under every width method.
                         ascii = isPrintableAscii(grapheme_bytes);
-                        cluster_width_cols = if (ascii) 1 else utf8.getWidthAt(grapheme_bytes, 0, text_buffer.tabWidth(), text_buffer.widthMethod());
+                        cluster_width_cols = if (ascii) 1 else utf8.getWidthAt(grapheme_bytes, 0, text_buffer.tabWidth(), text_buffer.controlDisplay(), text_buffer.widthMethod());
                         byte_offset = next_byte_offset;
                     }
                     // Only a glyph that this iteration draws, or a zero-width cluster, sets it again.
@@ -2663,6 +2663,9 @@ pub const OptimizedBuffer = struct {
 
                     if (is_blank) {
                         const useTransparentTextFastPath = opacity == 1.0 and ansi.alpha(drawBg) == 0;
+                        // A control is a cluster of its own. Its notation is one printable glyph per cell.
+                        const first_cp = utf8.decodeUtf8Unchecked(grapheme_bytes, 0).cp;
+                        const is_notation = utf8.controlNotationWidth(first_cp) == cluster_width_cols;
                         var tab_col: u32 = 0;
                         while (tab_col < cluster_width_cols) : (tab_col += 1) {
                             if (rendered_col_in_vline + tab_col >= horizontal_offset + viewport_width) break;
@@ -2671,7 +2674,7 @@ pub const OptimizedBuffer = struct {
                             if (tab_x >= @as(i32, @intCast(self.width))) break;
                             if (!self.isPointInScissor(tab_x, currentY)) continue;
 
-                            var char = DEFAULT_SPACE_CHAR;
+                            var char: u32 = if (is_notation) utf8.controlNotationGlyph(first_cp, tab_col) else DEFAULT_SPACE_CHAR;
                             var fg = drawFg;
                             if (is_tab and tab_col == 0 and indicator_width > 0 and indicator_width <= cluster_width_cols and
                                 @as(u64, rendered_col_in_vline) + indicator_width <= @as(u64, horizontal_offset) + viewport_width)
@@ -3345,7 +3348,7 @@ pub const OptimizedBuffer = struct {
         }
 
         const is_ascii = utf8.isAsciiOnly(text);
-        const titleLength = @as(i32, @intCast(utf8.calculateTextWidth(text, 2, is_ascii, self.width_method)));
+        const titleLength = @as(i32, @intCast(utf8.calculateTextWidth(text, 2, .hidden, is_ascii, self.width_method)));
         const minTitleSpace = 4;
 
         if (@as(i32, @intCast(width)) < titleLength + minTitleSpace) {

packages/native/src/context-abi.zig

@@ -3040,6 +3040,14 @@ test "Context editor transport copies styles selections positions history and li
     try std.testing.expectEqualStrings("\xe7\x95\x8c", bytes[0..count]);
     try std.testing.expectEqual(c.OT_OK, api.ot_edit_buffer_get_position(handle, &edit, c.OT_EDIT_POSITION_OFFSET, 3, 0, &position));
     try std.testing.expectEqual(3, position.col);
+    // a, the wide character, the tab, the line break, and z take 1 + 2 + 2 + 1 + 1 cells.
+    try std.testing.expectEqual(c.OT_OK, api.ot_edit_buffer_measure_text(handle, &edit, "a\xe7\x95\x8c\t\nz", 7, &count));
+    try std.testing.expectEqual(7, count);
+    try std.testing.expectEqual(c.OT_OK, api.ot_edit_buffer_get_position(handle, &edit, c.OT_EDIT_POSITION_COORDS, 1, 1, &position));
+    try std.testing.expectEqual(count, position.offset);
+    try std.testing.expectEqual(c.OT_OK, api.ot_edit_buffer_measure_text(handle, &edit, null, 0, &count));
+    try std.testing.expectEqual(0, count);
+    try std.testing.expectEqual(c.OT_WRONG_KIND, api.ot_edit_buffer_measure_text(handle, &view, "a", 1, &count));
     core.mutating = true;
     core.scene_measuring = true;
     try std.testing.expectEqual(c.OT_OK, api.ot_editor_view_get_info(handle, &view, 0, &info));

packages/native/src/context-editor-abi.zig

@@ -153,6 +153,13 @@ pub fn ot_edit_buffer_get_range(context: ?*Owner, id: ?*const c.ot_handle, by_co
     return c.OT_OK;
 }
 
+pub fn ot_edit_buffer_measure_text(context: ?*Owner, id: ?*const c.ot_handle, bytes: ?[*]const u8, byte_count: u32, out: ?*u32) callconv(.c) c.ot_status {
+    const owner = admit(context, true) catch |err| return fail(context, err);
+    if (id == null or out == null or (byte_count != 0 and bytes == null)) return fail(owner, error.InvalidOptions);
+    out.?.* = owner.core.editMeasureText(abi.handleFromC(id.?.*), if (bytes) |p| p[0..byte_count] else &.{}) catch |err| return fail(owner, err);
+    return c.OT_OK;
+}
+
 pub fn ot_edit_buffer_set_defaults(context: ?*Owner, id: ?*const c.ot_handle, mask: u32, options: ?*const c.ot_editor_style) callconv(.c) c.ot_status {
     const owner = admit(context, false) catch |err| return fail(context, err);
     if (id == null) return fail(owner, error.InvalidOptions);
@@ -613,7 +620,7 @@ test "Context editor ABI rejects malformed arguments without side effects" {
     style.struct_size = @sizeOf(c.ot_editor_style);
     style.abi_version = c.OT_CONTEXT_ABI_VERSION;
     const chunks = std.mem.zeroes([2]c.ot_styled_text_chunk);
-    for (0..18) |row| {
+    for (0..21) |row| {
         const status = switch (row) {
             0 => ot_edit_buffer_set_tab_width(&owner, &edit_id, 256),
             1 => ot_edit_buffer_command(&owner, &edit_id, c.OT_EDIT_DEBUG_ROPE + 1, 0),
@@ -622,10 +629,10 @@ test "Context editor ABI rejects malformed arguments without side effects" {
             4 => ot_edit_buffer_history(&owner, &edit_id, 0, &bytes, bytes.len - 1, &count),
             5 => abi.ot_edit_buffer_set_text(&owner, &edit_id, "x", 1, 2),
             6 => abi.ot_edit_buffer_set_text(&owner, &edit_id, null, 1, 0),
-            // C0, C1 and DEL bytes other than tab, CR and LF are rejected whole.
-            7 => abi.ot_edit_buffer_set_text(&owner, &edit_id, "a\x0cb", 3, 1),
-            8 => abi.ot_edit_buffer_insert_text(&owner, &edit_id, "a\x00", 2),
-            9 => abi.ot_edit_buffer_insert_text(&owner, &edit_id, "\xc2\x85", 2),
+            // Controls are text, but invalid UTF-8 beside them rejects the whole edit.
+            7 => abi.ot_edit_buffer_set_text(&owner, &edit_id, "a\x0c\xff", 3, 1),
+            8 => abi.ot_edit_buffer_insert_text(&owner, &edit_id, "\x00\xc2", 2),
+            9 => ot_editor_view_set_placeholder(&owner, &view_id, "\xc2\x85\x80", 3, &chunks, 0),
             10 => ot_edit_buffer_get_position(&owner, &edit_id, c.OT_EDIT_POSITION_LINE_START + 1, 0, 0, &position),
             11 => ot_edit_buffer_get_range(&owner, &edit_id, 2, 0, 0, 0, 1, &bytes, bytes.len, &count),
             12 => ot_edit_buffer_set_defaults(&owner, &edit_id, 0, &style),
@@ -634,6 +641,9 @@ test "Context editor ABI rejects malformed arguments without side effects" {
             15 => ot_editor_view_command(&owner, &view_id, c.OT_EDITOR_WRAP_MODE, c.OT_SCENE_WRAP_WORD + 1),
             16 => ot_editor_view_replace_selection(&owner, &view_id, "x", 1, null),
             17 => ot_editor_view_set_placeholder(&owner, &view_id, "a", 1, &chunks, chunks.len),
+            18 => ot_editor_view_replace_selection(&owner, &view_id, "\x1b\xff", 2, &count),
+            19 => ot_edit_buffer_measure_text(&owner, &edit_id, null, 1, &count),
+            20 => ot_edit_buffer_measure_text(&owner, &edit_id, "\x85", 1, &count),
             else => unreachable,
         };
         try std.testing.expectEqual(c.OT_INVALID_ARGUMENT, status);

packages/native/src/context.zig

@@ -2364,23 +2364,13 @@ pub const Context = struct {
     }
 
     pub fn validateTextBytes(text: *const text_buffer.UnifiedTextBuffer, bytes: []const u8, retained_bytes: u32) !void {
-        // Rope columns and line counts are u32. Tabs are the largest cell/byte
-        // expansion; reserve one count for the trailing logical line.
-        const bytes_max = (std.math.maxInt(u32) - 1) / @as(u32, @max(text.tabWidth(), 1));
+        // Rope columns and line counts are u32. Tabs and control notation (two cells per byte)
+        // are the largest cell/byte expansion; reserve one count for the trailing logical line.
+        const bytes_max = (std.math.maxInt(u32) - 1) / @as(u32, @max(text.tabWidth(), 2));
         if (retained_bytes > bytes_max or bytes.len > bytes_max - retained_bytes) return error.TextLimit;
         if (!std.unicode.utf8ValidateSlice(bytes)) return error.InvalidUnicode;
     }
 
-    fn validateEditTextBytes(text: *const text_buffer.UnifiedTextBuffer, bytes: []const u8, retained_bytes: u32) !void {
-        try validateTextBytes(text, bytes, retained_bytes);
-        // Cell-based editing cannot navigate standalone zero-width controls yet.
-        var codepoints = std.unicode.Utf8View.initUnchecked(bytes).iterator();
-        while (codepoints.nextCodepoint()) |codepoint| {
-            if ((codepoint < 0x20 and codepoint != '\t' and codepoint != '\r' and codepoint != '\n') or
-                (codepoint >= 0x7f and codepoint <= 0x9f)) return error.InvalidUnicode;
-        }
-    }
-
     pub fn sceneSetStyledText(self: *Context, handle: Handle, bytes: []const u8, chunks: []const StyledTextChunk) !void {
         try self.beginMutation();
         defer self.mutating = false;
@@ -2662,6 +2652,19 @@ pub const Context = struct {
         return buffer.copyTextRange(offsets.start, offsets.end, out);
     }
 
+    /// Lays `bytes` out as setText would, without storing them, and returns the cursor offset at their end.
+    pub fn editMeasureText(self: *Context, handle: Handle, bytes: []const u8) !u32 {
+        try self.checkSceneRead();
+        const buffer = (try self.getEditBuffer(handle)).buffer.tb;
+        // The byte limit keeps the widths and line breaks within u32.
+        try validateTextBytes(buffer, bytes, 0);
+        var layout = try buffer.textToSegments(self.allocator, bytes, 0, 0, false);
+        defer layout.segments.deinit(layout.allocator);
+        var break_count: u32 = 0;
+        for (layout.segments.items) |*segment| break_count += @intFromBool(segment.isBreak());
+        return layout.total_width_cols + break_count;
+    }
+
     pub fn textViewGetSelectedText(self: *Context, handle: Handle, out: []u8) !u32 {
         try self.checkSceneRead();
         const view = try self.getTextBufferView(handle);
@@ -3393,7 +3396,7 @@ pub const Context = struct {
         defer self.mutating = false;
         const edit = try self.getEditBuffer(handle);
         try edit.checkMutable();
-        try validateEditTextBytes(edit.buffer.tb, bytes, 0);
+        try validateTextBytes(edit.buffer.tb, bytes, 0);
         if (preserve_history) {
             try edit.buffer.replaceText(bytes);
         } else {
@@ -3406,7 +3409,7 @@ pub const Context = struct {
         defer self.mutating = false;
         const edit = try self.getEditBuffer(handle);
         try edit.checkMutable();
-        try validateEditTextBytes(edit.buffer.tb, bytes, edit.buffer.tb.getByteSize());
+        try validateTextBytes(edit.buffer.tb, bytes, edit.buffer.tb.getByteSize());
         try edit.buffer.insertText(bytes);
     }
 
@@ -3445,7 +3448,7 @@ pub const Context = struct {
             .delete_forward => try edit.buffer.deleteForward(),
             .backspace => try edit.buffer.backspace(),
             .new_line => {
-                try validateEditTextBytes(edit.buffer.tb, "\n", edit.buffer.tb.getByteSize());
+                try validateTextBytes(edit.buffer.tb, "\n", edit.buffer.tb.getByteSize());
                 try edit.buffer.insertText("\n");
             },
             .delete_line => try edit.buffer.deleteLine(),
@@ -3554,7 +3557,7 @@ pub const Context = struct {
         defer self.mutating = false;
         const view = try self.getEditorView(handle);
         try view.edit.checkMutable();
-        try validateEditTextBytes(view.edit.buffer.tb, bytes, view.edit.buffer.tb.getByteSize());
+        try validateTextBytes(view.edit.buffer.tb, bytes, view.edit.buffer.tb.getByteSize());
         const steps = try view.view.replaceSelectedText(bytes);
         view.invalidate();
         return steps;
@@ -3565,7 +3568,7 @@ pub const Context = struct {
         defer self.mutating = false;
         const view = try self.getEditorView(handle);
         try view.edit.checkMutable();
-        try validateEditTextBytes(view.edit.buffer.tb, bytes, 0);
+        try validateTextBytes(view.edit.buffer.tb, bytes, 0);
         var offset: usize = 0;
         for (chunks) |chunk| {
             if (chunk.byte_count == 0 or chunk.byte_count > bytes.len - offset or

packages/native/src/edit-buffer.zig

@@ -195,7 +195,10 @@ pub const EditBuffer = struct {
         const self = try allocator.create(EditBuffer);
         errdefer allocator.destroy(self);
 
-        const text_buffer = try UnifiedTextBuffer.initWithOptions(allocator, pool, link_pool, width_method, options);
+        // A cursor that moves by cell cannot stop on a zero-width control, so editors show notation.
+        var text_options = options;
+        text_options.control_display = .notation;
+        const text_buffer = try UnifiedTextBuffer.initWithOptions(allocator, pool, link_pool, width_method, text_options);
         errdefer text_buffer.deinit();
 
         const add_buffer = try AddBuffer.init(allocator, text_buffer, 65536);
@@ -273,12 +276,13 @@ pub const EditBuffer = struct {
             const chunk = seg.asText() orelse continue;
             const bytes = chunk.getBytes(self.tb.memRegistry());
             // The restored root's cached widths use the current policy, not the checkpoint's.
-            const next_col = old_col +| utf8.calculateTextWidth(bytes, old_tab_width, chunk.isAsciiOnly(), self.tb.widthMethod());
+            const next_col = old_col +| utf8.calculateTextWidth(bytes, old_tab_width, self.tb.controlDisplay(), chunk.isAsciiOnly(), self.tb.widthMethod());
             if (target_col <= next_col) {
                 const pos = utf8.findPosByWidth(
                     bytes,
                     target_col -| old_col,
                     old_tab_width,
+                    self.tb.controlDisplay(),
                     chunk.isAsciiOnly(),
                     false,
                     self.tb.widthMethod(),
@@ -286,6 +290,7 @@ pub const EditBuffer = struct {
                 return new_col +| utf8.calculateTextWidth(
                     bytes[0..pos.byte_offset],
                     self.tb.tabWidth(),
+                    self.tb.controlDisplay(),
                     chunk.isAsciiOnly(),
                     self.tb.widthMethod(),
                 );
@@ -339,7 +344,7 @@ pub const EditBuffer = struct {
         const chunk_bytes = chunk.getBytes(self.tb.memRegistry());
         const is_ascii_only = (chunk.flags & TextChunk.Flags.ASCII_ONLY) != 0;
 
-        const result = utf8.findPosByWidth(chunk_bytes, weight, self.tb.tabWidth(), is_ascii_only, false, self.tb.widthMethod());
+        const result = utf8.findPosByWidth(chunk_bytes, weight, self.tb.tabWidth(), self.tb.controlDisplay(), is_ascii_only, false, self.tb.widthMethod());
         const split_byte_offset = result.byte_offset;
 
         const left_chunk = self.tb.createChunk(
@@ -710,7 +715,7 @@ pub const EditBuffer = struct {
     /// cursor does not land on the second cell of a wide unit.
     pub fn snapColumn(self: *EditBuffer, row: u32, col: u32) u32 {
         const text = self.tb;
-        const bounds = iter_mod.getCursorUnitBoundsAt(text.rope(), text.memRegistry(), row, col, text.tabWidth(), text.widthMethod());
+        const bounds = iter_mod.getCursorUnitBoundsAt(text.rope(), text.memRegistry(), row, col, text.tabWidth(), text.controlDisplay(), text.widthMethod());
         return if (bounds) |unit| unit.start else col;
     }
 

packages/native/src/editor-view.zig

@@ -931,7 +931,7 @@ pub const EditorView = struct {
             self.edit_buffer.tb.pool,
             self.edit_buffer.tb.link_pool,
             self.edit_buffer.tb.width_method,
-            .{ .io = self.edit_buffer.tb.io, .logger = self.edit_buffer.tb.logger },
+            .{ .io = self.edit_buffer.tb.io, .logger = self.edit_buffer.tb.logger, .control_display = self.edit_buffer.tb.control_display },
         );
         errdefer if (self.placeholder_buffer == null) placeholder.deinit();
         const mem_id = try placeholder.replaceOwnedStyledText(

packages/native/src/encoded-unicode.zig

@@ -21,7 +21,7 @@ pub const EncodedUnicode = struct {
         var clusters: std.ArrayListUnmanaged(utf8.RenderClusterInfo) = .empty;
         defer clusters.deinit(allocator);
         const tab_width = 2;
-        try utf8.findRenderClusterInfo(allocator, input, tab_width, utf8.isAsciiOnly(input), width_method, &clusters);
+        try utf8.findRenderClusterInfo(allocator, input, tab_width, .hidden, utf8.isAsciiOnly(input), width_method, &clusters);
         var result: std.ArrayListUnmanaged(EncodedChar) = .empty;
         errdefer {
             release(pool, result.items);
@@ -41,7 +41,7 @@ pub const EncodedUnicode = struct {
                 byte_offset += std.unicode.utf8ByteSequenceLength(input[start]) catch unreachable;
             }
             const bytes = input[start..byte_offset];
-            const width = utf8.getWidthAt(bytes, 0, tab_width, width_method);
+            const width = utf8.getWidthAt(bytes, 0, tab_width, .hidden, width_method);
             if (width == 0) continue;
             if (!buffer.isPrintableGlyph(bytes)) {
                 result.appendNTimesAssumeCapacity(.{ .width = 1, .char = buffer.DEFAULT_SPACE_CHAR }, width);

packages/native/src/tests/buffer_test.zig

@@ -952,14 +952,15 @@ test "OptimizedBuffer drawTextChecked skips complete zero width UTF-8 codepoints
 }
 
 /// A cluster that a cell cannot hold and the text that every checked draw must render in its place.
-const UnprintableText = struct { bytes: []const u8, blank: []const u8, line_break: bool = false };
+/// An editor shows a control as notation instead.
+const UnprintableText = struct { bytes: []const u8, blank: []const u8, notation: ?[]const u8 = null, line_break: bool = false };
 const unprintable_texts = [_]UnprintableText{
-    .{ .bytes = "\x00", .blank = "" },
-    .{ .bytes = "\x07", .blank = "" },
-    .{ .bytes = "\x1b", .blank = "" },
-    .{ .bytes = "\x7f", .blank = "" },
-    .{ .bytes = "\u{85}", .blank = "" },
-    .{ .bytes = "\u{9b}", .blank = "" },
+    .{ .bytes = "\x00", .blank = "", .notation = "^@" },
+    .{ .bytes = "\x07", .blank = "", .notation = "^G" },
+    .{ .bytes = "\x1b", .blank = "", .notation = "^[" },
+    .{ .bytes = "\x7f", .blank = "", .notation = "^?" },
+    .{ .bytes = "\u{85}", .blank = "", .notation = "<85>" },
+    .{ .bytes = "\u{9b}", .blank = "", .notation = "<9b>" },
     .{ .bytes = "\n", .blank = "", .line_break = true },
     .{ .bytes = "\r", .blank = "", .line_break = true },
     .{ .bytes = "\r\n", .blank = "", .line_break = true },
@@ -1020,7 +1021,7 @@ fn expectNoControlCells(target: *OptimizedBuffer) !void {
     }
 }
 
-test "OptimizedBuffer checked text draws skip controls and blank clusters a cell cannot hold" {
+test "OptimizedBuffer checked text draws skip controls, editors show their notation, and blank clusters a cell cannot hold" {
     var pools = TestPools.init(std.testing.allocator);
     defer pools.deinit();
     // The blank rule starts where the pool stops storing a cluster.
@@ -1048,7 +1049,8 @@ test "OptimizedBuffer checked text draws skip controls and blank clusters a cell
                     .end => .{ "abcd", "" },
                 };
                 const with = try std.fmt.bufPrint(&with_buffer, "{s}{s}{s}", .{ prefix, unprintable.bytes, suffix });
-                const blank = try std.fmt.bufPrint(&blank_buffer, "{s}{s}{s}", .{ prefix, unprintable.blank, suffix });
+                const shown = if (path == .editor_view) unprintable.notation orelse unprintable.blank else unprintable.blank;
+                const blank = try std.fmt.bufPrint(&blank_buffer, "{s}{s}{s}", .{ prefix, shown, suffix });
                 for (clips) |clip| {
                     errdefer std.debug.print("path={t} text={any} position={t} clip={any}\n", .{ path, unprintable.bytes, position, clip });
                     for ([_]*OptimizedBuffer{ actual, expected }) |target| {

packages/native/src/tests/editor-view_test.zig

@@ -1485,7 +1485,7 @@ test "EditorView - placeholder with styled text" {
     try std.testing.expectEqual(@as(u32, 11), placeholder.getLength());
 }
 
-test "EditorView - placeholder renders to buffer when empty" {
+test "EditorView - placeholder renders to buffer when empty, with control notation" {
     var pools = TestPools.init(std.testing.allocator);
     defer pools.deinit();
 
@@ -1495,10 +1495,10 @@ test "EditorView - placeholder renders to buffer when empty" {
     var ev = try EditorView.init(std.testing.allocator, eb, 80, 10);
     defer ev.deinit();
 
-    try owned_styled.setPlaceholder(ev, &.{.{
-        .text = "Type something...",
-        .fg = ansi.rgbaFromFloats(0.5, 0.5, 0.5, 1.0),
-    }});
+    try owned_styled.setPlaceholder(ev, &.{
+        .{ .text = "Type\x1b\u{85}", .fg = ansi.rgbaFromFloats(0.5, 0.5, 0.5, 1.0) },
+        .{ .text = " something...", .fg = ansi.rgbaFromFloats(0.9, 0.9, 0.9, 1.0) },
+    });
 
     try std.testing.expect(ev.placeholder_buffer != null);
     try std.testing.expect(ev.placeholder_active);
@@ -1518,7 +1518,9 @@ test "EditorView - placeholder renders to buffer when empty" {
     const written = try opt_buffer.writeResolvedChars(&out_buffer, false);
     const result = out_buffer[0..written];
 
-    try std.testing.expect(std.mem.startsWith(u8, result, "Type something..."));
+    try std.testing.expect(std.mem.startsWith(u8, result, "Type^[<85> something..."));
+    // Style chunks measure the notation too, so the second style starts after it.
+    try std.testing.expect(!opt_buffer_mod.rgbaEqual(opt_buffer.buffer.fg[9], opt_buffer.buffer.fg[10]));
 
     try eb.insertText("Hello");
 
@@ -1530,7 +1532,7 @@ test "EditorView - placeholder renders to buffer when empty" {
     const result2 = out_buffer[0..written2];
 
     try std.testing.expect(std.mem.startsWith(u8, result2, "Hello"));
-    try std.testing.expect(!std.mem.startsWith(u8, result2, "Type something..."));
+    try std.testing.expect(!std.mem.startsWith(u8, result2, "Type^[<85> something..."));
 }
 
 test "EditorView - placeholder shrink clears tail and preserves background" {
@@ -2181,3 +2183,160 @@ test "EditorView - convert word selection to cell keeps the range and moves focu
     try std.testing.expectEqual(@as(u32, 0), converted_cursor.row);
     try std.testing.expectEqual(@as(u32, 9), converted_cursor.col);
 }
+
+test "EditorView - controls show Vim notation that the cursor stops on and deletes as one unit" {
+    var pools = TestPools.init(std.testing.allocator);
+    defer pools.deinit();
+    for (std.enums.values(utf8.WidthMethod)) |width_method| {
+        var codepoint: u21 = 0;
+        while (codepoint <= 0xa0) : (codepoint += 1) {
+            // Line feed and carriage return break lines; they are not cells.
+            if (codepoint == '\n' or codepoint == '\r') continue;
+            errdefer std.debug.print("width_method={t} codepoint=U+{X:0>4}\n", .{ width_method, codepoint });
+            var input: [4]u8 = undefined;
+            input[0] = 'a';
+            const length = try std.unicode.utf8Encode(codepoint, input[1..]);
+            input[1 + length] = 'b';
+            const text = input[0 .. length + 2];
+            const character = text[1 .. length + 1];
+            var shown_buffer: [4]u8 = undefined;
+            const shown = switch (codepoint) {
+                '\t' => "  ",
+                0...0x08, 0x0b...0x1f => try std.fmt.bufPrint(&shown_buffer, "^{c}", .{@as(u8, @intCast(codepoint + 0x40))}),
+                0x7f => "^?",
+                0x80...0x9f => try std.fmt.bufPrint(&shown_buffer, "<{x:0>2}>", .{codepoint}),
+                else => character,
+            };
+            const width: u32 = if (codepoint == 0xa0) 1 else @intCast(shown.len);
+            const eb = try EditBuffer.init(std.testing.allocator, &pools.graphemes, &pools.links, width_method, null);
+            defer eb.deinit();
+            const ev = try EditorView.init(std.testing.allocator, eb, 8, 1);
+            defer ev.deinit();
+            try eb.setText(text);
+            var frame = try opt_buffer_mod.OptimizedBuffer.init(std.testing.allocator, 8, 1, .{ .link_pool = &pools.links, .pool = &pools.graphemes, .width_method = width_method });
+            defer frame.deinit();
+            frame.clear(ansi.rgbColor(0, 0, 0, 255), null);
+            try frame.drawEditorViewChecked(ev, 0, 0);
+            var frame_bytes: [32]u8 = undefined;
+            var expected_frame: [32]u8 = undefined;
+            const expected = try std.fmt.bufPrint(&expected_frame, "a{s}b{s}", .{ shown, "      "[width..] });
+            try std.testing.expectEqualStrings(expected, frame_bytes[0..try frame.writeResolvedChars(&frame_bytes, false)]);
+
+            // The cursor stops before and after the unit, and copying its cells returns its bytes.
+            const stops = [_]u32{ 0, 1, 1 + width, 2 + width };
+            for (stops[1..]) |stop| {
+                eb.moveRight();
+                try std.testing.expectEqual(stop, eb.getPrimaryCursor().col);
+            }
+            var index: usize = stops.len - 1;
+            while (index > 0) : (index -= 1) {
+                eb.moveLeft();
+                try std.testing.expectEqual(stops[index - 1], eb.getPrimaryCursor().col);
+            }
+            var copied: [8]u8 = undefined;
+            ev.setSelection(1, 1 + width, null, null);
+            try std.testing.expectEqualStrings(character, copied[0..ev.getSelectedTextIntoBuffer(&copied)]);
+
+            // Each delete removes exactly the unit, and history restores the original bytes.
+            try eb.setCursor(0, 1);
+            try eb.deleteForward();
+            try std.testing.expectEqualStrings("ab", copied[0..eb.getText(&copied)]);
+            _ = try eb.undo();
+            try std.testing.expectEqualStrings(text, copied[0..eb.getText(&copied)]);
+            try eb.setCursor(0, 1 + width);
+            try eb.backspace();
+            try std.testing.expectEqualStrings("ab", copied[0..eb.getText(&copied)]);
+            _ = try eb.undo();
+            _ = try eb.redo();
+            try std.testing.expectEqualStrings("ab", copied[0..eb.getText(&copied)]);
+            _ = try eb.undo();
+            try std.testing.expectEqualStrings(text, copied[0..eb.getText(&copied)]);
+            if (codepoint == '\t' or width == 1) continue;
+
+            // Word and inclusive selections take the whole unit, and a move up from inside it snaps to its start.
+            for (0..2 + width) |x| {
+                _ = ev.setLocalSelectionBehavior(@intCast(x), 0, @intCast(x), 0, null, null, false, .word);
+                try std.testing.expectEqualStrings(text, copied[0..ev.getSelectedTextIntoBuffer(&copied)]);
+            }
+            ev.setSelectionInclusive(width, width, null, null);
+            try std.testing.expectEqualStrings(character, copied[0..ev.getSelectedTextIntoBuffer(&copied)]);
+            try eb.setCursor(0, 2 + width);
+            try eb.insertText("\nabcdef");
+            try eb.setCursor(1, width);
+            eb.moveUp();
+            try std.testing.expectEqual(@as(u32, 1), eb.getPrimaryCursor().col);
+        }
+    }
+}
+
+test "EditorView - wrapping never splits control notation" {
+    var pools = TestPools.init(std.testing.allocator);
+    defer pools.deinit();
+    // The first 16 bytes are ASCII, so layout takes its vector path.
+    const text = "ab\x0c-defgh\x07ij-kl\x00m\u{85}c\x00d-e\x7f";
+    const shown = "ab^L-defgh^Gij-kl^@m<85>c^@d-e^?";
+    // Cell offsets in `shown` where a unit ends; a row may end only there.
+    const unit_ends = [_]usize{ 1, 2, 4, 5, 6, 7, 8, 9, 10, 12, 13, 14, 15, 16, 17, 19, 20, 24, 25, 27, 28, 29, 30, 32 };
+    for ([_]text_buffer.WrapMode{ .char, .word }) |wrap_mode| {
+        // The widest unit, `<85>`, needs four cells.
+        for (4..shown.len + 1) |width| {
+            errdefer std.debug.print("wrap_mode={t} width={d}\n", .{ wrap_mode, width });
+            const eb = try EditBuffer.init(std.testing.allocator, &pools.graphemes, &pools.links, .unicode, null);
+            defer eb.deinit();
+            const ev = try EditorView.init(std.testing.allocator, eb, @intCast(width), shown.len);
+            defer ev.deinit();
+            ev.setWrapMode(wrap_mode);
+            try eb.setText(text);
+            var frame = try opt_buffer_mod.OptimizedBuffer.init(std.testing.allocator, @intCast(width), shown.len, .{ .link_pool = &pools.links, .pool = &pools.graphemes });
+            defer frame.deinit();
+            frame.clear(ansi.rgbColor(0, 0, 0, 255), null);
+            try frame.drawEditorViewChecked(ev, 0, 0);
+            var frame_bytes: [shown.len * (shown.len + 2)]u8 = undefined;
+            var rows = std.mem.splitScalar(u8, frame_bytes[0..try frame.writeResolvedChars(&frame_bytes, true)], '\n');
+            var joined: [shown.len]u8 = undefined;
+            var joined_len: usize = 0;
+            while (rows.next()) |row| {
+                const content = std.mem.trimEnd(u8, row, " ");
+                if (content.len == 0) continue;
+                @memcpy(joined[joined_len..][0..content.len], content);
+                joined_len += content.len;
+                try std.testing.expect(std.mem.indexOfScalar(usize, &unit_ends, joined_len) != null);
+            }
+            try std.testing.expectEqualStrings(shown, joined[0..joined_len]);
+        }
+    }
+}
+
+test "EditorView - cursor moves and backspace step over zero-width clusters" {
+    var pools = TestPools.init(std.testing.allocator);
+    defer pools.deinit();
+    // UAX #29 keeps a zero-width format character, or a mark after a control, in a cluster of its own.
+    // U+FFFD ends an Indic conjunct or emoji ZWJ sequence, so the characters after it do not join across it.
+    // The last text starts with 16 ASCII bytes, so layout takes its vector path.
+    const texts = [_][]const u8{ "a\u{200b}b", "a\u{feff}\u{2060}b", "a\x07\u{301}b", "\x1b\u{200d}\x1b", "日\u{fffd}", "क्\u{fffd}\u{301}ष", "👩\u{200d}\u{fffd}\u{200d}👩", "ab\x0c-defgh\x07ij-kl\x00m-n\x1b" };
+    for (std.enums.values(utf8.WidthMethod)) |width_method| {
+        for (texts) |text| {
+            errdefer std.debug.print("width_method={t} text={any}\n", .{ width_method, text });
+            const eb = try EditBuffer.init(std.testing.allocator, &pools.graphemes, &pools.links, width_method, null);
+            defer eb.deinit();
+            try eb.setText(text);
+            const end = eb.getTextBuffer().lineWidthAt(0);
+            var col: u32 = 0;
+            while (col < end) : (col = eb.getPrimaryCursor().col) {
+                eb.moveRight();
+                try std.testing.expect(eb.getPrimaryCursor().col > col);
+            }
+            while (col > 0) : (col = eb.getPrimaryCursor().col) {
+                eb.moveLeft();
+                try std.testing.expect(eb.getPrimaryCursor().col < col);
+            }
+            try eb.setCursor(0, end);
+            var bytes = text.len;
+            var out: [32]u8 = undefined;
+            while (bytes > 0) : (bytes = eb.getText(&out)) {
+                try eb.backspace();
+                try std.testing.expect(eb.getText(&out) < bytes);
+            }
+        }
+    }
+}

packages/native/src/tests/text-buffer-drawing_test.zig

@@ -2129,7 +2129,7 @@ test "drawTextBuffer - wide tab indicator occupies whole cells without moving se
         var output: [32]u8 = undefined;
         const written = try target.writeResolvedChars(&output, false);
         try std.testing.expectEqualStrings(case.text, output[0..written]);
-        try std.testing.expectEqual(@as(u32, 8), @import("../utf8.zig").calculateTextWidth(output[0..written], 2, false, .unicode));
+        try std.testing.expectEqual(@as(u32, 8), @import("../utf8.zig").calculateTextWidth(output[0..written], 2, .hidden, false, .unicode));
         try std.testing.expectEqual(selected, target.get(1, 0).?.bg);
         if (case.width == 8) {
             try std.testing.expectEqual(@as(u32, 'B'), target.get(3, 0).?.char);

packages/native/src/tests/text-buffer-iterators_test.zig

@@ -191,13 +191,13 @@ test "getGraphemeWidthAt - ASCII text" {
 
     try tb.setText("Hello");
 
-    try testing.expectEqual(@as(u32, 1), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 0, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 1), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 1, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 1), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 2, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 1), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 3, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 1), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 4, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 0), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 5, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 0), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 10, tb.tabWidth(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 1), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 0, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 1), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 1, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 1), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 2, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 1), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 3, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 1), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 4, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 0), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 5, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 0), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 10, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
 }
 
 test "getGraphemeWidthAt - emoji and wide characters" {
@@ -209,10 +209,10 @@ test "getGraphemeWidthAt - emoji and wide characters" {
 
     try tb.setText("a😀b");
 
-    try testing.expectEqual(@as(u32, 1), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 0, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 2), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 1, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 1), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 3, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 0), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 4, tb.tabWidth(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 1), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 0, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 2), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 1, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 1), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 3, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 0), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 4, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
 }
 
 test "getGraphemeWidthAt - multiple chunks" {
@@ -224,12 +224,12 @@ test "getGraphemeWidthAt - multiple chunks" {
 
     try tb.setText("Hello World");
 
-    try testing.expectEqual(@as(u32, 1), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 0, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 1), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 4, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 1), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 5, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 1), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 6, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 1), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 10, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 0), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 11, tb.tabWidth(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 1), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 0, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 1), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 4, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 1), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 5, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 1), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 6, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 1), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 10, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 0), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 11, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
 }
 
 test "getGraphemeWidthAt - empty line" {
@@ -241,7 +241,7 @@ test "getGraphemeWidthAt - empty line" {
 
     try tb.setText("");
 
-    try testing.expectEqual(@as(u32, 0), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 0, tb.tabWidth(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 0), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 0, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
 }
 
 test "getGraphemeWidthAt - at chunk boundary" {
@@ -253,7 +253,7 @@ test "getGraphemeWidthAt - at chunk boundary" {
 
     try tb.setText("abcdef");
 
-    try testing.expectEqual(@as(u32, 1), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 3, tb.tabWidth(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 1), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 3, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
 }
 
 test "getGraphemeWidthAt - after break segment" {
@@ -265,9 +265,9 @@ test "getGraphemeWidthAt - after break segment" {
 
     try tb.setText("abc\ndef");
 
-    try testing.expectEqual(@as(u32, 1), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 0, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 0), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 3, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 1), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 1, 0, tb.tabWidth(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 1), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 0, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 0), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 3, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 1), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 1, 0, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
 }
 
 test "getPrevGraphemeWidth - ASCII text" {
@@ -279,12 +279,12 @@ test "getPrevGraphemeWidth - ASCII text" {
 
     try tb.setText("Hello");
 
-    try testing.expectEqual(@as(u32, 0), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 0, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 1), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 1, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 1), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 2, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 1), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 3, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 1), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 4, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 1), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 5, tb.tabWidth(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 0), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 0, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 1), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 1, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 1), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 2, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 1), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 3, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 1), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 4, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 1), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 5, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
 }
 
 test "getPrevGraphemeWidth - emoji and wide characters" {
@@ -296,10 +296,10 @@ test "getPrevGraphemeWidth - emoji and wide characters" {
 
     try tb.setText("a😀b");
 
-    try testing.expectEqual(@as(u32, 0), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 0, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 1), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 1, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 2), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 3, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 1), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 4, tb.tabWidth(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 0), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 0, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 1), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 1, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 2), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 3, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 1), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 4, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
 }
 
 test "getPrevGraphemeWidth - at chunk boundary" {
@@ -311,9 +311,9 @@ test "getPrevGraphemeWidth - at chunk boundary" {
 
     try tb.setText("abcdef");
 
-    try testing.expectEqual(@as(u32, 1), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 3, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 1), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 4, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 1), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 5, tb.tabWidth(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 1), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 3, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 1), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 4, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 1), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 5, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
 }
 
 test "getPrevGraphemeWidth - emoji at chunk boundary" {
@@ -325,7 +325,7 @@ test "getPrevGraphemeWidth - emoji at chunk boundary" {
 
     try tb.setText("a😀b");
 
-    try testing.expectEqual(@as(u32, 2), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 3, tb.tabWidth(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 2), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 3, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
 }
 
 test "getPrevGraphemeWidth - multiple chunks" {
@@ -337,10 +337,10 @@ test "getPrevGraphemeWidth - multiple chunks" {
 
     try tb.setText("Hello 😀");
 
-    try testing.expectEqual(@as(u32, 1), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 1, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 1), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 5, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 1), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 6, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 2), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 8, tb.tabWidth(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 1), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 1, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 1), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 5, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 1), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 6, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 2), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 8, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
 }
 
 test "getPrevGraphemeWidth - empty line" {
@@ -352,7 +352,7 @@ test "getPrevGraphemeWidth - empty line" {
 
     try tb.setText("");
 
-    try testing.expectEqual(@as(u32, 0), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 0, tb.tabWidth(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 0), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 0, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
 }
 
 test "getPrevGraphemeWidth - col beyond line width" {
@@ -364,7 +364,7 @@ test "getPrevGraphemeWidth - col beyond line width" {
 
     try tb.setText("abc");
 
-    try testing.expectEqual(@as(u32, 1), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 100, tb.tabWidth(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 1), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 100, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
 }
 
 test "getPrevGraphemeWidth - multiline" {
@@ -376,11 +376,11 @@ test "getPrevGraphemeWidth - multiline" {
 
     try tb.setText("abc\n😀xyz");
 
-    try testing.expectEqual(@as(u32, 0), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 0, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 1), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 3, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 0), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 1, 0, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 2), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 1, 2, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 1), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 1, 3, tb.tabWidth(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 0), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 0, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 1), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 3, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 0), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 1, 0, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 2), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 1, 2, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 1), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 1, 3, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
 }
 
 test "getGraphemeWidthAt - CJK characters (Chinese)" {
@@ -392,11 +392,11 @@ test "getGraphemeWidthAt - CJK characters (Chinese)" {
 
     try tb.setText("a世界b");
 
-    try testing.expectEqual(@as(u32, 1), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 0, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 2), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 1, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 2), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 3, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 1), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 5, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 0), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 6, tb.tabWidth(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 1), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 0, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 2), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 1, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 2), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 3, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 1), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 5, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 0), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 6, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
 }
 
 test "getGraphemeWidthAt - various emoji including star" {
@@ -408,9 +408,9 @@ test "getGraphemeWidthAt - various emoji including star" {
 
     try tb.setText("🌟🎉");
 
-    try testing.expectEqual(@as(u32, 2), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 0, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 2), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 2, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 0), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 4, tb.tabWidth(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 2), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 0, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 2), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 2, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 0), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 4, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
 }
 
 test "getGraphemeWidthAt - tab characters" {
@@ -423,12 +423,12 @@ test "getGraphemeWidthAt - tab characters" {
 
     try tb.setText("a\tb\t\tc");
 
-    try testing.expectEqual(@as(u32, 1), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 0, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 4), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 1, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 1), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 5, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 4), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 6, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 4), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 10, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 1), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 14, tb.tabWidth(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 1), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 0, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 4), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 1, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 1), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 5, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 4), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 6, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 4), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 10, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 1), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 14, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
 }
 
 test "getGraphemeWidthAt - tab with different tab_width" {
@@ -441,10 +441,10 @@ test "getGraphemeWidthAt - tab with different tab_width" {
     try tb.setText("x\ty");
 
     tb.setTabWidth(2);
-    try testing.expectEqual(@as(u32, 2), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 1, 2, .unicode));
+    try testing.expectEqual(@as(u32, 2), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 1, 2, .hidden, .unicode));
 
     tb.setTabWidth(8);
-    try testing.expectEqual(@as(u32, 8), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 1, 8, .unicode));
+    try testing.expectEqual(@as(u32, 8), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 1, 8, .hidden, .unicode));
 }
 
 test "getGraphemeWidthAt - middle of wide character" {
@@ -456,11 +456,11 @@ test "getGraphemeWidthAt - middle of wide character" {
 
     try tb.setText("世");
 
-    try testing.expectEqual(@as(u32, 2), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 0, tb.tabWidth(), tb.widthMethod()));
-    const result = iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 1, tb.tabWidth(), tb.widthMethod());
+    try testing.expectEqual(@as(u32, 2), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 0, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    const result = iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 1, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod());
     try testing.expectEqual(@as(u32, 1), result);
 
-    const bounds = iter_mod.getGraphemeBoundsAt(tb.rope(), tb.memRegistry(), 0, 1, tb.tabWidth(), tb.widthMethod()).?;
+    const bounds = iter_mod.getGraphemeBoundsAt(tb.rope(), tb.memRegistry(), 0, 1, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()).?;
     try testing.expectEqual(@as(u32, 0), bounds.start);
     try testing.expectEqual(@as(u32, 2), bounds.end);
 }
@@ -474,7 +474,7 @@ test "getGraphemeWidthAt - invalid row" {
 
     try tb.setText("test");
 
-    try testing.expectEqual(@as(u32, 0), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 5, 0, tb.tabWidth(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 0), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 5, 0, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
 }
 
 test "getPrevGraphemeWidth - CJK characters" {
@@ -486,11 +486,11 @@ test "getPrevGraphemeWidth - CJK characters" {
 
     try tb.setText("a世界b");
 
-    try testing.expectEqual(@as(u32, 0), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 0, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 1), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 1, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 2), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 3, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 2), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 5, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 1), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 6, tb.tabWidth(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 0), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 0, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 1), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 1, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 2), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 3, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 2), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 5, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 1), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 6, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
 }
 
 test "getPrevGraphemeWidth - star emoji" {
@@ -502,9 +502,9 @@ test "getPrevGraphemeWidth - star emoji" {
 
     try tb.setText("x🌟y");
 
-    try testing.expectEqual(@as(u32, 1), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 1, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 2), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 3, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 1), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 4, tb.tabWidth(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 1), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 1, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 2), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 3, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 1), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 4, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
 }
 
 test "getPrevGraphemeWidth - tabs" {
@@ -517,9 +517,9 @@ test "getPrevGraphemeWidth - tabs" {
 
     try tb.setText("a\tb");
 
-    try testing.expectEqual(@as(u32, 1), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 1, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 4), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 5, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 1), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 6, tb.tabWidth(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 1), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 1, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 4), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 5, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 1), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 6, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
 }
 
 test "getPrevGraphemeWidth - invalid row" {
@@ -531,7 +531,7 @@ test "getPrevGraphemeWidth - invalid row" {
 
     try tb.setText("test");
 
-    try testing.expectEqual(@as(u32, 0), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 10, 5, tb.tabWidth(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 0), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 10, 5, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
 }
 
 test "getGraphemeWidthAt and getPrevGraphemeWidth - mixed content" {
@@ -544,16 +544,16 @@ test "getGraphemeWidthAt and getPrevGraphemeWidth - mixed content" {
 
     try tb.setText("Hi\t世🌟!");
 
-    try testing.expectEqual(@as(u32, 1), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 0, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 1), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 1, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 4), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 2, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 2), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 6, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 2), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 8, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 1), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 10, tb.tabWidth(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 1), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 0, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 1), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 1, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 4), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 2, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 2), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 6, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 2), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 8, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 1), iter_mod.getGraphemeWidthAt(tb.rope(), tb.memRegistry(), 0, 10, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
 
-    try testing.expectEqual(@as(u32, 1), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 1, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 1), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 2, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 4), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 6, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 2), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 8, tb.tabWidth(), tb.widthMethod()));
-    try testing.expectEqual(@as(u32, 2), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 10, tb.tabWidth(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 1), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 1, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 1), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 2, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 4), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 6, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 2), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 8, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
+    try testing.expectEqual(@as(u32, 2), iter_mod.getPrevGraphemeWidth(tb.rope(), tb.memRegistry(), 0, 10, tb.tabWidth(), tb.controlDisplay(), tb.widthMethod()));
 }

packages/native/src/tests/text-buffer-segment_test.zig

@@ -29,10 +29,10 @@ test "TextChunk layout cache retries and releases partial allocations" {
         defer cache.clear();
         var chunk: TextChunk = .{ .mem_id = mem_id, .byte_start = 0, .byte_end = text.len, .width_cols = 2560 };
 
-        try testing.expectError(error.OutOfMemory, chunk.getLayoutInfo(arena.allocator(), &cache, &registry, 2, .unicode));
+        try testing.expectError(error.OutOfMemory, chunk.getLayoutInfo(arena.allocator(), &cache, &registry, 2, .hidden, .unicode));
         try testing.expect(chunk.getCachedLayoutInfo(2, .unicode) == null);
         failing.fail_index = std.math.maxInt(usize);
-        const retried = try chunk.getLayoutInfo(arena.allocator(), &cache, &registry, 2, .unicode);
+        const retried = try chunk.getLayoutInfo(arena.allocator(), &cache, &registry, 2, .hidden, .unicode);
         try testing.expectEqual(@as(usize, 512), retried.wrap_breaks.len);
         try testing.expectEqual(@as(usize, 512), retried.cjk_breaks.len);
         try testing.expectEqual(retried.cjk_breaks.ptr, chunk.getCachedLayoutInfo(2, .unicode).?.cjk_breaks.ptr);
@@ -41,7 +41,7 @@ test "TextChunk layout cache retries and releases partial allocations" {
         try testing.expectEqual(failing.allocated_bytes, failing.freed_bytes);
 
         failing.fail_index = failing.alloc_index + fail_index;
-        try testing.expectError(error.OutOfMemory, chunk.getLayoutInfo(arena.allocator(), &cache, &registry, 2, .unicode));
+        try testing.expectError(error.OutOfMemory, chunk.getLayoutInfo(arena.allocator(), &cache, &registry, 2, .hidden, .unicode));
         cache.clear();
         try testing.expectEqual(failing.allocated_bytes, failing.freed_bytes);
     }
@@ -92,17 +92,17 @@ test "TextChunk.getLayoutInfo returns direct byte and column metadata" {
         .mem_id = mem_id,
         .byte_start = 0,
         .byte_end = @intCast(text.len),
-        .width_cols = @intCast(utf8.calculateTextWidth(text, 2, false, .unicode)),
+        .width_cols = @intCast(utf8.calculateTextWidth(text, 2, .hidden, false, .unicode)),
     };
 
-    const layout = try chunk.getLayoutInfo(arena.allocator(), &cache, &registry, 2, .unicode);
+    const layout = try chunk.getLayoutInfo(arena.allocator(), &cache, &registry, 2, .hidden, .unicode);
     try testing.expectEqual(@as(usize, 1), layout.wrap_breaks.len);
     try testing.expectEqual(@as(u32, 6), layout.wrap_breaks[0].byte_start);
     try testing.expectEqual(@as(u32, 4), layout.wrap_breaks[0].col_start);
     try testing.expectEqual(@as(u32, 7), layout.wrap_breaks[0].byteEnd());
     try testing.expectEqual(@as(u32, 5), layout.wrap_breaks[0].colEnd());
 
-    const cached = try chunk.getLayoutInfo(arena.allocator(), &cache, &registry, 2, .unicode);
+    const cached = try chunk.getLayoutInfo(arena.allocator(), &cache, &registry, 2, .hidden, .unicode);
     try testing.expectEqual(@intFromPtr(layout.wrap_breaks.ptr), @intFromPtr(cached.wrap_breaks.ptr));
 
     const zero_width_text = "a\u{200B}b";
@@ -111,9 +111,9 @@ test "TextChunk.getLayoutInfo returns direct byte and column metadata" {
         .mem_id = zero_width_mem_id,
         .byte_start = 0,
         .byte_end = @intCast(zero_width_text.len),
-        .width_cols = @intCast(utf8.calculateTextWidth(zero_width_text, 2, false, .unicode)),
+        .width_cols = @intCast(utf8.calculateTextWidth(zero_width_text, 2, .hidden, false, .unicode)),
     };
-    const zero_width_layout = try zero_width_chunk.getLayoutInfo(arena.allocator(), &cache, &registry, 2, .unicode);
+    const zero_width_layout = try zero_width_chunk.getLayoutInfo(arena.allocator(), &cache, &registry, 2, .hidden, .unicode);
     try testing.expectEqual(@as(usize, 1), zero_width_layout.wrap_breaks.len);
     try testing.expectEqual(@as(u32, 1), zero_width_layout.wrap_breaks[0].byte_start);
     try testing.expectEqual(@as(u32, 3), zero_width_layout.wrap_breaks[0].byte_len);
@@ -131,24 +131,24 @@ test "TextChunk keeps CJK line opportunities out of the word cache" {
     const text = "\u{65e5}\u{672c}\t\u{8a9e}\u{6587}";
     const mem_id = try registry.register(text, false);
     var chunk: TextChunk = .{ .mem_id = mem_id, .byte_start = 0, .byte_end = text.len, .width_cols = 10 };
-    const initial_words = try chunk.getWordLayoutInfo(arena.allocator(), &cache, &registry, 2, .unicode);
+    const initial_words = try chunk.getWordLayoutInfo(arena.allocator(), &cache, &registry, 2, .hidden, .unicode);
     try testing.expect(@sizeOf(@TypeOf(initial_words)) < @sizeOf(utf8.ChunkLayoutInfo));
     try testing.expectEqual(utf8.WordClass.cjk_word, initial_words.word_classes.first);
     try testing.expectEqual(utf8.WordClass.cjk_word, initial_words.word_classes.last);
     try testing.expectEqual(@as(usize, 1), initial_words.wrap_breaks.len);
     try testing.expectEqual(@as(usize, 0), chunk.cold.?.cjk_breaks.capacity);
-    _ = try chunk.getLayoutInfo(arena.allocator(), &cache, &registry, 2, .unicode);
+    _ = try chunk.getLayoutInfo(arena.allocator(), &cache, &registry, 2, .hidden, .unicode);
     try testing.expect(chunk.cold.?.wrap_breaks != null);
     for ([_]utf8.WidthMethod{ .unicode, .wcwidth }) |method| {
         for ([_]u8{ 2, 8 }) |tab_width| {
-            const words = try chunk.getWordLayoutInfo(arena.allocator(), &cache, &registry, tab_width, method);
+            const words = try chunk.getWordLayoutInfo(arena.allocator(), &cache, &registry, tab_width, .hidden, method);
             try testing.expectEqual(@as(usize, 1), words.wrap_breaks.len);
             try testing.expectEqual(utf8.LayoutWrapBreakKind.whitespace, words.wrap_breaks[0].kind);
             try testing.expectEqual(@as(u32, tab_width), words.wrap_breaks[0].width_cols);
-            const lines = try chunk.getLayoutInfo(arena.allocator(), &cache, &registry, tab_width, method);
+            const lines = try chunk.getLayoutInfo(arena.allocator(), &cache, &registry, tab_width, .hidden, method);
             try testing.expectEqual(@as(usize, 1), lines.wrap_breaks.len);
             try testing.expectEqual(@as(usize, 2), lines.cjk_breaks.len);
-            const cached_words = try chunk.getWordLayoutInfo(arena.allocator(), &cache, &registry, tab_width, method);
+            const cached_words = try chunk.getWordLayoutInfo(arena.allocator(), &cache, &registry, tab_width, .hidden, method);
             try testing.expectEqual(utf8.WordClass.cjk_word, cached_words.word_classes.first);
             try testing.expectEqual(utf8.WordClass.cjk_word, cached_words.word_classes.last);
             try testing.expectEqual(@intFromPtr(words.wrap_breaks.ptr), @intFromPtr(cached_words.wrap_breaks.ptr));
@@ -159,7 +159,7 @@ test "TextChunk keeps CJK line opportunities out of the word cache" {
     const greek = "\u{3bb}-\u{3bb}";
     const greek_id = try registry.register(greek, false);
     var greek_chunk: TextChunk = .{ .mem_id = greek_id, .byte_start = 0, .byte_end = greek.len, .width_cols = 3 };
-    _ = try greek_chunk.getWordLayoutInfo(arena.allocator(), &cache, &registry, 2, .unicode);
+    _ = try greek_chunk.getWordLayoutInfo(arena.allocator(), &cache, &registry, 2, .hidden, .unicode);
     try testing.expect(greek_chunk.getCachedLayoutInfo(2, .unicode) != null);
 }
 
@@ -197,7 +197,7 @@ test "findChunkLayoutInfo classifies direct byte and column break metadata" {
     defer breaks.deinit(testing.allocator);
 
     for (cases) |case| {
-        _ = try utf8.findChunkLayoutInfo(testing.allocator, case.text, case.tab_width, false, .unicode, &breaks);
+        _ = try utf8.findChunkLayoutInfo(testing.allocator, case.text, case.tab_width, .hidden, false, .unicode, &breaks);
         try testing.expectEqualDeep(case.expected, breaks.items);
     }
 }
@@ -207,7 +207,7 @@ test "walkChunkLayoutInfo keeps Prepend joined to an ASCII vector" {
     var breaks: std.ArrayListUnmanaged(utf8.LayoutWrapBreak) = .empty;
     defer breaks.deinit(testing.allocator);
 
-    _ = try utf8.findChunkLayoutInfo(testing.allocator, text, 2, false, .unicode, &breaks);
+    _ = try utf8.findChunkLayoutInfo(testing.allocator, text, 2, .hidden, false, .unicode, &breaks);
     try testing.expectEqual(@as(usize, 1), breaks.items.len);
     try testing.expectEqual(@as(u32, 0), breaks.items[0].byte_start);
     try testing.expectEqual(@as(u32, 3), breaks.items[0].byte_len);
@@ -222,7 +222,7 @@ test "walkChunkLayoutInfo ASCII prefixes keep their final grapheme intact" {
     for (0..32) |prefix_len| {
         const text = try std.fmt.allocPrint(testing.allocator, "{s} \u{301} \u{754c}abc", .{("a" ** 32)[0..prefix_len]});
         defer testing.allocator.free(text);
-        _ = try utf8.findChunkLayoutInfo(testing.allocator, text, 2, false, .unicode, &breaks);
+        _ = try utf8.findChunkLayoutInfo(testing.allocator, text, 2, .hidden, false, .unicode, &breaks);
         try testing.expectEqual(@as(usize, 3), breaks.items.len);
         try testing.expectEqual(@as(u32, @intCast(prefix_len)), breaks.items[0].byte_start);
         try testing.expectEqual(@as(u32, 3), breaks.items[0].byte_len);
@@ -244,7 +244,7 @@ test "walkChunkLayoutInfo preserves mixed-content whitespace graphemes" {
     defer breaks.deinit(testing.allocator);
 
     for (cases) |text| {
-        _ = try utf8.findChunkLayoutInfo(testing.allocator, text, 2, false, .unicode, &breaks);
+        _ = try utf8.findChunkLayoutInfo(testing.allocator, text, 2, .hidden, false, .unicode, &breaks);
         try testing.expectEqual(@as(usize, 1), breaks.items.len);
         try testing.expectEqual(@as(u32, 0), breaks.items[0].byte_start);
         try testing.expectEqual(@as(u32, @intCast(text.len)), breaks.items[0].byte_len);
@@ -278,7 +278,7 @@ test "walkChunkLayoutInfo stops streaming after visitor allocation failure" {
     };
     var ctx: Context = .{ .allocator = failing.allocator() };
     defer ctx.breaks.deinit(ctx.allocator);
-    _ = try utf8.walkChunkLayoutInfoComptime(text, 2, true, .unicode, &ctx, Context.visit);
+    _ = try utf8.walkChunkLayoutInfoComptime(text, 2, .hidden, true, .unicode, &ctx, Context.visit);
     try testing.expect(failing.has_induced_failure);
     try testing.expect(ctx.failed);
     try testing.expectEqual(@as(usize, 1), ctx.callback_count);
@@ -301,11 +301,11 @@ test "TextChunk.getLayoutInfo refreshes tab-dependent metadata" {
         .width_cols = 4,
     };
 
-    const first = try chunk.getLayoutInfo(arena.allocator(), &cache, &registry, 2, .unicode);
+    const first = try chunk.getLayoutInfo(arena.allocator(), &cache, &registry, 2, .hidden, .unicode);
     try testing.expectEqual(@as(u32, 2), first.wrap_breaks[0].width_cols);
     const capacity_after_first = arena.queryCapacity();
 
-    const second = try chunk.getLayoutInfo(arena.allocator(), &cache, &registry, 8, .unicode);
+    const second = try chunk.getLayoutInfo(arena.allocator(), &cache, &registry, 8, .hidden, .unicode);
     try testing.expectEqual(@as(u32, 8), second.wrap_breaks[0].width_cols);
     try testing.expectEqual(@intFromPtr(first.wrap_breaks.ptr), @intFromPtr(second.wrap_breaks.ptr));
     try testing.expectEqual(capacity_after_first, arena.queryCapacity());
@@ -326,15 +326,15 @@ test "TextChunk.getRenderClusters reuses tab-dependent cache storage" {
         .width_cols = 4,
     };
 
-    const first = try chunk.getRenderClusters(arena.allocator(), &registry, 2, .unicode);
+    const first = try chunk.getRenderClusters(arena.allocator(), &registry, 2, .hidden, .unicode);
     try testing.expectEqual(@as(u32, 2), first[0].width_cols);
     const capacity_after_first = arena.queryCapacity();
 
-    const wider = try chunk.getRenderClusters(arena.allocator(), &registry, 8, .unicode);
+    const wider = try chunk.getRenderClusters(arena.allocator(), &registry, 8, .hidden, .unicode);
     try testing.expectEqual(@as(u32, 8), wider[0].width_cols);
     try testing.expectEqual(@intFromPtr(first.ptr), @intFromPtr(wider.ptr));
 
-    const narrower = try chunk.getRenderClusters(arena.allocator(), &registry, 4, .unicode);
+    const narrower = try chunk.getRenderClusters(arena.allocator(), &registry, 4, .hidden, .unicode);
     try testing.expectEqual(@as(u32, 4), narrower[0].width_cols);
     try testing.expectEqual(@intFromPtr(first.ptr), @intFromPtr(narrower.ptr));
     try testing.expectEqual(capacity_after_first, arena.queryCapacity());

packages/native/src/tests/text-buffer-view_test.zig

@@ -4623,17 +4623,17 @@ fn splitsGrapheme(tb: *TextBuffer, mode: text_buffer_view.WrapMode) bool {
         };
         const bytes = chunk.getBytes(tb.memRegistry());
         if (previous) |left| if (left.len > 0 and bytes.len > 0) {
-            // Replay the left chunk's last grapheme so the break state knows its context.
+            // Replay the left chunk from its last grapheme with cells so the break state knows its context.
             var state: @import("uucode").grapheme.BreakState = .default;
             var last: ?u21 = null;
-            var pos = utf8.getPrevGraphemeStart(left, left.len, tb.tabWidth(), .unicode).?.start_offset;
+            var pos = if (utf8.getPrevGraphemeStart(left, left.len, tb.tabWidth(), .hidden, .unicode)) |grapheme| grapheme.start_offset else 0;
             while (pos < left.len) : (pos += utf8.decodeUtf8Unchecked(left, pos).len) {
                 const cp = utf8.decodeUtf8Unchecked(left, pos).cp;
                 _ = utf8.isGraphemeBreak(last, cp, &state, .unicode);
                 last = cp;
             }
             const first = utf8.decodeUtf8Unchecked(bytes, 0).cp;
-            const combining = first != 0x200D and utf8.zeroWidthPrefixLen(bytes, tb.tabWidth(), .unicode) > 0;
+            const combining = first != 0x200D and utf8.zeroWidthPrefixLen(bytes, tb.tabWidth(), .hidden, .unicode) > 0;
             const allow_combining = mode == .char or (!has_cjk_words and last.? <= 0x7F and utf8.isWordCodepoint(last.?));
             if (!(allow_combining and combining) and !utf8.isGraphemeBreak(last, first, &state, .unicode)) return true;
         };

packages/native/src/tests/text-buffer_test.zig

@@ -1933,7 +1933,7 @@ test "TextBuffer owned styled replacement reclaims styled plain and empty transi
             for (0..tb.rope().count()) |segment| {
                 if (tb.rope().get(@intCast(segment)).?.asText()) |chunk| {
                     _ = try tb.getLayoutInfoFor(chunk);
-                    _ = try chunk.getRenderClusters(tb.getAllocator(), tb.memRegistry(), tb.tabWidth(), tb.widthMethod());
+                    _ = try chunk.getRenderClusters(tb.getAllocator(), tb.memRegistry(), tb.tabWidth(), tb.controlDisplay(), tb.widthMethod());
                 }
             }
             const live = tracked.allocated_bytes - tracked.freed_bytes;

packages/native/src/tests/utf8_no_zwj_test.zig

@@ -5,9 +5,9 @@ const utf8 = @import("../utf8.zig");
 test "no_zwj: basic emoji ZWJ sequence split" {
     const text = "👩‍🚀"; // Woman + ZWJ + Rocket
 
-    const width_unicode = utf8.calculateTextWidth(text, 4, false, .unicode);
-    const width_no_zwj = utf8.calculateTextWidth(text, 4, false, .no_zwj);
-    const width_wcwidth = utf8.calculateTextWidth(text, 4, false, .wcwidth);
+    const width_unicode = utf8.calculateTextWidth(text, 4, .hidden, false, .unicode);
+    const width_no_zwj = utf8.calculateTextWidth(text, 4, .hidden, false, .no_zwj);
+    const width_wcwidth = utf8.calculateTextWidth(text, 4, .hidden, false, .wcwidth);
 
     // unicode: single grapheme cluster (width 2)
     try testing.expectEqual(@as(u32, 2), width_unicode);
@@ -22,8 +22,8 @@ test "no_zwj: basic emoji ZWJ sequence split" {
 test "no_zwj: family emoji split" {
     const text = "👨‍👩‍👧"; // Man + ZWJ + Woman + ZWJ + Girl
 
-    const width_unicode = utf8.calculateTextWidth(text, 4, false, .unicode);
-    const width_no_zwj = utf8.calculateTextWidth(text, 4, false, .no_zwj);
+    const width_unicode = utf8.calculateTextWidth(text, 4, .hidden, false, .unicode);
+    const width_no_zwj = utf8.calculateTextWidth(text, 4, .hidden, false, .no_zwj);
 
     // unicode: single grapheme (width 2)
     try testing.expectEqual(@as(u32, 2), width_unicode);
@@ -35,8 +35,8 @@ test "no_zwj: family emoji split" {
 test "no_zwj: combining marks still combined" {
     const text = "é"; // e + combining acute (U+0301)
 
-    const width_unicode = utf8.calculateTextWidth(text, 4, false, .unicode);
-    const width_no_zwj = utf8.calculateTextWidth(text, 4, false, .no_zwj);
+    const width_unicode = utf8.calculateTextWidth(text, 4, .hidden, false, .unicode);
+    const width_no_zwj = utf8.calculateTextWidth(text, 4, .hidden, false, .no_zwj);
 
     // Both should treat this as a single grapheme with width 1
     try testing.expectEqual(@as(u32, 1), width_unicode);
@@ -46,9 +46,9 @@ test "no_zwj: combining marks still combined" {
 test "no_zwj: skin tone modifiers still combined" {
     const text = "👋🏿"; // Wave + dark skin tone
 
-    const width_unicode = utf8.calculateTextWidth(text, 4, false, .unicode);
-    const width_no_zwj = utf8.calculateTextWidth(text, 4, false, .no_zwj);
-    const width_wcwidth = utf8.calculateTextWidth(text, 4, false, .wcwidth);
+    const width_unicode = utf8.calculateTextWidth(text, 4, .hidden, false, .unicode);
+    const width_no_zwj = utf8.calculateTextWidth(text, 4, .hidden, false, .no_zwj);
+    const width_wcwidth = utf8.calculateTextWidth(text, 4, .hidden, false, .wcwidth);
 
     // unicode: single grapheme (width 2)
     try testing.expectEqual(@as(u32, 2), width_unicode);
@@ -63,9 +63,9 @@ test "no_zwj: skin tone modifiers still combined" {
 test "no_zwj: flag emoji stays combined" {
     const text = "🇺🇸"; // US flag (two regional indicators)
 
-    const width_unicode = utf8.calculateTextWidth(text, 4, false, .unicode);
-    const width_no_zwj = utf8.calculateTextWidth(text, 4, false, .no_zwj);
-    const width_wcwidth = utf8.calculateTextWidth(text, 4, false, .wcwidth);
+    const width_unicode = utf8.calculateTextWidth(text, 4, .hidden, false, .unicode);
+    const width_no_zwj = utf8.calculateTextWidth(text, 4, .hidden, false, .no_zwj);
+    const width_wcwidth = utf8.calculateTextWidth(text, 4, .hidden, false, .wcwidth);
 
     // unicode: single flag grapheme (width 2)
     try testing.expectEqual(@as(u32, 2), width_unicode);
@@ -80,8 +80,8 @@ test "no_zwj: flag emoji stays combined" {
 test "no_zwj: mixed text with ZWJ emoji" {
     const text = "Hello👩‍🚀World"; // Hello + woman astronaut + World
 
-    const width_unicode = utf8.calculateTextWidth(text, 4, false, .unicode);
-    const width_no_zwj = utf8.calculateTextWidth(text, 4, false, .no_zwj);
+    const width_unicode = utf8.calculateTextWidth(text, 4, .hidden, false, .unicode);
+    const width_no_zwj = utf8.calculateTextWidth(text, 4, .hidden, false, .no_zwj);
 
     // unicode: Hello(5) + astronaut(2) + World(5) = 12
     try testing.expectEqual(@as(u32, 12), width_unicode);
@@ -98,8 +98,8 @@ test "no_zwj: findRenderClusterInfo splits ZWJ sequences" {
 
     const text = "Hi👩‍🚀Bye";
 
-    try utf8.findRenderClusterInfo(testing.allocator, text, 4, false, .unicode, &result_unicode);
-    try utf8.findRenderClusterInfo(testing.allocator, text, 4, false, .no_zwj, &result_no_zwj);
+    try utf8.findRenderClusterInfo(testing.allocator, text, 4, .hidden, false, .unicode, &result_unicode);
+    try utf8.findRenderClusterInfo(testing.allocator, text, 4, .hidden, false, .no_zwj, &result_no_zwj);
 
     // unicode: 1 grapheme (the whole ZWJ sequence)
     try testing.expectEqual(@as(usize, 1), result_unicode.items.len);
@@ -114,8 +114,8 @@ test "no_zwj: findRenderClusterInfo splits ZWJ sequences" {
 test "no_zwj: findWrapPosByWidth with ZWJ sequences" {
     const text = "AB👩‍🚀CD"; // A(1) B(1) woman(2) rocket(2) C(1) D(1)
 
-    const result_unicode = utf8.findWrapPosByWidth(text, 4, 4, false, .unicode);
-    const result_no_zwj = utf8.findWrapPosByWidth(text, 4, 4, false, .no_zwj);
+    const result_unicode = utf8.findWrapPosByWidth(text, 4, 4, .hidden, false, .unicode);
+    const result_no_zwj = utf8.findWrapPosByWidth(text, 4, 4, .hidden, false, .no_zwj);
 
     // unicode: stops after "AB👩‍🚀" = 4 columns (whole sequence)
     try testing.expectEqual(@as(u32, 4), result_unicode.columns_used);
@@ -128,8 +128,8 @@ test "no_zwj: findPosByWidth with ZWJ sequences" {
     const text = "AB👩‍🚀CD"; // A(1) B(1) woman(2) rocket(2) C(1) D(1)
 
     // With include_start_before=false
-    const start4_unicode = utf8.findPosByWidth(text, 4, 4, false, false, .unicode);
-    const start4_no_zwj = utf8.findPosByWidth(text, 4, 4, false, false, .no_zwj);
+    const start4_unicode = utf8.findPosByWidth(text, 4, 4, .hidden, false, false, .unicode);
+    const start4_no_zwj = utf8.findPosByWidth(text, 4, 4, .hidden, false, false, .no_zwj);
 
     // unicode: Woman+ZWJ+Rocket is one grapheme that ends at col 4, so it's included
     // Stops at 'C' (byte 13)
@@ -142,8 +142,8 @@ test "no_zwj: findPosByWidth with ZWJ sequences" {
     try testing.expectEqual(@as(u32, 4), start4_no_zwj.columns_used);
 
     // With include_start_before=true
-    const end4_unicode = utf8.findPosByWidth(text, 4, 4, false, true, .unicode);
-    const end4_no_zwj = utf8.findPosByWidth(text, 4, 4, false, true, .no_zwj);
+    const end4_unicode = utf8.findPosByWidth(text, 4, 4, .hidden, false, true, .unicode);
+    const end4_no_zwj = utf8.findPosByWidth(text, 4, 4, .hidden, false, true, .no_zwj);
 
     // unicode: includes whole sequence
     try testing.expectEqual(@as(u32, 4), end4_unicode.columns_used);
@@ -156,8 +156,8 @@ test "no_zwj: getWidthAt with ZWJ sequence" {
     const text = "👩‍🚀"; // Woman + ZWJ + Rocket
 
     // At woman emoji (byte 0)
-    const width_woman_unicode = utf8.getWidthAt(text, 0, 4, .unicode);
-    const width_woman_no_zwj = utf8.getWidthAt(text, 0, 4, .no_zwj);
+    const width_woman_unicode = utf8.getWidthAt(text, 0, 4, .hidden, .unicode);
+    const width_woman_no_zwj = utf8.getWidthAt(text, 0, 4, .hidden, .no_zwj);
 
     // unicode: whole sequence width
     try testing.expectEqual(@as(u32, 2), width_woman_unicode);
@@ -166,7 +166,7 @@ test "no_zwj: getWidthAt with ZWJ sequence" {
     try testing.expectEqual(@as(u32, 2), width_woman_no_zwj);
 
     // At ZWJ (byte 4)
-    const width_zwj_no_zwj = utf8.getWidthAt(text, 4, 4, .no_zwj);
+    const width_zwj_no_zwj = utf8.getWidthAt(text, 4, 4, .hidden, .no_zwj);
     try testing.expectEqual(@as(u32, 0), width_zwj_no_zwj); // ZWJ itself has width 0
 }
 
@@ -174,8 +174,8 @@ test "no_zwj: getPrevGraphemeStart with ZWJ sequence" {
     const text = "AB👩‍🚀CD";
 
     // From end of text (after 'D')
-    const r1_unicode = utf8.getPrevGraphemeStart(text, text.len, 4, .unicode);
-    const r1_no_zwj = utf8.getPrevGraphemeStart(text, text.len, 4, .no_zwj);
+    const r1_unicode = utf8.getPrevGraphemeStart(text, text.len, 4, .hidden, .unicode);
+    const r1_no_zwj = utf8.getPrevGraphemeStart(text, text.len, 4, .hidden, .no_zwj);
 
     try testing.expect(r1_unicode != null);
     try testing.expect(r1_no_zwj != null);
@@ -190,8 +190,8 @@ test "no_zwj: getPrevGraphemeStart with ZWJ sequence" {
 test "no_zwj: multiple ZWJ sequences" {
     const text = "👨‍👩‍👧👨‍👩‍👦"; // Family with girl + Family with boy
 
-    const width_unicode = utf8.calculateTextWidth(text, 4, false, .unicode);
-    const width_no_zwj = utf8.calculateTextWidth(text, 4, false, .no_zwj);
+    const width_unicode = utf8.calculateTextWidth(text, 4, .hidden, false, .unicode);
+    const width_no_zwj = utf8.calculateTextWidth(text, 4, .hidden, false, .no_zwj);
 
     // unicode: 2 families = 4 columns
     try testing.expectEqual(@as(u32, 4), width_unicode);
@@ -203,8 +203,8 @@ test "no_zwj: multiple ZWJ sequences" {
 test "no_zwj: ZWJ with skin tones" {
     const text = "👨🏿‍❤️‍👨🏻"; // Man with dark skin + ZWJ + heart + ZWJ + man with light skin
 
-    const width_unicode = utf8.calculateTextWidth(text, 4, false, .unicode);
-    const width_no_zwj = utf8.calculateTextWidth(text, 4, false, .no_zwj);
+    const width_unicode = utf8.calculateTextWidth(text, 4, .hidden, false, .unicode);
+    const width_no_zwj = utf8.calculateTextWidth(text, 4, .hidden, false, .no_zwj);
 
     // unicode: single couple grapheme (width 2)
     try testing.expectEqual(@as(u32, 2), width_unicode);
@@ -216,9 +216,9 @@ test "no_zwj: ZWJ with skin tones" {
 test "no_zwj: keycap sequences without ZWJ" {
     const text = "1️⃣"; // 1 + VS16 + combining enclosing keycap
 
-    const width_unicode = utf8.calculateTextWidth(text, 4, false, .unicode);
-    const width_no_zwj = utf8.calculateTextWidth(text, 4, false, .no_zwj);
-    const width_wcwidth = utf8.calculateTextWidth(text, 4, false, .wcwidth);
+    const width_unicode = utf8.calculateTextWidth(text, 4, .hidden, false, .unicode);
+    const width_no_zwj = utf8.calculateTextWidth(text, 4, .hidden, false, .no_zwj);
+    const width_wcwidth = utf8.calculateTextWidth(text, 4, .hidden, false, .wcwidth);
 
     // unicode and no_zwj should both treat this as a single grapheme
     try testing.expectEqual(@as(u32, 2), width_unicode);
@@ -232,8 +232,8 @@ test "no_zwj: keycap sequences without ZWJ" {
 test "no_zwj: rainbow flag without ZWJ" {
     const text = "🏳️‍🌈"; // White flag + VS16 + ZWJ + rainbow
 
-    const width_unicode = utf8.calculateTextWidth(text, 4, false, .unicode);
-    const width_no_zwj = utf8.calculateTextWidth(text, 4, false, .no_zwj);
+    const width_unicode = utf8.calculateTextWidth(text, 4, .hidden, false, .unicode);
+    const width_no_zwj = utf8.calculateTextWidth(text, 4, .hidden, false, .no_zwj);
 
     // unicode: single rainbow flag grapheme (width 2)
     try testing.expectEqual(@as(u32, 2), width_unicode);
@@ -245,9 +245,9 @@ test "no_zwj: rainbow flag without ZWJ" {
 test "no_zwj: Devanagari conjuncts still work" {
     const text = "क्ष"; // Ka + virama + Sha (conjunct)
 
-    const width_unicode = utf8.calculateTextWidth(text, 4, false, .unicode);
-    const width_no_zwj = utf8.calculateTextWidth(text, 4, false, .no_zwj);
-    const width_wcwidth = utf8.calculateTextWidth(text, 4, false, .wcwidth);
+    const width_unicode = utf8.calculateTextWidth(text, 4, .hidden, false, .unicode);
+    const width_no_zwj = utf8.calculateTextWidth(text, 4, .hidden, false, .no_zwj);
+    const width_wcwidth = utf8.calculateTextWidth(text, 4, .hidden, false, .wcwidth);
 
     // unicode: Devanagari renders as width 2 in terminals (Ka=1 + Sha=1)
     try testing.expectEqual(@as(u32, 2), width_unicode);

packages/native/src/tests/utf8_wcwidth_cursor_test.zig

@@ -5,70 +5,70 @@ const utf8 = @import("../utf8.zig");
 test "wcwidth: cursor movement through emoji with skin tone" {
     const text = "👋🏿"; // Wave + dark skin tone = 4 columns
 
-    const width_wave = utf8.getWidthAt(text, 0, 4, .wcwidth);
+    const width_wave = utf8.getWidthAt(text, 0, 4, .hidden, .wcwidth);
     try testing.expectEqual(@as(u32, 2), width_wave);
 
-    const width_skin = utf8.getWidthAt(text, 4, 4, .wcwidth);
+    const width_skin = utf8.getWidthAt(text, 4, 4, .hidden, .wcwidth);
     try testing.expectEqual(@as(u32, 2), width_skin);
 
-    const total = utf8.calculateTextWidth(text, 4, false, .wcwidth);
+    const total = utf8.calculateTextWidth(text, 4, .hidden, false, .wcwidth);
     try testing.expectEqual(@as(u32, 4), total);
 }
 
 test "wcwidth: cursor movement through ZWJ sequence" {
     const text = "👩‍🚀"; // Woman + ZWJ + Rocket = 4 columns (2+0+2)
 
-    const total = utf8.calculateTextWidth(text, 4, false, .wcwidth);
+    const total = utf8.calculateTextWidth(text, 4, .hidden, false, .wcwidth);
     try testing.expectEqual(@as(u32, 4), total);
 
-    const width_woman = utf8.getWidthAt(text, 0, 4, .wcwidth);
+    const width_woman = utf8.getWidthAt(text, 0, 4, .hidden, .wcwidth);
     try testing.expectEqual(@as(u32, 2), width_woman);
 
-    const width_zwj = utf8.getWidthAt(text, 4, 4, .wcwidth);
+    const width_zwj = utf8.getWidthAt(text, 4, 4, .hidden, .wcwidth);
     try testing.expectEqual(@as(u32, 0), width_zwj);
 
-    const width_rocket = utf8.getWidthAt(text, 7, 4, .wcwidth);
+    const width_rocket = utf8.getWidthAt(text, 7, 4, .hidden, .wcwidth);
     try testing.expectEqual(@as(u32, 2), width_rocket);
 }
 
 test "wcwidth: cursor movement through family emoji" {
     const text = "👨‍👩‍👧"; // Man + ZWJ + Woman + ZWJ + Girl = 6 columns (2+0+2+0+2)
 
-    const total = utf8.calculateTextWidth(text, 4, false, .wcwidth);
+    const total = utf8.calculateTextWidth(text, 4, .hidden, false, .wcwidth);
     try testing.expectEqual(@as(u32, 6), total);
 
-    const width_man = utf8.getWidthAt(text, 0, 4, .wcwidth);
+    const width_man = utf8.getWidthAt(text, 0, 4, .hidden, .wcwidth);
     try testing.expectEqual(@as(u32, 2), width_man);
 
-    const width_zwj1 = utf8.getWidthAt(text, 4, 4, .wcwidth);
+    const width_zwj1 = utf8.getWidthAt(text, 4, 4, .hidden, .wcwidth);
     try testing.expectEqual(@as(u32, 0), width_zwj1);
 
-    const width_woman = utf8.getWidthAt(text, 7, 4, .wcwidth);
+    const width_woman = utf8.getWidthAt(text, 7, 4, .hidden, .wcwidth);
     try testing.expectEqual(@as(u32, 2), width_woman);
 
-    const width_zwj2 = utf8.getWidthAt(text, 11, 4, .wcwidth);
+    const width_zwj2 = utf8.getWidthAt(text, 11, 4, .hidden, .wcwidth);
     try testing.expectEqual(@as(u32, 0), width_zwj2);
 
-    const width_girl = utf8.getWidthAt(text, 14, 4, .wcwidth);
+    const width_girl = utf8.getWidthAt(text, 14, 4, .hidden, .wcwidth);
     try testing.expectEqual(@as(u32, 2), width_girl);
 }
 
 test "wcwidth: getPrevGraphemeStart through emoji with skin tone" {
     const text = "A👋🏿B"; // A(1) + 👋(2) + 🏿(2) + B(1) = 6 columns
 
-    const r_end = utf8.getPrevGraphemeStart(text, text.len, 4, .wcwidth);
+    const r_end = utf8.getPrevGraphemeStart(text, text.len, 4, .hidden, .wcwidth);
     try testing.expect(r_end != null);
     try testing.expectEqual(@as(u32, 1), r_end.?.width);
 
-    const r_b = utf8.getPrevGraphemeStart(text, r_end.?.start_offset, 4, .wcwidth);
+    const r_b = utf8.getPrevGraphemeStart(text, r_end.?.start_offset, 4, .hidden, .wcwidth);
     try testing.expect(r_b != null);
     try testing.expectEqual(@as(u32, 2), r_b.?.width);
 
-    const r_skin = utf8.getPrevGraphemeStart(text, r_b.?.start_offset, 4, .wcwidth);
+    const r_skin = utf8.getPrevGraphemeStart(text, r_b.?.start_offset, 4, .hidden, .wcwidth);
     try testing.expect(r_skin != null);
     try testing.expectEqual(@as(u32, 2), r_skin.?.width);
 
-    const r_wave = utf8.getPrevGraphemeStart(text, r_skin.?.start_offset, 4, .wcwidth);
+    const r_wave = utf8.getPrevGraphemeStart(text, r_skin.?.start_offset, 4, .hidden, .wcwidth);
     try testing.expect(r_wave != null);
     try testing.expectEqual(@as(u32, 1), r_wave.?.width);
 }
@@ -76,19 +76,19 @@ test "wcwidth: getPrevGraphemeStart through emoji with skin tone" {
 test "wcwidth: getPrevGraphemeStart through ZWJ sequence" {
     const text = "X👩‍🚀Y"; // X(1) + 👩(2) + ZWJ(0) + 🚀(2) + Y(1) = 6 columns
 
-    const r_end = utf8.getPrevGraphemeStart(text, text.len, 4, .wcwidth);
+    const r_end = utf8.getPrevGraphemeStart(text, text.len, 4, .hidden, .wcwidth);
     try testing.expect(r_end != null);
     try testing.expectEqual(@as(u32, 1), r_end.?.width);
 
-    const r_y = utf8.getPrevGraphemeStart(text, r_end.?.start_offset, 4, .wcwidth);
+    const r_y = utf8.getPrevGraphemeStart(text, r_end.?.start_offset, 4, .hidden, .wcwidth);
     try testing.expect(r_y != null);
     try testing.expectEqual(@as(u32, 2), r_y.?.width);
 
-    const r_rocket = utf8.getPrevGraphemeStart(text, r_y.?.start_offset, 4, .wcwidth);
+    const r_rocket = utf8.getPrevGraphemeStart(text, r_y.?.start_offset, 4, .hidden, .wcwidth);
     try testing.expect(r_rocket != null);
     try testing.expectEqual(@as(u32, 2), r_rocket.?.width);
 
-    const r_woman = utf8.getPrevGraphemeStart(text, r_rocket.?.start_offset, 4, .wcwidth);
+    const r_woman = utf8.getPrevGraphemeStart(text, r_rocket.?.start_offset, 4, .hidden, .wcwidth);
     try testing.expect(r_woman != null);
     try testing.expectEqual(@as(u32, 1), r_woman.?.width);
 }
@@ -96,25 +96,25 @@ test "wcwidth: getPrevGraphemeStart through ZWJ sequence" {
 test "wcwidth: findPosByWidth through emoji sequence" {
     const text = "AB👋🏿CD"; // A(1) B(1) 👋(2) 🏿(2) C(1) D(1) = 8 columns
 
-    const pos_start = utf8.findPosByWidth(text, 3, 4, false, false, .wcwidth);
+    const pos_start = utf8.findPosByWidth(text, 3, 4, .hidden, false, false, .wcwidth);
     try testing.expectEqual(@as(u32, 2), pos_start.byte_offset);
 
-    const pos_end = utf8.findPosByWidth(text, 3, 4, false, true, .wcwidth);
+    const pos_end = utf8.findPosByWidth(text, 3, 4, .hidden, false, true, .wcwidth);
     try testing.expectEqual(@as(u32, 6), pos_end.byte_offset);
 }
 
 test "wcwidth: findWrapPosByWidth through emoji" {
     const text = "Hi👋🏿Bye"; // H(1) i(1) 👋(2) 🏿(2) B(1) y(1) e(1) = 10 columns
 
-    const wrap_4 = utf8.findWrapPosByWidth(text, 4, 4, false, .wcwidth);
+    const wrap_4 = utf8.findWrapPosByWidth(text, 4, 4, .hidden, false, .wcwidth);
     try testing.expectEqual(@as(u32, 6), wrap_4.byte_offset);
     try testing.expectEqual(@as(u32, 4), wrap_4.columns_used);
 
-    const wrap_5 = utf8.findWrapPosByWidth(text, 5, 4, false, .wcwidth);
+    const wrap_5 = utf8.findWrapPosByWidth(text, 5, 4, .hidden, false, .wcwidth);
     try testing.expectEqual(@as(u32, 6), wrap_5.byte_offset);
     try testing.expectEqual(@as(u32, 4), wrap_5.columns_used);
 
-    const wrap_6 = utf8.findWrapPosByWidth(text, 6, 4, false, .wcwidth);
+    const wrap_6 = utf8.findWrapPosByWidth(text, 6, 4, .hidden, false, .wcwidth);
     try testing.expectEqual(@as(u32, 10), wrap_6.byte_offset);
     try testing.expectEqual(@as(u32, 6), wrap_6.columns_used);
 }
@@ -122,79 +122,79 @@ test "wcwidth: findWrapPosByWidth through emoji" {
 test "wcwidth: combining marks have zero width" {
     const text = "e\u{0301}"; // e + combining acute
 
-    const width_e = utf8.getWidthAt(text, 0, 4, .wcwidth);
+    const width_e = utf8.getWidthAt(text, 0, 4, .hidden, .wcwidth);
     try testing.expectEqual(@as(u32, 1), width_e);
 
-    const width_combining = utf8.getWidthAt(text, 1, 4, .wcwidth);
+    const width_combining = utf8.getWidthAt(text, 1, 4, .hidden, .wcwidth);
     try testing.expectEqual(@as(u32, 0), width_combining);
 
-    const total = utf8.calculateTextWidth(text, 4, false, .wcwidth);
+    const total = utf8.calculateTextWidth(text, 4, .hidden, false, .wcwidth);
     try testing.expectEqual(@as(u32, 1), total);
 }
 
 test "wcwidth: CJK characters have width 2" {
     const text = "你好世界"; // 4 CJK characters
 
-    const total = utf8.calculateTextWidth(text, 4, false, .wcwidth);
+    const total = utf8.calculateTextWidth(text, 4, .hidden, false, .wcwidth);
     try testing.expectEqual(@as(u32, 8), total);
 
-    const width_char1 = utf8.getWidthAt(text, 0, 4, .wcwidth);
+    const width_char1 = utf8.getWidthAt(text, 0, 4, .hidden, .wcwidth);
     try testing.expectEqual(@as(u32, 2), width_char1);
 
-    const width_char2 = utf8.getWidthAt(text, 3, 4, .wcwidth);
+    const width_char2 = utf8.getWidthAt(text, 3, 4, .hidden, .wcwidth);
     try testing.expectEqual(@as(u32, 2), width_char2);
 }
 
 test "wcwidth: variation selectors have zero width" {
     const text = "☺\u{FE0F}"; // Smiling face + VS16
 
-    const width_face = utf8.getWidthAt(text, 0, 4, .wcwidth);
+    const width_face = utf8.getWidthAt(text, 0, 4, .hidden, .wcwidth);
     try testing.expectEqual(@as(u32, 1), width_face);
 
-    const width_vs = utf8.getWidthAt(text, 3, 4, .wcwidth);
+    const width_vs = utf8.getWidthAt(text, 3, 4, .hidden, .wcwidth);
     try testing.expectEqual(@as(u32, 0), width_vs);
 
-    const total = utf8.calculateTextWidth(text, 4, false, .wcwidth);
+    const total = utf8.calculateTextWidth(text, 4, .hidden, false, .wcwidth);
     try testing.expectEqual(@as(u32, 1), total);
 }
 
 test "wcwidth: flag emoji counts both regional indicators" {
     const text = "🇺🇸"; // US flag (two regional indicators)
 
-    const width_ri1 = utf8.getWidthAt(text, 0, 4, .wcwidth);
+    const width_ri1 = utf8.getWidthAt(text, 0, 4, .hidden, .wcwidth);
     try testing.expectEqual(@as(u32, 1), width_ri1);
 
-    const width_ri2 = utf8.getWidthAt(text, 4, 4, .wcwidth);
+    const width_ri2 = utf8.getWidthAt(text, 4, 4, .hidden, .wcwidth);
     try testing.expectEqual(@as(u32, 1), width_ri2);
 
-    const total = utf8.calculateTextWidth(text, 4, false, .wcwidth);
+    const total = utf8.calculateTextWidth(text, 4, .hidden, false, .wcwidth);
     try testing.expectEqual(@as(u32, 2), total);
 }
 
 test "wcwidth: mixed content with cursor movement" {
     const text = "A👋🏿B世C"; // A(1) 👋(2) 🏿(2) B(1) 世(2) C(1) = 9 columns
 
-    const r_end = utf8.getPrevGraphemeStart(text, text.len, 4, .wcwidth);
+    const r_end = utf8.getPrevGraphemeStart(text, text.len, 4, .hidden, .wcwidth);
     try testing.expect(r_end != null);
     try testing.expectEqual(@as(u32, 1), r_end.?.width);
 
-    const r_cjk = utf8.getPrevGraphemeStart(text, r_end.?.start_offset, 4, .wcwidth);
+    const r_cjk = utf8.getPrevGraphemeStart(text, r_end.?.start_offset, 4, .hidden, .wcwidth);
     try testing.expect(r_cjk != null);
     try testing.expectEqual(@as(u32, 2), r_cjk.?.width);
 
-    const r_b = utf8.getPrevGraphemeStart(text, r_cjk.?.start_offset, 4, .wcwidth);
+    const r_b = utf8.getPrevGraphemeStart(text, r_cjk.?.start_offset, 4, .hidden, .wcwidth);
     try testing.expect(r_b != null);
     try testing.expectEqual(@as(u32, 1), r_b.?.width);
 
-    const r_skin = utf8.getPrevGraphemeStart(text, r_b.?.start_offset, 4, .wcwidth);
+    const r_skin = utf8.getPrevGraphemeStart(text, r_b.?.start_offset, 4, .hidden, .wcwidth);
     try testing.expect(r_skin != null);
     try testing.expectEqual(@as(u32, 2), r_skin.?.width);
 
-    const r_wave = utf8.getPrevGraphemeStart(text, r_skin.?.start_offset, 4, .wcwidth);
+    const r_wave = utf8.getPrevGraphemeStart(text, r_skin.?.start_offset, 4, .hidden, .wcwidth);
     try testing.expect(r_wave != null);
     try testing.expectEqual(@as(u32, 2), r_wave.?.width);
 
-    const r_a = utf8.getPrevGraphemeStart(text, r_wave.?.start_offset, 4, .wcwidth);
+    const r_a = utf8.getPrevGraphemeStart(text, r_wave.?.start_offset, 4, .hidden, .wcwidth);
     try testing.expect(r_a != null);
     try testing.expectEqual(@as(u32, 1), r_a.?.width);
 }
@@ -204,7 +204,7 @@ test "wcwidth: findRenderClusterInfo with emoji" {
     defer result.deinit(testing.allocator);
 
     const text = "👋🏿"; // Wave + skin tone modifier
-    try utf8.findRenderClusterInfo(testing.allocator, text, 4, false, .wcwidth, &result);
+    try utf8.findRenderClusterInfo(testing.allocator, text, 4, .hidden, false, .wcwidth, &result);
 
     try testing.expectEqual(@as(usize, 1), result.items.len);
 
@@ -218,7 +218,7 @@ test "wcwidth: findRenderClusterInfo with ZWJ sequence" {
     defer result.deinit(testing.allocator);
 
     const text = "👩‍🚀"; // Woman + ZWJ + Rocket
-    try utf8.findRenderClusterInfo(testing.allocator, text, 4, false, .wcwidth, &result);
+    try utf8.findRenderClusterInfo(testing.allocator, text, 4, .hidden, false, .wcwidth, &result);
 
     try testing.expectEqual(@as(usize, 1), result.items.len);
 
@@ -232,7 +232,7 @@ test "wcwidth: findRenderClusterInfo with combining marks" {
     defer result.deinit(testing.allocator);
 
     const text = "e\u{0301}"; // e + combining acute
-    try utf8.findRenderClusterInfo(testing.allocator, text, 4, false, .wcwidth, &result);
+    try utf8.findRenderClusterInfo(testing.allocator, text, 4, .hidden, false, .wcwidth, &result);
 
     try testing.expectEqual(@as(usize, 1), result.items.len);
     try testing.expectEqual(@as(u32, 0), result.items[0].byte_start);
@@ -243,25 +243,25 @@ test "wcwidth: findRenderClusterInfo with combining marks" {
 test "wcwidth: tab width handling" {
     const text = "A\tB"; // A + tab + B
 
-    const total = utf8.calculateTextWidth(text, 4, false, .wcwidth);
+    const total = utf8.calculateTextWidth(text, 4, .hidden, false, .wcwidth);
     try testing.expectEqual(@as(u32, 6), total);
 
-    const tab_width = utf8.getWidthAt(text, 1, 4, .wcwidth);
+    const tab_width = utf8.getWidthAt(text, 1, 4, .hidden, .wcwidth);
     try testing.expectEqual(@as(u32, 4), tab_width);
 }
 
 test "wcwidth: boundary at wide character" {
     const text = "世X"; // 世(2) X(1) = 3 columns
 
-    const pos_start = utf8.findPosByWidth(text, 2, 4, false, false, .wcwidth);
+    const pos_start = utf8.findPosByWidth(text, 2, 4, .hidden, false, false, .wcwidth);
     try testing.expectEqual(@as(u32, 3), pos_start.byte_offset);
     try testing.expectEqual(@as(u32, 2), pos_start.columns_used);
 
-    const pos_end = utf8.findPosByWidth(text, 2, 4, false, true, .wcwidth);
+    const pos_end = utf8.findPosByWidth(text, 2, 4, .hidden, false, true, .wcwidth);
     try testing.expectEqual(@as(u32, 3), pos_end.byte_offset);
     try testing.expectEqual(@as(u32, 2), pos_end.columns_used);
 
-    const pos_3 = utf8.findPosByWidth(text, 3, 4, false, true, .wcwidth);
+    const pos_3 = utf8.findPosByWidth(text, 3, 4, .hidden, false, true, .wcwidth);
     try testing.expectEqual(@as(u32, 4), pos_3.byte_offset);
     try testing.expectEqual(@as(u32, 3), pos_3.columns_used);
 }

packages/native/src/tests/utf8_wcwidth_test.zig

@@ -6,7 +6,7 @@ test "findRenderClusterInfo wcwidth: empty string" {
     var result: std.ArrayListUnmanaged(utf8.RenderClusterInfo) = .empty;
     defer result.deinit(testing.allocator);
 
-    try utf8.findRenderClusterInfo(testing.allocator, "", 4, false, .wcwidth, &result);
+    try utf8.findRenderClusterInfo(testing.allocator, "", 4, .hidden, false, .wcwidth, &result);
     try testing.expectEqual(@as(usize, 0), result.items.len);
 }
 
@@ -14,7 +14,7 @@ test "findRenderClusterInfo wcwidth: ASCII-only returns empty" {
     var result: std.ArrayListUnmanaged(utf8.RenderClusterInfo) = .empty;
     defer result.deinit(testing.allocator);
 
-    try utf8.findRenderClusterInfo(testing.allocator, "hello world", 4, true, .wcwidth, &result);
+    try utf8.findRenderClusterInfo(testing.allocator, "hello world", 4, .hidden, true, .wcwidth, &result);
     try testing.expectEqual(@as(usize, 0), result.items.len);
 }
 
@@ -22,7 +22,7 @@ test "findRenderClusterInfo wcwidth: ASCII with tab" {
     var result: std.ArrayListUnmanaged(utf8.RenderClusterInfo) = .empty;
     defer result.deinit(testing.allocator);
 
-    try utf8.findRenderClusterInfo(testing.allocator, "hello\tworld", 4, false, .wcwidth, &result);
+    try utf8.findRenderClusterInfo(testing.allocator, "hello\tworld", 4, .hidden, false, .wcwidth, &result);
 
     // Should have one entry for the tab
     try testing.expectEqual(@as(usize, 1), result.items.len);
@@ -37,7 +37,7 @@ test "findRenderClusterInfo wcwidth: CJK characters" {
     defer result.deinit(testing.allocator);
 
     const text = "hello世界";
-    try utf8.findRenderClusterInfo(testing.allocator, text, 4, false, .wcwidth, &result);
+    try utf8.findRenderClusterInfo(testing.allocator, text, 4, .hidden, false, .wcwidth, &result);
 
     // Should have two entries for the CJK characters (each codepoint separately)
     try testing.expectEqual(@as(usize, 2), result.items.len);
@@ -60,7 +60,7 @@ test "findRenderClusterInfo wcwidth: emoji with skin tone - single grapheme clus
     defer result.deinit(testing.allocator);
 
     const text = "👋🏿"; // Wave + skin tone modifier
-    try utf8.findRenderClusterInfo(testing.allocator, text, 4, false, .wcwidth, &result);
+    try utf8.findRenderClusterInfo(testing.allocator, text, 4, .hidden, false, .wcwidth, &result);
 
     try testing.expectEqual(@as(usize, 1), result.items.len);
 
@@ -74,7 +74,7 @@ test "findRenderClusterInfo wcwidth: emoji with ZWJ - single grapheme cluster" {
     defer result.deinit(testing.allocator);
 
     const text = "👩‍🚀"; // Woman + ZWJ + Rocket (11 bytes total)
-    try utf8.findRenderClusterInfo(testing.allocator, text, 4, false, .wcwidth, &result);
+    try utf8.findRenderClusterInfo(testing.allocator, text, 4, .hidden, false, .wcwidth, &result);
 
     try testing.expectEqual(@as(usize, 1), result.items.len);
 
@@ -87,7 +87,7 @@ test "findRenderClusterInfo wcwidth: combining mark - part of base grapheme" {
     defer result.deinit(testing.allocator);
 
     const text = "e\u{0301}test"; // e + combining acute accent + test
-    try utf8.findRenderClusterInfo(testing.allocator, text, 4, false, .wcwidth, &result);
+    try utf8.findRenderClusterInfo(testing.allocator, text, 4, .hidden, false, .wcwidth, &result);
 
     try testing.expectEqual(@as(usize, 1), result.items.len);
     try testing.expectEqual(@as(u32, 0), result.items[0].byte_start);
@@ -103,8 +103,8 @@ test "findRenderClusterInfo wcwidth vs unicode: emoji with skin tone" {
 
     const text = "Hi👋🏿Bye";
 
-    try utf8.findRenderClusterInfo(testing.allocator, text, 4, false, .wcwidth, &result_wcwidth);
-    try utf8.findRenderClusterInfo(testing.allocator, text, 4, false, .unicode, &result_unicode);
+    try utf8.findRenderClusterInfo(testing.allocator, text, 4, .hidden, false, .wcwidth, &result_wcwidth);
+    try utf8.findRenderClusterInfo(testing.allocator, text, 4, .hidden, false, .unicode, &result_unicode);
 
     try testing.expectEqual(@as(usize, 1), result_wcwidth.items.len);
     try testing.expectEqual(@as(usize, 1), result_unicode.items.len);
@@ -127,8 +127,8 @@ test "findRenderClusterInfo wcwidth vs unicode: flag emoji" {
 
     const text = "🇺🇸"; // US flag (two regional indicators)
 
-    try utf8.findRenderClusterInfo(testing.allocator, text, 4, false, .wcwidth, &result_wcwidth);
-    try utf8.findRenderClusterInfo(testing.allocator, text, 4, false, .unicode, &result_unicode);
+    try utf8.findRenderClusterInfo(testing.allocator, text, 4, .hidden, false, .wcwidth, &result_wcwidth);
+    try utf8.findRenderClusterInfo(testing.allocator, text, 4, .hidden, false, .unicode, &result_unicode);
 
     try testing.expectEqual(@as(usize, 1), result_wcwidth.items.len);
     try testing.expectEqual(@as(usize, 1), result_unicode.items.len);
@@ -145,18 +145,18 @@ test "getWidthAt wcwidth: combining mark has zero width" {
     const text = "e\u{0301}"; // e + combining acute accent
 
     // In wcwidth mode, combining mark is a separate codepoint
-    const width_e = utf8.getWidthAt(text, 0, 8, .wcwidth);
+    const width_e = utf8.getWidthAt(text, 0, 8, .hidden, .wcwidth);
     try testing.expectEqual(@as(u32, 1), width_e); // Just 'e'
 
-    const width_combining = utf8.getWidthAt(text, 1, 8, .wcwidth);
+    const width_combining = utf8.getWidthAt(text, 1, 8, .hidden, .wcwidth);
     try testing.expectEqual(@as(u32, 0), width_combining); // Combining mark has width 0
 }
 
 test "calculateTextWidth wcwidth: emoji with skin tone counts both codepoints" {
     const text = "👋🏿"; // Wave + dark skin tone
 
-    const width_wcwidth = utf8.calculateTextWidth(text, 4, false, .wcwidth);
-    const width_unicode = utf8.calculateTextWidth(text, 4, false, .unicode);
+    const width_wcwidth = utf8.calculateTextWidth(text, 4, .hidden, false, .wcwidth);
+    const width_unicode = utf8.calculateTextWidth(text, 4, .hidden, false, .unicode);
 
     // wcwidth: counts both codepoints (2 + 2 = 4)
     try testing.expectEqual(@as(u32, 4), width_wcwidth);
@@ -168,8 +168,8 @@ test "calculateTextWidth wcwidth: emoji with skin tone counts both codepoints" {
 test "calculateTextWidth wcwidth: flag emoji counts both RIs" {
     const text = "🇺🇸"; // US flag
 
-    const width_wcwidth = utf8.calculateTextWidth(text, 4, false, .wcwidth);
-    const width_unicode = utf8.calculateTextWidth(text, 4, false, .unicode);
+    const width_wcwidth = utf8.calculateTextWidth(text, 4, .hidden, false, .wcwidth);
+    const width_unicode = utf8.calculateTextWidth(text, 4, .hidden, false, .unicode);
 
     // wcwidth: counts both regional indicators (1 + 1 = 2)
     try testing.expectEqual(@as(u32, 2), width_wcwidth);
@@ -185,8 +185,8 @@ test "calculateTextWidth wcwidth: flag emoji counts both RIs" {
 test "findWrapPosByWidth wcwidth: emoji with skin tone stops earlier" {
     const text = "Hi👋🏿Bye"; // H(1) i(1) wave(2) skin(2) B(1) y(1) e(1) = 10 cols wcwidth
 
-    const result_wcwidth = utf8.findWrapPosByWidth(text, 4, 4, false, .wcwidth);
-    const result_unicode = utf8.findWrapPosByWidth(text, 4, 4, false, .unicode);
+    const result_wcwidth = utf8.findWrapPosByWidth(text, 4, 4, .hidden, false, .wcwidth);
+    const result_unicode = utf8.findWrapPosByWidth(text, 4, 4, .hidden, false, .unicode);
 
     // wcwidth: stops after "Hi👋" = 4 columns (1+1+2)
     try testing.expectEqual(@as(u32, 6), result_wcwidth.byte_offset);
@@ -201,12 +201,12 @@ test "findPosByWidth wcwidth: emoji boundary behavior" {
     const text = "AB👋🏿CD"; // A(1) B(1) wave(2) skin(2) C(1) D(1)
 
     // With include_start_before=false (selection start)
-    const start3 = utf8.findPosByWidth(text, 3, 4, false, false, .wcwidth);
+    const start3 = utf8.findPosByWidth(text, 3, 4, .hidden, false, false, .wcwidth);
     // wcwidth: stops after "AB" at 2 columns (wave would exceed)
     try testing.expectEqual(@as(u32, 2), start3.byte_offset);
 
     // With include_start_before=true (selection end)
-    const end3 = utf8.findPosByWidth(text, 3, 4, false, true, .wcwidth);
+    const end3 = utf8.findPosByWidth(text, 3, 4, .hidden, false, true, .wcwidth);
     // wcwidth: includes wave since it starts at column 2 which is < 3
     try testing.expectEqual(@as(u32, 6), end3.byte_offset);
     try testing.expectEqual(@as(u32, 4), end3.columns_used);
@@ -216,13 +216,13 @@ test "getPrevGraphemeStart wcwidth: each codepoint separate" {
     const text = "Hi👋🏿";
 
     // From end of text (after skin tone)
-    const r_end = utf8.getPrevGraphemeStart(text, text.len, 4, .wcwidth);
+    const r_end = utf8.getPrevGraphemeStart(text, text.len, 4, .hidden, .wcwidth);
     try testing.expect(r_end != null);
     try testing.expectEqual(@as(usize, 6), r_end.?.start_offset); // Skin tone starts at byte 6
     try testing.expectEqual(@as(u32, 2), r_end.?.width);
 
     // From start of skin tone (byte 6)
-    const r_wave = utf8.getPrevGraphemeStart(text, 6, 4, .wcwidth);
+    const r_wave = utf8.getPrevGraphemeStart(text, 6, 4, .hidden, .wcwidth);
     try testing.expect(r_wave != null);
     try testing.expectEqual(@as(usize, 2), r_wave.?.start_offset); // Wave starts at byte 2
     try testing.expectEqual(@as(u32, 2), r_wave.?.width);
@@ -235,33 +235,33 @@ test "getPrevGraphemeStart wcwidth: each codepoint separate" {
 test "wcwidth: zero-width characters are handled correctly" {
     // ZWJ (Zero Width Joiner) should have width 0
     const text_zwj = "\u{200D}";
-    const width_zwj = utf8.calculateTextWidth(text_zwj, 4, false, .wcwidth);
+    const width_zwj = utf8.calculateTextWidth(text_zwj, 4, .hidden, false, .wcwidth);
     try testing.expectEqual(@as(u32, 0), width_zwj);
 
     // Combining marks should have width 0
     const text_combining = "e\u{0301}"; // e + combining acute
-    const width = utf8.calculateTextWidth(text_combining, 4, false, .wcwidth);
+    const width = utf8.calculateTextWidth(text_combining, 4, .hidden, false, .wcwidth);
     try testing.expectEqual(@as(u32, 1), width); // Only 'e' contributes
 }
 
 test "wcwidth: variation selectors" {
     // VS15 (text presentation) and VS16 (emoji presentation)
     const text_vs16 = "☺\u{FE0F}"; // Smiling face + VS16
-    const width_vs16 = utf8.calculateTextWidth(text_vs16, 4, false, .wcwidth);
+    const width_vs16 = utf8.calculateTextWidth(text_vs16, 4, .hidden, false, .wcwidth);
     try testing.expectEqual(@as(u32, 1), width_vs16); // Smiling face (1) + VS16 (0) = 1
 }
 
 test "wcwidth: regional indicators counted separately" {
     // Each regional indicator should contribute width 1
     const text = "🇺🇸"; // US flag = two regional indicators
-    const width = utf8.calculateTextWidth(text, 4, false, .wcwidth);
+    const width = utf8.calculateTextWidth(text, 4, .hidden, false, .wcwidth);
     try testing.expectEqual(@as(u32, 2), width); // Each RI has width 1
 }
 
 test "wcwidth: emoji ZWJ sequences split" {
     // Woman astronaut = woman + ZWJ + rocket
     const text = "👩‍🚀";
-    const width = utf8.calculateTextWidth(text, 4, false, .wcwidth);
+    const width = utf8.calculateTextWidth(text, 4, .hidden, false, .wcwidth);
     // Woman (2) + ZWJ (0) + Rocket (2) = 4
     try testing.expectEqual(@as(u32, 4), width);
 }
@@ -269,7 +269,7 @@ test "wcwidth: emoji ZWJ sequences split" {
 test "wcwidth: family emoji split into components" {
     // Family emoji with ZWJ
     const text = "👨‍👩‍👧"; // Man + ZWJ + Woman + ZWJ + Girl
-    const width = utf8.calculateTextWidth(text, 4, false, .wcwidth);
+    const width = utf8.calculateTextWidth(text, 4, .hidden, false, .wcwidth);
     // Man (2) + ZWJ (0) + Woman (2) + ZWJ (0) + Girl (2) = 6
     try testing.expectEqual(@as(u32, 6), width);
 }
@@ -277,21 +277,21 @@ test "wcwidth: family emoji split into components" {
 test "wcwidth: skin tone modifiers counted separately" {
     // Emoji with skin tone modifier
     const text = "👋🏻"; // Wave + light skin tone
-    const width = utf8.calculateTextWidth(text, 4, false, .wcwidth);
+    const width = utf8.calculateTextWidth(text, 4, .hidden, false, .wcwidth);
     // Wave (2) + Skin tone modifier (2) = 4
     try testing.expectEqual(@as(u32, 4), width);
 }
 
 test "wcwidth: CJK characters have width 2" {
     const text = "你好世界"; // 4 CJK characters
-    const width = utf8.calculateTextWidth(text, 4, false, .wcwidth);
+    const width = utf8.calculateTextWidth(text, 4, .hidden, false, .wcwidth);
     try testing.expectEqual(@as(u32, 8), width); // 4 * 2 = 8
 }
 
 test "wcwidth: mixed ASCII and emoji" {
     const text = "Hello👋World";
     // H(1) e(1) l(1) l(1) o(1) 👋(2) W(1) o(1) r(1) l(1) d(1) = 12
-    const width = utf8.calculateTextWidth(text, 4, false, .wcwidth);
+    const width = utf8.calculateTextWidth(text, 4, .hidden, false, .wcwidth);
     try testing.expectEqual(@as(u32, 12), width);
 }
 
@@ -299,7 +299,7 @@ test "wcwidth: findWrapPosByWidth with ZWJ sequences" {
     const text = "AB👩‍🚀CD"; // A(1) B(1) woman(2) ZWJ(0) rocket(2) C(1) D(1) = 8
 
     // Should wrap after woman emoji (before ZWJ)
-    const result = utf8.findWrapPosByWidth(text, 4, 4, false, .wcwidth);
+    const result = utf8.findWrapPosByWidth(text, 4, 4, .hidden, false, .wcwidth);
     try testing.expectEqual(@as(u32, 6), result.byte_offset); // After woman emoji
     try testing.expectEqual(@as(u32, 4), result.columns_used);
 }
@@ -309,13 +309,13 @@ test "wcwidth: findPosByWidth with skin tone modifier" {
 
     // With include_start_before=false, include codepoints that end at or before max_columns
     // Wave ends at column 4, which is at max_columns=4, so it's included
-    const start4 = utf8.findPosByWidth(text, 4, 4, false, false, .wcwidth);
+    const start4 = utf8.findPosByWidth(text, 4, 4, .hidden, false, false, .wcwidth);
     try testing.expectEqual(@as(u32, 6), start4.byte_offset); // After wave
     try testing.expectEqual(@as(u32, 4), start4.columns_used);
 
     // With include_start_before=true, include codepoints that start before max_columns
     // Wave starts at column 2 which is < 4, so it's included
-    const end4 = utf8.findPosByWidth(text, 4, 4, false, true, .wcwidth);
+    const end4 = utf8.findPosByWidth(text, 4, 4, .hidden, false, true, .wcwidth);
     try testing.expectEqual(@as(u32, 6), end4.byte_offset); // After wave
     try testing.expectEqual(@as(u32, 4), end4.columns_used);
 }
@@ -324,11 +324,11 @@ test "wcwidth: getWidthAt with combining marks" {
     const text = "e\u{0301}test"; // e + combining acute
 
     // Width at 'e' should be 1
-    const width_e = utf8.getWidthAt(text, 0, 4, .wcwidth);
+    const width_e = utf8.getWidthAt(text, 0, 4, .hidden, .wcwidth);
     try testing.expectEqual(@as(u32, 1), width_e);
 
     // Width at combining mark should be 0 (but next non-zero is 't')
-    const width_combining = utf8.getWidthAt(text, 1, 4, .wcwidth);
+    const width_combining = utf8.getWidthAt(text, 1, 4, .hidden, .wcwidth);
     try testing.expectEqual(@as(u32, 0), width_combining);
 }
 
@@ -336,19 +336,19 @@ test "wcwidth: getPrevGraphemeStart with ZWJ sequence" {
     const text = "AB👩‍🚀"; // A B woman ZWJ rocket
 
     // From end (after rocket)
-    const r1 = utf8.getPrevGraphemeStart(text, text.len, 4, .wcwidth);
+    const r1 = utf8.getPrevGraphemeStart(text, text.len, 4, .hidden, .wcwidth);
     try testing.expect(r1 != null);
     // Should point to rocket emoji (after ZWJ)
     try testing.expectEqual(@as(u32, 2), r1.?.width);
 
     // From rocket start, should go to ZWJ
-    const r2 = utf8.getPrevGraphemeStart(text, r1.?.start_offset, 4, .wcwidth);
+    const r2 = utf8.getPrevGraphemeStart(text, r1.?.start_offset, 4, .hidden, .wcwidth);
     try testing.expect(r2 != null);
 
     // Eventually should reach woman emoji
     var pos = text.len;
     var count: usize = 0;
-    while (utf8.getPrevGraphemeStart(text, pos, 4, .wcwidth)) |prev| {
+    while (utf8.getPrevGraphemeStart(text, pos, 4, .hidden, .wcwidth)) |prev| {
         pos = prev.start_offset;
         count += 1;
         if (count > 10) break; // Safety limit

packages/native/src/text-buffer-iterators.zig

@@ -199,7 +199,7 @@ pub const GraphemeBounds = struct {
 };
 
 /// Takes mutable rope for lazy marker cache rebuilding
-fn getTextUnitBoundsAt(rope: *UnifiedRope, mem_registry: *const MemRegistry, row: u32, col: u32, tab_width: u8, width_method: utf8.WidthMethod, cluster_graphemes: bool) ?GraphemeBounds {
+fn getTextUnitBoundsAt(rope: *UnifiedRope, mem_registry: *const MemRegistry, row: u32, col: u32, tab_width: u8, control_display: utf8.ControlDisplay, width_method: utf8.WidthMethod, cluster_graphemes: bool) ?GraphemeBounds {
     const line_width = lineWidthAt(rope, row);
     if (col >= line_width) return null;
 
@@ -217,14 +217,14 @@ fn getTextUnitBoundsAt(rope: *UnifiedRope, mem_registry: *const MemRegistry, row
                 const bytes = chunk.getBytes(mem_registry);
                 const is_ascii = (chunk.flags & TextChunk.Flags.ASCII_ONLY) != 0;
                 const pos = if (cluster_graphemes)
-                    utf8.findGraphemePosByWidth(bytes, local_col, tab_width, is_ascii, false, width_method)
+                    utf8.findGraphemePosByWidth(bytes, local_col, tab_width, control_display, is_ascii, false, width_method)
                 else
-                    utf8.findPosByWidth(bytes, local_col, tab_width, is_ascii, false, width_method);
+                    utf8.findPosByWidth(bytes, local_col, tab_width, control_display, is_ascii, false, width_method);
                 if (pos.byte_offset >= bytes.len) return null;
                 const width = if (cluster_graphemes)
-                    utf8.getGraphemeWidthAt(bytes, pos.byte_offset, tab_width, width_method)
+                    utf8.getGraphemeWidthAt(bytes, pos.byte_offset, tab_width, control_display, width_method)
                 else
-                    utf8.getWidthAt(bytes, pos.byte_offset, tab_width, width_method);
+                    utf8.getWidthAt(bytes, pos.byte_offset, tab_width, control_display, width_method);
                 const start = cols_before + pos.columns_used;
                 return .{ .start = start, .end = start + width };
             }
@@ -234,21 +234,21 @@ fn getTextUnitBoundsAt(rope: *UnifiedRope, mem_registry: *const MemRegistry, row
     return null;
 }
 
-pub fn getGraphemeBoundsAt(rope: *UnifiedRope, mem_registry: *const MemRegistry, row: u32, col: u32, tab_width: u8, width_method: utf8.WidthMethod) ?GraphemeBounds {
-    return getTextUnitBoundsAt(rope, mem_registry, row, col, tab_width, width_method, true);
+pub fn getGraphemeBoundsAt(rope: *UnifiedRope, mem_registry: *const MemRegistry, row: u32, col: u32, tab_width: u8, control_display: utf8.ControlDisplay, width_method: utf8.WidthMethod) ?GraphemeBounds {
+    return getTextUnitBoundsAt(rope, mem_registry, row, col, tab_width, control_display, width_method, true);
 }
 
-pub fn getCursorUnitBoundsAt(rope: *UnifiedRope, mem_registry: *const MemRegistry, row: u32, col: u32, tab_width: u8, width_method: utf8.WidthMethod) ?GraphemeBounds {
-    return getTextUnitBoundsAt(rope, mem_registry, row, col, tab_width, width_method, false);
+pub fn getCursorUnitBoundsAt(rope: *UnifiedRope, mem_registry: *const MemRegistry, row: u32, col: u32, tab_width: u8, control_display: utf8.ControlDisplay, width_method: utf8.WidthMethod) ?GraphemeBounds {
+    return getTextUnitBoundsAt(rope, mem_registry, row, col, tab_width, control_display, width_method, false);
 }
 
-pub fn getGraphemeWidthAt(rope: *UnifiedRope, mem_registry: *const MemRegistry, row: u32, col: u32, tab_width: u8, width_method: utf8.WidthMethod) u32 {
-    const bounds = getCursorUnitBoundsAt(rope, mem_registry, row, col, tab_width, width_method) orelse return 0;
+pub fn getGraphemeWidthAt(rope: *UnifiedRope, mem_registry: *const MemRegistry, row: u32, col: u32, tab_width: u8, control_display: utf8.ControlDisplay, width_method: utf8.WidthMethod) u32 {
+    const bounds = getCursorUnitBoundsAt(rope, mem_registry, row, col, tab_width, control_display, width_method) orelse return 0;
     return bounds.end -| col;
 }
 
 /// Takes mutable rope for lazy marker cache rebuilding
-pub fn getPrevGraphemeWidth(rope: *UnifiedRope, mem_registry: *const MemRegistry, row: u32, col: u32, tab_width: u8, width_method: utf8.WidthMethod) u32 {
+pub fn getPrevGraphemeWidth(rope: *UnifiedRope, mem_registry: *const MemRegistry, row: u32, col: u32, tab_width: u8, control_display: utf8.ControlDisplay, width_method: utf8.WidthMethod) u32 {
     if (col == 0) return 0;
 
     const line_width = lineWidthAt(rope, row);
@@ -270,7 +270,7 @@ pub fn getPrevGraphemeWidth(rope: *UnifiedRope, mem_registry: *const MemRegistry
                     // Exactly at chunk boundary - get last grapheme from previous chunk
                     const pc = prev_chunk.?;
                     const bytes = pc.chunk.getBytes(mem_registry);
-                    const prev = utf8.getPrevGraphemeStart(bytes, bytes.len, tab_width, width_method);
+                    const prev = utf8.getPrevGraphemeStart(bytes, bytes.len, tab_width, control_display, width_method);
                     if (prev) |res| {
                         return res.width;
                     }
@@ -281,7 +281,7 @@ pub fn getPrevGraphemeWidth(rope: *UnifiedRope, mem_registry: *const MemRegistry
                 const is_ascii = (chunk.flags & TextChunk.Flags.ASCII_ONLY) != 0;
                 const local_col: u32 = clamped_col - cols_before;
 
-                const here = utf8.findPosByWidth(bytes, local_col, tab_width, is_ascii, false, width_method);
+                const here = utf8.findPosByWidth(bytes, local_col, tab_width, control_display, is_ascii, false, width_method);
 
                 const grapheme_start_col = here.columns_used;
 
@@ -291,7 +291,7 @@ pub fn getPrevGraphemeWidth(rope: *UnifiedRope, mem_registry: *const MemRegistry
                     // We're in the middle of a wide grapheme cluster - need to look at previous chunk or grapheme
                     if (prev_chunk) |pc| {
                         const prev_bytes = pc.chunk.getBytes(mem_registry);
-                        const prev = utf8.getPrevGraphemeStart(prev_bytes, prev_bytes.len, tab_width, width_method);
+                        const prev = utf8.getPrevGraphemeStart(prev_bytes, prev_bytes.len, tab_width, control_display, width_method);
                         if (prev) |res| return res.width;
                     }
                     return 0;
@@ -301,7 +301,7 @@ pub fn getPrevGraphemeWidth(rope: *UnifiedRope, mem_registry: *const MemRegistry
 
                 if (offset_into_grapheme > 0) {
                     // We need to jump back: offset_into_grapheme + width of previous grapheme
-                    const prev = utf8.getPrevGraphemeStart(bytes, @intCast(here.byte_offset), tab_width, width_method);
+                    const prev = utf8.getPrevGraphemeStart(bytes, @intCast(here.byte_offset), tab_width, control_display, width_method);
                     if (prev) |res| {
                         const total_distance = offset_into_grapheme + res.width;
                         return total_distance;
@@ -309,7 +309,7 @@ pub fn getPrevGraphemeWidth(rope: *UnifiedRope, mem_registry: *const MemRegistry
                     return offset_into_grapheme;
                 }
 
-                const prev = utf8.getPrevGraphemeStart(bytes, @intCast(here.byte_offset), tab_width, width_method);
+                const prev = utf8.getPrevGraphemeStart(bytes, @intCast(here.byte_offset), tab_width, control_display, width_method);
                 if (prev) |res| {
                     return res.width;
                 }
@@ -332,6 +332,7 @@ pub fn extractTextBetweenOffsets(
     rope: *const UnifiedRope,
     mem_registry: *const MemRegistry,
     tab_width: u8,
+    control_display: utf8.ControlDisplay,
     start_offset: u32,
     end_offset: u32,
     out_buffer: ?[]u8,
@@ -345,6 +346,7 @@ pub fn extractTextBetweenOffsets(
     const Context = struct {
         mem_registry: *const MemRegistry,
         tab_width: u8,
+        control_display: utf8.ControlDisplay,
         out_buffer: ?[]u8,
         out_index: usize = 0,
         col_offset: u32,
@@ -381,12 +383,12 @@ pub fn extractTextBetweenOffsets(
             var byte_end: u32 = @intCast(chunk_bytes.len);
 
             if (local_start_col > 0) {
-                const start_result = utf8.findGraphemePosByWidth(chunk_bytes, local_start_col, ctx.tab_width, is_ascii_only, false, ctx.width_method);
+                const start_result = utf8.findGraphemePosByWidth(chunk_bytes, local_start_col, ctx.tab_width, ctx.control_display, is_ascii_only, false, ctx.width_method);
                 byte_start = start_result.byte_offset;
             }
 
             if (local_end_col < chunk.width_cols) {
-                const end_result = utf8.findGraphemePosByWidth(chunk_bytes, local_end_col, ctx.tab_width, is_ascii_only, true, ctx.width_method);
+                const end_result = utf8.findGraphemePosByWidth(chunk_bytes, local_end_col, ctx.tab_width, ctx.control_display, is_ascii_only, true, ctx.width_method);
                 byte_end = end_result.byte_offset;
             }
 
@@ -410,6 +412,7 @@ pub fn extractTextBetweenOffsets(
     var ctx: Context = .{
         .mem_registry = mem_registry,
         .tab_width = tab_width,
+        .control_display = control_display,
         .out_buffer = out_buffer,
         .col_offset = first.start_weight,
         .start = start_offset,

packages/native/src/text-buffer-segment.zig

@@ -43,6 +43,8 @@ const CachedMeasure = struct {
     result: WordMeasureSummary,
 };
 
+/// Cached layout is keyed by tab width and width method. A chunk belongs to one text buffer,
+/// whose control display never changes, so the keys omit it.
 pub const TextChunkColdState = struct {
     render_clusters: ?[]RenderClusterInfo = null,
     render_clusters_capacity: usize = 0,
@@ -221,6 +223,7 @@ pub const TextChunk = struct {
         allocator: Allocator,
         mem_registry: *const MemRegistry,
         tabwidth: u8,
+        control_display: utf8.ControlDisplay,
         width_method: utf8.WidthMethod,
     ) TextBufferError![]const RenderClusterInfo {
         if (self.isAsciiOnly()) return &[_]RenderClusterInfo{};
@@ -236,7 +239,7 @@ pub const TextChunk = struct {
                 .capacity = cold.render_clusters_capacity,
             };
             reusable.clearRetainingCapacity();
-            try utf8.findRenderClusterInfo(allocator, self.getBytes(mem_registry), tabwidth, self.isAsciiOnly(), width_method, &reusable);
+            try utf8.findRenderClusterInfo(allocator, self.getBytes(mem_registry), tabwidth, control_display, self.isAsciiOnly(), width_method, &reusable);
             cold.render_clusters = reusable.items;
             cold.render_clusters_capacity = reusable.capacity;
             cold.render_clusters_tab_width = tabwidth;
@@ -249,7 +252,7 @@ pub const TextChunk = struct {
         var render_cluster_list: std.ArrayListUnmanaged(RenderClusterInfo) = .empty;
         errdefer render_cluster_list.deinit(allocator);
 
-        try utf8.findRenderClusterInfo(allocator, chunk_bytes, tabwidth, self.isAsciiOnly(), width_method, &render_cluster_list);
+        try utf8.findRenderClusterInfo(allocator, chunk_bytes, tabwidth, control_display, self.isAsciiOnly(), width_method, &render_cluster_list);
 
         cold.render_clusters = render_cluster_list.items;
         cold.render_clusters_capacity = render_cluster_list.capacity;
@@ -265,9 +268,10 @@ pub const TextChunk = struct {
         cache: *ChunkLayoutCache,
         mem_registry: *const MemRegistry,
         tabwidth: u8,
+        control_display: utf8.ControlDisplay,
         width_method: utf8.WidthMethod,
     ) TextBufferError!ChunkLayoutInfo {
-        return self.getLayoutInfoForMode(false, allocator, cache, mem_registry, tabwidth, width_method);
+        return self.getLayoutInfoForMode(false, allocator, cache, mem_registry, tabwidth, control_display, width_method);
     }
 
     pub fn getWordLayoutInfo(
@@ -276,9 +280,10 @@ pub const TextChunk = struct {
         cache: *ChunkLayoutCache,
         mem_registry: *const MemRegistry,
         tabwidth: u8,
+        control_display: utf8.ControlDisplay,
         width_method: utf8.WidthMethod,
     ) TextBufferError!WordLayoutInfo {
-        return self.getLayoutInfoForMode(true, allocator, cache, mem_registry, tabwidth, width_method);
+        return self.getLayoutInfoForMode(true, allocator, cache, mem_registry, tabwidth, control_display, width_method);
     }
 
     fn getLayoutInfoForMode(
@@ -288,6 +293,7 @@ pub const TextChunk = struct {
         cache: *ChunkLayoutCache,
         mem_registry: *const MemRegistry,
         tabwidth: u8,
+        control_display: utf8.ControlDisplay,
         width_method: utf8.WidthMethod,
     ) TextBufferError!(if (word_only) WordLayoutInfo else ChunkLayoutInfo) {
         const cold = try self.getOrCreateCold(allocator);
@@ -330,9 +336,9 @@ pub const TextChunk = struct {
         }
         // ASCII has no line-only opportunities; both consumers share its scan.
         const word_classes = if (word_only or self.isAsciiOnly())
-            try utf8.findWordChunkLayoutInfo(cache.allocator, chunk_bytes, tabwidth, self.isAsciiOnly(), width_method, &wrap_breaks)
+            try utf8.findWordChunkLayoutInfo(cache.allocator, chunk_bytes, tabwidth, control_display, self.isAsciiOnly(), width_method, &wrap_breaks)
         else
-            try utf8.findLineAndWordChunkLayoutInfo(cache.allocator, chunk_bytes, tabwidth, false, width_method, &cjk_breaks, &wrap_breaks);
+            try utf8.findLineAndWordChunkLayoutInfo(cache.allocator, chunk_bytes, tabwidth, control_display, false, width_method, &cjk_breaks, &wrap_breaks);
         cold.wrap_breaks_tab_width = tabwidth;
         cold.line_breaks_ready = !word_only or !word_classes.has_cjk_breaks;
         cold.word_classes = word_classes;

packages/native/src/text-buffer-view.zig

@@ -1069,6 +1069,7 @@ pub const UnifiedTextBufferView = struct {
         var cols_before: u32 = 0;
         const mem_registry = self.text_buffer.memRegistry();
         const tab_width = self.text_buffer.tabWidth();
+        const control_display = self.text_buffer.controlDisplay();
         const width_method = self.text_buffer.widthMethod();
 
         while (seg_idx < rope.count()) : (seg_idx += 1) {
@@ -1083,23 +1084,25 @@ pub const UnifiedTextBufferView = struct {
                     bytes,
                     coords.col - cols_before,
                     tab_width,
+                    control_display,
                     is_ascii,
                     false,
                     width_method,
                 );
                 if (pos.byte_offset >= bytes.len) return null;
-                var width = utf8.getGraphemeWidthAt(bytes, pos.byte_offset, tab_width, width_method);
+                var width = utf8.getGraphemeWidthAt(bytes, pos.byte_offset, tab_width, control_display, width_method);
                 if (width == 0) {
                     const next = utf8.findGraphemePosByWidth(
                         bytes,
                         pos.columns_used + 1,
                         tab_width,
+                        control_display,
                         is_ascii,
                         false,
                         width_method,
                     );
                     if (next.byte_offset < bytes.len) {
-                        width = utf8.getGraphemeWidthAt(bytes, next.byte_offset, tab_width, width_method);
+                        width = utf8.getGraphemeWidthAt(bytes, next.byte_offset, tab_width, control_display, width_method);
                     }
                 }
                 const first_cp = utf8.decodeUtf8Unchecked(bytes, pos.byte_offset).cp;
@@ -1353,6 +1356,7 @@ pub const UnifiedTextBufferView = struct {
                     window,
                     keep_cols,
                     view.text_buffer.tabWidth(),
+                    view.text_buffer.controlDisplay(),
                     (chunk.chunk.flags & TextChunk.Flags.ASCII_ONLY) != 0,
                     view.text_buffer.widthMethod(),
                 );
@@ -1376,6 +1380,7 @@ pub const UnifiedTextBufferView = struct {
                     window,
                     drop_cols,
                     view.text_buffer.tabWidth(),
+                    view.text_buffer.controlDisplay(),
                     (chunk.chunk.flags & TextChunk.Flags.ASCII_ONLY) != 0,
                     true,
                     view.text_buffer.widthMethod(),
@@ -1857,6 +1862,7 @@ pub const UnifiedTextBufferView = struct {
                     slice_bytes,
                     max_width_cols,
                     wctx.text_buffer.tabWidth(),
+                    wctx.text_buffer.controlDisplay(),
                     is_ascii_only,
                     wctx.text_buffer.widthMethod(),
                 );
@@ -1869,7 +1875,7 @@ pub const UnifiedTextBufferView = struct {
 
                 // A zero-width prefix, such as an appended combining mark, stays with the preceding text.
                 if (!allow_forced_grapheme or max_width_cols == 0) {
-                    const prefix = if (is_ascii_only) 0 else utf8.zeroWidthPrefixLen(slice_bytes, wctx.text_buffer.tabWidth(), wctx.text_buffer.widthMethod());
+                    const prefix = if (is_ascii_only) 0 else utf8.zeroWidthPrefixLen(slice_bytes, wctx.text_buffer.tabWidth(), wctx.text_buffer.controlDisplay(), wctx.text_buffer.widthMethod());
                     return .{ .width_cols = 0, .bytes_used = @intCast(prefix) };
                 }
 
@@ -1877,6 +1883,7 @@ pub const UnifiedTextBufferView = struct {
                     slice_bytes,
                     max_width_cols,
                     wctx.text_buffer.tabWidth(),
+                    wctx.text_buffer.controlDisplay(),
                     is_ascii_only,
                     true,
                     wctx.text_buffer.widthMethod(),
@@ -1890,6 +1897,7 @@ pub const UnifiedTextBufferView = struct {
                     slice_bytes,
                     forced.columns_used,
                     wctx.text_buffer.tabWidth(),
+                    wctx.text_buffer.controlDisplay(),
                     is_ascii_only,
                     wctx.text_buffer.widthMethod(),
                 );
@@ -2125,7 +2133,7 @@ pub const UnifiedTextBufferView = struct {
                             const layout_allocator = storage.arena.allocator();
                             if (layout == null) {
                                 var breaks: std.ArrayListUnmanaged(utf8.LayoutWrapBreak) = .empty;
-                                const classes = utf8.findChunkLayoutInfo(layout_allocator, chunk_bytes, wctx.text_buffer.tabWidth(), chunk.isAsciiOnly(), wctx.text_buffer.widthMethod(), &breaks) catch break :save false;
+                                const classes = utf8.findChunkLayoutInfo(layout_allocator, chunk_bytes, wctx.text_buffer.tabWidth(), wctx.text_buffer.controlDisplay(), chunk.isAsciiOnly(), wctx.text_buffer.widthMethod(), &breaks) catch break :save false;
                                 layout = .{ .wrap_breaks = breaks.toOwnedSlice(layout_allocator) catch break :save false, .word_classes = classes };
                             }
                             storage.layouts.append(layout_allocator, layout.?) catch break :save false;
@@ -2177,6 +2185,7 @@ pub const UnifiedTextBufferView = struct {
                     const streamed_layout = utf8.walkChunkLayoutInfoComptime(
                         chunk_bytes,
                         wctx.text_buffer.tabWidth(),
+                        wctx.text_buffer.controlDisplay(),
                         chunk.isAsciiOnly(),
                         wctx.text_buffer.widthMethod(),
                         wctx,
@@ -2207,10 +2216,10 @@ pub const UnifiedTextBufferView = struct {
 
             /// One grapheme that is wider than the space left, plus the zero-width clusters
             /// after it, which the wrap search also keeps on the line.
-            fn forceCharGrapheme(comptime width_method: utf8.WidthMethod, bytes: []const u8, tab_width: u8, is_ascii_only: bool) utf8.PosByWidthResult {
-                const forced = utf8.findGraphemePosByWidth(bytes, 1, tab_width, is_ascii_only, true, width_method);
+            fn forceCharGrapheme(comptime width_method: utf8.WidthMethod, bytes: []const u8, tab_width: u8, control_display: utf8.ControlDisplay, is_ascii_only: bool) utf8.PosByWidthResult {
+                const forced = utf8.findGraphemePosByWidth(bytes, 1, tab_width, control_display, is_ascii_only, true, width_method);
                 if (forced.grapheme_count == 0) return forced;
-                const kept = utf8.findWrapPosByWidthGraphemeSafe(bytes, forced.columns_used, tab_width, is_ascii_only, width_method);
+                const kept = utf8.findWrapPosByWidthGraphemeSafe(bytes, forced.columns_used, tab_width, control_display, is_ascii_only, width_method);
                 return .{ .byte_offset = @max(forced.byte_offset, kept.byte_offset), .grapheme_count = forced.grapheme_count, .columns_used = forced.columns_used };
             }
 
@@ -2218,12 +2227,13 @@ pub const UnifiedTextBufferView = struct {
                 const chunk_bytes = chunk.getBytes(wctx.text_buffer.memRegistry());
                 const is_ascii_only = (chunk.flags & TextChunk.Flags.ASCII_ONLY) != 0;
                 const tab_width = wctx.text_buffer.tabWidth();
+                const control_display = wctx.text_buffer.controlDisplay();
                 var chunk_byte_offset: usize = 0;
                 var chunk_col_offset: u32 = 0;
                 // Zero-width clusters stay with the preceding grapheme, as in unsplit text,
                 // even when an edit put them at the start of a chunk.
                 if (!is_ascii_only) {
-                    chunk_byte_offset = utf8.zeroWidthPrefixLen(chunk_bytes, tab_width, width_method);
+                    chunk_byte_offset = utf8.zeroWidthPrefixLen(chunk_bytes, tab_width, control_display, width_method);
                     try addVirtualChunk(wctx, chunk, 0, @intCast(chunk_byte_offset), 0, 0);
                 }
 
@@ -2239,7 +2249,7 @@ pub const UnifiedTextBufferView = struct {
                             continue;
                         }
                         const remaining_bytes = chunk_bytes[chunk_byte_offset..];
-                        const force_result = forceCharGrapheme(width_method, remaining_bytes, tab_width, is_ascii_only);
+                        const force_result = forceCharGrapheme(width_method, remaining_bytes, tab_width, control_display, is_ascii_only);
                         if (force_result.grapheme_count > 0) {
                             try addVirtualChunk(wctx, chunk, @intCast(chunk_byte_offset), force_result.byte_offset, chunk_col_offset, force_result.columns_used);
                             chunk_col_offset += force_result.columns_used;
@@ -2255,6 +2265,7 @@ pub const UnifiedTextBufferView = struct {
                         remaining_bytes,
                         remaining_width_cols,
                         tab_width,
+                        control_display,
                         is_ascii_only,
                         width_method,
                     );
@@ -2264,7 +2275,7 @@ pub const UnifiedTextBufferView = struct {
                             try commitVirtualLine(wctx);
                             continue;
                         }
-                        const force_result = forceCharGrapheme(width_method, remaining_bytes, tab_width, is_ascii_only);
+                        const force_result = forceCharGrapheme(width_method, remaining_bytes, tab_width, control_display, is_ascii_only);
                         if (force_result.grapheme_count > 0) {
                             try addVirtualChunk(wctx, chunk, @intCast(chunk_byte_offset), force_result.byte_offset, chunk_col_offset, force_result.columns_used);
                             chunk_col_offset += force_result.columns_used;

packages/native/src/text-buffer.zig

@@ -67,6 +67,8 @@ pub const UnifiedTextBuffer = struct {
     link_tracker: ?link.LinkTracker,
 
     width_method: utf8.WidthMethod,
+    /// Fixed at creation: chunk layout caches omit it from their keys.
+    control_display: utf8.ControlDisplay,
 
     view_dirty_flags: std.ArrayListUnmanaged(bool),
     next_view_id: u32,
@@ -119,6 +121,10 @@ pub const UnifiedTextBuffer = struct {
         return self.tab_width;
     }
 
+    pub fn controlDisplay(self: *const Self) utf8.ControlDisplay {
+        return self.control_display;
+    }
+
     /// Accessor: return the internal allocator.
     pub fn getAllocator(self: *const Self) Allocator {
         return self.allocator;
@@ -158,11 +164,11 @@ pub const UnifiedTextBuffer = struct {
     }
 
     pub fn getGraphemeWidthAt(self: *const Self, row: u32, col: u32) u32 {
-        return iter_mod.getGraphemeWidthAt(@constCast(&self._rope), &self.mem_registry, row, col, self.tab_width, self.width_method);
+        return iter_mod.getGraphemeWidthAt(@constCast(&self._rope), &self.mem_registry, row, col, self.tab_width, self.control_display, self.width_method);
     }
 
     pub fn getPrevGraphemeWidth(self: *const Self, row: u32, col: u32) u32 {
-        return iter_mod.getPrevGraphemeWidth(@constCast(&self._rope), &self.mem_registry, row, col, self.tab_width, self.width_method);
+        return iter_mod.getPrevGraphemeWidth(@constCast(&self._rope), &self.mem_registry, row, col, self.tab_width, self.control_display, self.width_method);
     }
 
     pub fn textEndOffset(self: *const Self) u32 {
@@ -178,6 +184,7 @@ pub const UnifiedTextBuffer = struct {
             coords.row,
             coords.col,
             self.tab_width,
+            self.control_display,
             self.width_method,
         ) orelse return null;
         const line_start = offset - coords.col;
@@ -193,6 +200,7 @@ pub const UnifiedTextBuffer = struct {
             coords.row,
             coords.col,
             self.tab_width,
+            self.control_display,
             self.width_method,
         ) orelse return null;
         const line_start = offset - coords.col;
@@ -200,11 +208,11 @@ pub const UnifiedTextBuffer = struct {
     }
 
     pub fn getLayoutInfoFor(self: *const Self, chunk: *const TextChunk) TextBufferError!seg_mod.ChunkLayoutInfo {
-        return chunk.getLayoutInfo(self.allocator, @constCast(&self.layout_cache), &self.mem_registry, self.tab_width, self.width_method);
+        return chunk.getLayoutInfo(self.allocator, @constCast(&self.layout_cache), &self.mem_registry, self.tab_width, self.control_display, self.width_method);
     }
 
     pub fn getWordLayoutInfoFor(self: *const Self, chunk: *const TextChunk) TextBufferError!seg_mod.WordLayoutInfo {
-        return chunk.getWordLayoutInfo(self.allocator, @constCast(&self.layout_cache), &self.mem_registry, self.tab_width, self.width_method);
+        return chunk.getWordLayoutInfo(self.allocator, @constCast(&self.layout_cache), &self.mem_registry, self.tab_width, self.control_display, self.width_method);
     }
 
     /// Accessor: walk all lines and segments via callbacks.
@@ -220,6 +228,7 @@ pub const UnifiedTextBuffer = struct {
     pub const InitOptions = struct {
         io: std.Io = compatibility_io,
         logger: *const logger.Logger = logger.processLogger(),
+        control_display: utf8.ControlDisplay = .hidden,
     };
 
     pub fn init(
@@ -292,6 +301,7 @@ pub const UnifiedTextBuffer = struct {
             .link_pool = link_pool,
             .link_tracker = null,
             .width_method = width_method,
+            .control_display = options.control_display,
             .view_dirty_flags = view_dirty_flags,
             .next_view_id = 0,
             .free_view_ids = free_view_ids,
@@ -370,6 +380,7 @@ pub const UnifiedTextBuffer = struct {
         try self.initStorage(self.global_allocator, self.pool, self.link_pool, width_method, .{
             .io = self.io,
             .logger = self.logger,
+            .control_display = self.control_display,
         }, self.arena);
         self.mem_registry = registry;
         self.view_dirty_flags = view_dirty_flags;
@@ -471,7 +482,7 @@ pub const UnifiedTextBuffer = struct {
         // For grapheme-accurate width calculation (used by highlighting system),
         // use utf8.calculateTextWidth which properly handles grapheme clusters
         const is_ascii = utf8.isAsciiOnly(text);
-        return utf8.calculateTextWidth(text, self.tab_width, is_ascii, self.width_method);
+        return utf8.calculateTextWidth(text, self.tab_width, self.control_display, is_ascii, self.width_method);
     }
 
     /// Clear text while preserving highlights, registered bytes, and undo history.
@@ -824,6 +835,7 @@ pub const UnifiedTextBuffer = struct {
         var width_cursor = utf8.TextWidthCursor{
             .text = text,
             .tab_width = self.tab_width,
+            .control_display = self.control_display,
             .width_method = self.width_method,
         };
         var chunk_index: usize = 0;
@@ -1097,7 +1109,7 @@ pub const UnifiedTextBuffer = struct {
             flags |= TextChunk.Flags.HAS_TAB;
         }
 
-        const chunk_width = utf8.calculateTextWidth(chunk_bytes, self.tab_width, is_ascii, self.width_method);
+        const chunk_width = utf8.calculateTextWidth(chunk_bytes, self.tab_width, self.control_display, is_ascii, self.width_method);
 
         return .{
             .mem_id = mem_id,
@@ -1738,6 +1750,7 @@ pub const UnifiedTextBuffer = struct {
                     mutable.width_cols = utf8.calculateTextWidth(
                         bytes,
                         ctx.buffer.tab_width,
+                        ctx.buffer.control_display,
                         chunk.isAsciiOnly(),
                         ctx.buffer.width_method,
                     );
@@ -1791,6 +1804,7 @@ pub const UnifiedTextBuffer = struct {
             &self._rope,
             &self.mem_registry,
             self.tab_width,
+            self.control_display,
             start_offset,
             clamped_end,
             out_buffer,
@@ -1804,6 +1818,7 @@ pub const UnifiedTextBuffer = struct {
             &self._rope,
             &self.mem_registry,
             self.tab_width,
+            self.control_display,
             start_offset,
             @min(end_offset, self._rope.totalWeight()),
             null,

packages/native/src/utf8.zig

@@ -527,30 +527,66 @@ inline fn eawToWidth(cp: u21, eaw: uucode.types.EastAsianWidth) i16 {
     return 1;
 }
 
+/// How layout measures a control character: a C0 control other than tab, line feed, and
+/// carriage return, DEL, or a C1 control. Line feed and carriage return break lines.
+/// Grapheme segmentation always puts a control in a cluster of its own (UAX #29 GB4 and GB5).
+pub const ControlDisplay = enum(u8) {
+    /// Zero width. Read-only text hides controls.
+    hidden,
+    /// Vim notation: C0 controls and DEL show as `^@`..`^_` and `^?` in two cells, and C1
+    /// controls as lowercase hex `<80>`..`<9f>` in four. An editor places its cursor by cell,
+    /// so it shows every control as notation that the cursor can stop on and delete.
+    notation,
+};
+
+/// Cell count of the notation for `cp`, or 0 when `cp` has none, as tab, line feed, and carriage return.
+pub fn controlNotationWidth(cp: u21) u32 {
+    if (cp == '\t' or cp == '\n' or cp == '\r') return 0;
+    if (cp < 0x20 or cp == 0x7f) return 2;
+    if (cp >= 0x80 and cp < 0xa0) return 4;
+    return 0;
+}
+
+/// Printable ASCII byte of notation cell `index` for control `cp`.
+pub fn controlNotationGlyph(cp: u21, index: u32) u8 {
+    std.debug.assert(index < controlNotationWidth(cp));
+    const digits = "0123456789abcdef";
+    return switch (index) {
+        0 => if (cp < 0x80) '^' else '<',
+        // DEL is 0x7f and shows as `^?` (0x3f).
+        1 => if (cp < 0x80) @intCast(cp ^ 0x40) else digits[cp >> 4],
+        2 => digits[cp & 0xf],
+        else => '>',
+    };
+}
+
+inline fn controlWidth(cp: u21, control_display: ControlDisplay) u32 {
+    return switch (control_display) {
+        .hidden => 0,
+        .notation => controlNotationWidth(cp),
+    };
+}
+
 /// Calculate the display width of a byte in columns
 /// Used for ASCII-only fast paths
-inline fn asciiCharWidth(byte: u8, tab_width: u8) u32 {
+inline fn asciiCharWidth(byte: u8, tab_width: u8, control_display: ControlDisplay) u32 {
     if (byte == '\t') {
         return tab_width;
     } else if (byte >= 32 and byte <= 126) {
         return 1;
     }
-    return 0;
+    return controlWidth(byte, control_display);
 }
 
 /// Calculate the display width of a character (byte or codepoint) in columns
-inline fn charWidth(byte: u8, codepoint: u21, tab_width: u8) u32 {
-    if (byte == '\t') {
-        return tab_width;
-    } else if (byte < 0x80 and byte >= 32 and byte <= 126) {
-        return 1;
-    } else if (byte >= 0x80) {
-        if (codepoint > 0x10FFFF) return 0;
-        const eaw = uucode.get(.east_asian_width, codepoint);
-        const w = eawToWidth(codepoint, eaw);
-        return if (w > 0) @intCast(w) else 0;
-    }
-    return 0;
+inline fn charWidth(byte: u8, codepoint: u21, tab_width: u8, control_display: ControlDisplay) u32 {
+    if (byte < 0x80) return asciiCharWidth(byte, tab_width, control_display);
+    if (codepoint > 0x10FFFF) return 0;
+    const eaw = uucode.get(.east_asian_width, codepoint);
+    const w = eawToWidth(codepoint, eaw);
+    // eawToWidth marks a C1 control with -1.
+    if (w < 0) return controlWidth(codepoint, control_display);
+    return if (w > 0) @intCast(w) else 0;
 }
 
 /// Check if a codepoint is valid for grapheme break detection
@@ -564,42 +600,14 @@ inline fn isValidCodepoint(cp: u21) bool {
 /// - no_zwj mode: use grapheme breaks but treat ZWJ as a break (ignore joining)
 /// - unicode mode: use standard grapheme cluster segmentation
 pub inline fn isGraphemeBreak(prev_cp: ?u21, curr_cp: u21, break_state: *uucode.grapheme.BreakState, width_method: WidthMethod) bool {
-    // wcwidth mode uses Unicode grapheme clustering for proper rendering
-    // (ZWJ sequences, skin tone modifiers stay together), but width is
-    // calculated using wcwidth semantics (sum of codepoint widths)
-    if (width_method == .wcwidth) {
-        if (prev_cp == null) return true;
-
-        if (!isValidCodepoint(curr_cp)) return true;
-        if (!isValidCodepoint(prev_cp.?)) return true;
-        return uucode.grapheme.isBreak(prev_cp.?, curr_cp, break_state);
-    }
-
-    if (!isValidCodepoint(curr_cp)) return true;
-
-    // In no_zwj mode, treat ZWJ (U+200D) as NOT joining characters
-    // When we see ZWJ after a character, it's part of that character's grapheme
-    // But when we see a character after ZWJ, it starts a new grapheme
-    if (width_method == .no_zwj) {
-        const ZWJ: u21 = 0x200D;
-        if (prev_cp) |p| {
-            // If previous was ZWJ, current starts a new grapheme
-            // Don't call uucode.grapheme.isBreak because it will say no break
-            if (p == ZWJ) {
-                // Reset break state since we're forcing a break
-                break_state.* = .default;
-                return true;
-            }
-        }
-        // If current is ZWJ, don't break yet - it's part of previous grapheme
-        // (will have width 0 anyway)
-    }
-
-    if (prev_cp) |p| {
-        if (!isValidCodepoint(p)) return true;
-        return uucode.grapheme.isBreak(p, curr_cp, break_state);
-    }
-    return true;
+    const p = prev_cp orelse return true;
+    // U+FFFD stands alone, and in no_zwj mode a character after ZWJ starts a new grapheme. A forced
+    // break ends any joining sequence, as UAX #29 does, so the graphemes after it segment the same
+    // as when a scan starts there.
+    const forced = !isValidCodepoint(curr_cp) or !isValidCodepoint(p) or
+        (width_method == .no_zwj and p == 0x200D);
+    if (forced) break_state.* = .default;
+    return forced or uucode.grapheme.isBreak(p, curr_cp, break_state);
 }
 
 /// State for accumulating grapheme cluster width
@@ -777,12 +785,13 @@ pub fn findWrapPosByWidth(
     text: []const u8,
     max_columns: u32,
     tab_width: u8,
+    control_display: ControlDisplay,
     isASCIIOnly: bool,
     width_method: WidthMethod,
 ) WrapByWidthResult {
     switch (width_method) {
-        .unicode, .unicode_wide, .no_zwj => return findWrapPosByWidthUnicode(text, max_columns, tab_width, isASCIIOnly, width_method),
-        .wcwidth => return findWrapPosByWidthWCWidth(text, max_columns, tab_width, isASCIIOnly),
+        .unicode, .unicode_wide, .no_zwj => return findWrapPosByWidthUnicode(text, max_columns, tab_width, control_display, isASCIIOnly, width_method),
+        .wcwidth => return findWrapPosByWidthWCWidth(text, max_columns, tab_width, control_display, isASCIIOnly),
     }
 }
 
@@ -792,13 +801,14 @@ pub fn findWrapPosByWidthGraphemeSafe(
     text: []const u8,
     max_columns: u32,
     tab_width: u8,
+    control_display: ControlDisplay,
     isASCIIOnly: bool,
     width_method: WidthMethod,
 ) WrapByWidthResult {
     if (width_method == .wcwidth) {
-        return findWrapPosByWidthUnicode(text, max_columns, tab_width, isASCIIOnly, .wcwidth);
+        return findWrapPosByWidthUnicode(text, max_columns, tab_width, control_display, isASCIIOnly, .wcwidth);
     }
-    return findWrapPosByWidth(text, max_columns, tab_width, isASCIIOnly, width_method);
+    return findWrapPosByWidth(text, max_columns, tab_width, control_display, isASCIIOnly, width_method);
 }
 
 /// Find wrap position by width using Unicode grapheme cluster segmentation
@@ -806,6 +816,7 @@ fn findWrapPosByWidthUnicode(
     text: []const u8,
     max_columns: u32,
     tab_width: u8,
+    control_display: ControlDisplay,
     isASCIIOnly: bool,
     width_method: WidthMethod,
 ) WrapByWidthResult {
@@ -843,7 +854,7 @@ fn findWrapPosByWidthUnicode(
                     return .{ .byte_offset = @intCast(state.cluster_start), .grapheme_count = state.grapheme_count, .columns_used = state.columns_used };
                 }
 
-                const cp_width = asciiCharWidth(b, tab_width);
+                const cp_width = asciiCharWidth(b, tab_width, control_display);
                 if (!state.cluster_started) {
                     state.width_state = GraphemeWidthState.init(curr_cp, cp_width, width_method);
                     state.cluster_width = cp_width;
@@ -873,7 +884,7 @@ fn findWrapPosByWidthUnicode(
                 return .{ .byte_offset = @intCast(state.cluster_start), .grapheme_count = state.grapheme_count, .columns_used = state.columns_used };
             }
 
-            const cp_width = charWidth(b0, curr_cp, tab_width);
+            const cp_width = charWidth(b0, curr_cp, tab_width, control_display);
             if (!state.cluster_started) {
                 state.width_state = GraphemeWidthState.init(curr_cp, cp_width, width_method);
                 state.cluster_width = cp_width;
@@ -900,7 +911,7 @@ fn findWrapPosByWidthUnicode(
             return .{ .byte_offset = @intCast(state.cluster_start), .grapheme_count = state.grapheme_count, .columns_used = state.columns_used };
         }
 
-        const cp_width = charWidth(b0, curr_cp, tab_width);
+        const cp_width = charWidth(b0, curr_cp, tab_width, control_display);
         if (!state.cluster_started) {
             state.width_state = GraphemeWidthState.init(curr_cp, cp_width, width_method);
             state.cluster_width = cp_width;
@@ -930,6 +941,7 @@ fn findWrapPosByWidthWCWidth(
     text: []const u8,
     max_columns: u32,
     tab_width: u8,
+    control_display: ControlDisplay,
     isASCIIOnly: bool,
 ) WrapByWidthResult {
     if (text.len == 0 or max_columns == 0) {
@@ -961,7 +973,7 @@ fn findWrapPosByWidthWCWidth(
 
         if (pos + cp_len > text.len) break;
 
-        const cp_width = charWidth(b0, curr_cp, tab_width);
+        const cp_width = charWidth(b0, curr_cp, tab_width, control_display);
 
         // In wcwidth mode, stop if we've already used max_columns
         // (don't continue adding zero-width chars after reaching limit)
@@ -991,13 +1003,14 @@ pub fn findPosByWidth(
     text: []const u8,
     max_columns: u32,
     tab_width: u8,
+    control_display: ControlDisplay,
     isASCIIOnly: bool,
     include_start_before: bool,
     width_method: WidthMethod,
 ) PosByWidthResult {
     switch (width_method) {
-        .unicode, .unicode_wide, .no_zwj => return findPosByWidthUnicode(text, max_columns, tab_width, isASCIIOnly, include_start_before, width_method),
-        .wcwidth => return findPosByWidthWCWidth(text, max_columns, tab_width, isASCIIOnly, include_start_before),
+        .unicode, .unicode_wide, .no_zwj => return findPosByWidthUnicode(text, max_columns, tab_width, control_display, isASCIIOnly, include_start_before, width_method),
+        .wcwidth => return findPosByWidthWCWidth(text, max_columns, tab_width, control_display, isASCIIOnly, include_start_before),
     }
 }
 
@@ -1007,11 +1020,12 @@ pub fn findGraphemePosByWidth(
     text: []const u8,
     max_columns: u32,
     tab_width: u8,
+    control_display: ControlDisplay,
     isASCIIOnly: bool,
     include_start_before: bool,
     width_method: WidthMethod,
 ) PosByWidthResult {
-    return findPosByWidthUnicode(text, max_columns, tab_width, isASCIIOnly, include_start_before, width_method);
+    return findPosByWidthUnicode(text, max_columns, tab_width, control_display, isASCIIOnly, include_start_before, width_method);
 }
 
 /// Find position by column width using Unicode grapheme cluster segmentation
@@ -1019,6 +1033,7 @@ fn findPosByWidthUnicode(
     text: []const u8,
     max_columns: u32,
     tab_width: u8,
+    control_display: ControlDisplay,
     isASCIIOnly: bool,
     include_start_before: bool,
     width_method: WidthMethod,
@@ -1057,7 +1072,7 @@ fn findPosByWidthUnicode(
                     return .{ .byte_offset = @intCast(state.cluster_start), .grapheme_count = state.grapheme_count, .columns_used = state.columns_used };
                 }
 
-                const cp_width = asciiCharWidth(b, tab_width);
+                const cp_width = asciiCharWidth(b, tab_width, control_display);
                 if (!state.cluster_started) {
                     state.width_state = GraphemeWidthState.init(curr_cp, cp_width, width_method);
                     state.cluster_width = cp_width;
@@ -1087,7 +1102,7 @@ fn findPosByWidthUnicode(
                 return .{ .byte_offset = @intCast(state.cluster_start), .grapheme_count = state.grapheme_count, .columns_used = state.columns_used };
             }
 
-            const cp_width = charWidth(b0, curr_cp, tab_width);
+            const cp_width = charWidth(b0, curr_cp, tab_width, control_display);
             if (!state.cluster_started) {
                 state.width_state = GraphemeWidthState.init(curr_cp, cp_width, width_method);
                 state.cluster_width = cp_width;
@@ -1114,7 +1129,7 @@ fn findPosByWidthUnicode(
             return .{ .byte_offset = @intCast(state.cluster_start), .grapheme_count = state.grapheme_count, .columns_used = state.columns_used };
         }
 
-        const cp_width = charWidth(b0, curr_cp, tab_width);
+        const cp_width = charWidth(b0, curr_cp, tab_width, control_display);
         if (!state.cluster_started) {
             state.width_state = GraphemeWidthState.init(curr_cp, cp_width, width_method);
             state.cluster_width = cp_width;
@@ -1149,6 +1164,7 @@ fn findPosByWidthWCWidth(
     text: []const u8,
     max_columns: u32,
     tab_width: u8,
+    control_display: ControlDisplay,
     isASCIIOnly: bool,
     include_start_before: bool,
 ) PosByWidthResult {
@@ -1181,7 +1197,7 @@ fn findPosByWidthWCWidth(
 
         if (pos + cp_len > text.len) break;
 
-        const cp_width = charWidth(b0, curr_cp, tab_width);
+        const cp_width = charWidth(b0, curr_cp, tab_width, control_display);
         const cp_start_col = columns_used;
         const cp_end_col = columns_used + cp_width;
 
@@ -1209,7 +1225,7 @@ fn findPosByWidthWCWidth(
 
 /// Byte length of the zero-width grapheme clusters that start `text`, such as a combining
 /// mark that an edit placed in a chunk of its own.
-pub fn zeroWidthPrefixLen(text: []const u8, tab_width: u8, width_method: WidthMethod) usize {
+pub fn zeroWidthPrefixLen(text: []const u8, tab_width: u8, control_display: ControlDisplay, width_method: WidthMethod) usize {
     var pos: usize = 0;
     var cluster_start: usize = 0;
     var prev_cp: ?u21 = null;
@@ -1217,7 +1233,7 @@ pub fn zeroWidthPrefixLen(text: []const u8, tab_width: u8, width_method: WidthMe
     var width_state = GraphemeWidthState.init(0, 0, width_method);
     while (pos < text.len) {
         const decoded = decodeUtf8Unchecked(text, pos);
-        const cp_width = charWidth(text[pos], decoded.cp, tab_width);
+        const cp_width = charWidth(text[pos], decoded.cp, tab_width, control_display);
         if (prev_cp == null or isGraphemeBreak(prev_cp, decoded.cp, &break_state, width_method)) {
             if (width_state.width > 0) return cluster_start;
             cluster_start = pos;
@@ -1232,19 +1248,19 @@ pub fn zeroWidthPrefixLen(text: []const u8, tab_width: u8, width_method: WidthMe
 }
 
 /// Get width at byte offset - proxy function that dispatches based on width_method
-pub fn getWidthAt(text: []const u8, byte_offset: usize, tab_width: u8, width_method: WidthMethod) u32 {
+pub fn getWidthAt(text: []const u8, byte_offset: usize, tab_width: u8, control_display: ControlDisplay, width_method: WidthMethod) u32 {
     switch (width_method) {
-        .unicode, .unicode_wide, .no_zwj => return getWidthAtUnicode(text, byte_offset, tab_width, width_method),
-        .wcwidth => return getWidthAtWCWidth(text, byte_offset, tab_width),
+        .unicode, .unicode_wide, .no_zwj => return getWidthAtUnicode(text, byte_offset, tab_width, control_display, width_method),
+        .wcwidth => return getWidthAtWCWidth(text, byte_offset, tab_width, control_display),
     }
 }
 
-pub fn getGraphemeWidthAt(text: []const u8, byte_offset: usize, tab_width: u8, width_method: WidthMethod) u32 {
-    return getWidthAtUnicode(text, byte_offset, tab_width, width_method);
+pub fn getGraphemeWidthAt(text: []const u8, byte_offset: usize, tab_width: u8, control_display: ControlDisplay, width_method: WidthMethod) u32 {
+    return getWidthAtUnicode(text, byte_offset, tab_width, control_display, width_method);
 }
 
 /// Get width at byte offset using Unicode grapheme cluster segmentation
-fn getWidthAtUnicode(text: []const u8, byte_offset: usize, tab_width: u8, width_method: WidthMethod) u32 {
+fn getWidthAtUnicode(text: []const u8, byte_offset: usize, tab_width: u8, control_display: ControlDisplay, width_method: WidthMethod) u32 {
     if (byte_offset >= text.len) return 0;
 
     const b0 = text[byte_offset];
@@ -1259,7 +1275,7 @@ fn getWidthAtUnicode(text: []const u8, byte_offset: usize, tab_width: u8, width_
 
     var break_state: uucode.grapheme.BreakState = .default;
     var prev_cp: ?u21 = first_cp;
-    const first_width = charWidth(b0, first_cp, tab_width);
+    const first_width = charWidth(b0, first_cp, tab_width, control_display);
     var state = GraphemeWidthState.init(first_cp, first_width, width_method);
 
     var pos = byte_offset + first_len;
@@ -1274,7 +1290,7 @@ fn getWidthAtUnicode(text: []const u8, byte_offset: usize, tab_width: u8, width_
         const is_break = isGraphemeBreak(prev_cp, curr_cp, &break_state, width_method);
         if (is_break) break;
 
-        const cp_width = charWidth(b, curr_cp, tab_width);
+        const cp_width = charWidth(b, curr_cp, tab_width, control_display);
         state.addCodepoint(curr_cp, cp_width);
 
         prev_cp = curr_cp;
@@ -1286,7 +1302,7 @@ fn getWidthAtUnicode(text: []const u8, byte_offset: usize, tab_width: u8, width_
 
 /// Get width at byte offset using wcwidth-style codepoint-by-codepoint processing
 /// In wcwidth mode, each codepoint is treated independently - return its width directly
-fn getWidthAtWCWidth(text: []const u8, byte_offset: usize, tab_width: u8) u32 {
+fn getWidthAtWCWidth(text: []const u8, byte_offset: usize, tab_width: u8, control_display: ControlDisplay) u32 {
     if (byte_offset >= text.len) return 0;
 
     const b0 = text[byte_offset];
@@ -1297,7 +1313,7 @@ fn getWidthAtWCWidth(text: []const u8, byte_offset: usize, tab_width: u8) u32 {
         break :blk dec.cp;
     };
 
-    const first_width = charWidth(b0, first_cp, tab_width);
+    const first_width = charWidth(b0, first_cp, tab_width, control_display);
     return first_width;
 }
 
@@ -1306,16 +1322,17 @@ pub const PrevGraphemeResult = struct {
     width: u32,
 };
 
-/// Get previous grapheme start - proxy function that dispatches based on width_method
-pub fn getPrevGraphemeStart(text: []const u8, byte_offset: usize, tab_width: u8, width_method: WidthMethod) ?PrevGraphemeResult {
+/// Get the start of the last grapheme before byte_offset that takes cells - proxy function that
+/// dispatches based on width_method
+pub fn getPrevGraphemeStart(text: []const u8, byte_offset: usize, tab_width: u8, control_display: ControlDisplay, width_method: WidthMethod) ?PrevGraphemeResult {
     switch (width_method) {
-        .unicode, .unicode_wide, .no_zwj => return getPrevGraphemeStartUnicode(text, byte_offset, tab_width, width_method),
-        .wcwidth => return getPrevGraphemeStartWCWidth(text, byte_offset, tab_width),
+        .unicode, .unicode_wide, .no_zwj => return getPrevGraphemeStartUnicode(text, byte_offset, tab_width, control_display, width_method),
+        .wcwidth => return getPrevGraphemeStartWCWidth(text, byte_offset, tab_width, control_display),
     }
 }
 
 /// Get previous grapheme start using wcwidth-style codepoint-by-codepoint processing
-fn getPrevGraphemeStartWCWidth(text: []const u8, byte_offset: usize, tab_width: u8) ?PrevGraphemeResult {
+fn getPrevGraphemeStartWCWidth(text: []const u8, byte_offset: usize, tab_width: u8, control_display: ControlDisplay) ?PrevGraphemeResult {
     if (byte_offset == 0 or text.len == 0) return null;
     if (byte_offset > text.len) return null;
 
@@ -1330,7 +1347,7 @@ fn getPrevGraphemeStartWCWidth(text: []const u8, byte_offset: usize, tab_width:
             break :blk dec.cp;
         };
         const cp_len: usize = if (b < 0x80) 1 else decodeUtf8Unchecked(text, pos).len;
-        const cp_width = charWidth(b, curr_cp, tab_width);
+        const cp_width = charWidth(b, curr_cp, tab_width, control_display);
 
         if (cp_width > 0) {
             last_result = .{
@@ -1344,17 +1361,33 @@ fn getPrevGraphemeStartWCWidth(text: []const u8, byte_offset: usize, tab_width:
     return last_result;
 }
 
-/// Get previous grapheme start using Unicode grapheme cluster segmentation
-fn getPrevGraphemeStartUnicode(text: []const u8, byte_offset: usize, tab_width: u8, width_method: WidthMethod) ?PrevGraphemeResult {
+/// Get previous grapheme start using the grapheme segmentation that layout measures with.
+/// Like the wcwidth variant, skip zero-width clusters: a cursor has no cell to stop on in them.
+fn getPrevGraphemeStartUnicode(text: []const u8, byte_offset: usize, tab_width: u8, control_display: ControlDisplay, width_method: WidthMethod) ?PrevGraphemeResult {
     if (byte_offset == 0 or text.len == 0) return null;
     if (byte_offset > text.len) return null;
 
-    // For unicode/no_zwj modes, use grapheme cluster detection
+    // UAX #29 joins two ASCII characters only as CR LF, and LF takes no cells, so an ASCII character
+    // after ASCII that takes cells is the previous grapheme, and ASCII text needs no scan.
+    const last = text[byte_offset - 1];
+    const before: u8 = if (byte_offset > 1) text[byte_offset - 2] else 0;
+    if (last < 0x80 and before < 0x80) {
+        const width = asciiCharWidth(last, tab_width, control_display);
+        if (width > 0) return .{ .start_offset = byte_offset - 1, .width = width };
+    }
+
+    // The last grapheme usually takes cells, so find it without measuring the graphemes before it.
+    const last_start = lastGraphemeStart(text, byte_offset, width_method);
+    const last_width = getWidthAt(text, last_start, tab_width, control_display, width_method);
+    if (last_width > 0) return .{ .start_offset = last_start, .width = last_width };
+
+    // Otherwise measure each grapheme during one more scan to find the last one that takes cells.
     var break_state: uucode.grapheme.BreakState = .default;
     var pos: usize = 0;
     var prev_cp: ?u21 = null;
-    var prev_grapheme_start: usize = 0;
-    var second_to_last_grapheme_start: usize = 0;
+    var grapheme_start: usize = 0;
+    var width_state = GraphemeWidthState.init(0, 0, width_method);
+    var result: ?PrevGraphemeResult = null;
 
     while (pos < byte_offset) {
         const b = text[pos];
@@ -1365,42 +1398,44 @@ fn getPrevGraphemeStartUnicode(text: []const u8, byte_offset: usize, tab_width:
         };
 
         const cp_len: usize = if (b < 0x80) 1 else decodeUtf8Unchecked(text, pos).len;
-
-        if (isValidCodepoint(curr_cp)) {
-            const is_break = if (prev_cp) |p| blk: {
-                if (!isValidCodepoint(p)) break :blk true;
-                break :blk uucode.grapheme.isBreak(p, curr_cp, &break_state);
-            } else true;
-
-            if (is_break) {
-                second_to_last_grapheme_start = prev_grapheme_start;
-                prev_grapheme_start = pos;
-            }
-
-            prev_cp = curr_cp;
+        const cp_width = charWidth(b, curr_cp, tab_width, control_display);
+        if (isGraphemeBreak(prev_cp, curr_cp, &break_state, width_method)) {
+            if (width_state.width > 0) result = .{ .start_offset = grapheme_start, .width = width_state.width };
+            grapheme_start = pos;
+            width_state = GraphemeWidthState.init(curr_cp, cp_width, width_method);
+        } else {
+            width_state.addCodepoint(curr_cp, cp_width);
         }
 
+        prev_cp = curr_cp;
         pos += cp_len;
     }
 
-    if (prev_grapheme_start == 0 and byte_offset == 0) {
-        return null;
+    // Adding a code point never narrows a grapheme, so the part before byte_offset takes no cells either.
+    std.debug.assert(grapheme_start == last_start);
+    std.debug.assert(width_state.width == 0);
+    return result;
+}
+
+fn lastGraphemeStart(text: []const u8, byte_offset: usize, width_method: WidthMethod) usize {
+    var break_state: uucode.grapheme.BreakState = .default;
+    var pos: usize = 0;
+    var prev_cp: ?u21 = null;
+    var grapheme_start: usize = 0;
+    while (pos < byte_offset) {
+        const decoded = decodeUtf8Unchecked(text, pos);
+        if (isGraphemeBreak(prev_cp, decoded.cp, &break_state, width_method)) grapheme_start = pos;
+        prev_cp = decoded.cp;
+        pos += decoded.len;
     }
-
-    const start_offset = if (prev_grapheme_start < byte_offset) prev_grapheme_start else second_to_last_grapheme_start;
-    const width = getWidthAt(text, start_offset, tab_width, width_method);
-
-    return .{
-        .start_offset = start_offset,
-        .width = width,
-    };
+    return grapheme_start;
 }
 
 /// Calculate the display width of text - proxy function that dispatches based on width_method
-pub fn calculateTextWidth(text: []const u8, tab_width: u8, isASCIIOnly: bool, width_method: WidthMethod) u32 {
+pub fn calculateTextWidth(text: []const u8, tab_width: u8, control_display: ControlDisplay, isASCIIOnly: bool, width_method: WidthMethod) u32 {
     switch (width_method) {
-        .unicode, .unicode_wide, .no_zwj => return calculateTextWidthUnicode(text, tab_width, isASCIIOnly, width_method),
-        .wcwidth => return calculateTextWidthWCWidth(text, tab_width, isASCIIOnly),
+        .unicode, .unicode_wide, .no_zwj => return calculateTextWidthUnicode(text, tab_width, control_display, isASCIIOnly, width_method),
+        .wcwidth => return calculateTextWidthWCWidth(text, tab_width, control_display, isASCIIOnly),
     }
 }
 
@@ -1410,6 +1445,7 @@ pub fn calculateTextWidth(text: []const u8, tab_width: u8, isASCIIOnly: bool, wi
 pub const TextWidthCursor = struct {
     text: []const u8,
     tab_width: u8,
+    control_display: ControlDisplay,
     width_method: WidthMethod,
     byte_offset: usize = 0,
     columns: u32 = 0,
@@ -1430,7 +1466,7 @@ pub const TextWidthCursor = struct {
         while (self.byte_offset < end) {
             const b0 = self.text[self.byte_offset];
             if (b0 < 0x80 and (self.byte_offset + 1 == self.text.len or self.text[self.byte_offset + 1] < 0x80)) {
-                self.columns += asciiCharWidth(b0, self.tab_width);
+                self.columns += asciiCharWidth(b0, self.tab_width, self.control_display);
                 self.byte_offset += 1;
                 self.break_state = .default;
                 continue;
@@ -1438,14 +1474,14 @@ pub const TextWidthCursor = struct {
 
             const first = decodeUtf8Unchecked(self.text, self.byte_offset);
             var previous: u21 = if (self.byte_offset + first.len <= self.text.len) first.cp else 0xFFFD;
-            var state = GraphemeWidthState.init(previous, charWidth(b0, previous, self.tab_width), self.width_method);
+            var state = GraphemeWidthState.init(previous, charWidth(b0, previous, self.tab_width, self.control_display), self.width_method);
             self.byte_offset += @min(first.len, self.text.len - self.byte_offset);
             while (self.byte_offset < self.text.len) {
                 const byte = self.text[self.byte_offset];
                 const decoded = decodeUtf8Unchecked(self.text, self.byte_offset);
                 const cp: u21 = if (self.byte_offset + decoded.len <= self.text.len) decoded.cp else 0xFFFD;
                 if (isGraphemeBreak(previous, cp, &self.break_state, self.width_method)) break;
-                state.addCodepoint(cp, charWidth(byte, cp, self.tab_width));
+                state.addCodepoint(cp, charWidth(byte, cp, self.tab_width, self.control_display));
                 self.byte_offset += @min(decoded.len, self.text.len - self.byte_offset);
                 previous = cp;
             }
@@ -1456,7 +1492,7 @@ pub const TextWidthCursor = struct {
 };
 
 /// Calculate text width using Unicode grapheme cluster segmentation
-fn calculateTextWidthUnicode(text: []const u8, tab_width: u8, isASCIIOnly: bool, width_method: WidthMethod) u32 {
+fn calculateTextWidthUnicode(text: []const u8, tab_width: u8, control_display: ControlDisplay, isASCIIOnly: bool, width_method: WidthMethod) u32 {
     if (text.len == 0) return 0;
 
     // ASCII-only fast path
@@ -1486,10 +1522,10 @@ fn calculateTextWidthUnicode(text: []const u8, tab_width: u8, isASCIIOnly: bool,
                 total_width_cols += state.width;
             }
 
-            const cp_width = charWidth(b0, curr_cp, tab_width);
+            const cp_width = charWidth(b0, curr_cp, tab_width, control_display);
             state = GraphemeWidthState.init(curr_cp, cp_width, width_method);
         } else {
-            const cp_width = charWidth(b0, curr_cp, tab_width);
+            const cp_width = charWidth(b0, curr_cp, tab_width, control_display);
             state.addCodepoint(curr_cp, cp_width);
         }
 
@@ -1505,7 +1541,7 @@ fn calculateTextWidthUnicode(text: []const u8, tab_width: u8, isASCIIOnly: bool,
 }
 
 /// Calculate text width using wcwidth-style codepoint-by-codepoint processing
-fn calculateTextWidthWCWidth(text: []const u8, tab_width: u8, isASCIIOnly: bool) u32 {
+fn calculateTextWidthWCWidth(text: []const u8, tab_width: u8, control_display: ControlDisplay, isASCIIOnly: bool) u32 {
     if (text.len == 0) return 0;
 
     // ASCII-only fast path
@@ -1526,7 +1562,7 @@ fn calculateTextWidthWCWidth(text: []const u8, tab_width: u8, isASCIIOnly: bool)
         };
         const cp_len: usize = if (b0 < 0x80) 1 else decodeUtf8Unchecked(text, pos).len;
 
-        const cp_width = charWidth(b0, curr_cp, tab_width);
+        const cp_width = charWidth(b0, curr_cp, tab_width, control_display);
         total_width_cols += cp_width;
 
         pos += cp_len;
@@ -1671,6 +1707,7 @@ pub fn findChunkLayoutInfo(
     allocator: std.mem.Allocator,
     text: []const u8,
     tab_width: u8,
+    control_display: ControlDisplay,
     isASCIIOnly: bool,
     width_method: WidthMethod,
     wrap_breaks: *std.ArrayListUnmanaged(LayoutWrapBreak),
@@ -1687,7 +1724,7 @@ pub fn findChunkLayoutInfo(
         }
     };
     var ctx: Context = .{ .allocator = allocator, .wrap_breaks = wrap_breaks };
-    return walkChunkLayoutInfoComptime(text, tab_width, isASCIIOnly, width_method, &ctx, Context.append);
+    return walkChunkLayoutInfoComptime(text, tab_width, control_display, isASCIIOnly, width_method, &ctx, Context.append);
 }
 
 /// Editor queries do not need to materialize line-only CJK opportunities.
@@ -1695,6 +1732,7 @@ pub fn findWordChunkLayoutInfo(
     allocator: std.mem.Allocator,
     text: []const u8,
     tab_width: u8,
+    control_display: ControlDisplay,
     isASCIIOnly: bool,
     width_method: WidthMethod,
     wrap_breaks: *std.ArrayListUnmanaged(LayoutWrapBreak),
@@ -1710,7 +1748,7 @@ pub fn findWordChunkLayoutInfo(
             return true;
         }
     };
-    return walkChunkLayoutInfoGeneric(text, tab_width, isASCIIOnly, width_method, Visitor{ .allocator = allocator, .breaks = wrap_breaks });
+    return walkChunkLayoutInfoGeneric(text, tab_width, control_display, isASCIIOnly, width_method, Visitor{ .allocator = allocator, .breaks = wrap_breaks });
 }
 
 /// Retained layout stores word and line-only breaks once, in disjoint lists.
@@ -1718,6 +1756,7 @@ pub fn findLineAndWordChunkLayoutInfo(
     allocator: std.mem.Allocator,
     text: []const u8,
     tab_width: u8,
+    control_display: ControlDisplay,
     isASCIIOnly: bool,
     width_method: WidthMethod,
     line_breaks: *std.ArrayListUnmanaged(LayoutWrapBreak),
@@ -1740,12 +1779,13 @@ pub fn findLineAndWordChunkLayoutInfo(
             return true;
         }
     };
-    return walkChunkLayoutInfoGeneric(text, tab_width, isASCIIOnly, width_method, Visitor{ .allocator = allocator, .lines = line_breaks, .words = word_breaks });
+    return walkChunkLayoutInfoGeneric(text, tab_width, control_display, isASCIIOnly, width_method, Visitor{ .allocator = allocator, .lines = line_breaks, .words = word_breaks });
 }
 
 pub inline fn walkChunkLayoutInfoComptime(
     text: []const u8,
     tab_width: u8,
+    control_display: ControlDisplay,
     isASCIIOnly: bool,
     width_method: WidthMethod,
     context: anytype,
@@ -1759,12 +1799,13 @@ pub inline fn walkChunkLayoutInfoComptime(
             return callback(self.context, wrap_break);
         }
     };
-    return walkChunkLayoutInfoGeneric(text, tab_width, isASCIIOnly, width_method, Visitor{ .context = context });
+    return walkChunkLayoutInfoGeneric(text, tab_width, control_display, isASCIIOnly, width_method, Visitor{ .context = context });
 }
 
 fn walkChunkLayoutInfoGeneric(
     text: []const u8,
     tab_width: u8,
+    control_display: ControlDisplay,
     isASCIIOnly: bool,
     width_method: WidthMethod,
     visitor: anytype,
@@ -1832,7 +1873,7 @@ fn walkChunkLayoutInfoGeneric(
                     var i: usize = 0;
                     while (i + 1 < ascii_len) : (i += 1) {
                         const b = text[pos + i];
-                        const width = asciiCharWidth(b, tab_width);
+                        const width = asciiCharWidth(b, tab_width, control_display);
                         if (isAsciiWrapBreak(b)) {
                             if (!try emitLayoutWrapBreak(visitor, pos + i, 1, col, width, asciiLayoutWrapBreakKind(b))) return chunkWordClassEdges(text);
                         }
@@ -1844,7 +1885,7 @@ fn walkChunkLayoutInfoGeneric(
                     cluster_started = true;
                     cluster_start = pos + ascii_len - 1;
                     cluster_col_offset = col;
-                    cluster_width_state = GraphemeWidthState.init(last_cp, asciiCharWidth(last_b, tab_width), width_method);
+                    cluster_width_state = GraphemeWidthState.init(last_cp, asciiCharWidth(last_b, tab_width, control_display), width_method);
                     cluster_break_kind = asciiLayoutWrapBreakKind(last_b);
                     cluster_class = classifyWordClass(last_cp);
                     prev_cp = last_cp;
@@ -1860,7 +1901,7 @@ fn walkChunkLayoutInfoGeneric(
         std.debug.assert(decoded.len > 0 and decoded.len <= text.len - pos);
         const curr_class = classifyWordClass(decoded.cp);
         const is_break = isGraphemeBreak(prev_cp, decoded.cp, &break_state, width_method);
-        const cp_width = charWidth(b0, decoded.cp, tab_width);
+        const cp_width = charWidth(b0, decoded.cp, tab_width, control_display);
 
         if (is_break) {
             if (cluster_started) {
@@ -1925,13 +1966,14 @@ pub fn findRenderClusterInfo(
     allocator: std.mem.Allocator,
     text: []const u8,
     tab_width: u8,
+    control_display: ControlDisplay,
     isASCIIOnly: bool,
     width_method: WidthMethod,
     result: *std.ArrayListUnmanaged(RenderClusterInfo),
 ) !void {
     switch (width_method) {
-        .unicode, .unicode_wide, .no_zwj => try findRenderClusterInfoUnicode(allocator, text, tab_width, isASCIIOnly, width_method, result),
-        .wcwidth => try findRenderClusterInfoWCWidth(allocator, text, tab_width, isASCIIOnly, result),
+        .unicode, .unicode_wide, .no_zwj => try findRenderClusterInfoUnicode(allocator, text, tab_width, control_display, isASCIIOnly, width_method, result),
+        .wcwidth => try findRenderClusterInfoWCWidth(allocator, text, tab_width, control_display, isASCIIOnly, result),
     }
 }
 
@@ -1940,6 +1982,7 @@ fn findRenderClusterInfoUnicode(
     allocator: std.mem.Allocator,
     text: []const u8,
     tab_width: u8,
+    control_display: ControlDisplay,
     isASCIIOnly: bool,
     width_method: WidthMethod,
     result: *std.ArrayListUnmanaged(RenderClusterInfo),
@@ -2000,11 +2043,11 @@ fn findRenderClusterInfoUnicode(
                     cluster_is_tab = (b == '\t');
                     cluster_is_multibyte = false;
 
-                    const cp_width = asciiCharWidth(b, tab_width);
+                    const cp_width = asciiCharWidth(b, tab_width, control_display);
                     cluster_width_state = GraphemeWidthState.init(curr_cp, cp_width, width_method);
                 } else {
                     // Continuing cluster (shouldn't happen for ASCII, but handle it)
-                    const cp_width = asciiCharWidth(b, tab_width);
+                    const cp_width = asciiCharWidth(b, tab_width, control_display);
                     cluster_width_state.addCodepoint(curr_cp, cp_width);
                 }
 
@@ -2046,11 +2089,11 @@ fn findRenderClusterInfoUnicode(
                 cluster_is_tab = (b0 == '\t');
                 cluster_is_multibyte = (cp_len != 1);
 
-                const cp_width = charWidth(b0, curr_cp, tab_width);
+                const cp_width = charWidth(b0, curr_cp, tab_width, control_display);
                 cluster_width_state = GraphemeWidthState.init(curr_cp, cp_width, width_method);
             } else {
                 cluster_is_multibyte = cluster_is_multibyte or (cp_len != 1);
-                const cp_width = charWidth(b0, curr_cp, tab_width);
+                const cp_width = charWidth(b0, curr_cp, tab_width, control_display);
                 cluster_width_state.addCodepoint(curr_cp, cp_width);
             }
 
@@ -2091,11 +2134,11 @@ fn findRenderClusterInfoUnicode(
             cluster_is_tab = (b0 == '\t');
             cluster_is_multibyte = (cp_len != 1);
 
-            const cp_width = charWidth(b0, curr_cp, tab_width);
+            const cp_width = charWidth(b0, curr_cp, tab_width, control_display);
             cluster_width_state = GraphemeWidthState.init(curr_cp, cp_width, width_method);
         } else {
             cluster_is_multibyte = cluster_is_multibyte or (cp_len != 1);
-            const cp_width = charWidth(b0, curr_cp, tab_width);
+            const cp_width = charWidth(b0, curr_cp, tab_width, control_display);
             cluster_width_state.addCodepoint(curr_cp, cp_width);
         }
 
@@ -2121,6 +2164,7 @@ fn findRenderClusterInfoWCWidth(
     allocator: std.mem.Allocator,
     text: []const u8,
     tab_width: u8,
+    control_display: ControlDisplay,
     isASCIIOnly: bool,
     result: *std.ArrayListUnmanaged(RenderClusterInfo),
 ) !void {
@@ -2182,13 +2226,13 @@ fn findRenderClusterInfoWCWidth(
             cluster_start_col = col;
             cluster_is_tab = (b0 == '\t');
             cluster_is_multibyte = (cp_len != 1);
-            const cp_width = charWidth(b0, curr_cp, tab_width);
+            const cp_width = charWidth(b0, curr_cp, tab_width, control_display);
             cluster_width_state = GraphemeWidthState.init(curr_cp, cp_width, .wcwidth);
             cluster_started = true;
         } else {
             // Continuing cluster
             cluster_is_multibyte = cluster_is_multibyte or (cp_len != 1);
-            const cp_width = charWidth(b0, curr_cp, tab_width);
+            const cp_width = charWidth(b0, curr_cp, tab_width, control_display);
             cluster_width_state.addCodepoint(curr_cp, cp_width);
         }
 

packages/web/src/content/docs/extend/editing.mdx

@@ -134,7 +134,7 @@ click maps to the painted character. Reset the matching form when the selection
 | Move                | `moveCursorLeft`, `moveCursorRight`, `moveCursorUp`, `moveCursorDown`, `gotoLine`                      |
 | Set cursor          | `setCursor`, `setCursorToLineCol`, `setCursorByOffset`                                                 |
 | Read cursor         | `getCursorPosition`, `getNextWordBoundary`, `getPrevWordBoundary`, `getEOL`                            |
-| Convert coordinates | `offsetToPosition`, `positionToOffset`, `getLineStartOffset`                                           |
+| Convert coordinates | `offsetToPosition`, `positionToOffset`, `getLineStartOffset`, `measureText`                            |
 | Read text           | `getText`, `getTextRange`, `getTextRangeByCoords`, `getLineCount`                                      |
 | History             | `undo`, `redo`, `canUndo`, `canRedo`, `clearHistory`                                                   |
 | Style               | Default-style, syntax-style, and highlight methods shared with `TextBuffer`                            |
@@ -143,11 +143,32 @@ click maps to the painted character. Reset the matching form when the selection
 `setText()` resets history and the native add buffer. `replaceText()` creates an undo point. All replacement methods
 copy text into native-owned memory. The `Owned` variants retain the same public behavior without a borrowed-memory path.
 
-Text that you replace or insert must not contain control characters other than tab, carriage return, and line feed.
-The replacement and insertion methods throw `NativeError` with status `InvalidArgument` for C0 controls, DEL, and C1
-controls, and the edit buffer does not change. The same rule applies to editor placeholders, `Textarea.initialValue`,
-and `Input.value`. `Textarea` and `Input` remove these characters from pasted text and ignore key sequences that
-contain them.
+An editor keeps every control character and shows it as notation, as Vim does. C0 controls other than tab, carriage
+return, and line feed show as `^@` to `^_` in two cells, DEL as `^?` in two cells, and C1 controls as lowercase hex
+`<80>` to `<9f>` in four cells. The cursor stops before each control and deletes it as one unit, wrapping never splits
+its notation, and `getText()`, history, and selected text return the original characters. Tab, carriage return, and
+line feed keep their meaning. The same rule applies to editor placeholders, `Textarea.initialValue`, and `Input.value`.
+`Input.maxLength` counts the characters of the value, controls included. Read-only text, such as `TextRenderable`,
+still gives controls no cells. `Textarea` and `Input` ignore key sequences that contain control characters.
+
+Pasted text keeps its controls, but `Textarea` and `Input` remove terminal escape sequences from it with
+`stripAnsiSequences()`, the same way on Bun and Node: control sequences such as `ESC [31m`, control strings such as
+OSC titles and hyperlinks through BEL or ST, other escapes such as `ESC ( B`, and their C1 forms. A sequence ends at
+the first character that cannot continue it, and a lone ESC or a sequence that the paste cuts off is removed too, so a
+paste never inserts ESC.
+
+`measureText(text)` returns the global display offset at the end of `text` as the buffer would lay it out, without
+storing it. It uses the buffer's width method, tab width, and control notation, and counts each line break as one.
+JavaScript width functions such as `Bun.stringWidth()` and `string-width` give controls and tabs no cells and count
+no line breaks, so do not use them for editor offsets. To map a string index in the buffer text to an offset, measure
+the text before it. To map an offset to a string index, take the length of `getTextRange(0, offset)`:
+
+```typescript
+buffer.setText("\f@file")
+const text = buffer.getText()
+const offset = buffer.measureText(text.slice(0, text.indexOf("@"))) // 2: ^L takes two cells
+buffer.getTextRange(0, offset).length // 1: the index of "@"
+```
 
 The edit buffer emits native `"cursor-changed"` and `"content-changed"` events through its `EventEmitter` interface.
 `undo()` and `redo()` also emit `"cursorChanged"`. Native code reports each event during the edit call. Core then
@@ -239,8 +260,8 @@ global display-width offset and uses it to index a JavaScript string by UTF-16 c
 
 The result is unreliable after wide CJK text, emoji, line breaks, combining sequences, or other complex graphemes. It
 also returns a nearby character at some end-of-line and end-of-buffer positions. Use `logicalCursor`, `visualCursor`,
-`offsetToPosition()`, and `positionToOffset()` for display coordinates. Build a separate grapheme-to-UTF-16 map when a
-JavaScript string index is required.
+`offsetToPosition()`, and `positionToOffset()` for display coordinates. Use `measureText()` and `getTextRange()` to map
+between a JavaScript string index and an offset.
 
 ## Renderable base classes
 

packages/web/src/content/docs/reference/api-index.mdx

@@ -526,7 +526,7 @@ Each item names the changed API or behavior first. New APIs are in the tables ab
 - `SyntaxStyle.create()`, `fromStyles(styles)`, and `fromTheme(theme)`: now `create(owner)`, `fromStyles(styles, owner)`, and `fromTheme(theme, owner)`. Pass `renderer.nativeScene`, or a `ResourceContext` for standalone use.
 - `TextBuffer.create(widthMethod)` and `EditBuffer.create(widthMethod)`: now `create(widthMethod, owner)`.
 - `.ptr` on `TextBuffer`, `TextBufferView`, `EditBuffer`, `EditorView`, and `SyntaxStyle`: removed. Their constructors take Context handles. Use the `create()` factories.
-- `EditBuffer.setText()`, `insertText()`, and `replaceText()`, the same Textarea and Input methods, Textarea `initialValue`, `Input.value`, and the `placeholder` of both: now throw for control characters other than tab, carriage return, and line feed. Remove them first. Paste handlers remove them.
+- `EditBuffer.setText()`, `insertText()`, and `replaceText()`, the same Textarea and Input methods, Textarea `initialValue`, `Input.value`, and the `placeholder` of both: now show control characters other than tab, carriage return, and line feed as notation, such as `^L` or `<85>`. In 0.5.14, the cursor could not move past or delete them, and the terminal received them.
 - `TextBuffer.loadFile()`: now rejects invalid UTF-8.
 - Text buffer and text view offsets: must be integers. Other values throw `RangeError`.
 - Highlight `priority` above 255 or `hlRef` above 65535: now throws. In 0.5.14, the value wrapped.