native: grow the object handle table instead of failing at object_capacity (#1622)

A Context's native object table had a fixed size of `object_capacity`
slots, and a renderer creates its Context with 65,536. Every Markdown
table cell holds a text buffer and a view, so a long document or chat
transcript failed after about 32,500 cells with
`ot_text_buffer_view_create failed: ObjectLimit`, whether it was set at
once or streamed. Plain renderables hit the same limit at 65,534.

The table now starts at `object_capacity` and doubles when it is full,
up to 4,194,304 slots, or `object_capacity` if that is larger. Past
that, creation still returns `ObjectLimit`. Handles keep their slot
index and generation, so existing handles stay valid, and no code holds
a pointer into the slot array across an insert. The cap comes from
memory, not the handle format: a slot is 32 bytes, so the cap is 128
MiB, and each live slot owns a much larger native object. There is no
ABI change. The comments in `opentui.h` and the docs now call
`object_capacity` the initial size. A clipboard start that fails to grow
the table reports out of memory instead of limit exceeded.

The handle-table model test starts at 2 slots and checks growth to 4 and
5, old handles, stale handles, and `ObjectLimit` at the maximum. A
Markdown test renders a table that needs more objects than a 64-slot
renderer, and it fails on main with the issue's error. Tests that check
cleanup at `ObjectLimit` now pin the maximum. Reverting growth, the
doubling, the clamp, the `>=` maximum check, or growth inside `insert`
each fails a test.

Fixes #1611

packages/core/src/renderables/__tests__/Markdown.test.ts

@@ -22,6 +22,8 @@ import {
   TestRecorder,
 } from "../../testing.js"
 import { ManualClock } from "../../testing/manual-clock.js"
+import { createTestStdout } from "../../testing/test-streams.js"
+import { NativeSession } from "../../NativeSession.js"
 import { TextAttributes, type CapturedFrame } from "../../types.js"
 import type { SimpleHighlight } from "../../lib/tree-sitter/types.js"
 import { stringWidth } from "../../platform/runtime.js"
@@ -919,6 +921,23 @@ test("table with many columns", async () => {
   `)
 })
 
+test("a table may hold more native objects than the renderer's initial object capacity", async () => {
+  // Each cell holds a text buffer and a view, so 4 x 21 cells need more than 64 slots.
+  const stdout = createTestStdout(20, 4)
+  const nativeSession = new NativeSession(stdout, { context: { objectCapacity: 64, renderCellsMax: 1_000_000 } })
+  const setup = await createTestRenderer({ width: 20, height: 4, stdout, nativeSession, bufferedOutput: "stdout" })
+  try {
+    const rows = Array.from({ length: 21 }, (_, r) => `|${r}|${r}|${r}|${r}|${r === 0 ? "\n|-|-|-|-|" : ""}`)
+    const syntaxStyle = SyntaxStyle.create(setup.renderer.nativeScene)
+    setup.renderer.root.add(new MarkdownRenderable(setup.renderer, { content: rows.join("\n"), syntaxStyle }))
+    await setup.renderOnce()
+    expect(setup.captureCharFrame()).toContain("│0   │0   │0   │0  │")
+  } finally {
+    setup.renderer.destroy()
+    await setup.renderer.closed
+  }
+})
+
 test("no tables returns original content", async () => {
   const markdown = `# Just a heading
 

packages/core/src/tests/image-pool.test.ts

@@ -111,7 +111,7 @@ describe("NativeImagePool", () => {
   test("failed publications can retry without leaking native images", () => {
     const lib = resolveRenderLib()
     const pixels = Uint8Array.of(1, 2, 3, 255)
-    // One slot and one published handle fill the owner; a leaked object would fail a publication or the refill.
+    // One slot and one published handle fill the owner; a leaked object would make the refill grow the table.
     const owner = new ResourceContext({ objectCapacity: 2, renderCellsMax: 1 })
     const retain = lib.imageRetain
     const pool = new NativeImagePool({ width: 1, height: 1, capacity: 1, owner })
@@ -127,10 +127,8 @@ describe("NativeImagePool", () => {
       frame?.dispose()
       pool.dispose()
     }
-    const fill = () => NativeImage.fromRgba(pixels, 1, 1, 4, { owner })
-    fill()
-    fill()
-    expect(fill).toThrow("ObjectLimit")
+    const fill = () => NativeImage.fromRgba(pixels, 1, 1, 4, { owner }) as unknown as { handle: { slot: number } }
+    expect([fill().handle.slot, fill().handle.slot].sort()).toEqual([0, 1])
     owner.destroy()
   })
 

packages/native/include/opentui.h

@@ -2074,9 +2074,11 @@ typedef struct ot_session_pump_result {
 /* Zero-initialize, then set struct_size to sizeof(ot_context_options) and
  * abi_version to OT_CONTEXT_ABI_VERSION. Version 1 requires the exact size,
  * zero flags, zero reserved fields, and positive resource limits.
- * object_capacity counts native object slots, not bytes. render_cells_max
- * bounds each renderer's width * height in terminal cells, not bytes or total
- * context memory. Neither limit uses a default when zero.
+ * object_capacity is the initial count of native object slots, not bytes. A full
+ * table doubles, up to 4194304 slots or object_capacity, whichever is larger;
+ * past that, creation returns OT_OBJECT_LIMIT. render_cells_max bounds each
+ * renderer's width * height in terminal cells, not bytes or total context
+ * memory. Neither limit uses a default when zero.
  * Allocator and I/O injection are available only through the direct Zig API. */
 typedef struct ot_context_options {
     uint32_t struct_size;
@@ -2526,7 +2528,7 @@ ot_status ot_session_get_renderer_state(
  * rendering. A live scene preparation or painted draft returns OT_FRAME_BUSY;
  * use ot_scene_frame_acquire_buffer_lease for painted-frame access.
  * The output requires the exact size/version and zero reserved. Rejection leaves
- * it unchanged and acquires no lease. Leases share object_capacity; C contexts
+ * it unchanged and acquires no lease. Leases take object slots; C contexts
  * allow at most 4096 leases and 64 MiB of distinct leased storage, including
  * retired arrays and tracker capacity. Shared Context pools are not in that charge.
  * Release every accepted lease, even after Session resize or destruction. */

packages/native/src/clipboard/host.zig

@@ -1567,7 +1567,7 @@ fn startImmediateOperation(
     selection: Selection,
     out_handle: *Handle,
 ) StartStatus {
-    service.objects.checkCapacity() catch return .limit_exceeded;
+    service.objects.checkCapacity() catch |err| return if (err == error.OutOfMemory) .out_of_memory else .limit_exceeded;
     const owned_request = service.allocator.dupe(u8, request) catch return .out_of_memory;
     const operation = service.allocator.create(Operation) catch {
         if (owned_request.len > 0) service.allocator.free(owned_request);
@@ -1829,13 +1829,22 @@ test "clipboard status values are stable" {
 }
 
 test "clipboard service preserves a configured native operation limit" {
-    var objects = try handles.Table.init(std.testing.allocator, 4);
+    var objects = try handles.Table.init(std.testing.allocator, 1);
     defer objects.deinit();
     const operation_limit = 2;
     const service = try createService(std.testing.allocator, &objects, operation_limit, PROVIDER_TRANSFERS_MAX_DEFAULT, null, 0);
     defer destroyTestService(&objects, service);
 
     var operations: [operation_limit]Handle = undefined;
+    // The full table grows for the first operation; a table at its maximum reports
+    // limit_exceeded, and a failed growth reports out_of_memory.
+    objects.slot_count_max = 1;
+    try std.testing.expectEqual(StartStatus.limit_exceeded, startClearOperation(&objects, service, 0, 0, &operations[0]));
+    objects.slot_count_max = handles.slot_count_max_default;
+    var failing = std.testing.FailingAllocator.init(std.testing.allocator, .{ .fail_index = 0, .resize_fail_index = 0 });
+    objects.allocator = failing.allocator();
+    try std.testing.expectEqual(StartStatus.out_of_memory, startClearOperation(&objects, service, 0, 0, &operations[0]));
+    objects.allocator = std.testing.allocator;
     for (&operations) |*operation| {
         try std.testing.expectEqual(StartStatus.ok, startClearOperation(&objects, service, 0, 0, operation));
     }

packages/native/src/context-handles.zig

@@ -52,18 +52,24 @@ pub const DestroyToken = struct {
     ptr: *anyopaque,
 };
 
+/// Handles could address 2^32 slots, but at 32 bytes each that is 128 GiB. Growth stops at
+/// 2^22 slots (128 MiB): each live slot owns a native object far larger than its slot.
+pub const slot_count_max_default = 1 << 22;
+
 /// Single-owner storage. Different tables may be used on different threads.
 pub const Table = struct {
     allocator: std.mem.Allocator,
     context_id: u64,
-    slots: []Slot,
-    free_head: ?u32,
+    // Growth moves the array, so callers hold slot indexes, not slot pointers, across insert.
+    slots: []Slot = &.{},
+    free_head: ?u32 = null,
     live_count: u32 = 0,
+    slot_count_max: u32 = slot_count_max_default,
 
+    /// The table starts with `capacity` slots and doubles when full, up to
+    /// slot_count_max. A larger `capacity` never grows.
     pub fn init(allocator: std.mem.Allocator, capacity: u32) Error!Table {
         std.debug.assert(capacity > 0);
-        const slots = try allocator.alloc(Slot, capacity);
-        errdefer allocator.free(slots);
         var id = last_context_id.load(.monotonic);
         while (true) {
             if (id == std.math.maxInt(u64)) return error.ContextLimit;
@@ -71,15 +77,9 @@ pub const Table = struct {
                 id = current;
             } else break;
         }
-        for (slots, 0..) |*slot, index| {
-            slot.* = .{ .next_free = if (index + 1 < slots.len) @intCast(index + 1) else null };
-        }
-        return .{
-            .allocator = allocator,
-            .context_id = id + 1,
-            .slots = slots,
-            .free_head = 0,
-        };
+        var table: Table = .{ .allocator = allocator, .context_id = id + 1 };
+        try table.grow(capacity);
+        return table;
     }
 
     pub fn deinit(self: *Table) void {
@@ -88,12 +88,26 @@ pub const Table = struct {
         self.* = undefined;
     }
 
-    pub fn checkCapacity(self: *const Table) Error!void {
-        if (self.free_head == null) return error.ObjectLimit;
+    /// Makes a vacant slot available, so the next insert cannot fail.
+    pub fn checkCapacity(self: *Table) Error!void {
+        if (self.free_head != null) return;
+        if (self.slots.len >= self.slot_count_max) return error.ObjectLimit;
+        try self.grow(@min(self.slot_count_max, @as(u32, @intCast(self.slots.len)) *| 2));
+    }
+
+    fn grow(self: *Table, count_new: u32) Error!void {
+        std.debug.assert(self.free_head == null and count_new > self.slots.len);
+        const count_old: u32 = @intCast(self.slots.len);
+        self.slots = try self.allocator.realloc(self.slots, count_new);
+        for (self.slots[count_old..], count_old..) |*slot, index| {
+            slot.* = .{ .next_free = if (index + 1 < count_new) @intCast(index + 1) else null };
+        }
+        self.free_head = count_old;
     }
 
     pub fn insert(self: *Table, kind: Kind, ptr: *anyopaque) Error!Handle {
-        const index = self.free_head orelse return error.ObjectLimit;
+        try self.checkCapacity();
+        const index = self.free_head.?;
         const slot = &self.slots[index];
         std.debug.assert(slot.state == .vacant);
         std.debug.assert(self.live_count < self.slots.len);
@@ -159,10 +173,11 @@ pub const Table = struct {
     }
 };
 
-// The model mirrors one slot per table slot. Slots 0 and 1 start one destroy away
-// from generation retirement, so the random walk also covers exhausted slots.
+// The model mirrors one slot per table slot. The table starts with two slots and grows to
+// 4 and then 5 (the clamp). Slots 0 and 1 start one destroy away from generation
+// retirement, so the random walk also covers exhausted slots.
 const Model = struct {
-    const capacity = 4;
+    const capacity = 5;
     const kinds = [_]Kind{ .session, .buffer, .image };
     const State = enum { vacant, alive, destroying, retired };
 
@@ -204,7 +219,9 @@ const Model = struct {
             vacant = vacant or state == .vacant;
         }
         try std.testing.expectEqual(live, self.table.live_count);
-        if (vacant) try self.table.checkCapacity() else try std.testing.expectError(error.ObjectLimit, self.table.checkCapacity());
+        // Only insert may grow the table; a full table is checked only where it cannot grow.
+        if (self.table.free_head != null) try self.table.checkCapacity() else if (!vacant) try std.testing.expectError(error.ObjectLimit, self.table.checkCapacity());
+        try std.testing.expect(std.mem.findScalar(usize, &.{ 2, 4, 5 }, self.table.slots.len) != null);
         for (kinds) |kind| {
             var cursor: usize = 0;
             var listed: u32 = 0;
@@ -289,8 +306,9 @@ const Model = struct {
 
 test "Table matches a reference model under seeded random operations" {
     for (0..16) |seed| {
-        var table = try Table.init(std.testing.allocator, Model.capacity);
+        var table = try Table.init(std.testing.allocator, 2);
         defer table.deinit();
+        table.slot_count_max = Model.capacity;
         var model: Model = .{ .table = &table };
         // Runs before table.deinit, so a failed expectation does not trip its live_count assertion.
         defer model.releaseAll();

packages/native/src/context.zig

@@ -426,7 +426,7 @@ pub const SceneTextSelectionOptions = struct {
 };
 
 pub const Options = struct {
-    /// All checked handles, including leases, share this capacity.
+    /// Initial slots for all checked handles, including leases. See handles.Table.init.
     object_capacity: u32 = 4096,
     render_cells_max: u32 = 1_000_000,
     lease_count_max: u32 = buf.BufferLease.count_max_default,

packages/native/src/tests/context-image_test.zig

@@ -10,6 +10,7 @@ test "Context checked image failed creation leaves capacity identities and admis
     var failing = testing.FailingAllocator.init(testing.allocator, .{});
     const owner = try context.Context.init(failing.allocator(), testing.io, .{ .object_capacity = 1 });
     defer owner.deinit() catch unreachable;
+    owner.objects.slot_count_max = 1;
     const pixels = [_]u8{ 1, 2, 3, 255 };
     for (0..2) |offset| {
         failing.fail_index = failing.alloc_index + offset;

packages/native/src/tests/context_test.zig

@@ -354,6 +354,8 @@ test "Context handles distinguish context, kind, stale generation, and limits" {
     try std.testing.expectEqual(old.slot, foreign.slot);
     try std.testing.expectError(error.WrongContext, first.raw().getSession(foreign));
     try std.testing.expectError(error.WrongKind, first.raw().getTextBuffer(old));
+    // An object_capacity above slot_count_max keeps the table at that size.
+    first.objects.slot_count_max = 0;
     try std.testing.expectError(error.ObjectLimit, first.createSession(.{}));
     try first.destroy(old);
     const replacement = try first.createSession(.{});

packages/native/src/tests/scene_flush_test.zig

@@ -26,6 +26,7 @@ const Fixture = struct {
     fn init(self: *Fixture, allocator: std.mem.Allocator) !void {
         self.owner.owner_thread = std.Thread.getCurrentId();
         self.owner.core = try context.Context.init(allocator, self.owner.io_threaded.io(), .{ .object_capacity = 3 });
+        self.owner.core.objects.slot_count_max = 3;
         self.session = try self.owner.core.createSession(.{ .chunk_size = 4096 });
         try self.owner.core.attachSessionRenderer(self.session, 16, 4, .{ .remote_mode = .remote });
         const root = try self.owner.core.sceneCreateNode(self.session, 0, 1);

packages/native/src/tests/scene_frame_test.zig

@@ -278,6 +278,7 @@ test "Scene painted lease limits and failed initialization release all provision
     }, [_]context.Error{ error.ObjectLimit, error.LeaseLimit, error.LeaseBytesLimit }) |limits, expected| {
         const f = try Fixture.init(testing.allocator, 4, 1, .{ .limits = limits, .output = transport });
         defer f.deinit();
+        f.owner.objects.slot_count_max = limits.object_capacity;
         const frame = try f.owner.sceneFrameStep(f.id, null, options);
         try testing.expectError(expected, f.owner.sceneFrameAcquireBufferLease(f.id, frame, .next));
         try testing.expectEqual(@as(u32, 0), f.owner.lease_count);

packages/native/src/tests/scene_test.zig

@@ -422,6 +422,7 @@ test "Scene owns and iteratively destroys registered nodes including detached bo
     var failing = testing.FailingAllocator.init(testing.allocator, .{});
     const f = try Fixture.init(failing.allocator(), 2, 2, .{ .limits = .{ .object_capacity = 513 } });
     defer f.deinit();
+    f.owner.objects.slot_count_max = 513;
     var last = f.root;
     for (1..512) |index| {
         last = try f.owner.sceneCreateNode(f.id, 1, @intCast(index + 1));

packages/native/src/tests/session_test.zig

@@ -320,6 +320,7 @@ test "Session creation validates finite limits and unwinds every allocation fail
     var failing = testing.FailingAllocator.init(testing.allocator, .{});
     const owner = try context.Context.init(failing.allocator(), std.Io.failing, .{ .object_capacity = 1 });
     defer owner.deinit() catch unreachable;
+    owner.objects.slot_count_max = 1;
     const allocated = failing.allocated_bytes;
     for ([_]session.Options{
         .{ .chunk_size = 0 },

packages/web/src/content/docs/extend/editing.mdx

@@ -215,8 +215,9 @@ The public highlight methods are `addHighlight()`, `addHighlightByCharRange()`,
 
 ## Resource limits
 
-The Context's `objectCapacity` bounds resource handles, including buffers and views. Native text storage also requires
-memory and can reject allocation. A JavaScript string parameter does not imply unlimited capacity.
+The Context's resource handles, including buffers and views, start at `objectCapacity` slots and grow to at most
+4,194,304, or `objectCapacity` if it is larger. Native text storage also requires memory and can reject allocation. A
+JavaScript string parameter does not imply unlimited capacity.
 
 Each native text buffer also has 255 byte-source registrations. A Context-owned `TextBuffer` reserves one, and each
 nonempty `append()` uses another. After 254 appends without replacement, further appends reject with `OutOfMemory`,

packages/web/src/content/docs/native/c.mdx

@@ -79,7 +79,7 @@ int main(void) {
     CHECK(ot_session_attach_renderer(context, &session, &renderer));
 ```
 
-The Context allows eight native objects and at most 36 cells in each drawing buffer or renderer.
+The Context starts with eight native object slots and allows at most 36 cells in each drawing buffer or renderer.
 The Session allocates two 4,096-byte output chunks.
 These limits suit this small example. They are not a total memory budget.
 `remote = 1` stops the renderer from inheriting local terminal environment settings.

packages/web/src/content/docs/native/resources.mdx

@@ -267,7 +267,8 @@ Keep the allocator, `std.Io`, callback data, and borrowed environment state aliv
 
 `ot_context_options` sets two required limits:
 
-- `object_capacity` counts native resource slots, including checked leases.
+- `object_capacity` is the initial count of native resource slots, including checked leases. A full table doubles, up
+  to 4,194,304 slots or `object_capacity`, whichever is larger.
 - `render_cells_max` limits each drawing buffer or renderer size, not the sum of cells across the Context.
 
 Both limits must be positive. Zero does not select a default.

packages/web/src/content/docs/reference/buffer-api.mdx

@@ -36,9 +36,10 @@ try {
 `create(width, height, widthMethod, options)` requires positive cell dimensions and an explicit `owner`.
 For renderer-owned resources, pass `renderer.nativeScene`. For standalone work, create a `ResourceContext`.
 
-`ResourceContext` creates a native Context without a renderer or terminal Session. `objectCapacity` limits the number
-of native objects, including the cell leases that `withBuffers()` holds. `renderCellsMax` limits the cell count
-(`width * height`) of each drawing buffer. Destroy the Context only after every `withBuffers()` callback returns.
+`ResourceContext` creates a native Context without a renderer or terminal Session. `objectCapacity` sets the initial
+number of native object slots, including the cell leases that `withBuffers()` holds; the Context grows it as needed.
+`renderCellsMax` limits the cell count (`width * height`) of each drawing buffer. Destroy the Context only after every
+`withBuffers()` callback returns.
 Destroying the Context releases its remaining resources.
 
 Read [How Core uses native](/docs/native/core#resources-through-core) for Core resource factories and their native owners.

packages/web/src/content/docs/reference/native-image.mdx

@@ -116,7 +116,7 @@ interface ImageCreateOptions {
 `ImageLoadOptions`, `PixelImportOptions`, and `NativeImagePoolOptions` also include `owner`.
 Pass `renderer.nativeScene.resourceContext` to create an image in the renderer's Context.
 
-Without `owner`, these methods share one automatic Context with 4,096 object slots.
+Without `owner`, these methods share one automatic Context that starts with 4,096 object slots.
 Derived images, retained handles, and raw-pixel leases keep that Context alive.
 The last release destroys it. You control the lifetime of an explicit `ResourceContext`.