feat(packages/codegen): add full source map support (#25585)

## Summary

Align `oxc-codegen` source maps with Rust `oxc_codegen`, using the Rust implementation as the conformance oracle.

This PR:

- changes `printSync` to return `{ code, map? }`;
- returns a standard Source Map v3 object;
- exposes Rust’s generated source map through `codegen_conformance`;
- compares the complete JS and Rust source maps;
- supports Oxc `start`/`end` offsets and TS-ESTree `range` offsets;
- uses `sourceText` to convert UTF-16 offsets and recover original identifier spellings;
- validates both maps and no-maps builds in both `preserveParens` modes;
- aligns mapping placement across JavaScript, TypeScript, JSX, templates, imports, destructuring, delimiters, and postfix chains;
- handles astral characters and all ECMAScript line terminators.
`loc`-only ASTs are not supported for source maps.

## API change

```ts
const { code, map } = printSync(program, {
  sourcemap: true,
  sourceFileName: "input.js",
  sourceText,
});
```

`map` is a Source Map v3 object:

```ts
interface SourceMap {
  version: 3;
  mappings: string;
  names: string[];
  sources: string[];
  sourcesContent?: string[];
}
```

## Implementation

Mapped writes collect only:

- the generated UTF-16 offset;
- the original source offset;
- the optional original identifier name.

After printing, the generator builds line tables for the generated output and `sourceText`, converts the offsets to line/column positions, suppresses duplicate source positions, and encodes the result as base64 VLQ.

The no-maps builds continue to remove mapping-specific writes at build time.

## Validation flow

```mermaid
flowchart TB
    fixture["Fixture<br/>filename + sourceText + parse options"]

    fixture --> rustParse
    fixture --> jsParse

    subgraph rust["Rust oracle"]
        direction TB
        rustParse["Parse with oxc_parser"]
        normalize["Normalize Rust-only AST information"]
        rustPrint["Print with oxc_codegen<br/>source maps enabled"]
        rustResult["Expected result<br/>{ code, map }"]

        rustParse --> normalize
        normalize --> rustPrint
        rustPrint --> rustResult
    end

    subgraph js["JavaScript candidate"]
        direction TB
        jsParse["Parse ESTree with oxc-parser<br/>start/end or range offsets"]

        plainPrint["Print with no-maps build"]
        plainResult["Actual code"]

        mappedPrint["Print with maps-enabled build"]
        deferred["Record generated offsets,<br/>source offsets, and names"]
        encode["Convert UTF-16 offsets to positions<br/>deduplicate and encode VLQ"]
        mappedResult["Actual result<br/>{ code, map }"]

        jsParse --> plainPrint
        plainPrint --> plainResult

        jsParse --> mappedPrint
        mappedPrint --> deferred
        deferred --> encode
        encode --> mappedResult
    end

    rustParse --> parseGate
    jsParse --> parseGate

    rustResult --> codeGate
    plainResult --> codeGate
    mappedResult --> codeGate

    rustResult --> mapGate
    mappedResult --> mapGate

    parseGate["Parse agreement<br/>Rust success requires no JS errors"]
    codeGate["Code equality<br/>both JS builds equal Rust"]
    mapGate["Source Map v3 equality<br/>mappings + names + sources + content"]

    parseGate --> pass["Fixture passes"]
    codeGate --> pass
    mapGate --> pass
```

Each fixture runs with both `preserveParens: false` and `preserveParens: true`.

## AST normalization

Rust’s AST contains information that ESTree cannot represent. The Rust AST is normalized before printing so the comparison measures code generation rather than representation differences.

This includes:

- parenthesized function and arrow-expression flags;
- `import "m"` versus `import {} from "m"`;
- absent versus empty import-attribute clauses;
- `assert` versus `with` attribute keywords;
- the unavailable ESTree span for a complete `with` clause.
This commit is contained in:
camc314 committed 2026-08-18 11:41:34 +00:00
1 parent 228e8e0f85
commit 300763f328
26 files changed
+1207 -358

No files matched your search

Generated
+1
View File
@@ -1920,6 +1920,7 @@ dependencies = [
"oxc_codegen",
"oxc_napi",
"oxc_parser",
"oxc_sourcemap",
"oxc_syntax",
]
+22 -16
View File
@@ -68,7 +68,6 @@ The conformance suites check this on every fixture. It is what makes the two pri
- No minify mode.
- No support for comments.
- No symbol mangling.
- Source map support is implemented, but not well-tested.
Many of these features can be added in future, and we should be able to do so without _much_ impact on performance.
@@ -300,7 +299,7 @@ This is why the debug build is the one to run the conformance suites against - `
| `print_ts.js` | true | false | 39 KB |
| `print_ts_maps.js` | true | true | 40 KB |
`index.ts` picks one from the caller's `ts` and `sourceMap` options and `require`s it on first use -
`index.ts` picks one from the caller's `ts` and `sourcemap` options and `require`s it on first use -
`require` rather than `import()`, because `printSync` is synchronous.
A caller printing only JavaScript never loads, parses or compiles the TypeScript printers at all.
@@ -505,12 +504,14 @@ Each of these produced a small perf bump individually, ~10% gain in aggregate.
### Source maps computed at the end
During printing, a mapped write records only an output _offset_ and the node's original position.
During printing, a mapped write records only an output offset and the node's original UTF-16 offset
from `start` / `end`. Nodes without offsets are not mapped, and `sourceText` is required to convert
source offsets to line/column positions.
`emitMappings` then walks the output once at the end, counting newlines, to turn those offsets into
generated line/column. Tracking a line and column throughout would cost every write in every build.
The `Mapping` objects handed to `addMapping` are reused across calls rather than allocated per mapping.
`generateSourceMap` then walks the output once at the end, counting ECMAScript line terminators,
builds the equivalent line table for `sourceText`, turns both sets of offsets into line/column, and
encodes the mappings as base64 VLQ. Tracking lines and columns throughout would cost every write in
every build.
#### Potential future improvement
@@ -564,6 +565,7 @@ so that arm can run.
- `import "m";` versus `import {} from "m";`
- An absent `with` clause versus an empty `with {}`
- `assert {...}` versus `with {...}`
- The span of a `with {...}` clause (only its individual attributes have ESTree locations)
The conformance harness normalizes the Rust AST down to what ESTree can express before printing,
rather than expecting the JS side to reproduce information it was never given.
@@ -571,9 +573,9 @@ See `Normalize` in `tasks/codegen_conformance/src/lib.rs`.
### Positions, and other producers
- **`loc`, not spans.** Oxc's AST records byte spans. Source mappings need line/column, which other
producers put in `loc`. Mapped writes check the node for `loc` rather than checking the build.
- **Acorn and TS-ESLint ASTs are accepted**, and differ from Oxc's in a handful of places
- **Oxc offsets only.** `start` / `end` offsets are converted to UTF-16 line/column once at the end.
Source maps require `sourceText`; `loc` and `range` are not supported inputs.
- **TS-ESLint ASTs are accepted**, and differ from Oxc's in a handful of places
(`TSMappedType`, `TSImportType`, `TSEnumDeclaration`, `TSModuleDeclaration`).
[`print/types.ts`] holds the widened node types and documents each difference.
@@ -581,14 +583,18 @@ See `Normalize` in `tasks/codegen_conformance/src/lib.rs`.
`oxc-codegen` is tested against all Test262, Acorn-JSX, and TypeScript test cases - about 62,000 fixtures.
Every fixture is printed twice, and the two must agree byte for byte:
Every fixture is printed three times:
1. In Rust, via the `oxc-codegen-conformance` NAPI addon (`tasks/codegen_conformance`).
2. In JS, by this package.
1. In Rust, with a source map, via the `oxc-codegen-conformance` NAPI addon (`tasks/codegen_conformance`).
2. In JS, through the no-maps build.
3. In JS, through the separately compiled maps build.
Both sides parse the same source text with the same `SourceType`, derived by the same function,
so the only thing under test is the printing. Fixtures which do not parse cleanly are skipped,
rather than passing quietly.
All three generated outputs must agree byte for byte. Every decoded generated/original line, column,
and optional original name from the maps build must also agree with Rust, including ordering and
duplicate suppression.
Both sides parse the same source text with the same `SourceType`, derived by the same function.
Fixtures which do not parse cleanly are skipped, rather than passing quietly.
Every fixture is checked in **both `preserveParens` modes**. They are different paths through the printer -
with the wrappers present it must decide what to re-emit, without them it must re-derive every parenthesis
+23 -2
View File
@@ -79,10 +79,28 @@ const { code } = printSync(program, {
### `printSync(node, options?)`
```ts
function printSync(node: Node, options?: Options): { code: string };
function printSync(node: Node, options?: Options): { code: string; map?: SourceMap };
```
Prints a complete `Program` or a single statement and returns the generated source code.
Prints a complete `Program` or a single statement and returns the generated source code and,
when requested, a standard Source Map v3 object.
```js
import { printSync } from "oxc-codegen";
import { parseSync } from "oxc-parser";
const sourceText = "const answer=6*7";
const { program } = parseSync("input.js", sourceText);
const { code, map } = printSync(program, {
sourcemap: true,
sourceFileName: "input.js",
sourceText,
});
```
Source-map mappings require `sourceText` and nodes with valid Oxc `start` / `end` offsets. A
manually constructed AST without offsets can still be printed, but its source map has an empty
`mappings` string.
### Options
@@ -92,6 +110,9 @@ Prints a complete `Program` or a single statement and returns the generated sour
| `startingIndentLevel` | `number` | `0` | Starting indent level, from `0` to `1000` |
| `jsx` | `boolean` | `false` | Enable TSX-safe printing for ambiguous TypeScript syntax |
| `ts` | `boolean` | `false` | Select the printer that supports TypeScript nodes |
| `sourcemap` | `boolean` | `false` | Return a Source Map v3 object in `map` |
| `sourceFileName` | `string` | `""` | Original source filename recorded in the source map |
| `sourceText` | `string` | - | Original text required for source-map mappings and content |
## Why pure JavaScript?
+1 -1
View File
@@ -1,6 +1,6 @@
// Benchmarks. Run with `pnpm run bench`.
//
// Source maps are not measured. `printSync` is given no `sourceMap`, so it selects the builds
// Source maps are not measured. `printSync` is given no `sourcemap`, so it selects the builds
// compiled without source map support, which is what most callers use.
//
// Imports from `dist`, which must hold a RELEASE build. `pnpm run build-dev` replaces `dist` with a debug build,
+6 -6
View File
@@ -14,7 +14,7 @@
// pass through the same functions, every function becomes polymorphic, which is a large slow-down.
// Having specialized code paths for JS and TS-shape ASTs avoids this problem - most functions remain monomorphic.
//
// Each combination is its own build, lazy-loaded depending on the `sourceMap` and `ts` options.
// Each combination is its own build, lazy-loaded depending on the `sourcemap` and `ts` options.
//
// `require` rather than `import()` because the printer is synchronous.
// All builds are ESM, which `require` can load on the Node versions in this package's `engines` field.
@@ -22,10 +22,10 @@
import { createRequire } from "node:module";
import { State } from "./state.ts";
import type { Options } from "./print/options.ts";
import type { CodegenResult, Options } from "./print/options.ts";
import type * as ESTree from "../../../npm/oxc-types/types.d.ts";
export type { Mapping, Options, Position, SourceMapGenerator } from "./print/options.ts";
export type { CodegenResult, Options, SourceMap } from "./print/options.ts";
/**
* All printer builds are compiled from `print/index.ts`, so they share its exports.
@@ -40,7 +40,7 @@ const require = createRequire(import.meta.url);
const EMPTY_OPTIONS: Options = {};
/**
* Printer builds, indexed by `(ts ? 1 : 0) + (sourceMap ? 2 : 0)`. Lazily loaded as needed.
* Printer builds, indexed by `(ts ? 1 : 0) + (sourcemap ? 2 : 0)`. Lazily loaded as needed.
*/
const PRINTER_PATHS = [
"./print_js.js",
@@ -61,7 +61,7 @@ const printers: (PrintModule["printSync"] | null)[] = [null, null, null, null];
* @param options - Printing options (optional)
* @returns Object holding the generated code
*/
export function printSync(node: ESTree.Node, options?: Options): { code: string } {
export function printSync(node: ESTree.Node, options?: Options): CodegenResult {
// The printer is built 4 times, over whether the AST may contain TypeScript and whether
// source mappings are wanted. This picks the build the options call for and loads it on first use,
// so a caller printing only JavaScript never pays for the TypeScript printers.
@@ -70,7 +70,7 @@ export function printSync(node: ESTree.Node, options?: Options): { code: string
options = EMPTY_OPTIONS;
} else {
if (options.ts === true) index = 1;
if (options.sourceMap != null) index |= 2;
if (options.sourcemap === true) index |= 2;
}
let print = printers[index];
@@ -62,7 +62,7 @@ function printObjectAssignmentTarget(node: ESTree.ObjectAssignmentTarget, state:
const property = properties[i];
if (property.type === "RestElement") {
write(state, "...", CAT_OTHER);
writeWithMap(state, "...", CAT_OTHER, property);
printAssignmentTarget(property.argument, state);
} else {
printAssignmentTargetProperty(property, state);
@@ -155,7 +155,7 @@ function printArrayAssignmentTarget(node: ESTree.ArrayAssignmentTarget, state: S
if (rest !== null) {
if (length > 0) write(state, " ", CAT_OTHER);
write(state, "...", CAT_OTHER);
writeWithMap(state, "...", CAT_OTHER, rest);
printAssignmentTarget(rest.argument, state);
}
@@ -85,7 +85,7 @@ function printObjectBindingPattern(node: ESTree.ObjectPattern, state: State): vo
if (i > 0) write(state, ", ", CAT_OTHER);
const property = properties[i];
if (property.type === "RestElement") {
write(state, "...", CAT_OTHER);
writeWithMap(state, "...", CAT_OTHER, property);
printBindingPattern(property.argument, state);
} else {
printBindingProperty(property, state);
+14 -6
View File
@@ -9,9 +9,12 @@ import {
CAT_QUESTION,
CAT_START_OF_STMT,
debugAssertLastFresh,
markWithMap,
write,
writeNoLast,
writeWithMap,
writeWithMapEnd,
writeWithMapNoLast,
} from "./write.ts";
import { printExpression } from "./expression.ts";
import { printFunctionBody, printParenParams } from "./function.ts";
@@ -157,7 +160,8 @@ function printClassBody(node: ClassBodyNode, state: State): void {
const { body } = node;
const { length } = body;
if (length === 0) {
writeWithMap(state, "{}", CAT_OTHER, node);
writeWithMapNoLast(state, "{", node);
writeWithMapEnd(state, "}", CAT_OTHER, node);
return;
}
@@ -209,13 +213,14 @@ function printClassBody(node: ClassBodyNode, state: State): void {
state.indentLevel--;
printIndent(state);
write(state, "}", CAT_OTHER);
writeWithMapEnd(state, "}", CAT_OTHER, node);
}
/**
* Print a method, including getters, setters, constructors and their modifiers.
*/
function printMethodDefinition(node: MethodDefinitionNode, state: State): void {
markWithMap(state, node);
const { decorators } = node;
if (decorators != null && decorators.length > 0) printDecorators(decorators, state);
@@ -286,6 +291,7 @@ function printMethodDefinition(node: MethodDefinitionNode, state: State): void {
* Print a class field, with its modifiers and initializer.
*/
function printPropertyDefinition(node: PropertyDefinitionNode, state: State): void {
markWithMap(state, node);
const { decorators } = node;
if (decorators != null && decorators.length > 0) printDecorators(decorators, state);
@@ -352,11 +358,12 @@ function printStaticBlock(node: ESTree.StaticBlock, state: State): void {
const { body } = node;
const { length } = body;
if (length === 0) {
write(state, "{}", CAT_OTHER);
writeWithMapNoLast(state, "{", node);
writeWithMapEnd(state, "}", CAT_OTHER, node);
return;
}
write(state, "{\n", CAT_OTHER);
writeWithMap(state, "{\n", CAT_OTHER, node);
state.indentLevel++;
@@ -367,13 +374,14 @@ function printStaticBlock(node: ESTree.StaticBlock, state: State): void {
state.indentLevel--;
printIndent(state);
write(state, "}", CAT_OTHER);
writeWithMapEnd(state, "}", CAT_OTHER, node);
}
/**
* Print an `accessor` field.
*/
function printAccessorProperty(node: AccessorPropertyNode, state: State): void {
markWithMap(state, node);
const { decorators } = node;
if (decorators != null && decorators.length > 0) printDecorators(decorators, state);
@@ -398,7 +406,7 @@ function printAccessorProperty(node: AccessorPropertyNode, state: State): void {
}
printSpaceBeforeIdentifier(state);
writeWithMap(state, "accessor", CAT_IDENT, node);
write(state, "accessor", CAT_IDENT);
if (node.computed) {
write(state, " [", CAT_OTHER);
+48 -11
View File
@@ -51,9 +51,13 @@ import {
} from "./precedence.ts";
import {
debugAssertLastFresh,
markWithMap,
markWithMapAfter,
markWithMapAtStartOffset,
write,
writeNoLast,
writeWithMap,
writeWithMapEnd,
writeWithMapNoLast,
} from "./write.ts";
import { escapeScriptCloseTag } from "./string.ts";
@@ -158,6 +162,7 @@ export function printExpression(
printTemplateLiteral(node, state);
break;
case "TaggedTemplateExpression":
markWithMap(state, node);
printExpression(node.tag, state, PREC_POSTFIX, ctx & CTX_FORBID_CALL);
if (TS) printTypeArguments(node.typeArguments, state);
printTemplateLiteral(node.quasi, state);
@@ -191,6 +196,18 @@ export function printExpression(
write(state, "(", CAT_OTHER);
printExpression(inner, state, PREC_LOWEST, CTX_NONE);
write(state, ")", CAT_OTHER);
if (SOURCEMAPS && precedence === PREC_POSTFIX) {
markWithMapAfter(state, inner);
const wrappers: ESTree.ParenthesizedExpression[] = [];
let wrapper = expression;
while (wrapper.type === "ParenthesizedExpression") {
wrappers.push(wrapper);
wrapper = wrapper.expression;
}
for (let index = wrappers.length - 1; index >= 0; index--) {
markWithMapAfter(state, wrappers[index]);
}
}
} else {
printExpression(expression, state, precedence, ctx);
}
@@ -222,6 +239,13 @@ export function printExpression(
default:
throw new Error(`Unknown expression type: ${node.type}`);
}
// Rust adds an exclusive-end mapping after a postfix operand which ended in `)` or `]`, so the
// punctuation of a following call/member/tag chain resolves to the operand rather than one
// character to its left.
if (SOURCEMAPS && precedence === PREC_POSTFIX && state.lastWasPostfixClose) {
markWithMapAfter(state, node);
}
}
/**
@@ -310,7 +334,7 @@ function printCallExpression(
if (TS) printTypeArguments(node.typeArguments, state);
printArguments(node.arguments, state);
printArguments(node, node.arguments, state);
if (wrap) write(state, ")", CAT_OTHER);
}
@@ -320,10 +344,15 @@ function printCallExpression(
*
* Arguments print at `PREC_COMMA`, so a sequence expression parenthesizes itself instead of reading as more arguments.
*/
function printArguments(args: ESTree.Argument[], state: State): void {
function printArguments(
node: ESTree.CallExpression | ESTree.NewExpression,
args: ESTree.Argument[],
state: State,
): void {
const { length } = args;
if (length === 0) {
write(state, "()", CAT_OTHER);
writeNoLast(state, "(");
writeWithMapEnd(state, ")", CAT_OTHER, node);
return;
}
@@ -334,14 +363,14 @@ function printArguments(args: ESTree.Argument[], state: State): void {
const arg = args[i];
if (arg.type === "SpreadElement") {
write(state, "...", CAT_OTHER);
writeWithMap(state, "...", CAT_OTHER, arg);
printExpression(arg.argument, state, PREC_COMMA, CTX_NONE);
} else {
printExpression(arg, state, PREC_COMMA, CTX_NONE);
}
}
write(state, ")", CAT_OTHER);
writeWithMapEnd(state, ")", CAT_OTHER, node);
}
/**
@@ -359,6 +388,7 @@ export function printPrivateInExpression(
const wrap = precedence >= PREC_COMPARE;
if (wrap) write(state, "(", CAT_OTHER);
markWithMap(state, node);
writeWithMapNoLast(state, "#", node.left);
write(state, node.left.name, CAT_IDENT);
write(state, " in ", CAT_OTHER);
@@ -405,7 +435,7 @@ function printObjectExpression(node: ESTree.ObjectExpression, state: State): voi
write(state, " ", CAT_OTHER);
}
write(state, "}", CAT_OTHER);
writeWithMapEnd(state, "}", CAT_OTHER, node);
if (wrap) write(state, ")", CAT_OTHER);
}
@@ -428,6 +458,7 @@ function printObjectProperty(node: ESTree.ObjectPropertyKind, state: State): voi
value.type === "FunctionExpression" ||
(TS && value.type === "TSEmptyBodyFunctionExpression")
) {
markWithMap(state, node);
const { kind } = node;
const isGetter = kind === "get";
const isAccessor = isGetter || kind === "set";
@@ -544,7 +575,7 @@ function printArrayExpression(node: ESTree.ArrayExpression, state: State): void
const element = elements[i];
if (element != null) {
if (element.type === "SpreadElement") {
write(state, "...", CAT_OTHER);
writeWithMap(state, "...", CAT_OTHER, element);
printExpression(element.argument, state, PREC_COMMA, CTX_NONE);
} else {
printExpression(element, state, PREC_COMMA, CTX_NONE);
@@ -562,7 +593,7 @@ function printArrayExpression(node: ESTree.ArrayExpression, state: State): void
printIndent(state);
}
write(state, "]", CAT_OTHER);
writeWithMapEnd(state, "]", CAT_OTHER, node);
}
/**
@@ -589,6 +620,7 @@ function printAssignmentExpression(
if (wrap) write(state, "(", CAT_OTHER);
markWithMap(state, node);
printAssignmentTarget(left, state);
write(state, PADDED_ASSIGN_OPERATORS[node.operator], CAT_OTHER);
printExpression(node.right, state, PREC_COMMA, ctx);
@@ -620,6 +652,7 @@ function printUpdateExpression(
writeWithMap(state, node.operator, operatorCode, node);
printExpression(node.argument, state, PREC_PREFIX, ctx);
} else {
markWithMap(state, node);
printExpression(node.argument, state, PREC_POSTFIX, ctx);
printSpaceBeforeOperator(state, operatorCode);
write(state, node.operator, operatorCode);
@@ -800,7 +833,7 @@ function printNewExpression(node: ESTree.NewExpression, state: State, precedence
writeWithMap(state, "new ", CAT_OTHER, node);
printExpression(node.callee, state, PREC_NEW, CTX_FORBID_CALL);
if (TS) printTypeArguments(node.typeArguments, state);
printArguments(node.arguments, state);
printArguments(node, node.arguments, state);
if (wrap) write(state, ")", CAT_OTHER);
}
@@ -817,7 +850,7 @@ function printTemplateLiteral(node: ESTree.TemplateLiteral, state: State): void
const { quasis, expressions } = node;
const { length } = expressions;
const firstQuasi = quasis[0];
writeWithMapNoLast(state, templateQuasiRaw(firstQuasi), firstQuasi);
writeNoLast(state, templateQuasiRaw(firstQuasi));
for (let i = 0; i < length; i++) {
write(state, "${", CAT_OTHER);
@@ -825,7 +858,11 @@ function printTemplateLiteral(node: ESTree.TemplateLiteral, state: State): void
writeNoLast(state, "}");
const quasi = quasis[i + 1];
writeWithMapNoLast(state, templateQuasiRaw(quasi), quasi);
const raw = templateQuasiRaw(quasi);
// A TS-shaped Oxc ESTree quasi includes the substitution's closing `}` in its span; the
// JS-shaped ESTree quasi and Oxc's Rust AST both start at the raw template text.
if (raw.length > 0) markWithMapAtStartOffset(state, quasi, TS ? 1 : 0);
writeNoLast(state, raw);
}
write(state, "`", CAT_OTHER);
+15 -2
View File
@@ -7,9 +7,12 @@ import {
CAT_OTHER,
CAT_START_OF_STMT,
debugAssertLastFresh,
markWithMapNoName,
write,
writeNoLast,
writeWithMap,
writeWithMapEnd,
writeWithMapNoLast,
} from "./write.ts";
import { printDecorators } from "./class.ts";
import { printSpaceBeforeIdentifier } from "./space.ts";
@@ -98,7 +101,16 @@ function printParams(params: ESTree.ParamPattern[], state: State): void {
const param = params[i];
// Oxc stores TypeScript's special `this` parameter separately from formal parameters and
// prints it without a source mapping.
if (TS && param.type === "Identifier" && param.name === "this") {
write(state, "this", CAT_IDENT);
if (param.typeAnnotation != null) printTypeAnnotation(param.typeAnnotation, state);
continue;
}
const { decorators } = param;
markWithMapNoName(state, decorators != null && decorators.length > 0 ? decorators[0] : param);
if (decorators != null && decorators.length > 0) printDecorators(decorators, state);
if (TS && param.type === "TSParameterProperty") {
@@ -136,7 +148,8 @@ export function printFunctionBody(body: ESTree.FunctionBody, state: State): void
// `body` is a BlockStatement holding directives + statements.
const statements = body.body;
if (statements.length === 0) {
writeWithMap(state, "{}", CAT_OTHER, body);
writeWithMapNoLast(state, "{", body);
writeWithMapEnd(state, "}", CAT_OTHER, body);
return;
}
@@ -146,7 +159,7 @@ export function printFunctionBody(body: ESTree.FunctionBody, state: State): void
state.indentLevel--;
printIndent(state);
write(state, "}", CAT_OTHER);
writeWithMapEnd(state, "}", CAT_OTHER, body);
}
/**
+6 -10
View File
@@ -10,10 +10,10 @@
// Reference: `oxc/crates/oxc_codegen/src/{gen.rs,lib.rs,str.rs,binary_expr_visitor.rs}`
import { debugAssert, typeAssertIs } from "../asserts.ts";
import { emitMappings } from "./source_map.ts";
import { generateSourceMap } from "./source_map.ts";
import { printProgram, printStatement } from "./statement.ts";
import type { Options } from "./options.ts";
import type { CodegenResult, Options } from "./options.ts";
import type { State } from "../state.ts";
import type * as ESTree from "../../../../npm/oxc-types/types.d.ts";
@@ -29,7 +29,7 @@ import type * as ESTree from "../../../../npm/oxc-types/types.d.ts";
* @param options - The same options `state` was created from, for the parts only this build acts on
* @returns Object holding the generated code
*/
export function printSync(node: ESTree.Node, state: State, options: Options): { code: string } {
export function printSync(node: ESTree.Node, state: State, options: Options): CodegenResult {
if (node.type === "Program") {
printProgram(node, state);
} else {
@@ -39,15 +39,11 @@ export function printSync(node: ESTree.Node, state: State, options: Options): {
// This is removed by minifier in non-sourcemap builds
if (SOURCEMAPS) {
debugAssert(
options.sourceMap != null,
"`options.sourceMap` should be defined when `SOURCEMAPS` is `true`",
);
emitMappings(state, options.sourceMap);
debugAssert(options.sourcemap === true, "`options.sourcemap` should be true in a maps build");
return { code: state.output, map: generateSourceMap(state, options) };
}
return { code: state.output };
}
export type { Mapping, Options, Position, SourceMapGenerator } from "./options.ts";
export type { CodegenResult, Options, SourceMap } from "./options.ts";
+16 -8
View File
@@ -1,7 +1,14 @@
// JSX.
// Port of `gen.rs` JSX section.
import { CAT_OTHER, write, writeNoLast, writeWithMap, writeWithMapNoLast } from "./write.ts";
import {
CAT_OTHER,
write,
writeNoLast,
writeWithMap,
writeWithMapEnd,
writeWithMapNoLast,
} from "./write.ts";
import { printExpression } from "./expression.ts";
import { CTX_NONE } from "./operators.ts";
import { PREC_COMMA, PREC_LOWEST } from "./precedence.ts";
@@ -31,9 +38,10 @@ export function printJSXElement(node: ESTree.JSXElement, state: State): void {
writeNoLast(state, " ");
const attribute = attributes[i];
if (attribute.type === "JSXSpreadAttribute") {
write(state, "{...", CAT_OTHER);
writeWithMapNoLast(state, "{", attribute);
write(state, "...", CAT_OTHER);
printExpression(attribute.argument, state, PREC_COMMA, CTX_NONE);
writeNoLast(state, "}");
writeWithMapEnd(state, "}", CAT_OTHER, attribute);
} else {
printJSXAttribute(attribute, state);
}
@@ -75,9 +83,9 @@ function printJSXElementName(
printJSXElementName(node.property, state);
break;
case "JSXNamespacedName":
writeWithMapNoLast(state, node.namespace.name, node);
writeWithMapNoLast(state, node.namespace.name, node.namespace);
writeNoLast(state, ":");
writeNoLast(state, node.name.name);
writeWithMapNoLast(state, node.name.name, node.name);
break;
case "ThisExpression":
writeWithMapNoLast(state, "this", node);
@@ -97,9 +105,9 @@ function printJSXAttribute(node: ESTree.JSXAttribute, state: State): void {
// and the next real write reads `last`.
const { name } = node;
if (name.type === "JSXNamespacedName") {
writeWithMapNoLast(state, name.namespace.name, name);
writeWithMapNoLast(state, name.namespace.name, name.namespace);
writeNoLast(state, ":");
writeNoLast(state, name.name.name);
writeWithMapNoLast(state, name.name.name, name.name);
} else {
writeWithMapNoLast(state, name.name, name);
}
@@ -132,7 +140,7 @@ function printJSXAttributeValue(node: ESTree.JSXAttributeValue | UnknownNode, st
const text = raw != null ? raw.slice(1, -1) : String(node.value);
const quote = text.includes('"') ? "'" : '"';
writeWithMapNoLast(state, quote, node);
writeNoLast(state, quote);
writeNoLast(state, text);
writeNoLast(state, quote);
break;
+8 -3
View File
@@ -4,6 +4,7 @@ import {
CAT_IDENT,
CAT_OTHER,
CAT_START_OF_DEFAULT_EXPORT,
markWithMapAtStartOffset,
write,
writeNoLast,
writeWithMap,
@@ -89,7 +90,7 @@ export function printImportDeclaration(node: ESTree.ImportDeclaration, state: St
write(state, ", ", CAT_OTHER);
}
writeWithMap(state, "* as ", CAT_OTHER, specifier);
write(state, "* as ", CAT_OTHER);
writeWithMap(state, specifier.local.name, CAT_IDENT, specifier.local);
write(state, " ", CAT_OTHER);
break;
@@ -139,7 +140,11 @@ function printImportAttributes(
const { length } = attributes;
if (length === 0) return;
write(state, " with { ", CAT_OTHER);
// ESTree omits the `WithClause` wrapper. The Rust reference normalizes its mapping anchor to the
// first attribute, which is the first location both representations carry.
write(state, " ", CAT_OTHER);
markWithMapAtStartOffset(state, attributes[0], 0);
write(state, "with { ", CAT_OTHER);
for (let i = 0; i < length; i++) {
if (i > 0) write(state, ", ", CAT_OTHER);
@@ -147,7 +152,7 @@ function printImportAttributes(
const attribute = attributes[i];
const { key } = attribute;
if (key.type === "Identifier") {
writeWithMap(state, key.name, CAT_IDENT, key);
write(state, key.name, CAT_IDENT);
} else {
printString(state, key.value, key);
}
+20 -37
View File
@@ -1,41 +1,18 @@
// Printing options, and the source map types the `sourceMap` option brings with it.
// Printing options and returned output types.
/**
* Source map generator, compatible with `SourceMapGenerator` from the `source-map` package.
*/
export interface SourceMapGenerator {
addMapping(mapping: Mapping): void;
file?: string;
_file?: string;
/** Standard Source Map v3 output, compatible with Rollup and other JavaScript tooling. */
export interface SourceMap {
version: 3;
mappings: string;
names: string[];
sources: string[];
sourcesContent?: string[];
}
/**
* Source map mapping emitted through `Options["sourceMap"]`.
*
* Note: the `Mapping` objects passed to `addMapping` are reused across calls;
* implementations must copy any values they need to retain.
*/
export interface Mapping {
original: Position;
generated: Position;
name: string | undefined;
source: string;
}
/**
* A `Mapping` before the per-mapping fields are filled in.
*/
export type MutableMapping = Omit<Mapping, "original" | "source"> & {
original: Position | null;
source: string | undefined;
};
/**
* Position in the original source, as found on a node's `loc.start`.
*/
export interface Position {
line: number;
column: number;
/** Result returned by `printSync`. */
export interface CodegenResult {
code: string;
map?: SourceMap;
}
/**
@@ -66,8 +43,14 @@ export interface Options {
*/
ts?: boolean;
/** Original source text. Required for source-map mappings. */
sourceText?: string;
/**
* If present, source mappings are emitted through it.
* Generate and return a source map in `CodegenResult.map`.
*/
sourceMap?: SourceMapGenerator;
sourcemap?: boolean;
/** Original source filename recorded in the returned source map. */
sourceFileName?: string;
}
+348 -37
View File
@@ -1,55 +1,366 @@
// Source map generation.
//
// `write` records the output offset and original position of every mapped node as it goes,
// and this converts those offsets to generated line/column in one pass at the end.
// Mapped writes record output offsets and original positions while printing. This module converts
// them to generated positions and encodes a standard Source Map v3 in one pass at the end.
import { debugAssert, typeAssertIs } from "../asserts.ts";
import { Buffer } from "node:buffer";
import type { Mapping, MutableMapping, SourceMapGenerator } from "./options.ts";
import { debugAssert } from "../asserts.ts";
import type { Options, SourceMap } from "./options.ts";
import type { State } from "../state.ts";
/**
* Convert the offsets recorded during printing into mappings, and hand them to the generator.
*
* Printing records only an output offset per mapped node, which is why this walks the output once
* at the end counting newlines, rather than tracking a line and column throughout.
*
* @param sourceMap - Generator the mappings are added to, which the caller was given in its options
*/
export function emitMappings(state: State, sourceMap: SourceMapGenerator): void {
/** Convert deferred mapping data into a standard Source Map v3 object. */
export function generateSourceMap(state: State, options: Options): SourceMap {
debugAssert(
state.mapOffsets !== null && state.mapPositions !== null && state.mapNames !== null,
"Source map arrays should exist when a `sourceMap` was given",
state.mapPositions !== null,
"Source map positions should exist when sourcemap generation is enabled",
);
const { output, mapOffsets, mapPositions, mapNames } = state;
const source = sourceMap.file || sourceMap._file;
const { output, mapPositions, mapNames } = state;
const mappingCount = mapPositions.length >> 1;
const { sourceText } = options;
debugAssert(
sourceText !== undefined || mappingCount === 0,
"Mappings should only be recorded when sourceText is available",
);
if (sourceText === undefined || mappingCount === 0) {
const map: SourceMap = {
version: 3,
mappings: "",
names: [],
sources: [options.sourceFileName ?? ""],
};
if (sourceText !== undefined) map.sourcesContent = [sourceText];
return map;
}
// The `generated` and `mapping` objects are reused across `addMapping` calls to avoid generating ephemeral objects
const generated = { line: 1, column: 0 };
const mapping: MutableMapping = { original: null, generated, name: undefined, source };
let mappingBuffer: Buffer = Buffer.allocUnsafe(
Math.max(MIN_MAPPING_BUFFER_LENGTH, mappingCount * ESTIMATED_BYTES_PER_MAPPING),
);
let mappingLength = 0;
const names: string[] = [];
let nameIds: Map<string, number> | undefined;
let mapNameEntryIndex = 0;
let nextNamedMappingIndex = (mapNames?.[0] as number | undefined) ?? Infinity;
let line = 1;
let sourceLineStarts: number[] | undefined;
let sourceScanOffset = 0;
let sourceLine = 0;
let sourceLineStart = 0;
let replayedSourceScanTotal = 0;
let lineStart = 0;
let nextNewline = output.indexOf("\n");
if (nextNewline === -1) nextNewline = Infinity;
// Proving an output contains only `\n` takes a full scan. Amortize it only when enough mappings
// will benefit; sparse maps and huge one-line literals stay on the single-pass regexp path.
const useOutputLineFeedFastPath =
mappingCount >= MIN_LF_FAST_PATH_MAPPINGS &&
output.length <= mappingCount * MAX_LF_FAST_PATH_CHARS_PER_MAPPING &&
!hasUncommonLineTerminator(output);
// Require mappings to cover a substantial part of the source, so looking for the first line
// break cannot scan a huge unmapped suffix. Reordered inputs conservatively take the slow path.
const useSourceLineBoundaryCache =
mappingCount >= MIN_LF_FAST_PATH_MAPPINGS &&
sourceText.length <= mappingCount * MAX_LF_FAST_PATH_CHARS_PER_MAPPING &&
mapPositions[mapPositions.length - 1] * 2 >= sourceText.length;
const useSourceLineFeedFastPath =
useSourceLineBoundaryCache && hasOnlyLineFeedsAndCrLf(sourceText);
let nextLineStart = findNextLineStart(output, 0, useOutputLineFeedFastPath);
let nextSourceLineStart = useSourceLineBoundaryCache
? findNextLineStart(sourceText, 0, useSourceLineFeedFastPath)
: Infinity;
let hasSegmentOnLine = false;
let previousGeneratedColumn = 0;
let previousOriginalLine = 0;
let previousOriginalColumn = 0;
let previousNameId = 0;
const { length } = mapOffsets;
for (let i = 0; i < length; i++) {
const offset = mapOffsets[i];
while (offset > nextNewline) {
line++;
lineStart = nextNewline + 1;
const next = output.indexOf("\n", lineStart);
nextNewline = next === -1 ? Infinity : next;
for (let index = 0, positionIndex = 0; index < mappingCount; index++, positionIndex += 2) {
const offset = mapPositions[positionIndex];
while (offset >= nextLineStart) {
lineStart = nextLineStart;
nextLineStart = findNextLineStart(output, lineStart, useOutputLineFeedFastPath);
previousGeneratedColumn = 0;
hasSegmentOnLine = false;
if (mappingLength === mappingBuffer.length) {
mappingBuffer = growMappingBuffer(mappingBuffer, mappingLength, mappingLength + 1);
}
mappingBuffer[mappingLength++] = 59; // `;`
}
generated.line = line;
generated.column = offset - lineStart;
mapping.original = mapPositions[i];
mapping.name = mapNames[i];
const generatedColumn = offset - lineStart;
let sourceOffset = mapPositions[positionIndex + 1];
// Rust end mappings use `span.end - 1`, which may land inside a multi-byte code point. The JS
// equivalent can land on a low surrogate; normalize it back to the code point's start.
if (sourceOffset > 0) {
const char = sourceText.charCodeAt(sourceOffset);
if (
char >= 0xdc00 &&
char <= 0xdfff &&
sourceText.charCodeAt(sourceOffset - 1) >= 0xd800 &&
sourceText.charCodeAt(sourceOffset - 1) <= 0xdbff
) {
sourceOffset--;
}
}
let originalLine: number;
let originalColumn: number;
if (sourceLineStarts === undefined) {
if (sourceOffset >= sourceScanOffset) {
// Mappings almost always advance through the source. Scan only the text between this
// mapping and the last one instead of building a line table for the whole source.
if (useSourceLineBoundaryCache) {
while (sourceOffset >= nextSourceLineStart) {
sourceLineStart = nextSourceLineStart;
nextSourceLineStart = findNextLineStart(
sourceText,
sourceLineStart,
useSourceLineFeedFastPath,
);
sourceLine++;
}
sourceScanOffset = sourceOffset;
} else {
while (sourceScanOffset < sourceOffset) {
const char = sourceText.charCodeAt(sourceScanOffset);
if (char === 13 && sourceText.charCodeAt(sourceScanOffset + 1) === 10) {
// An offset on the `\n` of `\r\n` still belongs to the preceding line.
if (sourceScanOffset + 1 === sourceOffset) break;
sourceScanOffset += 2;
sourceLine++;
sourceLineStart = sourceScanOffset;
} else if (char === 10 || char === 13 || char === 0x2028 || char === 0x2029) {
sourceScanOffset++;
sourceLine++;
sourceLineStart = sourceScanOffset;
} else {
sourceScanOffset++;
}
}
}
} else if (
sourceOffset >= sourceLineStart ||
(sourceScanOffset - sourceOffset <= MAX_BACKWARD_SOURCE_SCAN &&
replayedSourceScanTotal + sourceScanOffset - sourceOffset <=
Math.max(MAX_REPLAYED_SOURCE_SCAN, sourceText.length))
) {
// Parent/end mappings and locally reordered nodes can step backwards. These moves are
// normally within the current line or a nearby one, so a short reverse scan is cheaper
// than allocating a complete line table.
// A same-line backtrack needs no scan and leaves the forward cursor where it is. When the
// mapping crosses a line, count the text a later forward step must replay; once replaying
// costs as much as building a line table, the fallback below becomes cheaper.
if (sourceOffset < sourceLineStart) {
replayedSourceScanTotal += sourceScanOffset - sourceOffset;
while (sourceOffset < sourceLineStart) {
let index = sourceLineStart - 1;
if (sourceText.charCodeAt(index) === 10 && sourceText.charCodeAt(index - 1) === 13) {
index -= 2;
} else {
index--;
}
while (index >= 0) {
const char = sourceText.charCodeAt(index);
if (char === 10 || char === 13 || char === 0x2028 || char === 0x2029) break;
index--;
}
sourceLineStart = index + 1;
sourceLine--;
}
sourceScanOffset = sourceOffset;
if (useSourceLineBoundaryCache) {
nextSourceLineStart = findNextLineStart(
sourceText,
sourceLineStart,
useSourceLineFeedFastPath,
);
}
}
} else {
// A heavily transformed AST can jump arbitrarily far backwards or oscillate repeatedly.
// Build the table lazily once reverse scanning crosses either limit.
sourceLineStarts = getLineStarts(sourceText);
sourceLine = findSourceLine(sourceLineStarts, sourceOffset, previousOriginalLine);
sourceLineStart = sourceLineStarts[sourceLine];
sourceScanOffset = sourceOffset;
}
typeAssertIs<Mapping>(mapping);
sourceMap.addMapping(mapping);
originalLine = sourceLine;
originalColumn = sourceOffset - sourceLineStart;
} else {
originalLine = findSourceLine(sourceLineStarts, sourceOffset, previousOriginalLine);
originalColumn = sourceOffset - sourceLineStarts[originalLine];
}
if (mappingLength + MAX_MAPPING_SEGMENT_LENGTH > mappingBuffer.length) {
mappingBuffer = growMappingBuffer(
mappingBuffer,
mappingLength,
mappingLength + MAX_MAPPING_SEGMENT_LENGTH,
);
}
if (hasSegmentOnLine) mappingBuffer[mappingLength++] = 44; // `,`
mappingLength = writeVlq(
mappingBuffer,
mappingLength,
generatedColumn - previousGeneratedColumn,
);
// All mappings point into the one source file, so the source-index delta is always zero (`A`).
mappingBuffer[mappingLength++] = 65;
mappingLength = writeVlq(mappingBuffer, mappingLength, originalLine - previousOriginalLine);
mappingLength = writeVlq(mappingBuffer, mappingLength, originalColumn - previousOriginalColumn);
if (index === nextNamedMappingIndex) {
const name = mapNames![mapNameEntryIndex + 1] as string;
nameIds ??= new Map<string, number>();
let nameId = nameIds.get(name);
if (nameId === undefined) {
nameId = names.length;
names.push(name);
nameIds.set(name, nameId);
}
mappingLength = writeVlq(mappingBuffer, mappingLength, nameId - previousNameId);
previousNameId = nameId;
mapNameEntryIndex += 2;
nextNamedMappingIndex = (mapNames![mapNameEntryIndex] as number | undefined) ?? Infinity;
}
hasSegmentOnLine = true;
previousGeneratedColumn = generatedColumn;
previousOriginalLine = originalLine;
previousOriginalColumn = originalColumn;
}
const map: SourceMap = {
version: 3,
mappings: mappingBuffer.toString("ascii", 0, mappingLength),
names,
sources: [options.sourceFileName ?? ""],
};
if (sourceText !== undefined) map.sourcesContent = [sourceText];
return map;
}
/** Build UTF-16 offsets to the start of every ECMAScript source line. */
function getLineStarts(sourceText: string): number[] {
const lineStarts = [0];
for (let index = 0; index < sourceText.length; index++) {
const char = sourceText.charCodeAt(index);
if (char !== 10 && char !== 13 && char !== 0x2028 && char !== 0x2029) continue;
if (char === 13 && sourceText.charCodeAt(index + 1) === 10) index++;
lineStarts.push(index + 1);
}
return lineStarts;
}
/** Find the source line containing `sourceOffset`, starting close to the previous result. */
function findSourceLine(lineStarts: number[], sourceOffset: number, previousLine: number): number {
// Source mappings normally advance through the original file. Search the next few lines
// linearly, as V8 optimizes this small predictable loop well, then fall back for large jumps or
// transformed ASTs whose source locations move backwards.
if (sourceOffset >= lineStarts[previousLine]) {
const end = Math.min(previousLine + LINE_SEARCH_ITERATIONS + 1, lineStarts.length);
for (let line = previousLine + 1; line < end; line++) {
if (lineStarts[line] > sourceOffset) return line - 1;
}
if (end === lineStarts.length) return lineStarts.length - 1;
} else {
const end = Math.max(previousLine - LINE_SEARCH_ITERATIONS, 0);
for (let line = previousLine - 1; line >= end; line--) {
if (lineStarts[line] <= sourceOffset) return line;
}
}
let low = 0;
let high = lineStarts.length;
while (low < high) {
const middle = (low + high) >> 1;
if (lineStarts[middle] <= sourceOffset) low = middle + 1;
else high = middle;
}
return low - 1;
}
/** Write one signed source-map delta as base64 VLQ, returning the next buffer position. */
function writeVlq(buffer: Buffer, index: number, value: number): number {
let vlq = value < 0 ? -value * 2 + 1 : value * 2;
if (vlq <= MAX_BITWISE_VLQ) {
do {
let digit = vlq & 31;
vlq >>>= 5;
if (vlq > 0) digit |= 32;
buffer[index++] = BASE64_CODES[digit];
} while (vlq > 0);
return index;
}
do {
let digit = vlq % 32;
vlq = Math.floor(vlq / 32);
if (vlq > 0) digit += 32;
buffer[index++] = BASE64_CODES[digit];
} while (vlq > 0);
return index;
}
/** Grow the mappings buffer to contain `requiredLength`, preserving its written prefix. */
function growMappingBuffer(buffer: Buffer, writtenLength: number, requiredLength: number): Buffer {
let capacity = buffer.length * 2;
while (capacity < requiredLength) capacity *= 2;
const next = Buffer.allocUnsafe(capacity);
buffer.copy(next, 0, 0, writtenLength);
return next;
}
/** Find the UTF-16 offset after the next ECMAScript line terminator. */
function findNextLineStart(output: string, from: number, useLineFeedFastPath: boolean): number {
if (useLineFeedFastPath) {
const index = output.indexOf("\n", from);
return index === -1 ? Infinity : index + 1;
}
// Let the regexp engine scan long generated lines. A JS `charCodeAt` loop is much slower for
// large literals, while `lastIndex` avoids allocating a substring just to start the search at
// `from`.
NEXT_LINE_TERMINATOR_REGEX.lastIndex = from;
const match = NEXT_LINE_TERMINATOR_REGEX.exec(output);
return match === null ? Infinity : match.index + match[0].length;
}
/** Whether output contains a line terminator other than `\n`. */
function hasUncommonLineTerminator(output: string): boolean {
// V8's specialized substring search is substantially faster than a regexp scan here, even when
// all three searches miss. This also avoids allocating regexp match state for the common path.
return (
output.indexOf("\r") !== -1 ||
output.indexOf("\u2028") !== -1 ||
output.indexOf("\u2029") !== -1
);
}
/** Whether every source line terminator can be found by searching for `\n`. */
function hasOnlyLineFeedsAndCrLf(sourceText: string): boolean {
if (sourceText.indexOf("\u2028") !== -1 || sourceText.indexOf("\u2029") !== -1) return false;
let carriageReturn = sourceText.indexOf("\r");
while (carriageReturn !== -1) {
if (sourceText.charCodeAt(carriageReturn + 1) !== 10) return false;
carriageReturn = sourceText.indexOf("\r", carriageReturn + 1);
}
return true;
}
const BASE64_CHARS = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
const BASE64_CODES = /* @__PURE__ */ Buffer.from(BASE64_CHARS, "ascii");
const LINE_SEARCH_ITERATIONS = 16;
const MIN_LF_FAST_PATH_MAPPINGS = 64;
const MAX_LF_FAST_PATH_CHARS_PER_MAPPING = 256;
const MAX_BACKWARD_SOURCE_SCAN = 4096;
const MAX_REPLAYED_SOURCE_SCAN = 16384;
const MAX_BITWISE_VLQ = 0x7fffffff;
const MIN_MAPPING_BUFFER_LENGTH = 64;
// Real-world fixtures use 5.2–5.5 bytes per mapping. Leave some headroom and grow for outliers.
const ESTIMATED_BYTES_PER_MAPPING = 6;
const MAX_MAPPING_SEGMENT_LENGTH = 64;
// `\r\n` must be one line terminator, rather than two.
const NEXT_LINE_TERMINATOR_REGEX = /\r\n|[\r\n\u2028\u2029]/g;
+21 -11
View File
@@ -8,9 +8,11 @@ import {
CAT_OP_UN_NOT,
CAT_OTHER,
CAT_START_OF_STMT,
markWithMap,
write,
writeNoLast,
writeWithMap,
writeWithMapEnd,
writeWithMapNoLast,
} from "./write.ts";
import { printClass } from "./class.ts";
@@ -83,7 +85,9 @@ export function printDirectivesAndStatements(
const inner = withoutParens(stmt.expression);
typeAssertIs<LiteralExtras>(inner);
if (inner.type === "Literal" && typeof inner.value === "string") {
const mapsIndent = state.indentLevel > 0 || state.pendingIndentAsSpace;
printIndent(state);
if (mapsIndent) markWithMap(state, stmt);
writeNoLast(state, "(");
printString(state, inner.value, inner);
write(state, ");\n", CAT_OTHER);
@@ -222,13 +226,14 @@ export function printStatement(node: ESTree.Statement | UnknownNode, state: Stat
case "LabeledStatement":
printIndent(state);
printSpaceBeforeIdentifier(state);
markWithMap(state, node);
writeWithMap(state, node.label.name, CAT_IDENT, node.label);
write(state, ":", CAT_OTHER);
printBody(node.body, state);
break;
case "EmptyStatement":
printIndent(state);
write(state, ";\n", CAT_OTHER);
writeWithMap(state, ";\n", CAT_OTHER, node);
break;
case "ImportDeclaration":
printImportDeclaration(node, state);
@@ -288,13 +293,13 @@ export function printStatement(node: ESTree.Statement | UnknownNode, state: Stat
break;
case "TSExportAssignment":
printIndent(state);
writeWithMap(state, "export = ", CAT_OTHER, node);
write(state, "export = ", CAT_OTHER);
printExpression(node.expression, state, PREC_LOWEST, CTX_NONE);
write(state, ";\n", CAT_OTHER);
break;
case "TSNamespaceExportDeclaration":
printIndent(state);
writeWithMap(state, "export as namespace ", CAT_OTHER, node);
write(state, "export as namespace ", CAT_OTHER);
writeWithMap(state, node.id.name, CAT_IDENT, node.id);
write(state, ";\n", CAT_OTHER);
break;
@@ -311,7 +316,9 @@ export function printStatement(node: ESTree.Statement | UnknownNode, state: Stat
* The indent is written first, so the mark is the last thing to touch `last` before the expression.
*/
function printExpressionStatement(node: ESTree.ExpressionStatement, state: State): void {
const mapsIndent = state.indentLevel > 0 || state.pendingIndentAsSpace;
printIndent(state);
if (mapsIndent) markWithMap(state, node);
state.last = CAT_START_OF_STMT;
printExpression(node.expression, state, PREC_LOWEST, CTX_NONE);
write(state, ";\n", CAT_OTHER);
@@ -380,7 +387,8 @@ function printBlockStatement(block: ESTree.BlockStatement, state: State): void {
const { body } = block;
const { length } = body;
if (length === 0) {
writeWithMap(state, "{}", CAT_OTHER, block);
writeWithMapNoLast(state, "{", block);
writeWithMapEnd(state, "}", CAT_OTHER, block);
return;
}
@@ -393,7 +401,7 @@ function printBlockStatement(block: ESTree.BlockStatement, state: State): void {
state.indentLevel--;
printIndent(state);
write(state, "}", CAT_OTHER);
writeWithMapEnd(state, "}", CAT_OTHER, block);
}
/**
@@ -416,12 +424,13 @@ function printIf(node: ESTree.IfStatement, state: State): void {
printBlockStatement(consequent, state);
write(state, alternate != null ? " " : "\n", CAT_OTHER);
} else if (wrapToAvoidAmbiguousElse(consequent)) {
write(state, ") {\n", CAT_OTHER);
write(state, ") ", CAT_OTHER);
writeWithMap(state, "{\n", CAT_OTHER, consequent);
state.indentLevel++;
printStatement(consequent, state);
state.indentLevel--;
printIndent(state);
write(state, "}", CAT_OTHER);
writeWithMapEnd(state, "}", CAT_OTHER, consequent);
write(state, alternate != null ? " " : "\n", CAT_OTHER);
} else {
write(state, ")", CAT_OTHER);
@@ -542,11 +551,12 @@ function printSwitchStatement(node: ESTree.SwitchStatement, state: State): void
const { cases } = node;
const { length } = cases;
if (length === 0) {
write(state, "{}\n", CAT_OTHER);
writeWithMapNoLast(state, "{", node);
writeWithMapEnd(state, "}\n", CAT_OTHER, node);
return;
}
write(state, "{\n", CAT_OTHER);
writeWithMap(state, "{\n", CAT_OTHER, node);
state.indentLevel++;
for (let i = 0; i < length; i++) {
@@ -555,7 +565,7 @@ function printSwitchStatement(node: ESTree.SwitchStatement, state: State): void
state.indentLevel--;
printIndent(state);
write(state, "}\n", CAT_OTHER);
writeWithMapEnd(state, "}\n", CAT_OTHER, node);
}
/**
@@ -626,7 +636,7 @@ function printDoWhileStatement(node: ESTree.DoWhileStatement, state: State): voi
write(state, " ", CAT_OTHER);
} else if (body.type === "EmptyStatement") {
printIndent(state);
write(state, ";\n", CAT_OTHER);
writeWithMap(state, ";\n", CAT_OTHER, body);
} else {
write(state, "\n", CAT_OTHER);
state.indentLevel++;
+4 -5
View File
@@ -11,7 +11,6 @@
// in a handful of places. The `*Node` aliases widen the properties where they do, and the branches
// which read a property Oxc's AST doesn't have at all assert its type at the point of use.
import type { Position } from "./options.ts";
import type * as ESTree from "../../../../npm/oxc-types/types.d.ts";
/** A node whose type the printer doesn't handle. */
@@ -111,11 +110,11 @@ export interface TSEnumDeclarationLegacyMembers {
}
/**
* A node, as passed to `writeWithMap` / `writeWithMapNoLast`, which record where the node started for source maps.
*
* `loc` is not part of Oxc's AST - that records byte spans - but producers such as Acorn add it.
* A node carrying the offsets needed for source mappings.
*/
export interface MappableNode {
type: string;
loc?: { start: Position } | null;
name?: unknown;
start?: number;
end?: number;
}
+38 -32
View File
@@ -11,6 +11,7 @@ import {
write,
writeNoLast,
writeWithMap,
writeWithMapEnd,
writeWithMapNoLast,
} from "./write.ts";
import { printExpression } from "./expression.ts";
@@ -88,59 +89,59 @@ function printTSType(
break;
case "TSStringKeyword":
printSpaceBeforeIdentifier(state);
writeWithMap(state, "string", CAT_IDENT, node);
write(state, "string", CAT_IDENT);
break;
case "TSNumberKeyword":
printSpaceBeforeIdentifier(state);
writeWithMap(state, "number", CAT_IDENT, node);
write(state, "number", CAT_IDENT);
break;
case "TSBooleanKeyword":
printSpaceBeforeIdentifier(state);
writeWithMap(state, "boolean", CAT_IDENT, node);
write(state, "boolean", CAT_IDENT);
break;
case "TSAnyKeyword":
printSpaceBeforeIdentifier(state);
writeWithMap(state, "any", CAT_IDENT, node);
write(state, "any", CAT_IDENT);
break;
case "TSVoidKeyword":
printSpaceBeforeIdentifier(state);
writeWithMap(state, "void", CAT_IDENT, node);
write(state, "void", CAT_IDENT);
break;
case "TSUnknownKeyword":
printSpaceBeforeIdentifier(state);
writeWithMap(state, "unknown", CAT_IDENT, node);
write(state, "unknown", CAT_IDENT);
break;
case "TSNeverKeyword":
printSpaceBeforeIdentifier(state);
writeWithMap(state, "never", CAT_IDENT, node);
write(state, "never", CAT_IDENT);
break;
case "TSUndefinedKeyword":
printSpaceBeforeIdentifier(state);
writeWithMap(state, "undefined", CAT_IDENT, node);
write(state, "undefined", CAT_IDENT);
break;
case "TSNullKeyword":
printSpaceBeforeIdentifier(state);
writeWithMap(state, "null", CAT_IDENT, node);
write(state, "null", CAT_IDENT);
break;
case "TSObjectKeyword":
printSpaceBeforeIdentifier(state);
writeWithMap(state, "object", CAT_IDENT, node);
write(state, "object", CAT_IDENT);
break;
case "TSSymbolKeyword":
printSpaceBeforeIdentifier(state);
writeWithMap(state, "symbol", CAT_IDENT, node);
write(state, "symbol", CAT_IDENT);
break;
case "TSBigIntKeyword":
printSpaceBeforeIdentifier(state);
writeWithMap(state, "bigint", CAT_IDENT, node);
write(state, "bigint", CAT_IDENT);
break;
case "TSIntrinsicKeyword":
printSpaceBeforeIdentifier(state);
writeWithMap(state, "intrinsic", CAT_IDENT, node);
write(state, "intrinsic", CAT_IDENT);
break;
case "TSThisType":
printSpaceBeforeIdentifier(state);
writeWithMap(state, "this", CAT_IDENT, node);
write(state, "this", CAT_IDENT);
break;
case "TSLiteralType":
printTSLiteral(node.literal, state);
@@ -249,7 +250,7 @@ function printTSType(
break;
case "TSJSDocUnknownType":
printSpaceBeforeIdentifier(state);
writeWithMap(state, "unknown", CAT_IDENT, node);
write(state, "unknown", CAT_IDENT);
break;
case "TSJSDocNullableType":
if (node.postfix) {
@@ -439,7 +440,8 @@ function printTSTypeLiteral(node: ESTree.TSTypeLiteral, state: State): void {
const { members } = node;
const { length } = members;
if (length === 0) {
writeWithMap(state, "{}", CAT_OTHER, node);
writeWithMapNoLast(state, "{", node);
writeWithMapEnd(state, "}", CAT_OTHER, node);
return;
}
@@ -454,7 +456,7 @@ function printTSTypeLiteral(node: ESTree.TSTypeLiteral, state: State): void {
state.indentLevel--;
printIndent(state);
write(state, "}", CAT_OTHER);
writeWithMapEnd(state, "}", CAT_OTHER, node);
}
/**
@@ -531,7 +533,7 @@ function printSignatureKey(key: ESTree.PropertyKey, state: State, ctx: number):
writeWithMap(state, key.name, CAT_IDENT, key);
break;
case "PrivateIdentifier":
writeWithMap(state, key.name, CAT_IDENT, key);
write(state, key.name, CAT_IDENT);
break;
case "Literal":
typeAssertIs<LiteralExtras>(key);
@@ -579,7 +581,7 @@ export function printTSIndexSignature(node: ESTree.TSIndexSignature, state: Stat
write(state, "[", CAT_OTHER);
const parameter = node.parameters[0];
writeWithMap(state, parameter.name, CAT_IDENT, parameter);
write(state, parameter.name, CAT_IDENT);
write(state, ": ", CAT_OTHER);
@@ -903,7 +905,7 @@ function printTSImportTypeQualifier(node: ESTree.TSImportTypeQualifier, state: S
write(state, ".", CAT_OTHER);
write(state, node.right.name, CAT_IDENT);
} else {
writeWithMap(state, node.name, CAT_IDENT, node);
write(state, node.name, CAT_IDENT);
}
}
@@ -949,7 +951,7 @@ export function printTSModuleDeclaration(node: TSModuleDeclarationNode, state: S
if (node.declare) write(state, "declare ", CAT_OTHER);
const kind = node.kind != null ? node.kind : node.global ? "global" : "module";
writeWithMap(state, kind, CAT_IDENT, node);
write(state, kind, CAT_IDENT);
if (kind !== "global") {
write(state, " ", CAT_OTHER);
@@ -979,7 +981,8 @@ export function printTSModuleDeclaration(node: TSModuleDeclarationNode, state: S
function printModuleBlock(body: ESTree.TSModuleBlock, state: State): void {
const statements = body.body;
if (statements.length === 0) {
writeWithMap(state, "{}", CAT_OTHER, body);
writeWithMapNoLast(state, "{", body);
writeWithMapEnd(state, "}", CAT_OTHER, body);
return;
}
@@ -990,7 +993,7 @@ function printModuleBlock(body: ESTree.TSModuleBlock, state: State): void {
state.indentLevel--;
printIndent(state);
write(state, "}", CAT_OTHER);
writeWithMapEnd(state, "}", CAT_OTHER, body);
}
/**
@@ -1009,7 +1012,7 @@ export function printTSInterfaceDeclaration(
if (node.declare) write(state, "declare ", CAT_OTHER);
writeWithMap(state, "interface ", CAT_OTHER, node);
write(state, "interface ", CAT_OTHER);
writeWithMap(state, node.id.name, CAT_IDENT, node.id);
@@ -1033,7 +1036,8 @@ export function printTSInterfaceDeclaration(
const members = node.body.body;
const { length } = members;
if (length === 0) {
writeWithMap(state, "{}", CAT_OTHER, node.body);
writeWithMapNoLast(state, "{", node.body);
writeWithMapEnd(state, "}", CAT_OTHER, node.body);
return;
}
@@ -1048,7 +1052,7 @@ export function printTSInterfaceDeclaration(
state.indentLevel--;
printIndent(state);
write(state, "}", CAT_OTHER);
writeWithMapEnd(state, "}", CAT_OTHER, node.body);
}
/**
@@ -1062,7 +1066,7 @@ export function printTSTypeAliasDeclaration(
): void {
if (node.declare) write(state, "declare ", CAT_OTHER);
writeWithMap(state, "type ", CAT_OTHER, node);
write(state, "type ", CAT_OTHER);
writeWithMap(state, node.id.name, CAT_IDENT, node.id);
@@ -1129,21 +1133,23 @@ export function printTSEnumDeclaration(node: TSEnumDeclarationNode, state: State
if (node.declare) write(state, "declare ", CAT_OTHER);
if (node.const) write(state, "const ", CAT_OTHER);
writeWithMap(state, "enum ", CAT_OTHER, node);
write(state, "enum ", CAT_OTHER);
writeWithMap(state, node.id.name, CAT_IDENT, node.id);
write(state, " ", CAT_OTHER);
typeAssertIs<TSEnumDeclarationLegacyMembers>(node);
const body = node.body != null ? node.body : node;
const members = node.body != null ? node.body.members : node.members;
const { length } = members;
if (length === 0) {
write(state, "{}", CAT_OTHER);
writeWithMapNoLast(state, "{", body);
writeWithMapEnd(state, "}", CAT_OTHER, body);
return;
}
write(state, "{\n", CAT_OTHER);
writeWithMap(state, "{\n", CAT_OTHER, body);
state.indentLevel++;
const lastIndex = length - 1;
@@ -1155,7 +1161,7 @@ export function printTSEnumDeclaration(node: TSEnumDeclarationNode, state: State
state.indentLevel--;
printIndent(state);
write(state, "}", CAT_OTHER);
writeWithMapEnd(state, "}", CAT_OTHER, body);
}
/**
@@ -1198,7 +1204,7 @@ export function printTSImportEqualsDeclaration(
node: ESTree.TSImportEqualsDeclaration,
state: State,
): void {
writeWithMap(state, "import ", CAT_OTHER, node);
write(state, "import ", CAT_OTHER);
if (node.importKind === "type") write(state, "type ", CAT_OTHER);
+252 -25
View File
@@ -3,7 +3,7 @@
// These live together because the categories exist only to be stored by `write` -
// `state.last` records what was written last by category, not the last character itself.
import { debugAssert, typeAssertIs } from "../asserts.ts";
import { debugAssert } from "../asserts.ts";
import type { MappableNode } from "./types.ts";
import type { State } from "../state.ts";
@@ -221,6 +221,7 @@ export function write(state: State, code: string, last: Category): void {
debugAssertCategoryMatches(state, code, last);
state.last = last;
updatePostfixClose(state, code);
state.output += code;
if (DEBUG) {
@@ -232,8 +233,8 @@ export function write(state: State, code: string, last: Category): void {
/**
* Append `code` to the output, record what it ends with, and record a source mapping for `node`.
*
* The mapping is only recorded where the caller asked for source maps and `node` carries a `loc` -
* Oxc's own AST records byte spans instead, so the check is on the node, not the build.
* The mapping is only recorded where the caller asked for source maps, supplied `sourceText`, and
* `node` carries Oxc `start` / `end` offsets.
*
* Builds without source map support have no use for this. For those builds, TSDown plugin rewrites every call
* into `write` and drops the `node` argument, leaving this unreferenced for the minifier to remove.
@@ -246,19 +247,10 @@ export function writeWithMap(state: State, code: string, last: Category, node: M
debugAssert(code.length > 0, "`code` should not be an empty string");
debugAssertCategoryMatches(state, code, last);
if (SOURCEMAPS && node.loc != null) {
debugAssert(
state.mapOffsets !== null && state.mapPositions !== null && state.mapNames !== null,
"Source map arrays should exist when source maps are enabled",
);
typeAssertIs<{ name?: string }>(node);
state.mapOffsets.push(state.output.length);
state.mapPositions.push(node.loc.start);
state.mapNames.push(node.name);
}
recordSourceMapping(state, node, false);
state.last = last;
updatePostfixClose(state, code);
state.output += code;
if (DEBUG) {
@@ -280,6 +272,7 @@ export function writeWithMap(state: State, code: string, last: Category, node: M
* @param code - Text to append, which unlike `write` may be empty
*/
export function writeNoLast(state: State, code: string): void {
updatePostfixClose(state, code);
state.output += code;
if (DEBUG) {
@@ -300,18 +293,9 @@ export function writeNoLast(state: State, code: string): void {
* @param node - Node this text came from
*/
export function writeWithMapNoLast(state: State, code: string, node: MappableNode): void {
if (SOURCEMAPS && node.loc != null) {
debugAssert(
state.mapOffsets !== null && state.mapPositions !== null && state.mapNames !== null,
"Source map arrays should exist when source maps are enabled",
);
typeAssertIs<{ name?: string }>(node);
state.mapOffsets.push(state.output.length);
state.mapPositions.push(node.loc.start);
state.mapNames.push(node.name);
}
recordSourceMapping(state, node, false);
updatePostfixClose(state, code);
state.output += code;
if (DEBUG) {
@@ -320,6 +304,247 @@ export function writeWithMapNoLast(state: State, code: string, node: MappableNod
}
}
/**
* Append `code`, recording a mapping for the last source character in `node` immediately before it.
*
* Rust uses this for emitted closing delimiters, including synthesized ones. The node end offset is
* exclusive, so move back by one source code point to match Rust's byte-span lookup.
*/
export function writeWithMapEnd(
state: State,
code: string,
last: Category,
node: MappableNode,
): void {
debugAssert(code.length > 0, "`code` should not be an empty string");
debugAssertCategoryMatches(state, code, last);
recordSourceMapping(state, node, 1);
state.last = last;
updatePostfixClose(state, code);
state.output += code;
if (DEBUG) {
state.lastIsStale = false;
state.lastCharWritten = code[code.length - 1];
}
}
/** Record a start mapping at the current output position without writing anything. */
export function markWithMap(state: State, node: MappableNode): void {
recordSourceMapping(state, node, false);
}
/** Record a start mapping without attaching an identifier name. */
export function markWithMapNoName(state: State, node: MappableNode): void {
recordSourceMapping(state, node, 3);
}
/** Record a mapping for `node`'s end offset at the current output position. */
export function markWithMapAfter(state: State, node: MappableNode): void {
recordSourceMapping(state, node, 2);
}
/** Record a mapping a fixed number of columns after `node`'s start offset. */
export function markWithMapAtStartOffset(
state: State,
node: MappableNode,
columnOffset: number,
): void {
if (!SOURCEMAPS) return;
const { sourceText } = state;
if (sourceText === undefined) return;
const { start, end } = node;
if (
typeof start !== "number" ||
typeof end !== "number" ||
!Number.isSafeInteger(start) ||
!Number.isSafeInteger(end) ||
start < 0 ||
end < start ||
start === end
) {
return;
}
debugAssert(
state.mapPositions !== null,
"Source map positions should exist when source maps are enabled",
);
const sourceOffset = start + columnOffset;
if (!(sourceOffset >= 0 && sourceOffset <= sourceText.length)) return;
if (state.mapPositions[state.mapPositions.length - 1] === sourceOffset) return;
state.mapPositions.push(state.output.length, sourceOffset);
}
/** Record one mapping, if source maps and a non-empty location are available. */
function recordSourceMapping(state: State, node: MappableNode, location: false | 1 | 2 | 3): void {
if (!SOURCEMAPS) return;
const { sourceText } = state;
if (sourceText === undefined) return;
const { start, end } = node;
if (
typeof start !== "number" ||
typeof end !== "number" ||
!Number.isSafeInteger(start) ||
!Number.isSafeInteger(end) ||
start < 0 ||
end < start ||
start === end
) {
return;
}
debugAssert(
state.mapPositions !== null,
"Source map positions should exist when source maps are enabled",
);
let sourceOffset: number;
if (location === 1) {
sourceOffset = end - 1;
} else if (location === 2) {
sourceOffset = end;
} else {
sourceOffset = start;
}
if (!(sourceOffset >= 0 && sourceOffset <= sourceText.length)) return;
// `oxc_codegen` suppresses consecutive source positions as it records them. Do this before
// recovering a name or retaining the mapping, since member-level marks commonly duplicate keys.
if (state.mapPositions[state.mapPositions.length - 1] === sourceOffset) return;
let name: string | undefined;
if (location === false) {
const printedName = typeof node.name === "string" ? node.name : undefined;
if (printedName !== undefined) {
// Almost every identifier is printed exactly as it appeared in the source. Avoid scanning it
// with Unicode property regexps or allocating a source substring in that common case.
const nameEnd = start + printedName.length;
const originalName =
printedName.length > 0 &&
end <= sourceText.length &&
nameEnd <= end &&
sourceText.startsWith(printedName, start) &&
(nameEnd === end || isDefinitelyIdentifierBoundary(sourceText.charCodeAt(nameEnd)))
? printedName
: originalNameFromSource(sourceText, node, start, end);
// A transformed or hand-authored AST can carry ranges unrelated to `sourceText`. Preserve
// the existing fallback in that case instead of recording an arbitrary source substring.
name =
originalName === undefined
? printedName
: originalName === printedName
? undefined
: originalName;
} else {
name = printedName;
}
}
const mappingIndex = state.mapPositions.length >> 1;
state.mapPositions.push(state.output.length, sourceOffset);
if (name !== undefined) {
(state.mapNames ??= []).push(mappingIndex, name);
}
}
/** Recover the original identifier spelling from validated source offsets. */
function originalNameFromSource(
sourceText: string,
node: MappableNode,
start: number,
end: number,
): string | undefined {
if (end > sourceText.length) return undefined;
// JSX identifiers admit `-`, which is not an ECMAScript identifier character. Their spans do
// not absorb TypeScript annotations, so the ESTree end offset is already exact.
if (node.type === "JSXIdentifier") {
const originalName = sourceText.slice(start, end);
return JSX_IDENTIFIER_REGEX.test(originalName) ? originalName : undefined;
}
let index = start;
if (index < end && sourceText.charCodeAt(index) === 35) index++; // `#` in a private identifier
const identifierStart = index;
while (index < end) {
const matcher = index === identifierStart ? IDENT_START_REGEX : IDENT_CONTINUE_REGEX;
if (sourceText.charCodeAt(index) === 92) {
const length = unicodeEscapeLength(sourceText, index, end);
if (length === 0) break;
const codePoint = unicodeEscapeCodePoint(sourceText, index, length);
if (codePoint > 0x10ffff || !matcher.test(String.fromCodePoint(codePoint))) break;
index += length;
continue;
}
const codePoint = sourceText.codePointAt(index) as number;
const char = String.fromCodePoint(codePoint);
if (!matcher.test(char)) break;
index += char.length;
}
return index === identifierStart ? undefined : sourceText.slice(start, index);
}
/** Return the UTF-16 length of a `\\u` identifier escape within `end`, or `0` if invalid. */
function unicodeEscapeLength(sourceText: string, index: number, end: number): number {
if (sourceText.charCodeAt(index + 1) !== 117) return 0; // `u`
const firstHex = index + 2;
if (sourceText.charCodeAt(firstHex) === 123) {
let cursor = firstHex + 1;
const firstDigit = cursor;
while (cursor < end && isHexDigit(sourceText.charCodeAt(cursor))) cursor++;
return cursor > firstDigit && sourceText.charCodeAt(cursor) === 125 ? cursor - index + 1 : 0;
}
const escapeEnd = firstHex + 4;
if (escapeEnd > end) return 0;
for (let cursor = firstHex; cursor < escapeEnd; cursor++) {
if (!isHexDigit(sourceText.charCodeAt(cursor))) return 0;
}
return escapeEnd - index;
}
/** Decode the code point in a syntactically valid identifier escape. */
function unicodeEscapeCodePoint(sourceText: string, index: number, length: number): number {
const braced = sourceText.charCodeAt(index + 2) === 123;
const digitsStart = index + (braced ? 3 : 2);
const digitsEnd = index + length - (braced ? 1 : 0);
return Number.parseInt(sourceText.slice(digitsStart, digitsEnd), 16);
}
/** Whether `code` is an ASCII hexadecimal digit. */
function isHexDigit(code: number): boolean {
return (code >= 48 && code <= 57) || (code >= 65 && code <= 70) || (code >= 97 && code <= 102);
}
/** Whether an ASCII character definitely cannot continue any supported identifier spelling. */
function isDefinitelyIdentifierBoundary(code: number): boolean {
return (
code <= 0x7f &&
!((code >= 48 && code <= 57) || (code >= 65 && code <= 90) || (code >= 97 && code <= 122)) &&
code !== 36 && // `$`
code !== 45 && // `-` in JSX identifiers
code !== 92 && // `\` starting a Unicode escape
code !== 95 // `_`
);
}
/** Track the final byte category Rust's postfix source-map hook checks, without reading `output`. */
function updatePostfixClose(state: State, code: string): void {
if (SOURCEMAPS && code.length > 0) {
const last = code.charCodeAt(code.length - 1);
state.lastWasPostfixClose = last === 41 || last === 93; // `)` or `]`
}
}
/**
* Assert that `state.last` still describes what was written last.
*
@@ -334,6 +559,8 @@ export function debugAssertLastFresh(state: State): void {
/** Matches a character which can continue an identifier. */
const IDENT_CONTINUE_REGEX = /[\p{ID_Continue}$\u200C\u200D]/u;
const IDENT_START_REGEX = /[\p{ID_Start}$_]/u;
const JSX_IDENTIFIER_REGEX = /^[\p{ID_Start}$_](?:[\p{ID_Continue}$-]|\u200C|\u200D)*$/u;
/**
* Assert that `last` truthfully describes the end of `code`.
+17 -11
View File
@@ -11,7 +11,7 @@
import { CAT_OTHER } from "./print/write.ts";
import type { Category } from "./print/write.ts";
import type { Options, Position } from "./print/options.ts";
import type { Options } from "./print/options.ts";
/**
* The string one level of indentation prints as. Set from the `indent` option, a tab by default.
@@ -67,6 +67,10 @@ export class State {
// means for spacing.
declare last: Category;
// Whether the last write ended in `)` or `]`. Only maps builds read this, to mirror Rust's
// trailing mapping for a postfix operand before chained punctuation.
declare lastWasPostfixClose: boolean;
// `true` between a `writeNoLast` and the write which follows it,
// i.e. while `last` describes something other than what was written last.
// Only used in debug builds. See `debugAssertLastFresh`.
@@ -76,11 +80,13 @@ export class State {
// Only used in debug builds. See `debugAssertCategoryMatches`.
declare lastCharWritten: string;
// Deferred source mappings.
// Either all 3 are arrays, or all 3 are `null` (when the printer was given no `sourceMap`).
declare mapOffsets: number[] | null;
declare mapPositions: Position[] | null;
declare mapNames: (string | undefined)[] | null;
// Deferred source mappings. Generated/source offset pairs exist when source maps are enabled.
// Names are sparse, so their index/name pairs exist only if a mapping carries an original name.
declare mapPositions: number[] | null;
declare mapNames: (number | string)[] | null;
// Original source text, used to preserve names in source maps when the caller provides it.
declare sourceText: string | undefined;
constructor(options: Options) {
this.output = "";
@@ -126,6 +132,8 @@ export class State {
// directly would flatten V8's rope representation on every append-then-read (quadratic),
// which is why the category is tracked rather than derived.
this.last = CAT_OTHER;
this.lastWasPostfixClose = false;
this.sourceText = options.sourceText;
// Debug-only fields for checking `last` is correct on both writes and reads
if (DEBUG) {
@@ -134,15 +142,13 @@ export class State {
}
// `writeWithMap` records the output offset and original position of every mapped node,
// and `emitMappings` converts the offsets to generated line/column in one pass at the end
if (options.sourceMap == null) {
this.mapOffsets = null;
// and `generateSourceMap` encodes them in one pass at the end.
if (options.sourcemap !== true) {
this.mapPositions = null;
this.mapNames = null;
} else {
this.mapOffsets = [];
this.mapPositions = [];
this.mapNames = [];
this.mapNames = null;
}
}
}
+220 -101
View File
@@ -1,11 +1,11 @@
// Source map tests.
//
// Source maps are the one part of the printer whose output is not the printed code, so the
// conformance suites say nothing about them. These check the mappings a print emits are
// self-consistent and point where they claim to, and that turning source maps on does not change
// the code that comes out.
// The conformance checker compares the complete Source Map v3 object against Rust `oxc_codegen`.
// The remaining tests here check invariants which give a more local failure than a full mapping
// diff: positions stay in bounds and ordered, indentation is respected, identifier text agrees,
// and turning source maps on does not change the generated code.
//
// `oxc-parser` emits no `loc`, so it is synthesized here from `start` / `end` offsets.
// Mappings use Oxc `start` / `end` offsets.
import { existsSync, readFileSync } from "node:fs";
import { join as pathJoin } from "node:path";
@@ -13,9 +13,10 @@ import { parseSync } from "oxc-parser";
import { beforeAll, describe, expect, test } from "vitest";
import { printSync } from "../dist/index.js";
import { checkFixture, getEcmaScriptLineTable } from "./utils/common.ts";
import type { Program } from "oxc-parser";
import type { Mapping, Position, SourceMapGenerator } from "../dist/index.js";
import type { SourceMap } from "../dist/index.js";
// Same directory the benchmarks download their fixtures to. Whichever are already cached are used;
// the inline fixtures below always run.
@@ -73,6 +74,133 @@ namespace NS {
export default NS;
`;
const INLINE_UNICODE = 'const smile = "😀";\r\nconst café = `first\u2028second`;\nsmile + café;';
describe("Rust conformance", () => {
test.each([
{ name: "inline.js", code: INLINE_JS, lang: "js" as const },
{ name: "inline.ts", code: INLINE_TS, lang: "ts" as const },
{
name: "unicode.js",
code: INLINE_UNICODE,
lang: "js" as const,
},
{
name: "escaped-name.js",
code: "const \\u0061 = 1; \\u0061;",
lang: "js" as const,
},
{ name: "astral-new.js", code: "new 𐐀;", lang: "js" as const },
{
name: "large-line-gap.js",
code: `\`${"\n".repeat(200)}\`;\nvalue;`,
lang: "js" as const,
},
{
name: "sparse-uncommon-line.js",
code: `value;${" ".repeat(20000)}\u2028`,
lang: "js" as const,
},
{
name: "many-crlf.js",
code: Array.from({ length: 100 }, (_, index) => `x${index};`).join("\r\n"),
lang: "js" as const,
},
])("$name mappings match oxc_codegen", ({ name, code, lang }) => {
expect(checkFixture(name, code, lang, "module")).toBe(true);
});
test("invalid source offsets fall back to the printed name", () => {
const code = "const name = 0;";
const program = parseProgram("invalid-offset.js", code);
const statement = program.body[0];
if (statement.type !== "VariableDeclaration") throw new Error("Expected variable declaration");
const identifier = statement.declarations[0].id;
if (identifier.type !== "Identifier") throw new Error("Expected identifier");
// Simulate a transformed AST whose old offsets are no longer valid for the source text.
identifier.end = code.length + 1;
const { map } = printSync(program, {
sourcemap: true,
sourceFileName: "invalid-offset.js",
sourceText: code,
});
expect(decodeSourceMap(map!)).toContainEqual(expect.objectContaining({ name: "name" }));
});
test("in-bounds non-identifier offsets fall back to the printed name", () => {
const code = "const name = 0;";
const program = parseProgram("stale-offset.js", code);
const statement = program.body[0];
if (statement.type !== "VariableDeclaration") throw new Error("Expected variable declaration");
const identifier = statement.declarations[0].id;
if (identifier.type !== "Identifier") throw new Error("Expected identifier");
// Simulate a transformed AST whose stale offsets are in bounds but start on whitespace.
identifier.start = code.indexOf(" ");
identifier.end = identifier.start + 1;
const { map } = printSync(program, {
sourcemap: true,
sourceFileName: "stale-offset.js",
sourceText: code,
});
const mappings = decodeSourceMap(map!);
expect(mappings).toContainEqual(expect.objectContaining({ name: "name" }));
expect(mappings).not.toContainEqual(expect.objectContaining({ name: "" }));
});
test("stale identifier offsets preserve the original longer spelling", () => {
const code = "const ab = 0;";
const program = parseProgram("renamed.js", code);
const statement = program.body[0];
if (statement.type !== "VariableDeclaration") throw new Error("Expected variable declaration");
const identifier = statement.declarations[0].id;
if (identifier.type !== "Identifier") throw new Error("Expected identifier");
identifier.name = "a";
const { map } = printSync(program, {
sourcemap: true,
sourceFileName: "renamed.js",
sourceText: code,
});
expect(map?.names).toEqual(["ab"]);
});
test("handles transformed ASTs whose source locations move backwards", () => {
const code = `const first = 1;\n${"\n".repeat(5000)}const second = 2;`;
const program = parseProgram("reordered.js", code);
program.body.reverse();
const { map } = printSync(program, {
sourcemap: true,
sourceFileName: "reordered.js",
sourceText: code,
});
const mappings = decodeSourceMap(map!);
expect(mappings.find((mapping) => mapping.generatedLine === 1)?.originalLine).toBe(5002);
expect(mappings.find((mapping) => mapping.generatedLine === 2)?.originalLine).toBe(1);
});
test("returns a source map only when requested", () => {
const code = "const value = 1;";
const program = parseProgram("return-map.js", code);
expect(printSync(program).map).toBeUndefined();
const { map } = printSync(program, {
sourcemap: true,
sourceFileName: "return-map.js",
sourceText: code,
});
expect(map).toMatchObject({
version: 3,
sources: ["return-map.js"],
sourcesContent: [code],
});
});
});
interface Fixture {
name: string;
/** Source text, or `null` if the fixture is a cached file which has not been downloaded */
@@ -95,6 +223,7 @@ function cached(name: string): string | null {
const FIXTURES: Fixture[] = [
{ name: "inline.js", code: INLINE_JS, ts: false, jsx: false },
{ name: "inline.ts", code: INLINE_TS, ts: true, jsx: false },
{ name: "unicode.js", code: INLINE_UNICODE, ts: false, jsx: false },
{ name: "react.development.js", code: cached("react.development.js"), ts: false, jsx: false },
{
name: "RadixUIAdoptionSection.jsx",
@@ -108,60 +237,7 @@ const FIXTURES: Fixture[] = [
// --- Helpers --------------------------------------------------------------------------------
/**
* Build a function converting a source offset to a line and column.
*
* @param source - Source text
* @returns Function taking an offset and returning a 1-based line and 0-based column
*/
function lineColTable(source: string): (offset: number) => Position {
const lineStarts = [0];
for (let i = 0; i < source.length; i++) {
if (source.charCodeAt(i) === 10) lineStarts.push(i + 1);
}
return (offset) => {
let lo = 0,
hi = lineStarts.length - 1;
while (lo < hi) {
const mid = (lo + hi + 1) >> 1;
if (lineStarts[mid] <= offset) lo = mid;
else hi = mid - 1;
}
return { line: lo + 1, column: offset - lineStarts[lo] };
};
}
/**
* Add a `loc` to every node in the AST, computed from its `start` / `end` offsets.
*
* @param root - AST to walk
* @param posOf - Function converting an offset to a line and column
*/
function addLocs(root: object, posOf: (offset: number) => Position): void {
const seen = new Set<object>();
const stack: unknown[] = [root];
while (stack.length > 0) {
const node = stack.pop();
if (node === null || typeof node !== "object") continue;
if (seen.has(node)) continue;
seen.add(node);
if (Array.isArray(node)) {
for (const child of node) stack.push(child);
continue;
}
const record = node as Record<string, unknown>;
if (typeof record.type === "string" && typeof record.start === "number") {
record.loc = { start: posOf(record.start), end: posOf(record.end as number) };
}
for (const key in record) {
if (key === "loc" || key === "parent") continue;
const child = record[key];
if (child !== null && typeof child === "object") stack.push(child);
}
}
}
/** A mapping, copied out of the `Mapping` object the printer reuses across calls. */
/** One decoded mapping used by the local invariant checks. */
interface Recorded {
generatedLine: number;
generatedColumn: number;
@@ -171,39 +247,76 @@ interface Recorded {
source: string;
}
/** Collects the mappings a print emits. This is the `sourceMap` option's whole interface. */
class Collector implements SourceMapGenerator {
file: string;
mappings: Recorded[] = [];
/** Decode a Source Map v3 mapping string for the local invariant checks below. */
function decodeSourceMap(map: SourceMap): Recorded[] {
const mappings: Recorded[] = [];
let sourceId = 0;
let originalLine = 0;
let originalColumn = 0;
let nameId = 0;
constructor(file: string) {
this.file = file;
}
const lines = map.mappings.split(";");
for (let generatedLine = 0; generatedLine < lines.length; generatedLine++) {
const line = lines[generatedLine];
if (line === "") continue;
addMapping(mapping: Mapping): void {
// `Mapping` objects are reused across calls - must copy
this.mappings.push({
generatedLine: mapping.generated.line,
generatedColumn: mapping.generated.column,
originalLine: mapping.original.line,
originalColumn: mapping.original.column,
name: mapping.name,
source: mapping.source,
});
let generatedColumn = 0;
for (const segment of line.split(",")) {
const values = decodeVlqSegment(segment);
generatedColumn += values[0];
sourceId += values[1];
originalLine += values[2];
originalColumn += values[3];
if (values.length === 5) nameId += values[4];
const source = map.sources[sourceId];
if (source === undefined) throw new Error(`Invalid source ID ${sourceId}`);
mappings.push({
generatedLine: generatedLine + 1,
generatedColumn,
originalLine: originalLine + 1,
originalColumn,
name: values.length === 5 ? map.names[nameId] : undefined,
source,
});
}
}
return mappings;
}
/** Decode one comma-delimited source-map segment. */
function decodeVlqSegment(segment: string): number[] {
const values: number[] = [];
let value = 0;
let factor = 1;
for (const char of segment) {
const digit = BASE64_CHARS.indexOf(char);
if (digit === -1) throw new Error(`Invalid base64 VLQ digit ${JSON.stringify(char)}`);
value += (digit % 32) * factor;
if (digit < 32) {
values.push(value % 2 === 1 ? -Math.floor(value / 2) : Math.floor(value / 2));
value = 0;
factor = 1;
} else {
factor *= 32;
}
}
if (factor !== 1) throw new Error("Unterminated base64 VLQ value");
return values;
}
const BASE64_CHARS = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
/**
* Parse a fixture and add a `loc` to every node.
* Parse a fixture.
*
* @param name - Fixture filename
* @param code - Source text
* @returns The AST
*/
function parseWithLocs(name: string, code: string): Program {
function parseProgram(name: string, code: string): Program {
const { program, errors } = parseSync(name, code, PARSE_OPTIONS);
if (errors.length > 0) throw new Error(`parse ${name}: ${errors[0].message}`);
addLocs(program, lineColTable(code));
return program;
}
@@ -229,21 +342,26 @@ for (const fixture of FIXTURES) {
beforeAll(() => {
const code = fixture.code as string;
const program = parseWithLocs(fixture.name, code);
const collector = new Collector(fixture.name);
const program = parseProgram(fixture.name, code);
const { jsx, ts } = fixture;
// Both builds through the public API - `printSync` picks the maps build when given a
// `sourceMap`, and the no-maps build when not
const withMaps = printSync(program, { jsx, ts, sourceMap: collector }).code;
const withoutMaps = printSync(program, { jsx, ts }).code;
// Both builds through the public API - `printSync` picks the maps build when `sourcemap`
// is true, and the no-maps build when it is not.
const { code: withMaps, map } = printSync(program, {
jsx,
ts,
sourcemap: true,
sourceFileName: fixture.name,
sourceText: code,
});
const { code: withoutMaps } = printSync(program, { jsx, ts });
printed = {
withMaps,
withoutMaps,
mappings: collector.mappings,
outLines: withMaps.split("\n"),
srcLines: code.split("\n"),
mappings: decodeSourceMap(map!),
outLines: getEcmaScriptLineTable(withMaps).lines,
srcLines: getEcmaScriptLineTable(code).lines,
};
});
@@ -296,9 +414,7 @@ for (const fixture of FIXTURES) {
});
test("every mapping names the source file", () => {
const wrong = printed.mappings.find(
(mapping) => mapping.source !== fixture.name && mapping.source !== undefined,
);
const wrong = printed.mappings.find((mapping) => mapping.source !== fixture.name);
expect(wrong).toBeUndefined();
});
@@ -337,15 +453,16 @@ describe("indent option", () => {
test.each(
INDENT_CASES.map((indent) => ({ label: JSON.stringify(indent) ?? "undefined", indent })),
)("indent=$label", ({ indent }) => {
const program = parseWithLocs("indent.js", INLINE_JS);
const collector = new Collector("indent.js");
const out = printSync(program, {
indent,
sourceMap: collector,
}).code;
const program = parseProgram("indent.js", INLINE_JS);
const { code: out, map } = printSync(program, {
indent: indent as string | undefined,
sourcemap: true,
sourceFileName: "indent.js",
sourceText: INLINE_JS,
});
const expectedIndent = indent ?? "\t";
const lines = out.split("\n");
const expectedIndent = typeof indent === "string" && /^[ \t]+$/.test(indent) ? indent : "\t";
const { lines } = getEcmaScriptLineTable(out);
const indented = lines.filter((line) => line.startsWith("\t") || line.startsWith(" "));
expect(indented.length).toBeGreaterThan(0);
@@ -357,8 +474,10 @@ describe("indent option", () => {
}
// No mapping points into an indent run
const mappings = decodeSourceMap(map!);
expect(mappings.length).toBeGreaterThan(0);
let insideIndent = null;
for (const mapping of collector.mappings) {
for (const mapping of mappings) {
const lead = /^[\t ]*/.exec(lines[mapping.generatedLine - 1])![0].length;
if (mapping.generatedColumn !== 0 && mapping.generatedColumn < lead) {
insideIndent = `${mapping.generatedLine}:${mapping.generatedColumn}`;
+47 -6
View File
@@ -1,14 +1,15 @@
// Shared machinery for the conformance tests.
//
// Each test takes one fixture's source text and prints it twice:
// Each test takes one fixture's source text and prints it three times:
//
// 1. In Rust, with `oxc_parser` + `oxc_codegen`, via the `oxc_codegen_conformance` Node addon.
// 2. In JS, by parsing with `oxc-parser` (the copy in this repo) and printing with this package.
// 2. In JS without source maps, by parsing with `oxc-parser` and printing with this package.
// 3. In JS with source maps, through the separately compiled maps-enabled build.
//
// The two must agree byte for byte. Both sides are given the same source text and the same `lang`
// Each must agree byte for byte. Both sides are given the same source text and the same `lang`
// and `sourceType`, and the addon derives its `SourceType` with `oxc_napi::get_source_type` -
// the same function `oxc-parser` uses. So the two printers are handed the same AST and the only
// thing under test is the printing.
// things under test are printing and decoded source map positions and names.
//
// Fixtures which do not parse cleanly have no AST to print, so the addon returns `null` for them
// and the test is reported as skipped rather than passing quietly.
@@ -44,6 +45,31 @@ export type SourceTypeOption = "script" | "module" | "unambiguous";
// so both are worth checking.
const PRESERVE_PARENS_MODES = [false, true];
/**
* Split source text on every ECMAScript line terminator and retain each line's UTF-16 start offset.
*/
export function getEcmaScriptLineTable(sourceText: string): {
lines: string[];
lineStarts: number[];
} {
const lines: string[] = [];
const lineStarts = [0];
let lineStart = 0;
for (let index = 0; index < sourceText.length; index++) {
const char = sourceText.charCodeAt(index);
if (char !== 10 && char !== 13 && char !== 0x2028 && char !== 0x2029) continue;
lines.push(sourceText.slice(lineStart, index));
if (char === 13 && sourceText.charCodeAt(index + 1) === 10) index++;
lineStart = index + 1;
lineStarts.push(lineStart);
}
lines.push(sourceText.slice(lineStart));
return { lines, lineStarts };
}
/**
* Check this package prints a fixture exactly as Rust `oxc_codegen` does.
*
@@ -97,8 +123,23 @@ export function checkFixture(
// rather than a printer one, and saying so is more use than a diff of printed output.
expect(errors, "Rust parsed this fixture cleanly but `oxc-parser` did not").toEqual([]);
const actual = printSync(program, { ts, jsx }).code;
expect(actual, `preserveParens: ${preserveParens}`).toBe(expected);
const { code: actual } = printSync(program, { ts, jsx });
expect(actual, `preserveParens: ${preserveParens}`).toBe(expected.code);
// Source maps use the maps-enabled build, which is compiled separately from the normal
// printer. Source offsets are converted at the end. Compare both its code and complete
// Source Map v3 output against Rust.
const { code: actualWithSourceMap, map } = printSync(program, {
ts,
jsx,
sourcemap: true,
sourceFileName: filename,
sourceText,
});
expect(actualWithSourceMap, `preserveParens: ${preserveParens}, sourceMap: code`).toBe(
expected.code,
);
expect(map, `preserveParens: ${preserveParens}, sourceMap`).toEqual(expected.map);
checked = true;
}
@@ -13,7 +13,7 @@ import type { Plugin } from "rolldown";
* write(state, "declare ", CAT_OTHER);
* ```
*
* `writeWithMap` and `writeWithMapNoLast` exist to record a source mapping for the node they are given.
* The mapped writes and `markWithMap` exist to record a source mapping for the node they are given.
* A build without source map support has nothing to record, so the call becomes the plain write
* it would otherwise be, and the node argument goes with it - it would still be evaluated,
* and held live across the call, for a function which ignores it.
@@ -31,6 +31,12 @@ const REWRITES = {
// `write` takes the `last` category between the code and the node, `writeNoLast` does not
writeWithMap: { arity: 4, plain: "write" },
writeWithMapNoLast: { arity: 3, plain: "writeNoLast" },
writeWithMapEnd: { arity: 4, plain: "write" },
// A standalone mark has no non-sourcemap equivalent. `void 0` is removed by the minifier.
markWithMap: { arity: 2, plain: null },
markWithMapNoName: { arity: 2, plain: null },
markWithMapAfter: { arity: 2, plain: null },
markWithMapAtStartOffset: { arity: 3, plain: null },
} as const;
const WRITE_MODULE = "./write.ts";
@@ -52,6 +58,7 @@ const plugin: Plugin = {
// Plain names this file will need in scope once its calls are rewritten
const needed = new Set<string>();
let rewrote = false;
new Visitor({
CallExpression(node) {
@@ -67,14 +74,19 @@ const plugin: Plugin = {
);
}
magicString.overwrite(callee.start, callee.end, plain);
// Remove `, node`, from the end of the argument before it to the end of it
magicString.remove(args[arity - 2].end, args[arity - 1].end);
needed.add(plain);
rewrote = true;
if (plain === null) {
magicString.overwrite(node.start, node.end, "void 0");
} else {
magicString.overwrite(callee.start, callee.end, plain);
// Remove `, node`, from the end of the argument before it to the end of it
magicString.remove(args[arity - 2].end, args[arity - 1].end);
needed.add(plain);
}
},
}).visit(program);
if (needed.size !== 0) {
if (rewrote) {
// Rewrite the import to drop the mapped names and carry whatever the rewrites now need.
// `write.ts` declares them itself and imports nothing, so it never reaches this.
// A file may import from `write.ts` more than once - the types separately from the values.
+2 -6
View File
@@ -23,9 +23,10 @@ doctest = false
oxc_allocator = { workspace = true }
oxc_ast = { workspace = true }
oxc_ast_visit = { workspace = true }
oxc_codegen = { workspace = true }
oxc_codegen = { workspace = true, features = ["sourcemap"] }
oxc_napi = { workspace = true }
oxc_parser = { workspace = true }
oxc_sourcemap = { workspace = true, features = ["napi"] }
oxc_syntax = { workspace = true }
napi = { workspace = true }
@@ -33,8 +34,3 @@ napi-derive = { workspace = true }
[build-dependencies]
napi-build = { workspace = true }
# `napi` is not named in the source - the `#[napi]` macro expands to code which uses it, which
# `cargo-shear` cannot see through.
[package.metadata.cargo-shear]
ignored = ["napi"]
+21 -2
View File
@@ -1,7 +1,8 @@
/* auto-generated by NAPI-RS */
/* eslint-disable */
/**
* Parse `source_text` with `oxc_parser` and print the AST with `oxc_codegen`.
* Parse `source_text` with `oxc_parser` and print the AST with `oxc_codegen`, including a source
* map.
*
* Returns `null` if the source text could not be parsed without errors, which tells the caller
* there is nothing to compare for this fixture.
@@ -13,7 +14,15 @@
*
* Returns any panics as errors to avoid aborting the whole Node process.
*/
export declare function codegen(filename: string, sourceText: string, options?: Options | undefined | null): string | null
export declare function codegen(filename: string, sourceText: string, options?: Options | undefined | null): CodegenResult | null
/** Generated code and its source mappings. */
export interface CodegenResult {
/** Generated source text. */
code: string
/** Source map generated by `oxc_codegen`. */
map: SourceMap
}
/**
* Options describing how a fixture should be parsed.
@@ -58,3 +67,13 @@ export declare const enum Severity {
Warning = 'Warning',
Advice = 'Advice'
}
export interface SourceMap {
file?: string
mappings: string
names: Array<string>
sourceRoot?: string
sources: Array<string>
sourcesContent?: Array<string>
version: number
x_google_ignoreList?: Array<number>
}
+36 -11
View File
@@ -2,14 +2,18 @@
//!
//! This is the reference the `oxc-codegen` package's conformance tests are checked against.
//! For each fixture, the tests parse the source twice - once here in Rust, and once with `oxc-parser`
//! on the JS side - and require both printers to produce the same output.
//! on the JS side - and require both printers to produce the same output and decoded source map
//! positions.
//!
//! Both sides are given the same source text and the same `SourceType`, via [`oxc_napi::get_source_type`],
//! which is the same function `oxc-parser` itself uses to turn a filename and `lang` / `sourceType` options
//! into a `SourceType`. So the ASTs the two printers are handed are the same AST, and the only thing
//! under test is the printing.
//! into a `SourceType`. So the ASTs the two printers are handed are the same AST, and the only things
//! under test are printing and source map generation.
use std::panic::{AssertUnwindSafe, catch_unwind};
use std::{
panic::{AssertUnwindSafe, catch_unwind},
path::PathBuf,
};
use napi_derive::napi;
@@ -22,6 +26,7 @@ use oxc_ast_visit::{VisitMut, walk_mut};
use oxc_codegen::{Codegen, CodegenOptions, CommentOptions};
use oxc_napi::get_source_type;
use oxc_parser::{ParseOptions, Parser};
use oxc_sourcemap::napi::SourceMap;
use oxc_syntax::scope::ScopeFlags;
/// Options describing how a fixture should be parsed.
@@ -39,7 +44,17 @@ pub struct Options {
pub preserve_parens: Option<bool>,
}
/// Parse `source_text` with `oxc_parser` and print the AST with `oxc_codegen`.
/// Generated code and its source mappings.
#[napi(object)]
pub struct CodegenResult {
/// Generated source text.
pub code: String,
/// Source map generated by `oxc_codegen`.
pub map: SourceMap,
}
/// Parse `source_text` with `oxc_parser` and print the AST with `oxc_codegen`, including a source
/// map.
///
/// Returns `null` if the source text could not be parsed without errors, which tells the caller
/// there is nothing to compare for this fixture.
@@ -57,7 +72,7 @@ pub fn codegen(
filename: String,
source_text: String,
options: Option<Options>,
) -> napi::Result<Option<String>> {
) -> napi::Result<Option<CodegenResult>> {
let options =
options.unwrap_or(Options { lang: None, source_type: None, preserve_parens: None });
@@ -82,7 +97,7 @@ fn codegen_impl(
lang: Option<&str>,
source_type_option: Option<&str>,
preserve_parens: bool,
) -> Option<String> {
) -> Option<CodegenResult> {
let source_type = get_source_type(filename, lang, source_type_option);
let allocator = Allocator::default();
@@ -105,9 +120,15 @@ fn codegen_impl(
let mut program = ret.program;
Normalize { preserve_parens }.visit_program(&mut program);
let codegen_options =
CodegenOptions { comments: CommentOptions::disabled(), ..CodegenOptions::default() };
Some(Codegen::new().with_options(codegen_options).build(&program).code)
let codegen_options = CodegenOptions {
comments: CommentOptions::disabled(),
source_map_path: Some(PathBuf::from(filename)),
..CodegenOptions::default()
};
let ret = Codegen::new().with_options(codegen_options).build(&program);
let map = ret.map.expect("source map generation was enabled").into();
Some(CodegenResult { code: ret.code, map })
}
/// Erases the parts of Oxc's AST which its own ESTree representation cannot carry.
@@ -180,7 +201,11 @@ fn normalize_with_clause(
if let Some(inner) = with_clause {
inner.keyword = WithClauseKeyword::With;
if inner.with_entries.is_empty() {
if let Some(first) = inner.with_entries.first() {
// ESTree exposes the entries but not the `WithClause` wrapper or its `{` span. Use the
// first location both representations can carry as the clause's mapping anchor.
inner.span.start = first.span.start;
} else {
*with_clause = None;
}
}