Infrastructure and build tooling upgrade (#537)
- Build lib/mappings.wasm with wasm-pack and wasm-bindgen
The previous build pipeline could not be run at all on a current Rust toolchain. Three of its four tools failed outright:
- cargo build:
undefined symbol: mapping_callback, because the import blocks lacked#[link(wasm_import_module = "env")]and modern rust-lld rejects undefined symbols- wasm-gc 0.1.6:
InvalidTableReference(128)- wasm-nm (which drives wasm-snip):
Not an unsigned 1-bit integer: 128Only wasm-opt still worked, so the committed binary was not reproducible from its own sources. CI never noticed because the
wasmjob only ran a barecargo buildand never exercised build.py.Replace build.py and who-calls.py with build.sh, which drives wasm-pack. wasm-pack vendors its own wasm-bindgen and wasm-opt, reducing the four hand-installed binaries to one. Two targets are built, selected by the existing package.json#browser field:
nodejsfor CommonJS andwebfor browsers. Both emit the same .wasm; build.sh asserts this. The shipped filename stays lib/mappings.wasm so URLs passed to SourceMapConsumer.initialize keep working.Port the ABI to wasm-bindgen. allocate_mappings, parse_mappings, free_mappings and get_last_error collapse into parseMappings() plus a generated free(), so JS no longer hand-manages a buffer or memcpys into exports.memory.buffer. Callbacks are passed per-call, which removes the callbackStack and withMappingCallback machinery. Mapping construction stays in JS to keep every mapping on one hidden class.
Note that a JS exception thrown from a mapping callback unwinds through the wasm frame and skips Rust’s borrow guard, leaving the handle permanently borrowed so that free() then fails. The callback boundary is therefore declared
catchand every query method returns Result<(), JsValue>, releasing the borrow before the exception propagates. Any future &mut self method that calls into JS needs the same treatment.Also drop the unused
randdependency, turn offdebugin [profile.release] (it was responsible for 1.6 MB of debuginfo in a shipped artifact), and set opt-level=”z”, lto, codegen-units=1 and panic=”abort”. lib/mappings.wasm goes from 48,526 to 28,556 bytes.Commit both Cargo.lock files, pin the toolchain in rust-toolchain.toml, and add a
wasm-distCI job that rebuilds the committed artifacts and fails if they differ byte-for-byte. Verified that the job exits non-zero when the Rust sources drift.Bump eslint’s ecmaVersion to 2020 for the dynamic import() in read-wasm-browser.js, and keep the generated glue out of eslint –fix and prettier so neither rewrites it and breaks the byte comparison.
- Strip import.meta.url from the generated browser glue
import.metais a hard parse error in webpack 4 and other pre-2020 bundlers:ERROR in ./lib/mappings-glue-browser.mjs 336:56 Module parse failed: Unexpected token (336:56) > module_or_path = new URL('mappings.wasm', import.meta.url);This broke
npm run buildin bench/, and would equally break any consumer of this package that bundles with webpack 4.The branch it appears in is unreachable here: read-wasm-browser.js validates its input and always passes the URL or bytes explicitly, so
module_or_pathis never undefined. Replace it at build time with a throw carrying the same message read-wasm-browser.js uses, and extend build.sh’s post-build guard to assert noimport.metasurvives.
- Drop the dead ‘url’ core module fallback in lib/url.js
lib/url.jsfell back torequire("url").URLwhenURLwas not a global, which was only the case on Node 8. This package requires Node= 12 and
URLhas been global since Node 10, so the fallback was unreachable.It was not harmless, though: bundlers that no longer polyfill Node core modules fail outright on it. Under webpack 5 this breaks
npm run buildin bench/ with:Module not found: Error: Can't resolve 'url' in '.../lib' BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.The comment above it also claimed the file was overridden via
package.json#browserto uselib/url-browser.js. That stopped being true in b95ecf0, which removedlib/url-browser.jswithout removing the browser mapping’s last reader or this comment.
- Upgrade the benchmark bundler to webpack 5
webpack 4 could not parse the
import.meta.urlthat wasm-bindgen emits into the browser glue. That specific blocker is gone now that build.sh strips it, but webpack 4 is six years old and its webpack-cli 3 pairing is long unmaintained, so move the benchmark to webpack 5.The upgrade surfaced one real bug in the library, fixed separately in 64191de: webpack 5 no longer polyfills Node core modules, which exposed a dead
require("url")fallback in lib/url.js.Commit a lockfile as well. The benchmark had none, so
npm installhere previously resolved whatever was current, which is how the webpack 4 tree drifted far enough to break in the first place.
- Do not track a lockfile for the benchmark
Reverts the lockfile added in ba82cf4. The benchmark is a private, non-published helper and was never intended to pin its own dependency tree;
package.json‘s ranges are enough for it.
- Install wasm-pack directly instead of via jetli/wasm-pack-action
Download the official v0.15.0 release binary and verify it against a pinned SHA-256, so the reproducibility check doesn’t depend on a third-party action.
- Require Node.js 18 or later
The wasm-pack-built lib/mappings.wasm uses WebAssembly multi-value and reference types, which Node.js 12, 14 and 16 cannot load. Bump engines.node to >= 18 and test on Node 18, 20 and 22 in CI.
- Run the Rust CI matrix on its intended toolchains
wasm-mappings/rust-toolchain.toml pins stable 1.98.1, and rustup prefers it over the directory override actions-rs sets on the repo root. As a result every matrix job ran on 1.98.1, and the nightly-only benches failed with “#![feature] may not be used on the stable release channel”.
Set RUSTUP_TOOLCHAIN from the matrix in CI, default the bench job to nightly when run locally, and correct the toolchain file’s comment.
- Pin binaryen 132 for the reproducible wasm build
wasm-pack uses a wasm-opt from PATH when there is one and otherwise downloads its own (binaryen 117 for wasm-pack 0.15.0). The committed lib/mappings.wasm was optimized with binaryen 132, so the wasm-dist CI job, which has no system wasm-opt, produced different bytes and failed.
Make build.sh require binaryen 132 on PATH, install that release in CI (verified against a pinned SHA-256), and document the requirement in CONTRIBUTING.md.
- Run ci/script.sh from wasm-mappings/ regardless of the caller’s cwd
The script relied on CI’s working-directory default. Run from the repo root, the wasm-dist job failed with “./build.sh: No such file or directory”, and rustup did not pick up wasm-mappings/rust-toolchain.toml.
- Move source-map-mappings to Rust edition 2024
Set
editionin [package] (under [profile.release] Cargo ignored it and the crate stayed on 2015), name the anonymous parameters in ComparatorFunction::compare, and usedyn Iteratorin the quickcheck tests. lib/mappings.wasm is byte-for-byte unchanged.
- Fix cargo warnings in the wasm-mappings crates
- Drop the deprecated std::u32 / std::i64 module imports.
- Assign instead of discarding a mem::replace result.
- Store the never-read observer as PhantomData.
- Spell out elided lifetimes in two return types.
- Upgrade quickcheck 0.5 -> 1.1 (0.5 trips a future-incompat warning) and port the Arbitrary impls to the 1.x API.
- Set edition 2024 on source-map-mappings-wasm-api, which had none.
The source changes alter codegen slightly, so lib/mappings.wasm is rebuilt (28556 -> 28563 bytes) with the pinned toolchain and binaryen.
- Disable the byte comparison in the wasm-dist CI job
The job still rebuilds lib/mappings.wasm and its glue, but no longer fails when the result differs from the committed artifacts.
- Re-enable the wasm-dist byte comparison for lib/mappings.wasm
Partially undoes dd28c67: CI again fails if rebuilding from source does not reproduce the committed lib/mappings.wasm. The generated glue files remain unchecked.
- Check the generated glue files in wasm-dist again
Restores the byte comparison for lib/mappings-glue.js, .d.ts and the browser .mjs/.d.mts alongside lib/mappings.wasm, so ci/script.sh is back to how it was before dd28c67.
- Remap source paths so the wasm build is machine-independent
Panic locations embed absolute source paths in lib/mappings.wasm, which differ between machines (checkout location, $CARGO_HOME, and whether rustup’s rust-src is installed). A build on macOS was 28563 bytes while GitHub CI produced 28471, failing the wasm-dist check. Pass –remap-path-prefix for each so the paths are fixed, and rebuild lib/mappings.wasm (now 28443 bytes). trim-paths would do this but is not stable in Cargo 1.98.1.
- Update docs and comments for the Node.js 18 requirement
Bump the Node.js version in CONTRIBUTING.md and the lib/url.js comment to 18, and use .catch() in read-wasm-browser.js.
- Bring the contributing docs up to date
- Drop the
npm run build/ dist/source-map.js instructions; there is no JS build step since dist/ was removed in #362.- Refresh the sample
npm testoutput.- Pin wasm-pack 0.15.0 in the install command, matching CI.
- Document the binaryen 132 requirement in wasm-mappings/CONTRIBUTING.md, which still claimed wasm-pack was the only tool needed.
- Point wasm-pack links at wasm-bindgen.github.io; the rustwasm.github.io page is gone.
Co-authored-by: Claude Opus 5 (1M context) noreply@anthropic.com
版权所有:中国计算机学会技术支持:开源发展技术委员会
京ICP备13000930号-9
京公网安备 11010802047560号
Source Map
This is a library to generate and consume the source map format described here.
Use with Node
Use on the Web
Table of Contents
Examples
Consuming a source map
Generating a source map
In depth guide: Compiling to JavaScript, and Debugging with Source Maps
With SourceNode (high level API)
With SourceMapGenerator (low level API)
API
Get a reference to the module:
SourceMapConsumer
A
SourceMapConsumerinstance represents a parsed source map which we can query for information about the original file positions by giving it a file position in the generated source.SourceMapConsumer.initialize(options)
When using
SourceMapConsumeroutside of node.js, for example on the Web, it needs to know from what URL to loadlib/mappings.wasm. You must inform it by callinginitializebefore constructing anySourceMapConsumers.The options object has the following properties:
"lib/mappings.wasm": AStringcontaining the URL of thelib/mappings.wasmfile, or anArrayBufferwith the contents oflib/mappings.wasm.new SourceMapConsumer(rawSourceMap)
The only parameter is the raw source map (either as a string which can be
JSON.parse‘d, or an object). According to the spec, source maps have the following attributes:version: Which version of the source map spec this map is following.sources: An array of URLs to the original source files.names: An array of identifiers which can be referenced by individual mappings.sourceRoot: Optional. The URL root from which all sources are relative.sourcesContent: Optional. An array of contents of the original source files.mappings: A string of base64 VLQs which contain the actual mappings.file: Optional. The generated filename this source map is associated with.x_google_ignoreList: Optional. An additional extension field which is an array of indices refering to urls in the sources array. This is used to identify third-party sources, that the developer might want to avoid when debugging. Read moreThe promise of the constructed souce map consumer is returned.
When the
SourceMapConsumerwill no longer be used anymore, you must call itsdestroymethod.Alternatively, you can use
SourceMapConsumer.withto avoid needing to remember to calldestroy.SourceMapConsumer.with
Construct a new
SourceMapConsumerfromrawSourceMapandsourceMapUrl(see theSourceMapConsumerconstructor for details. Then, invoke theasync function f(SourceMapConsumer) -> Twith the newly constructed consumer, wait forfto complete, calldestroyon the consumer, and returnf‘s return value.You must not use the consumer after
fcompletes!By using
with, you do not have to remember to manually calldestroyon the consumer, since it will be called automatically oncefcompletes.SourceMapConsumer.prototype.destroy()
Free this source map consumer’s associated wasm data that is manually-managed.
Alternatively, you can use
SourceMapConsumer.withto avoid needing to remember to calldestroy.SourceMapConsumer.prototype.computeColumnSpans()
Compute the last column for each generated mapping. The last column is inclusive.
SourceMapConsumer.prototype.originalPositionFor(generatedPosition)
Returns the original source, line, and column information for the generated source’s line and column positions provided. The only argument is an object with the following properties:
line: The line number in the generated source. Line numbers in this library are 1-based (note that the underlying source map specification uses 0-based line numbers – this library handles the translation).column: The column number in the generated source. Column numbers in this library are 0-based.bias: EitherSourceMapConsumer.GREATEST_LOWER_BOUNDorSourceMapConsumer.LEAST_UPPER_BOUND. Specifies whether to return the closest element that is smaller than or greater than the one we are searching for, respectively, if the exact element cannot be found. Defaults toSourceMapConsumer.GREATEST_LOWER_BOUND.and an object is returned with the following properties:
source: The original source file, or null if this information is not available.line: The line number in the original source, or null if this information is not available. The line number is 1-based.column: The column number in the original source, or null if this information is not available. The column number is 0-based.name: The original identifier, or null if this information is not available.SourceMapConsumer.prototype.generatedPositionFor(originalPosition)
Returns the generated line and column information for the original source, line, and column positions provided. The only argument is an object with the following properties:
source: The filename of the original source.line: The line number in the original source. The line number is 1-based.column: The column number in the original source. The column number is 0-based.and an object is returned with the following properties:
line: The line number in the generated source, or null. The line number is 1-based.column: The column number in the generated source, or null. The column number is 0-based.SourceMapConsumer.prototype.allGeneratedPositionsFor(originalPosition)
Returns all generated line and column information for the original source, line, and column provided. If no column is provided, returns all mappings corresponding to a either the line we are searching for or the next closest line that has any mappings. Otherwise, returns all mappings corresponding to the given line and either the column we are searching for or the next closest column that has any offsets.
The only argument is an object with the following properties:
source: The filename of the original source.line: The line number in the original source. The line number is 1-based.column: Optional. The column number in the original source. The column number is 0-based.and an array of objects is returned, each with the following properties:
line: The line number in the generated source, or null. The line number is 1-based.column: The column number in the generated source, or null. The column number is 0-based.SourceMapConsumer.prototype.hasContentsOfAllSources()
Return true if we have the embedded source content for every source listed in the source map, false otherwise.
In other words, if this method returns
true, thenconsumer.sourceContentFor(s)will succeed for every sourcesinconsumer.sources.SourceMapConsumer.prototype.sourceContentFor(source[, returnNullOnMissing])
Returns the original source content for the source provided. The only argument is the URL of the original source file.
If the source content for the given source is not found, then an error is thrown. Optionally, pass
trueas the second param to havenullreturned instead.SourceMapConsumer.prototype.eachMapping(callback, context, order)
Iterate over each mapping between an original source/line/column and a generated line/column in this source map.
callback: The function that is called with each mapping. Mappings have the form{ source, generatedLine, generatedColumn, originalLine, originalColumn, name }.context: Optional. If specified, this object will be the value ofthisevery time thatcallbackis called.order: EitherSourceMapConsumer.GENERATED_ORDERorSourceMapConsumer.ORIGINAL_ORDER. Specifies whether you want to iterate over the mappings sorted by the generated file’s line/column order or the original’s source/line/column order, respectively. Defaults toSourceMapConsumer.GENERATED_ORDER.SourceMapGenerator
An instance of the SourceMapGenerator represents a source map which is being built incrementally.
new SourceMapGenerator([startOfSourceMap])
You may pass an object with the following properties:
file: The filename of the generated source that this source map is associated with.sourceRoot: A root for all relative URLs in this source map.skipValidation: Optional. Whentrue, disables validation of mappings as they are added. This can improve performance but should be used with discretion, as a last resort. Even then, one should avoid using this flag when running tests, if possible.SourceMapGenerator.fromSourceMap(sourceMapConsumer)
Creates a new
SourceMapGeneratorfrom an existingSourceMapConsumerinstance.sourceMapConsumerThe SourceMap.SourceMapGenerator.prototype.addMapping(mapping)
Add a single mapping from original source line and column to the generated source’s line and column for this source map being created. The mapping object should have the following properties:
generated: An object with the generated line and column positions.original: An object with the original line and column positions.source: The original source file (relative to the sourceRoot).name: An optional original token name for this mapping.SourceMapGenerator.prototype.setSourceContent(sourceFile, sourceContent)
Set the source content for an original source file.
sourceFilethe URL of the original source file.sourceContentthe content of the source file.SourceMapGenerator.prototype.applySourceMap(sourceMapConsumer[, sourceFile[, sourceMapPath]])
Applies a SourceMap for a source file to the SourceMap. Each mapping to the supplied source file is rewritten using the supplied SourceMap. Note: The resolution for the resulting mappings is the minimum of this map and the supplied map.
sourceMapConsumer: The SourceMap to be applied.sourceFile: Optional. The filename of the source file. If omitted, sourceMapConsumer.file will be used, if it exists. Otherwise an error will be thrown.sourceMapPath: Optional. The dirname of the path to the SourceMap to be applied. If relative, it is relative to the SourceMap.This parameter is needed when the two SourceMaps aren’t in the same directory, and the SourceMap to be applied contains relative source paths. If so, those relative source paths need to be rewritten relative to the SourceMap.
If omitted, it is assumed that both SourceMaps are in the same directory, thus not needing any rewriting. (Supplying
'.'has the same effect.)SourceMapGenerator.prototype.toString()
Renders the source map being generated to a string.
SourceNode
SourceNodes provide a way to abstract over interpolating and/or concatenating snippets of generated JavaScript source code, while maintaining the line and column information associated between those snippets and the original source code. This is useful as the final intermediate representation a compiler might use before outputting the generated JS and source map.
new SourceNode([line, column, source[, chunk[, name]]])
line: The original line number associated with this source node, or null if it isn’t associated with an original line. The line number is 1-based.column: The original column number associated with this source node, or null if it isn’t associated with an original column. The column number is 0-based.source: The original source’s filename; null if no filename is provided.chunk: Optional. Is immediately passed toSourceNode.prototype.add, see below.name: Optional. The original identifier.SourceNode.fromStringWithSourceMap(code, sourceMapConsumer[, relativePath])
Creates a SourceNode from generated code and a SourceMapConsumer.
code: The generated code.sourceMapConsumerThe SourceMap for the generated code.relativePathThe optional path that relative sources insourceMapConsumershould be relative to.SourceNode.prototype.add(chunk)
Add a chunk of generated JS to this source node.
chunk: A string snippet of generated JS code, another instance ofSourceNode, or an array where each member is one of those things.SourceNode.prototype.prepend(chunk)
Prepend a chunk of generated JS to this source node.
chunk: A string snippet of generated JS code, another instance ofSourceNode, or an array where each member is one of those things.SourceNode.prototype.setSourceContent(sourceFile, sourceContent)
Set the source content for a source file. This will be added to the
SourceMapin thesourcesContentfield.sourceFile: The filename of the source file.sourceContent: The content of the source file.SourceNode.prototype.walk(fn)
Walk over the tree of JS snippets in this node and its children. The walking function is called once for each snippet of JS and is passed that snippet and the its original associated source’s line/column location.
fn: The traversal function.SourceNode.prototype.walkSourceContents(fn)
Walk over the tree of SourceNodes. The walking function is called for each source file content and is passed the filename and source content.
fn: The traversal function.SourceNode.prototype.join(sep)
Like
Array.prototype.joinexcept for SourceNodes. Inserts the separator between each of this source node’s children.sep: The separator.SourceNode.prototype.replaceRight(pattern, replacement)
Call
String.prototype.replaceon the very right-most source snippet. Useful for trimming white space from the end of a source node, etc.pattern: The pattern to replace.replacement: The thing to replace the pattern with.SourceNode.prototype.toString()
Return the string representation of this source node. Walks over the tree and concatenates all the various snippets together to one string.
SourceNode.prototype.toStringWithSourceMap([startOfSourceMap])
Returns the string representation of this tree of source nodes, plus a SourceMapGenerator which contains all the mappings between the generated and original sources.
The arguments are the same as those to
new SourceMapGenerator.