Bundling, import.meta and url rewriting

[Jamesernator]

One of the major goals of this proposal is to have a simpler bundling target, however like existing bundling the current proposal breaks the way that import.meta.url (and potentially import.meta.resolve in future) work.

For example suppose we do this is a module today:

// https://domain.tld/path/to/lib/renderIcon.js

const iconURL = new URL("./img.png", import.meta.url);

export default function renderIcon() {
  const img = document.createElement("img");
  img.src = iconURL;
  return img;
}

// https://domain.tld/path/to/main.js
import renderIcon from "./lib/renderIcon.js";

// do something with renderIcon

With a naive bundling strategy one would generate the following module:

module "#./lib/renderIcon.js" {
  const iconURL = new URL("./img.png", import.meta.url);
  
  export default function renderIcon() {
    const img = document.createElement("img");
    img.src = iconURL;
    return img;
  }
}

import renderIcon from "#./lib/renderIcon.js";

// do something with renderIcon

Now this obviously breaks as now new URL("./img.png", import.meta.url) points to the wrong URL as import.meta.url is now within a different folder. In order to resolve this, bundlers will need to repair import.meta.url, and would have to do so basically eternally into the future.

This becomes particularly problematic when import.meta.url is used both for loading modules and for resources, for example suppose we have the module:

const resolveShader = (name: string) => new URL(`./shaders/${ name }.js`, import.meta.url);
const resolveTexture = (name: string) => new URL(`./textures/${ name }.js`, import.meta.url);

export default function createBackground(shaderName: string, textureName: string) {
  assert(shaderName.match(IDENTIFIER_NAME));
  assert(shaderName.match(IDENTIFIER_NAME));
  
  const shader = await import(resolveShader(shaderName));
  const texture = await fetch(resolveTexture(shaderName));
}

Now suppose a bundle is generated:

module "#./shaders/shader1.js" {
  // ...
}

module "#./shaders/shader2.js" {
  // ...
}

module "./createBackground.js" {
  const resolveShader = (name: string) => new URL(`./shaders/${ name }.js`, import.meta.url);
  const resolveTexture = (name: string) => new URL(`./textures/${ name }.js`, import.meta.url);

  export default function createBackground(shaderName: string, textureName: string) {
    assert(shaderName.match(IDENTIFIER_NAME));
    assert(shaderName.match(IDENTIFIER_NAME));
  
    const shader = await import(resolveShader(shaderName));
    const texture = await fetch(resolveTexture(shaderName));
  }
}

There is not really any reasonble simple transform for import.meta.url that can ensure it continues to work for resources, but also resolves internally to the module for the JS parts.

This is exacerbated with proposals like import.meta.resolve that add more complexity about what these resolutions even mean.

Similarly import rewriting makes things more complicated as now specifiers potentially need extra transforms in places like import() to function correctly.

---

Now I have an idea for a solution that I mention here based on another issue, and in fact this issue is raised in response to concerns I had from thinking about that.

In particular I'd propose that module fragments should be able to be identified with arbitrary strings rather than just #fragment names. These names would be treated as if they were canonical urls resolved in the same way as specifiers are today.

For example suppose we have the following module:

// https://domain.tld/path/to/mod.js

module "./lib/assert.js" {
  // is https://domain.tld/path/to/lib/assert.js
  console.log(import.meta.url);

  export default function assert() {
  
  }
}

import assert from "./lib/assert.js";

In this case, we would treat the fragment module as being at https://domain.tld/path/to/mod.js for the purposes of import.meta.url and friends. Similarly for any import/import() emerging from within mod.js, we treat this module fragment as being canonical for that URL (unless overriden by say an import map). This way features like import.meta.url are preserved as is, specifier resolution is simple and relatively obvious, and things like bundling are almost as simple as inlining.

Now these canonicalizations of URLs would only apply inside the module, if canonicalization of urls is desired outside an import map could be used to map them to this module as per usual.

This also means bare specifiers can be used without import maps, if they are inlined into the module. However import maps would still take precendence (although that may map it directly back which would be permissible).

[littledan]

I agree with your analysis that the fragment-based approach forces bundlers to continue to rewrite URLs of ES modules (not just for import.meta.url but for all imports, unless you use an import map entry per module, which is not the intended deployment mode for import maps). The question is whether we consider this problem bad enough to be worth the costs that would come from allowing more general specifiers.

[nayeemrmn]

It's important to note that the meta-context of a module is changed by bundling it. Thinking host-agnostically about import.meta and all of its theoretical use cases, it's an unavoidable requirement that a bundler which preserves runtime behaviour has to hardcode the value of import.meta according to the host environment's specification.

For this reason I think the goal of coincidentally preserving the value of import.meta.url is a bit misconceived. And inline module identification which works by overriding external resource specifiers has no basis except for achieving this.

[lewisl9029]

Just wanted to voice my support for @Jamesernator's idea around using arbitrary strings analogous to how import specifiers work currently.

From the perspective of someone designing a new bundler, an extremely desirable capability this enables is bundling module code as-is without any transforms while preserving most (afaik all?) semantics.

This means bundlers can just string together bytes from different modules blindly interleaved with some data in between for the declaration boilerplate, without having to modify or even inspect them. It makes bundling simple and cheap enough to be performed on-demand at request time (say by CDN nodes) without introducing any additional latency or memory/compute usage to buffer individual module contents in order to transform them.

I'd love to hear more about the specific costs and limitations people are concerned about with this approach and help think about how we can solve for them, because the capability it enables feels extremely compelling, and it'd be a real shame if we missed the opportunity to enable it here and had to wait for another round of proposals.