PATCH: Fix @forge/bundler ignoring package.json "exports" field when bundling sandbox apps

Forge Forge Platform and Tools
1

Hi Forge Team, :wave:

Recently I was running into some issues where I couldn’t bundle my Forge app when deploying or tunnelling. The error I got was:

Error: Bundling failed: Module not found: Error: Can't resolve 'package-name' in '/forge-app-directory/src'

[stack trace]

After patching @forge/bundler to spit out webpack stats, there was additional debugging information from the failed resolution. Perhaps the error handling could additionally include errors.details from Webpack in @forge/bundler rather than only a stack trace from the CLI.

  "errors": [
    {
      "moduleIdentifier": "/Users/skooch/.nvm/versions/node/v18.16.1/lib/node_modules/@forge/cli/node_modules/babel-loader/lib/index.js??ruleSet[1].rules[0].use[0]!/Users/skooch/projects/forge-app-directory/src/index.js",
      "moduleName": "./src/index.js",
      "loc": "5:0-29",
      "message": "Module not found: Error: Can't resolve 'package-name' in '/Users/skooch/projects/forge-app-directory/src'",
      "moduleId": 4066,
      "moduleTrace": [],
      "details": "
resolve 'package-name' in '/Users/skooch/projects/forge-app-directory/src'
  Parsed request is a module
  using description file: /Users/skooch/projects/forge-app-directory/package.json (relative path: ./src)
    Field 'browser' doesn't contain a valid alias configuration
    resolve as module
      /Users/skooch/projects/forge-app-directory/src/node_modules doesn't exist or is not a directory
      looking for modules in /Users/skooch/projects/forge-app-directory/node_modules
        single file module
          using description file: /Users/skooch/projects/forge-app-directory/package.json (relative path: ./node_modules/package-name)
            no extension
              Field 'browser' doesn't contain a valid alias configuration
              /Users/skooch/projects/forge-app-directory/node_modules/package-name is not a file
            .ts
              Field 'browser' doesn't contain a valid alias configuration
              /Users/skooch/projects/forge-app-directory/node_modules/package-name.ts doesn't exist
            .tsx
              Field 'browser' doesn't contain a valid alias configuration
              /Users/skooch/projects/forge-app-directory/node_modules/package-name.tsx doesn't exist
            .js
              Field 'browser' doesn't contain a valid alias configuration
              /Users/skooch/projects/forge-app-directory/node_modules/package-name.js doesn't exist
            .jsx
              Field 'browser' doesn't contain a valid alias configuration
              /Users/skooch/projects/forge-app-directory/node_modules/package-name.jsx doesn't exist
            .json
              Field 'browser' doesn't contain a valid alias configuration
              /Users/skooch/projects/forge-app-directory/node_modules/package-name.json doesn't exist

Knowing that other packages were still bundling fine. I decided to review the webpack configurations generation code in the config/ directory, I discovered this line here in config/sandbox.js:L102:

webpackConfig.resolve = Object.assign(Object.assign({}, webpackConfig.resolve), { mainFields: ['main', 'module'], exportsFields: [], alias: getCustomModulesAliases() });

Clearly, it makes sense to override the resolve configuration so you can ensure your aliases apply. However while mainFields was set to the default, exportsFields was not, meaning that Webpack would ignore that field in a package.json.

I checked my installed package and sure enough, the package.json was using exports to specify the entrypoint, not main or module.

[...]
"type": "module",
"exports": "./dist/index.js",
[...]

This is not against the spec - but it is only supported in Node 12 and above. I would assume that Forge does not use any version older than that given the engines specification in most of the Forge packages is >=12.13.1.

I patched config/sandbox.js:L102 in @forge/bundler@4.10.2 to:

webpackConfig.resolve = Object.assign(Object.assign({}, webpackConfig.resolve), { mainFields: ['main', 'module'], exportsFields: ['exports'], alias: getCustomModulesAliases() });

And it all started to work just fine - with the exception of forge tunnel because I’d have to patch the dependency in the container. So I’ve also patched my dependency’s package.json to add a main field.

I’ve checked to see if @forge/bundler@4.10.3-next.15 has this issue fixed, and I couldn’t see anything but dependency updates in the changelog.

Here’s an npm why output so you can see I’m using the latest version of said package:

âžś  forge-app-directory git:(develop) âś— npm why @forge/bundler
@forge/bundler@4.10.2 dev
node_modules/@forge/cli/node_modules/@forge/bundler
  @forge/bundler@"4.10.2" from @forge/cli@6.14.1
  node_modules/@forge/cli
    dev @forge/cli@"^6.14.1" from the root project
  @forge/bundler@"4.10.2" from @forge/tunnel@3.6.2
  node_modules/@forge/cli/node_modules/@forge/tunnel
    @forge/tunnel@"3.6.2" from @forge/cli@6.14.1
    node_modules/@forge/cli
      dev @forge/cli@"^6.14.1" from the root project

I’m using Node v18.16.1 (npm v9.5.1)

I haven’t checked for any regression on this issue, so you may want to do that and backport this fix as required. I couldn’t find a repository for this package on Bitbucket or GitHub to submit a PR against, otherwise I would have done that.

Hope this helps all who may have had packaging issues while using Forge! Looking forward to getting the fix in!

2 Likes
2

Hello, and thank you for the bug report and patch!

This is unfortunately a tricky situation. Many packages use main, exports and related fields like browser in various ways when they want to provide (usually) two versions of the code, for a Node environment and for browsers.

Forge sandbox runtime is not a browser environment, but it doesn’t have all the features of a Node environment either. This is why we created a custom bundler configuration that tries to pick the most compatible versions for most third-party packages.

We have previously found that reading exports instead of main leads to Webpack pulling in code that relies on browser-only features, and the resulting Forge function crashing during snapshotting. This is why I don’t want to change the configuration from what it currently is - it might break other developers’ use cases.

Right now our team is working on a new native Node.js runtime, which is available as an EAP. Its bundler configuration is much less invasive so I encourage you to sign up and give it a try - most likely your package will work properly there.

In meanwhile I suggest patching the package with the main field. Is it a public package we can take a look at somewhere?

2 Likes
3

Thanks for responding, really appreciate the in depth explanation.

I understand that, that behaviour is taken into account under “conditional exports” as part of the “exports” standard (but I know package maintainers are another thing entirely). Modules: Packages | Node.js v21.2.0 Documentation

Wouldn’t it be good to prefer “main”, unless “main” is missing, in which case we read whatever we can get in “exports” preferring conditionals like “node” or what have you depending on the behaviour of the Forge runtime’s V8 Isolate?

I don’t see how this would significantly change the behaviour given that there’s no officially prescribed workaround, it would at least enable more packages to be used and protect from issues arising in the future since “exports” will likely continue to become the preferred entrypoint specifier.

If that is the penultimate desire though, then perhaps we could at least put something in the Forge error message to be a bit more informative?

Let me know your thoughts so I can adjust my patch.

I’m aware and I’d love to do that - but unfortunately my organisation’s needs dictate that we can’t use EAP features.

Absolutely, the package that caused me this issue was: GitHub - sindresorhus/p-queue: Promise queue with concurrency control

I’ll open a PR against it soon.

2 Likes
4

It doesn’t look like webpack supports preferring main to exports: exports fields is preferred over other package entry fields […].

It’s fairly likely that even with a preference over main exports is the right field to look at, if we can figure out how to resolve different subfields in there properly.

Still, this is a workaround for another package, and at minimum we would need to check whether it has a new version that has exports compatible with the sandbox runtime. And even then, someone else’s application would stop working unless they upgrade that package, which isn’t nice.

What we are doing now is actively getting away from these kinds of battles where a fix for one application breaks another (or 10). This is why we don’t plan changing the sandbox bundling configuration, instead focusing all our efforts on the new runtime which uses defaults as much as possible.

I’m sorry for such a disappointing answer - hope we can make the situation better on all fronts when the new runtime is in preview and GA.

P.S. You won’t be able to switch your production application to the new runtime while it’s in EAP - instead, deploy another one from the same codebase and see how well/fast that works.

2 Likes