If there is a # in the path to the Theia app, no VS Code extensions load

[kittaakos]

Bug Description:

Steps to Reproduce:

1. Download TheiaBlueprint Version 1.32.0 (1.32.0.92)
2. Put the app into a folder so that there is a # in the path. For example, /Users/a.kitta/Desktop/with #
3. Start Theia,
4. Open a folder and create an index.js file.
5. Type something. There is no coloring. No VS Code extensions have loaded.

% ./TheiaBlueprint.app/Contents/MacOS/TheiaBlueprint
Configuration directory URI: 'file:///Users/a.kitta/.theia'
Configuring to accept webviews on '^.+\.webview\..+$' hostname.
2023-01-12T09:53:14.096Z root INFO Backend Object.initialize: 4.9 ms [Finished 0.799 s after backend start]
2023-01-12T09:53:14.096Z root INFO Backend Object.configure: 4.1 ms [Finished 0.799 s after backend start]
2023-01-12T09:53:14.096Z root INFO Backend MessagingContribution.onStart: 1.5 ms [Finished 0.799 s after backend start]
2023-01-12T09:53:14.096Z root INFO Backend OSBackendApplicationContribution.configure: 0.4 ms [Finished 0.814 s after backend start]
2023-01-12T09:53:14.096Z root INFO Backend PluginLocalizationBackendContribution.configure: 0.1 ms [Finished 0.815 s after backend start]
2023-01-12T09:53:14.097Z root INFO Backend PluginLocalizationBackendContribution.initialize: 15.4 ms [Finished 0.815 s after backend start]
2023-01-12T09:53:14.097Z root INFO Backend ElectronTokenBackendContribution.configure: 0.3 ms [Finished 0.815 s after backend start]
2023-01-12T09:53:14.098Z root INFO Backend MiniBrowserBackendSecurityWarnings.initialize: 0.1 ms [Finished 0.815 s after backend start]
2023-01-12T09:53:14.098Z root INFO Backend PluginDeployerContribution.initialize: 0.4 ms [Finished 0.816 s after backend start]
2023-01-12T09:53:14.099Z root INFO Backend NodeFileUploadService.configure: 1.0 ms [Finished 0.816 s after backend start]
2023-01-12T09:53:14.099Z root INFO Backend WebviewBackendSecurityWarnings.initialize: 0.1 ms [Finished 0.816 s after backend start]
2023-01-12T09:53:14.099Z root INFO Backend FileDownloadEndpoint.configure: 0.4 ms [Finished 0.817 s after backend start]
2023-01-12T09:53:14.100Z root INFO Backend HostedPluginLocalizationService.initialize: 0.2 ms [Finished 0.817 s after backend start]
2023-01-12T09:53:14.100Z root INFO Backend MetricsBackendApplicationContribution.configure: 0.3 ms [Finished 0.818 s after backend start]
2023-01-12T09:53:14.100Z root INFO Backend HostedPluginReader.initialize: 0.2 ms [Finished 0.818 s after backend start]
2023-01-12T09:53:14.100Z root INFO Backend MiniBrowserEndpoint.configure: 0.4 ms [Finished 0.819 s after backend start]
2023-01-12T09:53:14.100Z root INFO Backend PluginApiContribution.configure: 0.2 ms [Finished 0.819 s after backend start]
2023-01-12T09:53:14.100Z root INFO Backend HostedPluginReader.configure: 0.2 ms [Finished 0.819 s after backend start]
2023-01-12T09:53:14.100Z root INFO Backend HostedPluginReader.configure: 0.0 ms [Finished 0.819 s after backend start]
2023-01-12T09:53:14.100Z root INFO Backend TheiaLauncherServiceEndpoint.configure: 0.1 ms [Finished 0.819 s after backend start]
2023-01-12T09:53:14.101Z root INFO Theia app listening on http://localhost:60443.
2023-01-12T09:53:14.102Z root WARN The local plugin referenced by local-dir:/Users/a.kitta/.theia-blueprint/plugins does not exist.
2023-01-12T09:53:14.103Z root WARN The local plugin referenced by local-dir:/Users/a.kitta/Desktop/with #/TheiaBlueprint.app/Contents/Resources/app/plugins does not exist.
2023-01-12T09:53:14.103Z root WARN The local plugin referenced by local-dir:/Users/a.kitta/.theia/plugins does not exist.
2023-01-12T09:53:14.105Z root INFO [MS-CEINTL.vscode-language-pack-zh-hans-1.53.2.vsix]: trying to decompress into "/private/var/folders/z1/xkw1yh5n7rz4n8djprp1mdn80000gn/T/vscode-unpacked/MS-CEINTL.vscode-language-pack-zh-hans-1.53.2.vsix"...
2023-01-12T09:53:14.106Z root INFO [MS-CEINTL.vscode-language-pack-zh-hans-1.53.2.vsix]: already found
2023-01-12T09:53:14.106Z root INFO PluginTheiaDirectoryHandler: accepting plugin with path /private/var/folders/z1/xkw1yh5n7rz4n8djprp1mdn80000gn/T/vscode-unpacked/MS-CEINTL.vscode-language-pack-zh-hans-1.53.2.vsix
2023-01-12T09:53:14.107Z root ERROR Failed to load plugin dependencies from '/private/var/folders/z1/xkw1yh5n7rz4n8djprp1mdn80000gn/T/vscode-unpacked/MS-CEINTL.vscode-language-pack-zh-hans-1.53.2.vsix' path Error: ENOENT: no such file or directory, open '/private/var/folders/z1/xkw1yh5n7rz4n8djprp1mdn80000gn/T/vscode-unpacked/MS-CEINTL.vscode-language-pack-zh-hans-1.53.2.vsix/package.json'
2023-01-12T09:53:14.107Z root INFO Resolve plugins list: 8.5 ms [Finished 0.826 s after backend start]
2023-01-12T09:53:14.107Z root INFO Deploy plugins list: 8.5 ms [Finished 0.826 s after backend start]
2023-01-12T09:53:14.157Z root INFO Backend DefaultWorkspaceServer.onStart: 13.6 ms [Finished 0.828 s after backend start]
2023-01-12T09:53:14.157Z root INFO Backend TaskBackendApplicationContribution.onStart: 0.2 ms [Finished 0.828 s after backend start]
2023-01-12T09:53:14.157Z root WARN Failed to resolve module: @types/body-parser
2023-01-12T09:53:14.157Z root WARN Failed to resolve module: @types/cookie
2023-01-12T09:53:14.157Z root WARN Failed to resolve module: @types/dompurify
2023-01-12T09:53:14.157Z root WARN Failed to resolve module: @types/express
2023-01-12T09:53:14.157Z root WARN Failed to resolve module: @types/fs-extra
2023-01-12T09:53:14.157Z root WARN Failed to resolve module: @types/lodash.debounce
2023-01-12T09:53:14.157Z root WARN Failed to resolve module: @types/lodash.throttle
2023-01-12T09:53:14.157Z root WARN Failed to resolve module: @types/markdown-it
2023-01-12T09:53:14.157Z root WARN Failed to resolve module: @types/react
2023-01-12T09:53:14.158Z root WARN Failed to resolve module: @types/react-dom
2023-01-12T09:53:14.158Z root WARN Failed to resolve module: @types/route-parser
2023-01-12T09:53:14.158Z root WARN Failed to resolve module: @types/safer-buffer
2023-01-12T09:53:14.158Z root WARN Failed to resolve module: @types/ws
2023-01-12T09:53:14.158Z root WARN Failed to resolve module: @types/yargs
2023-01-12T09:53:14.158Z root WARN Failed to resolve module: yargs
2023-01-12T09:53:14.158Z root WARN Failed to resolve module: @types/multer
2023-01-12T09:53:14.158Z root WARN Failed to resolve module: @types/rimraf
2023-01-12T09:53:14.158Z root WARN Failed to resolve module: @types/tar-fs
2023-01-12T09:53:14.158Z root WARN Failed to resolve module: @types/uuid
2023-01-12T09:53:14.158Z root WARN Failed to resolve module: @types/p-queue
2023-01-12T09:53:14.158Z root WARN Failed to resolve module: @types/diff
2023-01-12T09:53:14.158Z root WARN Failed to resolve module: @types/mime-types
2023-01-12T09:53:14.158Z root WARN Failed to resolve module: @types/mime
2023-01-12T09:53:14.158Z root WARN Failed to resolve module: @types/request
2023-01-12T09:53:14.158Z root INFO Backend MetricsBackendApplicationContribution.onStart: 48.1 ms [Finished 0.876 s after backend start]
2023-01-12T09:53:14.158Z root INFO Backend MiniBrowserEndpoint.onStart: 0.9 ms [Finished 0.877 s after backend start]
2023-01-12T09:53:14.158Z root INFO Finished starting backend application: 0.0 ms [Finished 0.877 s after backend start]
10:53:14.179 › Checking for update
10:53:14.791 › Update for version 1.32.0 is not available (latest version: 1.32.0, downgrade is disallowed).
Opening channel for service path '/services/electron-window'.
2023-01-12T09:53:15.718Z root INFO Git askpass helper is listening on http://127.0.0.1:60451.
Opening channel for service path '/services/theia-updater'.
2023-01-12T09:53:15.809Z root WARN Frontend J.configure took longer than the expected maximum 100 milliseconds: 204.1 ms [Finished 1.495 s after frontend start]
2023-01-12T09:53:15.839Z root INFO Using Git [2.32.1 (Apple Git-133)] from the PATH. (/usr/bin/git)
2023-01-12T09:53:15.967Z root ERROR Uncaught Exception:  Error [ERR_HTTP_HEADERS_SENT]: Cannot set headers after they are sent to the client
2023-01-12T09:53:15.968Z root ERROR Error [ERR_HTTP_HEADERS_SENT]: Cannot set headers after they are sent to the client
    at new NodeError (node:internal/errors:370:5)
    at ServerResponse.setHeader (node:_http_outgoing:573:11)
    at ServerResponse.header (/Users/a.kitta/Desktop/with #/TheiaBlueprint.app/Contents/Resources/app/node_modules/express/lib/response.js:794:10)
    at ServerResponse.send (/Users/a.kitta/Desktop/with #/TheiaBlueprint.app/Contents/Resources/app/node_modules/express/lib/response.js:174:12)
    at ServerResponse.json (/Users/a.kitta/Desktop/with #/TheiaBlueprint.app/Contents/Resources/app/node_modules/express/lib/response.js:278:15)
    at TheiaLauncherServiceEndpoint.isInitialized (/Users/a.kitta/Desktop/with #/TheiaBlueprint.app/Contents/Resources/app/node_modules/theia-blueprint-launcher/lib/node/launcher-endpoint.js:85:22)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
2023-01-12T09:53:15.972Z root INFO Start frontend contributions: 525.1 ms [Finished 1.732 s after frontend start]
2023-01-12T09:53:15.972Z root INFO Changed application state from 'init' to 'started_contributions'.
2023-01-12T09:53:16.000Z root INFO Changed application state from 'started_contributions' to 'attached_shell'.
2023-01-12T09:53:16.001Z root INFO >>> Restoring the layout state...
2023-01-12T09:53:16.386Z root WARN Linked preference "workbench.colorCustomizations" not found. Source: "editor.bracketPairColorization.enabled"
2023-01-12 10:53:16.465 TheiaBlueprint Helper (Renderer)[50439:2105588] CoreText note: Client requested name ".NewYork-Regular", it will get Times-Roman rather than the intended font. All system UI font access should be through proper APIs such as CTFontCreateUIFontForLanguage() or +[NSFont systemFontOfSize:].
2023-01-12 10:53:16.465 TheiaBlueprint Helper (Renderer)[50439:2105588] CoreText note: Set a breakpoint on CTFontLogSystemFontNameRequest to debug.
2023-01-12T09:53:16.527Z root INFO [7f6e2c6b-bc79-408b-95a4-578264203f27] Waiting for backend deployment: 564.9 ms [Finished 2.296 s after frontend start]
2023-01-12T09:53:16.527Z root INFO [7f6e2c6b-bc79-408b-95a4-578264203f27] Sync of 0 plugins: 0.0 ms [Finished 2.296 s after frontend start]
2023-01-12T09:53:16.578Z root INFO Checking whether '--no-optional-locks' can be used with the current Git executable. Minimum required version is '2.15.0'.
2023-01-12T09:53:16.611Z root INFO '--no-optional-locks' is a valid Git option for the current Git version: '2.32.1 (Apple Git-133)'.
2023-01-12T09:53:16.647Z root INFO <<< The layout has been successfully restored.
2023-01-12T09:53:16.649Z root INFO Initialize the workbench layout: 667.9 ms [Finished 2.416 s after frontend start]
2023-01-12T09:53:16.650Z root INFO Changed application state from 'attached_shell' to 'initialized_layout'.
2023-01-12T09:53:16.651Z root INFO [7f6e2c6b-bc79-408b-95a4-578264203f27] Load contributions of 0 plugins: 0.1 ms [Finished 2.417 s after frontend start]
2023-01-12T09:53:16.690Z root INFO Application launcher was already initialized.
2023-01-12T09:53:16.994Z root INFO [7f6e2c6b-bc79-408b-95a4-578264203f27] Start of 0 plugins: 335.2 ms [Finished 2.763 s after frontend start]
2023-01-12T09:53:17.512Z root INFO Frontend application start: 1794.8 ms [Finished 4.231 s after backend start]
2023-01-12T09:53:17.512Z root INFO Replace loading indicator with ready workbench UI (animation): 860.0 ms [Finished 3.280 s after frontend start]
2023-01-12T09:53:17.512Z root INFO Changed application state from 'initialized_layout' to 'ready'.
2023-01-12T09:53:17.766Z root ERROR h.registerSchemas is taking more than 500.0 ms, new schemas are ignored.

As the final step, remove the # from the path to Theia and start it. Now, the extensions load, and there is syntax coloring.

Downstream issue: arduino/arduino-ide#1124

Additional Information

• Operating System:
• Theia Version: TheiaBlueprint Version 1.32.0 (1.32.0.92)

[kittaakos]

I cannot even build Theia from the sources when there is # in the path to the local git repo nodejs/node-gyp#2556

Workaround update for the build:

• It works if I check out the source to a path that does not contain #,
• Build it,
• Rename the folder and add # to the path,
• Build Theia again -> this way it works.

Update 2:

• after the successful yarn command,
• yarn build fails:

@theia/example-electron: ERROR in ../../../../with / sync ^\.\/.*\.json$
@theia/example-electron: ENOENT: no such file or directory, scandir '/Users/a.kitta/Desktop/with '
@theia/example-electron: caused by plugins in ContextModule.resolveDependencies
@theia/example-electron: Error: ENOENT: no such file or directory, scandir '/Users/a.kitta/Desktop/with '
@theia/example-electron:  @ ../../../../with #/theia/packages/core/lib/browser/keyboard/browser-keyboard-layout-provider.js 365:13-82
@theia/example-electron:  @ ../../../../with #/theia/packages/core/lib/browser/keyboard/index.js 30:13-58
@theia/example-electron:  @ ../../../../with #/theia/packages/core/lib/browser/index.js 30:13-34
@theia/example-electron:  @ ../../../../with #/theia/examples/electron/src-gen/frontend/index.js 34:36-70
@theia/example-electron: webpack 5.75.0 compiled with 1 error in 21247 ms

[kittaakos]

To sum up, there are three separate issues:

• It's impossible to build Theia from the sources if there is a space or # in the path to the local repo. Running yarn fails. This is an upstream issue: fix: failure when build directory contains spaces (#65) nodejs/node-gyp#2556
• If you adjust the local Theia repo path to contain neither spaces nor #-es, and yarn succeeds, but yarn build fails.

@theia/example-electron: ERROR in ../../../../with / sync ^\.\/.*\.json$
@theia/example-electron: ENOENT: no such file or directory, scandir '/Users/a.kitta/Desktop/with '
@theia/example-electron: caused by plugins in ContextModule.resolveDependencies
@theia/example-electron: Error: ENOENT: no such file or directory, scandir '/Users/a.kitta/Desktop/with '
@theia/example-electron:  @ ../../../../with #/theia/packages/core/lib/browser/keyboard/browser-keyboard-layout-provider.js 365:13-82
@theia/example-electron:  @ ../../../../with #/theia/packages/core/lib/browser/keyboard/index.js 30:13-58
@theia/example-electron:  @ ../../../../with #/theia/packages/core/lib/browser/index.js 30:13-34
@theia/example-electron:  @ ../../../../with #/theia/examples/electron/src-gen/frontend/index.js 34:36-70
@theia/example-electron: webpack 5.75.0 compiled with 1 error in 21247 ms

It seems to be a webpack problem, and maybe the Theia webpack generator does not handle # in the path gracefully. I did not investigate it.

• The third problem is if the local plugins folder contains a # in the path, the VS Code extensions inside this folder do not load. This is the real issue because it's not a dev thing. The plugin resolver fails to load if users put any bundled Theia application into a folder where a # is in the path.

If I am in this folder:

/Users/a.kitta/Desktop/one two # three

and start Theia with this command:

./TheiaBlueprint.app/Contents/MacOS/TheiaBlueprint

the local plugin folder resolution will fail:

2023-01-12T13:06:54.537Z root WARN The local plugin referenced by local-dir:/Users/a.kitta/.theia-blueprint/plugins does not exist.
2023-01-12T13:06:54.537Z root WARN The local plugin referenced by local-dir:/Users/a.kitta/.theia/plugins does not exist.
2023-01-12T13:06:54.537Z root WARN The local plugin referenced by local-dir:/Users/a.kitta/Desktop/one two # three/TheiaBlueprint.app/Contents/Resources/app/plugins does not exist.

The problem was correctly pointed out by @r3inbowari here: arduino/arduino-ide#1124 (comment)

After manually modifying the /TheiaBlueprint.app/Contents/Resources/app/node_modules/@theia/plugin-ext/lib/main/node/resolvers/local-plugin-deployer-resolver.js JS module to log additional info out, I can see the following extended log:

2023-01-12T13:16:29.834Z root INFO resolveLocalPluginPath - localUri: local-dir:/Users/a.kitta/.theia-blueprint/plugins
2023-01-12T13:16:29.834Z root INFO resolveLocalPluginPath - fsPath: /Users/a.kitta/.theia-blueprint/plugins
2023-01-12T13:16:29.834Z root INFO resolveLocalPluginPath - localUri: local-dir:/Users/a.kitta/.theia-blueprint/extensions
2023-01-12T13:16:29.834Z root INFO resolveLocalPluginPath - fsPath: /Users/a.kitta/.theia-blueprint/extensions
2023-01-12T13:16:29.834Z root INFO resolveLocalPluginPath - localUri: local-dir:/Users/a.kitta/Desktop/one%20two%20#%20three%2FTheiaBlueprint.app%2FContents%2FResources%2Fapp%2Fplugins
2023-01-12T13:16:29.834Z root INFO resolveLocalPluginPath - fsPath: /Users/a.kitta/Desktop/one two 
2023-01-12T13:16:29.834Z root INFO resolveLocalPluginPath - localUri: local-dir:/Users/a.kitta/.theia/plugins
2023-01-12T13:16:29.834Z root INFO resolveLocalPluginPath - fsPath: /Users/a.kitta/.theia/plugins

The local-dir URI is incorrectly mapped to the filesystem path:

local-dir:/Users/a.kitta/Desktop/one%20two%20#%20three%2FTheiaBlueprint.app%2FContents%2FResources%2Fapp%2Fplugins

Actual:

/Users/a.kitta/Desktop/one two 

Expected:

/Users/a.kitta/Desktop/one two # three

If I use any online URL decoder, it works.

local-dir:/Users/a.kitta/Desktop/one%20two%20#%20three%2FTheiaBlueprint.app%2FContents%2FResources%2Fapp%2Fplugins

local-dir:/Users/a.kitta/Desktop/one two # three/TheiaBlueprint.app/Contents/Resources/app/plugins

---

The easiest way to reproduce the third issue is to modify the plugins folder inside the Theia repo and start the electron app, for example, from the sources:

When I am on 73f32f7, the following change is sufficient to trigger the bug:

diff --git a/.gitignore b/.gitignore
index 8fcd5853d..74142fcc0 100644
--- a/.gitignore
+++ b/.gitignore
@@ -22,7 +22,7 @@ package-backup.json
 .history
 .Trash-*
 packages/plugin/typedoc
-plugins
+plugins*
 gh-pages
 .vscode/ipch
 dev-packages/electron/compile_commands.json
diff --git a/package.json b/package.json
index 3fcfc46ca..752b2b6da 100644
--- a/package.json
+++ b/package.json
@@ -96,7 +96,7 @@
     "examples/*",
     "packages/*"
   ],
-  "theiaPluginsDir": "plugins",
+  "theiaPluginsDir": "plugins #",
   "theiaPlugins": {
     "eclipse-theia.builtin-extension-pack": "https://open-vsx.org/api/eclipse-theia/builtin-extension-pack/1.50.1/file/eclipse-theia.builtin-extension-pack-1.50.1.vsix",
     "vscode.theme-abyss": "https://open-vsx.org/api/vscode/theme-abyss/1.62.3/file/vscode.theme-abyss-1.62.3.vsix",

[kittaakos]

It's not possible to provide a workaround downstream with DI because the implementations are not bound:

theia/packages/plugin-ext/src/main/node/plugin-ext-backend-module.ts

Lines 57 to 59 in 73f32f7

	bind(PluginDeployerResolver).to(LocalDirectoryPluginDeployerResolver).inSingletonScope();
	bind(PluginDeployerResolver).to(GithubPluginDeployerResolver).inSingletonScope();
	bind(PluginDeployerResolver).to(HttpPluginDeployerResolver).inSingletonScope();

[colin-grant-work]

It's not possible to provide a workaround downstream with DI because the implementations are not bound:

@kittaakos , it would be possible to work around it downstream, if you would like to do so while awaiting an upstream fix, using a combination of extensions and contributions filters: create a contribution filter that removes the malfunctioning resolver, then bind PluginDeployerResolver to a fixed version of the same class.

Although that may fail if the classes that use the PluginDeployerResolvers use @multiInject rather than a ContributionProvider - I'm not entirely certain of the details.

[kittaakos]

Thank you so much for the hint, @colin-grant-work. I will try it, and if it works downstream, I will share the temp workaround.

[colin-grant-work]

Even if that doesn't work directly, because it looks like PluginDeployerImpl does use multiInject, creating a fixed version of the misbehaving resolver and then rebind the PluginDeployer to a version of PluginDeployerImpl where this line:

theia/packages/plugin-ext/src/main/node/plugin-deployer-impl.ts

Line 340 in 911c0ca

const foundPluginResolver = this.pluginResolvers.find(pluginResolver => pluginResolver.accept(pluginId));

Says

const foundPluginResolver = this.pluginResolvers.find(pluginResolver => !(pluginResolver instanceof BadResolver) && pluginResolver.accept(pluginId));

would also work.

[kittaakos]

it would be possible to work around it downstream

It is. I appreciate your pointer, @colin-grant-work. Here is a hack that works in a bundled Theia app.