Skip to content

Conversation

jonatanklosko
Copy link

Currently ERB (Ruby templates) are highlighted wrongly as if they where entirely Ruby. This bug probably applies to other mixed grammars.

This was an issue in the past and was fixed in #105, but during subsequent refactoring a regression was introduced.

Copy link

changeset-bot bot commented Aug 14, 2025

⚠️ No Changeset found

Latest commit: b5bcc6b

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@@ -21,7 +21,7 @@ export const useTokenize = ({ prism, code, grammar, language }: Options) => {
}

prism.hooks.run("before-tokenize", prismConfig)
prismConfig.tokens = prism.tokenize(code, grammar)
prismConfig.tokens = prism.tokenize(prismConfig.code, prismConfig.grammar)
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See the original Prism highlight implementation as a reference: https://github.com/PrismJS/prism/blob/v1.30.0/prism.js#L661-L670.

Calling before-tokenize above modifies prismConfig.code. For example, given this snippet:

<div class="flex gap-2">
  <%= link_to "Controller error", page_controller_error_path,
      class: "btn btn-error btn-soft" %>
  <%= link_to "View error", page_view_error_path,
      class: "btn btn-error btn-soft" %>
  <%= link_to "View sytnax error", page_view_syntax_error_path,
      class: "btn btn-error btn-soft" %>
  <%= link_to "Missing view error", page_missing_view_error_path,
      class: "btn btn-error btn-soft" %>
  <%= link_to "Route error", "/page/unknown-route",
      class: "btn btn-error btn-soft" %>
</div>

before-tokenize rewrites the code to:

<div class="flex gap-2">
  ___ERB0___
  ___ERB1___
  ___ERB2___
  ___ERB3___
  ___ERB4___
</div>

it is expected that we pass the modified code to prism.tokenize, rather than the original code, as currently done.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant