Research: more nuanced RegEx highlighting

[binyamin]

Is your request related to a specific problem you're having?

I have a block of code with a rather long regex. The entire regex is one color.

The solution you'd prefer / feature you'd like to see added

Break a regex into smaller components, maybe with an enclosing .hljs-regexp class. Those components might be general, or regex-specific.

Additional context

(Theme: Nord)
In Highlight.js

In VS Code

[joshgoebel]

This would likely fall under #2500.

maybe with an enclosing .hljs-regexp class.

Yes, what would be needed is actual subclasses (ie, regex.subgroup) (in a parser sense, not necessary CSS)... such as [forgive my naming]:

• sub grouping ()
• character group []
• count modifier (*,?,+)
• escape sequences?
• pins (^, $)

We don't have a way to codify such subclasses yet, but I'm open to figuring that out - it just hasn't become necessary yet.

My worry here is this would require pretty accurate almost real parsing of regex (which traditionally we try to avoid because of the complexity and maintenance) - and that may or may not be possible. If someone came up with a rough prototype I'd definitely review it. But if the parsing can't be fully accurate (ie, it misidentifies things regularly or simply can't handle some regex) then I'm not very interested in trying to fix regex parsing bugs in perpetuity. And of course then themes would need to be altered to support this on a case by case basis unless the missing rules could be added programmatically.

ALSO worth noting: This could easily be done with a 3rd party plugin if it's decided this does not belong in core. You'd simply redefine hljs.REGEXP_MODE to be whatever you desire [adding the rich detail you're requesting vs custom css classes] - and then be responsible for providing a CSS theme that provided the new styles (or make that a task for the user). Almost all the grammars defer to REGEXP_MODE for highlighting regex, so by improving this rule all grammars would gain the new detailed highlighting.

So either way seems perhaps something worth exploring.

[joshgoebel]

@binyamin You want to try and see what you can come up with?

[binyamin]

@joshgoebel I'd love to. I don't know the code at all, but I can take a look.

[joshgoebel]

Actually it may be much simpler and you might not need a full parser at all...

• first match escapes (to avoid false positives on special chars) \(
• then match special characters one at a time (or in combos), styling them
• you will need special handling when inside [] since the rules in there are a bit different than outside...

But that might get you really far... We already deal with some of this (we have to to know when a regex truly ends), we just don't highlight it... and of course you'll have to be super careful with your own escaping of things since you're writing regex to parse regex.

[michaelblyons]

What are the odds that there could be a stand-alone Regex language identifier?

[joshgoebel]

See, #2771... though it's far too ambitious for being helpful to resolve this issue and more likely to end up as a 3rd party grammar module.

[joshgoebel]

I'm going to add "help welcome" but this really needs a champion who is willing to invest some serious time to work thru all the questions we'd need to resolve to make something mergeable and of benefit to all themes. If someone wanted to do something smaller in scope for themselves I'd recommend writing a plugin-extension that just modified the libraries regex mode (and then added their own custom CSS classes for whatever theme they were using) - which would automatically cover many grammars (but not all).

If no one steps up this will auto-close in a few months as I don't see a need to leave it open forever.