TypeScript support for JSX

[thorn0]

Would be great if JSX recognized TypeScript constructs.

[syranide]

JSXTransformer uses fb-esprima which is a JSX-enhanced dialect of esprima, which is a complete JavaScript parser. So for JSX to support TypeScript would require extending or replacing esprima with a complete and compatible parser for TypeScript. Unless such a thing already exists, doing this would most likely require significant initial and ongoing effort.

Unless you are willing to put in that effort, it's unlikely that this is going to happen any time soon or at all in the foreseeable future. After all, JSX is only sugar for very similar looking and functionally equivalent JS code.

[thorn0]

Why does JSXTransformer need a JS parser at all? Why can't it just replace its XML-like constructs with JS code wherever it finds them (except string and comments)?

[sophiebits]

Contrived example:

<input value={this.props.text + "}"} />

The value shouldn't end at the first }; you need at least some JS parsing logic to understand that.

[syranide]

@spicyj Or just something as simple as "<tag />" should not be parsed by JSX.

@thorn0 You are probably right in that it isn't strictly necessary, but it aids in generating valid code, as we now actually makes sure that the JS parses before move on.

[sebmarkbage]

Technically you don't need a parser but you need a tokenizer.

Sweet.js macros operate on the token stream before it hits the parser. JSX almost works with Sweet.js except there are special rules for regular expressions. Closing tags doesn't work with the / token depending on context. The regexp resolution rules are hardcoded into the tokenizer. Sweet.js probably can't support our whitespace rules neither.

However, if we treat JSX as strictly part of a custom tokenization stream, we can do the transformation purely on the token level, just like Sweet.js. That should leave it compatible with any other language extension that doesn't need special tokenization rules. The resulting code would even be compatible with Sweet.js macros.

What do you think @jeffmo ?

[syranide]

@sebmarkbage As long as we assume that all opening brackets must have a corresponding closing bracket it should work technically. It will also have to assume that expressions, object notation, array notation, etc is compatible or the output will have to be configurable.

But we will lose the ability to detect and report syntax errors before we inject {expressions} into the output stream, so eventual errors in the output stream will likely be a lot less comprehensible as a result, I'm unsure about how bad they actually can become in the worst case.

There's no denying that there are advantages to operating on the token stream like this, I just fear that it will be at the expense of making JSX less user friendly and thus less appealing, JSX is just sugar so I would never sacrifice the convenience of really good error message with the convenience of something that looks like HTML. Hopefully I'm wrong and it's not bad at all, but I think it's a very fine line between JSX being a very nice optional feature and a nice toy.

JSX is really compelling to me, primarily because it's cleaner than the JavaScript equivalent without any immediate drawback (--watch makes it painless enough), but it still lacks features that vanilla JavaScript offers, which means that you have to revert to JavaScript from time to time, so I still stick to JavaScript, but I can see that I may switch to JSX in the future. But I would never if I end up with crappy error messages that plagues so many template frameworks. But that's my opinion.

[sebmarkbage]

Even if it was true that we can't get good enough error messages using a token-transform alone you can still have the parser provide context to the tokenizer.

If your parser can provide a compatible context , that's great, you get better error messages for free.

If it can't and you have to pipe the raw transformed strings, you still have the option to do that. If you want to use TypeScript or HipsterScript you can.

We can't add a convenient extension to the language and expect that nobody else will add other extensions. People will want them both. If you have to choose between TypeScript or JSX then JSX will certainly be seen as the toy.

Unless there is an ambiguity problem ofc. TypeScript may not be compatible for other reasons. That's why I like macros because they can be contextual extensions. An identifier can be treated differently depending on if it's a known React/JSX component or a Type/Class/Interface.

[thorn0]

TypeScript is similar to Sweet.js macros in that it's a superset of JS, just another convenient extension. However, yes, there might be incompatibilities as the type cast operator looks like this in TypeScript:

this.span = <HTMLSpanElement>document.createElement('span');

Oops...

[syranide]

@sebmarkbage You're more experienced in this area, but it seems like you would have always to explicitly extend the parser of the target language or use the standalone token-transform version, the likelihood of their and our parsers and tokenizers being compatible seems remote (unless esprima is the parser in the JavaScript world). Or am I missing something?

If that's the case, I haven't looked into the details, but it seems like a simple fix to just implement an optional parseProgram/parseExpression in fb-esprima which has no knowledge of the language features at all other than that brackets must be balanced for the sake of XJS expression containers. Token-transform for free really. The only issue is that string/comment syntax would have to be compatible, or the tokenizer would need to be configurable as well.

I could probably even put together a proof-of-concept if it's worth pursuing that path.

[jordwalke]

Apologies for just chiming in without reading the discussion. I just thought I'd let you know the result of a previous discussion I had with someone who started adding JSX support into TypeScript. We found it was difficult to distinguish type parameters Array<Things> from JSX. My suggestion was to require that all JSX be wrapped in parens like (<Typeahead />). This makes it easier to parse, and most of the time we end up wrapping our JSX in parens anyways to guard against Automatic Semicolon Insertion, so people are used to it.

[syranide]

@thorn0 @jordwalke It seems like Array<Things> shouldn't be an issue to solve if you're extending the actual parser (but perhaps you're not), as the parser should not be in a state where it would consider JSX to be valid. However, <A>1</A> seems intuitively harder to solve as we cannot know if <A> is JSX or a type-cast... until we've found the (possible) closing </A>, but that does not seem reasonable.

It seems to me that your suggestion (<A>) has the same issue unless you explicitly disallow type-casts as the first instruction inside parens, which isn't perfect, but should very rarely be an issue. So it's a surprisingly neat distinction/solution to a major ambiguity.

[sebmarkbage]

I was worried about maintaining two versions. In theory you could make a tokenizer that can be a drop in replacement in an esprima based typescript parser. But that's certainly more difficult.

This could be a nice quick fix. You can always add special look ahead/behind rules. It doesn't have to be pure.

On Jan 2, 2014, at 7:14 AM, Andreas Svensson notifications@github.com wrote:

@thorn0 @jordwalke It seems like Array shouldn't be an issue to solve if you're extending the actual parser (but perhaps you're not), as the parser should not be in a state where it would consider JSX to be valid. However, 1 seems intuitively harder to solve as we cannot know if is JSX or a type-cast until we've found the closing but does not seem reasonable.

It seems to me that your suggestion has the same issue unless you're explicitly disallowing type-casts as the first instruction inside parens, which isn't perfect, but should very rarely be an issue. So it's a surprisingly neat distinction/solution to a major ambiguity.

—
Reply to this email directly or view it on GitHub.

[syranide]

@sebmarkbage I'm playing with a proof-of-concept and it was a simple as I had hoped, the major issue that I hadn't foreseen is that it's basically not possible unambiguously detect the initial tag, without the parser to provide context or without any additional starting token/constraint. ...<abc is that a comparison perhaps?

If a language supports a feature like <abc> then we're basically all out of luck unless we require an initial parenthesis as @jordwalke suggested. It could make certain edge-cases in existing scripts break though (x(<string>y())) and there would be no obvious no-op way of escaping it for the user, so it's not 100% safe with just parens. The benefit would be that it wouldn't really look out-of-place in any language as you can't really get away from the mathematical parenthesis in any language.

So I'm unsure how flexible/verbose we should make it, the major issue being that if we require an additional token for the starting tag, it has to be repeated for every branch of conditional expressions with tags.

[sebmarkbage]

This is already a problem in JavaScript with regular expressions. You can still implement JS highlighting without a proper parser because you can use look behind to disambiguate. It's not easy to enumerate all the potential cases you'll have to look for though. Hopefully they're bounded.

Sweet.js has a pretty good write up on this for regular expressions. Since those are allowed in similar places as JSX I figure the solution would be similar. https://github.com/mozilla/sweet.js/wiki/design

[sebmarkbage]

Of course the extended language could add features that you can't account for. So it may not always work. That's why it would be great to have at least a little bit of feedback from the parser. E.g. if a regular expression is allowed, then can we also assume that < is the start of a tag.