Packager: Transform Times Out on Large Json File

[mattbierner]

Description

Try to require a large json file, such as that found in the cmu-pronouncing-dictionary module

Reproduction

const cmu = require('cmu-pronouncing-dictionary')

Here's the actual 4.5MB file: https://github.com/zeke/cmu-pronouncing-dictionary/master/index.json

I also tried changing the json to a regular js module but hit the same issue. Here's the log:

Bundling `index.ios.js`
  Transforming modules  ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░  99.5% (406/407)...Error: TimeoutError: transforming XXX/build/cmu.js took longer than 301 seconds.
You can adjust timeout via the 'transformTimeoutInterval' option
    at _transform.then.catch.error (XXX/node_modules/react-native/packager/src/JSTransformer/index.js:163:30)
    at process._tickCallback (internal/process/next_tick.js:103:7)
(node:36165) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 669): Error: TimeoutError: transforming XXX/build/cmu.js took longer than 301 seconds.
You can adjust timeout via the 'transformTimeoutInterval' option

Solution

Address the performance issue that is causing the processing of this file to take so long

Additional Information

• React Native version: 0.41
• Platform: iOS
• Operating System: MacOS 10.11.6

[mattbierner]

Actually this does not seem specific to json. I converted the file to js by prefixing it with export default but still ran into this issue.

[ide]

My guess is that the packager is passing both JS and JSON files to Babel when it should be passing only the former.

[redpandatronicsuk]

I had the same issue, also using cmu-pronouncing-dictionary :), add

"ignore": [
        "*.json"
    ]

to .babelrc and it should fix it.

UPDATE:
In RN 0.42 this doesn't seem to work anymore. When ignoring the JSONs in the transform step it produces this error:

UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 952): SyntaxError: TransformError: /path/to/file.json: Cannot read property 'tokens' of undefined

[iyawnis]

Is there any known workaround for this issue? I need to load a 8MB file. If I load it over bridge it takes 15 seconds on Android, even though iOS takes about half.

[cogwheel]

We are using ClojureScript with Reagent + Re-frame + React Native. For production builds, all of the clojure code compiles down to a single 3 MB JavaScript file which times out after 5 minutes. I'm still testing to see how long it will take on my machine (i7-6700k). A freshly generated (with re-natal) ClojureScript project takes 23 seconds to bundle.

When running with Figwheel, the clojurescript is compiled to a separate .js file and the index.android.js is a stub. The packager handles this in a fraction of a second, and the app runs fine when Figwheel sends the large chunk of code to the device.

Edit: I've increased the timeout to 30 minutes and it still isn't finishing.

[jeaye]

Still open, still an issue, and no comments from the maintainers? Can we please get some info on this?

[pesterhazy]

Apparently this has seen some changes in 0.45.

This .babelrc allows me to compile our ClojureScript-compiled index.ios.js in <25s:

{"presets": ["react-native"], "ignore": ["index.ios.js"]}

I wasn't able to ignore all js files, as that seems to affect node modules as well.

[mihirk]

Any solution to this?
I have a huge 70MB, JSON object I have tried to require it as both JSON file and JS file, packager throws the same TimeOutError.

[pesterhazy]

I've built a work-around for this long-standing packager bug. The idea is to build an empty "husk" index.ios.husk.js file, which includes requires and a placeholder. The placeholder is then replaced with the actual bundle contents in a second step. That way the packager doesn't run into OOMs or timeouts in the transformer/whole-program-optimization phases.

This repo shows how it works: https://github.com/pesterhazy/re-natal-husk

[jeaye]

I've added a test repo, showing how simple it is to kill the packager with only a few megs of compiled CLJS code: https://github.com/jeaye/react-native-packager-bug

The repo uses a CLJS macro to generate code which compiles to a few megs; following the steps in the readme will reproduce the timeout or out-of-memory error 100% of the time.

[cogwheel]

Currently we're working around this problem by using an optimized prod-build, but this is suboptimal during development for obvious reasons. We absolutely need to have an app working untethered from a dev machine, so we're expending quite a bit of effort just trying to get a production build working that we won't need for months.