[BUG] Install fails if "bin" script doesn't exist

[t1m0thyj]

Current Behavior:

In a workspace, npm install fails for a package with a "bin" script defined in package.json that doesn't exist. NPM tries to chmod the non-existent "bin" script, resulting in ENOENT error.

npm ERR! code ENOENT
npm ERR! syscall chmod
npm ERR! path /home/timothy/dev/npm7-bin-link-issue/node_modules/cli/lib/index.js
npm ERR! errno -2
npm ERR! enoent ENOENT: no such file or directory, chmod '/home/timothy/dev/npm7-bin-link-issue/node_modules/cli/lib/index.js'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent

Expected Behavior:

npm install should succeed and NPM should not attempt to chmod "bin" scripts if they don't exist.

Steps To Reproduce:

The scenario of a non-existent "bin" script is easy to encounter in a TypeScript monorepo using NPM workspaces, where a "bin" script may point to a .js file that doesn't exist until the TS is transpiled to JS.

To reproduce the error, clone this minimal sample repo and run npm install. Or follow the steps below:

1. Create an NPM project: npm init -y
2. Define the workspaces property in top-level package.json:
   "workspaces": [ "./packages/*" ]
3. Create the directory structure "packages/cli" and generate a package.json there: cd packages/cli && npm init -y
4. Define bin property in "packages/cli/package.json" and point to a non-existent script:
   "bin": { "test-cli": "./nonexistent.js" }
5. Run npm install at top level of project and see ENOENT

Environment:

• OS: Ubuntu 18.04
• Node: 15.8.0
• npm: 7.5.1

[chuanqisun]

I ran into the same issue. Seems to have been caused by workspace.
npm 7.11.2
node 16.1.0
Ubuntu 20.04.1 LTS (running as WSL)

[t1m0thyj]

For others encountering this issue, I've temporarily worked around it by adding a preinstall script to the package that has a "bin" script, which creates an empty "bin" script if none exists yet: https://github.com/zowe/zowe-cli/blob/da013f8b6897a16325343466c73f42de3b434325/packages/cli/scripts/preinstall.js

[ruyadorno]

Hi @t1m0thyj thank you so much for bringing this to our attention and opening npm/bin-links#23

I'm really curious about this statement:

The scenario of a non-existent "bin" script is easy to encounter in a TypeScript monorepo using NPM workspaces, where a "bin" script may point to a .js file that doesn't exist until the TS is transpiled to JS.

Is that a common scenario in the wider TypeScript community? I'm interested in understanding better the reasons and problems that particular setup is aiming to solve.

---

Meanwhile, I looked at your minimal sample repo and from an npm7 point of view it stands out the fact that the example is not using the prepare lifecycle script in order to make sure the build script gets run prior to the install / bin-linking, I made an example PR showing the way I would instinctively approach it here: https://github.com/t1m0thyj/npm7-bin-link-issue/pull/1 - but I'm aware that the decision to not use the prepare lifecycle script might have been intentional, so I'd like to hear from you.

[ljharb]

@ruyadorno it's common in any package that uses babel also; the "bin" script would be a .js file that doesn't exist prior to the build step running as well.

[ruyadorno]

right, thanks @ljharb! yeah my first thought is that the installer should def be more reliable anyways but I just want to make sure I fully understand the problem space first.

it's common in any package that uses babel also

in that case how is the babel ecosystem working around it today?

the "bin" script would be a .js file that doesn't exist prior to the build step running as well.

and from your pov is there a particular reason for not hooking the build step with the prepare lifecycle script? (assuming we're talking about nested workspaces here)

[ljharb]

@ruyadorno sure. All of my build steps use prepublishOnly, not prepare, which seems valid. Build output doesn't need to exist until publish time - or until i want sibling workspace packages to be able to consume it (ie, at runtime) - but certainly does not need to ever exist at install time.

[chuanqisun]

I might have a chicken-and-egg problem. I can't generate the .js file in prepare step because I need to npm install the build tools first. But I can't npm install because the .js file doesn't exist. My current workaround is cd into the nested package folder, npm install && npm run build to generate that .js file, then cd back to the monorepo root and run npm install. Any suggestions or workarounds?

The problematic package.json line
My documented workaround

[t1m0thyj]

Hi @t1m0thyj thank you so much for bringing this to our attention and opening npm/bin-links#23

I'm really curious about this statement:

The scenario of a non-existent "bin" script is easy to encounter in a TypeScript monorepo using NPM workspaces, where a "bin" script may point to a .js file that doesn't exist until the TS is transpiled to JS.

Is that a common scenario in the wider TypeScript community? I'm interested in understanding better the reasons and problems that particular setup is aiming to solve.

Meanwhile, I looked at your minimal sample repo and from an npm7 point of view it stands out the fact that the example is not using the prepare lifecycle script in order to make sure the build script gets run prior to the install / bin-linking, I made an example PR showing the way I would instinctively approach it here: t1m0thyj/npm7-bin-link-issue#1 - but I'm aware that the decision to not use the prepare lifecycle script might have been intentional, so I'd like to hear from you.

@ruyadorno Sorry for the delayed response. I don't claim to know how the wider TypeScript community handles this. I also don't immediately see an issue with using the prepare lifecycle script as you've suggested, so I'll consult with my team about whether it would work for our use case.

However, requiring to use the prepare lifecycle script feels like more of a workaround than the ideal solution to me. It also seems like other folks who've commented on this issue have reasons why they can't use it.

I suppose this comes down to the question of how npm should handle bins that don't exist at the end of the install process:

1. Should npm assume that something is wrong if a bin doesn't exist, and fail with an error as it does now?
2. Should npm issue a warning when a bin doesn't exist, since it may be a problem but doesn't necessarily warrant a fatal error?
3. Should npm assume that it doesn't matter if a bin doesn't exist, and fail silently when this happens?

The current behavior is #1, and my PR npm/bin-links#23 implements #3. But after further consideration, perhaps #2 offers a compromise that would be the best option?