0.33.0 - 'npm ci --omit=dev --cpu=x64 --os=linux --libc=glibc' installs both glibc and musl libs

[tmcgannon]

Possible install-time or require-time problem

• I have read the documentation relating to installation.

Are you using the latest version of sharp?

• I am using the latest version of sharp as reported by npm view sharp dist-tags.latest.

If you cannot confirm this, please upgrade to the latest version and try again before opening an issue.

If you are using another package which depends on a version of sharp that is not the latest, please open an issue against that package instead.

Are you using a supported runtime?

• I am using Node.js 18 with a version >= 18.17.0
• I am using Node.js 20 with a version >= 20.3.0
• I am using Node.js 21 or later

If you cannot confirm any of these, please upgrade to the latest version and try again before opening an issue.

Are you using a supported package manager?

• I am using npm >= 9.6.5
• I am using yarn >= 3.2.0
• I am using pnpm >= 7.1.0

If you cannot confirm any of these, please upgrade to the latest version and try again before opening an issue.

What is the complete output of running npm install --verbose --foreground-scripts sharp in an empty directory?

npm verb cli /usr/local/bin/node /usr/local/bin/npm
npm info using npm@10.2.3
npm info using node@v20.10.0
npm verb title npm install sharp
npm verb argv "install" "--loglevel" "verbose" "--foreground-scripts" "sharp"
npm verb logfile logs-max:10 dir:/Users/tmcgannon/.npm/_logs/2023-11-29T22_24_45_285Z-
npm verb logfile /Users/tmcgannon/.npm/_logs/2023-11-29T22_24_45_285Z-debug-0.log
npm http fetch GET 200 https://registry.npmjs.org/sharp 182ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/semver 47ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/detect-libc 126ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/color 134ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-darwin-x64 354ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-libvips-linuxmusl-x64 369ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-libvips-linux-arm 450ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-wasm32 465ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-darwin-arm64 495ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-linux-s390x 609ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-libvips-linux-x64 611ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-libvips-linuxmusl-arm64 611ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-linuxmusl-x64 610ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-win32-x64 610ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-libvips-darwin-x64 613ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-linux-arm 613ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-win32-ia32 653ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-linuxmusl-arm64 662ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-libvips-darwin-arm64 666ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-libvips-linux-s390x 667ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-linux-x64 669ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-linux-arm64 708ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-libvips-linux-arm64 765ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/color-convert 50ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@emnapi%2fruntime 266ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/color-name 44ms (cache revalidated)
npm verb reify failed optional dependency /Users/tmcgannon/node_modules/@img/sharp-win32-x64
npm verb reify failed optional dependency /Users/tmcgannon/node_modules/@img/sharp-win32-ia32
npm verb reify failed optional dependency /Users/tmcgannon/node_modules/@img/sharp-wasm32
npm verb reify failed optional dependency /Users/tmcgannon/node_modules/@emnapi/runtime
npm verb reify failed optional dependency /Users/tmcgannon/node_modules/@img/sharp-linuxmusl-x64
npm verb reify failed optional dependency /Users/tmcgannon/node_modules/@img/sharp-linuxmusl-arm64
npm verb reify failed optional dependency /Users/tmcgannon/node_modules/@img/sharp-linux-x64
npm verb reify failed optional dependency /Users/tmcgannon/node_modules/@img/sharp-linux-s390x
npm verb reify failed optional dependency /Users/tmcgannon/node_modules/@img/sharp-linux-arm64
npm verb reify failed optional dependency /Users/tmcgannon/node_modules/@img/sharp-linux-arm
npm verb reify failed optional dependency /Users/tmcgannon/node_modules/@img/sharp-libvips-linuxmusl-x64
npm verb reify failed optional dependency /Users/tmcgannon/node_modules/@img/sharp-libvips-linuxmusl-arm64
npm verb reify failed optional dependency /Users/tmcgannon/node_modules/@img/sharp-libvips-linux-x64
npm verb reify failed optional dependency /Users/tmcgannon/node_modules/@img/sharp-libvips-linux-s390x
npm verb reify failed optional dependency /Users/tmcgannon/node_modules/@img/sharp-libvips-linux-arm64
npm verb reify failed optional dependency /Users/tmcgannon/node_modules/@img/sharp-libvips-linux-arm
npm verb reify failed optional dependency /Users/tmcgannon/node_modules/@img/sharp-libvips-darwin-x64
npm verb reify failed optional dependency /Users/tmcgannon/node_modules/@img/sharp-darwin-x64
npm http fetch POST 200 https://registry.npmjs.org/-/npm/v1/security/advisories/bulk 89ms
npm info run sharp@0.33.0 install node_modules/sharp node install/check

> sharp@0.33.0 install
> node install/check

npm http fetch GET 200 https://registry.npmjs.org/xml2js 48ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/axios 59ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/tough-cookie 60ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/jsonwebtoken 62ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/dottie 106ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@azure%2fms-rest-js 67ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/azurite 91ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/sequelize 168ms (cache revalidated)
npm info run sharp@0.33.0 install { code: 0, signal: null }

added 8 packages, and audited 297 packages in 4s

58 packages are looking for funding
  run `npm fund` for details

8 vulnerabilities (7 moderate, 1 high)

To address issues that do not require attention, run:
  npm audit fix

To address all issues (including breaking changes), run:
  npm audit fix --force

What is the output of running npx envinfo --binaries --system --npmPackages=sharp --npmGlobalPackages=sharp?

➜  test npx envinfo --binaries --system --npmPackages=sharp --npmGlobalPackages=sharp

  System:
    OS: macOS 14.1
    CPU: (8) arm64 Apple M1
    Memory: 83.52 MB / 8.00 GB
    Shell: 5.9 - /bin/zsh
  Binaries:
    Node: 20.10.0 - /usr/local/bin/node
    npm: 10.2.3 - /usr/local/bin/npm
    bun: 1.0.14 - ~/.bun/bin/bun

Problem description

When I run npm ci --omit=dev --cpu=x64 --os=linux --libc=glibc, it installs both glibc and musl versioned libraries:

Expected behavior

Only sharp-libvips-linux-x64 and sharp-linux-x64 should be installed.

[lovell]

Please see npm/cli#6914

[adriaanmeuris]

I'm probably missing something, but I'm seeing the same thing happening when using either pnpm or yarn which do allow to set the supported architectures.

Example package.json file

{
  "name": "sharp-binaries",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "express": "4.18.2",
    "sharp": "0.33.0"
  },
  "os": ["linux"],
  "cpu": ["x64"],
  "libc": ["glibc"],
  "pnpm": {
    "supportedArchitectures": {
      "os": ["linux"],
      "cpu": ["x64"],
      "libc": ["glibc"]
    }
  }
}

PNPM

Uses the pnpm property in package.json to setup supported architectures as defined here.

Dockerfile

FROM node:18.18-alpine3.18
RUN corepack enable && corepack prepare pnpm@latest --activate
COPY package*.json ./
RUN pnpm i
COPY index.js ./
CMD ["node", "index.js"]

Build targetting linux/amd64

docker build . --no-cache --platform=linux/amd64 --tag 'sharp-binaries:latest'

Results
I'm seeing 4 packages (using Docker desktop) where I'd expect to see 2 (sharp-libvips-linux-x64 and sharp-linux-x64):

Yarn

Uses os, cpu and libc properties in package.json as described here.

Dockerfile

FROM node:18.18-alpine3.18
COPY package*.json ./
RUN yarn install
COPY index.js ./
CMD ["node", "index.js"]

Build targetting linux/amd64

docker build . --no-cache --platform=linux/amd64 --tag 'sharp-binaries:latest'

Results
I'm seeing 19 packages where I'd expect to see 2 (sharp-libvips-linux-x64 and sharp-linux-x64)

[lovell]

@adriaanmeuris For pnpm, please see pnpm/pnpm#7214 - if the latest version of pnpm is installing the wrong dependencies, then please ask at the pnpm repo.

For yarn, it looks like you're using version 1.

$ docker run --rm -it node:18.18-alpine3.18 yarn --version
1.22.19

Please use at least version 3 of yarn, but avoid its "plug'n'play" linker.

Version 1 of yarn is no longer actively developed and is missing so many useful newer features. See also evanw/esbuild#2949

[adriaanmeuris]

Thanks @lovell, I can confirm this works with yarn@3 without PnP linker.

For pnpm it indeed seems like an issue with installing the right dependencies as mentioned here.

[tmcgannon]

Thanks, I will track the npm issue.

[klausbadelt]

For whoever ends up here - the --libc npm feature is now merged npm/cli/pull/6914