stream: suggestion stream._writeMany

[ronag]

Currently the stream.writev API has two drawbacks:

• It requires adding properties to the input array (allBuffers).
• It will always creates an array copy.
• The array entries must be allocated objects.

It would be nice if user space could just create the correctly formatted array without having to perform any allocations, copies and transformations.

In order to not break anything I suggest a new signature writev maybe called writeMany?

Which would look something like:

Stream._writeMany = function (chunks, allBuffers, cb) {
}

e.g.

const chunks = [];
for (const { chunk } of data) {
  chunks.push(chunk);
}
w._writeMany(chunks, true, cb);

const chunks = [];
for (const { chunk, encoding } of data) {
  chunks.push(chunk, encoding);
}
w._writeMany(chunks, false, cb);

Looking at the current Writable implementation, the writev scenario will always create two "unnecessary" array copies in order to pass the chunks. Furthermore all the array entries are allocated objects.

[mscdex]

allBuffers is an implementation detail and not something that should be requested from the end user and relied upon.

[ronag]

allBuffers is an implementation detail and not something that should be requested from the end user and relied upon

True, but by exposing it we enable some optimizations that are otherwise not possible.

[mscdex]

We would still need to validate that the value for allBuffers is correct, which would negate any gain.

[ronag]

We would still need to validate that the value for allBuffers is correct, which would negate any gain.

I'm not sure I follow?