MSBuild crashing in the build

[jkoritzinsky]

Build Information

Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=410579
Build error leg or test failing: Build / Mono Product Build windows x64 release / Build product
Pull request: #91768

Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "",
  "ErrorPattern": "Child Node \"[0-9]+\" exited prematurely.",
  "BuildRetry": true,
  "ExcludeConsoleLog": false
}

Report

Build	Definition	Step Name	Console log	Pull Request
1374580	dotnet/runtime	Copy native test components to test output folder	Log	#126592
2947489	dotnet-runtime	Build product	Log	#59958
1372453	dotnet/runtime	Build product	Log	#126715
2947186	dotnet-runtime	Build product	Log	#59616
1370423	dotnet/runtime	Build product	Log	#126322
2945115	dotnet-runtime	Build product	Log
1364403	dotnet/runtime	Copy native test components to test output folder	Log	#126419
1358369	dotnet/runtime	Build product	Log	#126322
1358346	dotnet/runtime	Build product	Log	#124544
2934504	dotnet-runtime	Build product	Log
2933801	dotnet-runtime	Build product		#57892
1343631	dotnet/runtime	Build product	Log	#125790
2928111	dotnet-runtime	Build product	Log
2928051	dotnet-runtime	Build product
2927944	dotnet-runtime	Build product	Log
2927728	dotnet-runtime	Build product
2927606	dotnet-runtime	Build product	Log
2927190	dotnet-runtime	Build product	Log
2926739	dotnet-runtime	Build product	Log
2926621	dotnet-runtime	Build product
2926620	dotnet-runtime	Build product

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	6	21

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=410579
Error message validated: Child Node "[0-9]+" exited prematurely.
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 9/19/2023 6:06:29 PM UTC

Tagging subscribers to this area: @dotnet/runtime-infrastructure
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Build Information

Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=410579
Build error leg or test failing: Build / Mono Product Build windows x64 release / Build product
Pull request: #91768

Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "",
  "ErrorPattern": "Child Node \"[0-9]+\" exited prematurely.",
  "BuildRetry": true,
  "ExcludeConsoleLog": false
}

Author:	jkoritzinsky
Assignees:	-
Labels:	blocking-clean-ci, area-Infrastructure, untriaged, Known Build Error
Milestone:	-

[ViktorHofer]

190 hits per month

cc @rainersigwald @rokonec

[lewing]

reup for 97 hits per month

[MichalStrehovsky]

We now have more useful information in the logs for these. Two of the available logs have this in it:

2024-02-19T21:13:25.8115210Z MSBUILD : error MSB4166: Child node "2" exited prematurely. Shutting down. Diagnostic information may be found in files in "/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/MSBuildTemprunner/" and will be named MSBuild_*.failure.txt. This location can be changed by setting the MSBUILDDEBUGPATH environment variable to a different directory.
2024-02-19T21:13:25.8126770Z MSBUILD : error MSB4166: /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/MSBuildTemprunner/MSBuild_pid-50918_4ee39488d5ca4c7d9fb049f19f43c144.failure.txt:
2024-02-19T21:13:25.8128200Z MSBUILD : error MSB4166: UNHANDLED EXCEPTIONS FROM PROCESS 50918:
2024-02-19T21:13:25.8129250Z MSBUILD : error MSB4166: =====================
2024-02-19T21:13:25.8140670Z MSBUILD : error MSB4166: 2/19/2024 9:13:25 PM
2024-02-19T21:13:25.8241810Z MSBUILD : error MSB4166: System.IO.IOException: No buffer space available
2024-02-19T21:13:25.8260730Z MSBUILD : error MSB4166:  ---> System.Net.Sockets.SocketException (55): No buffer space available
2024-02-19T21:13:25.8363710Z MSBUILD : error MSB4166:    at System.IO.Pipes.PipeStream.WriteCore(ReadOnlySpan`1 buffer)
2024-02-19T21:13:25.8364490Z MSBUILD : error MSB4166:    --- End of inner exception stack trace ---
2024-02-19T21:13:25.8465500Z MSBUILD : error MSB4166:    at System.IO.Pipes.PipeStream.WriteCore(ReadOnlySpan`1 buffer)
2024-02-19T21:13:25.8569320Z MSBUILD : error MSB4166:    at System.IO.Pipes.PipeStream.Write(Byte[] buffer, Int32 offset, Int32 count)
2024-02-19T21:13:25.8670730Z MSBUILD : error MSB4166:    at Microsoft.Build.BackEnd.NodeEndpointOutOfProcBase.RunReadLoop(Stream localReadPipe, Stream localWritePipe, ConcurrentQueue`1 localPacketQueue, AutoResetEvent localPacketAvailable, AutoResetEvent localTerminatePacketPump)
2024-02-19T21:13:25.8773480Z MSBUILD : error MSB4166: ===================

@rainersigwald does this ring any bell?

[rainersigwald]

All of the recent hits seem to be dotnet/msbuild#10342, which should be fixed in the preview7 GA build. Is there a timeline for updating to that? Anything my team can help with?