JsonSerializer.Deserialize failing for some types in net9.0 that worked in net8.0.

[chuckries]

Description

The following snippet succeeds when compiled and run against net8.0, but throws an exception when compiled and run against net9.0

using System.Text.Json;
using System.Text.Json.Nodes;

string json = """
    { "names": ["Chuck"] }
    """;

var dict = JsonSerializer.Deserialize<Dictionary<string, JsonValue>>(json);

Reproduction Steps

1. create a new net8.0 console app and add the following code:

using System.Text.Json;
using System.Text.Json.Nodes;

string json = """
    { "names": ["Chuck"] }
    """;

var dict = JsonSerializer.Deserialize<Dictionary<string, JsonValue>>(json);

2. run the app
3. change net8.0 to net9.0
4. run the app

Expected behavior

The serializaiton succeeds

Actual behavior

in net9.0, the deserialization throws an exception:
System.InvalidOperationException: 'The element cannot be an object or array.'

Regression?

Yes, this works in net8 but not in net9

Known Workarounds

Deserializing to Dictionary<string, JsonNode> appears to work.

Configuration

dotnet --list-sdks
8.0.406 [C:\Program Files\dotnet\sdk]
9.0.200 [C:\Program Files\dotnet\sdk]
9.0.300-preview.0.25127.19 [C:\Program Files\dotnet\sdk]

dotnet --list-runtimes
Microsoft.AspNetCore.App 6.0.36 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 8.0.13 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 9.0.2 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.NETCore.App 6.0.36 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 8.0.13 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 9.0.2 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.WindowsDesktop.App 6.0.36 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 8.0.13 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 9.0.2 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

Other information

No response

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-system-text-json, @gregsdennis
See info in area-owners.md if you want to be subscribed.

[bbartels]

Looks like it was a deliberate breaking change:

runtime/src/libraries/System.Text.Json/src/System/Text/Json/Nodes/JsonValue.cs

Line 189 in 8a9d492

// Force usage of JsonArray and JsonObject instead of supporting those in an JsonValue.

[bbartels]

Actually, probably not deliberate. Issue introduced here here: d9b9676#diff-31898f08cdb2d11dc917abf3e66c4ac11e6ec296091977b22e22a8bb21542537R31

[eiriktsarpalis]

FYI @jeffhandley @PranavSenthilnathan