Inactive spawned NetworkObject is not destroyed when the NetworkManager shuts down

How to reproduce:

1. Open the attached "IN-154021" project
2. Open the Assets > "SampleScene" scene
3. Enter Play mode
4. In the Game view, click the "1. Start Host" button
5. Click the "2. Spawn" button
6. Click the "3. SetActive(false)" button
7. Click the "4. Shutdown" button
8. Observe the Hierarchy

Actual result: The inactive "Cube(Clone)" GameObject stays in the scene after the shutdown
Expected result: The "Cube(Clone)" GameObject is destroyed on shutdown, the same as when it is active

Reproducible with: 1.8.1 (6000.0.0f1), 3.1.0 (6000.7.0b5, 7000.0.0a8)

Reproducible on: Windows 11 (25H2), macOS 14.2.1 (by user)
Not reproducible on: No other environments tested