Enter/Exit Play mode loading times are a lot longer when a lot of colliders are packed into a Prefab

Reproduction steps:
1. Open the attached “BugRepro” project
2. Open the “PackedScene” scene
3. Enter/Exit the Play mode
4. Observe the loading time values in the Console
5. Open the “UnpackedScene” scene
6. Enter/Exit the Play mode
7. Observe the loading time values in the Console and compare them to the previous values

Expected result: Loading time values in steps 4 and 7 should be approximately the same
Actual result: Loading time values in step 4 are a lot bigger than in step 7

Reproducible with: 2021.3.33f1, 2022.3.14f1, 2023.2.1f1, 2023.3.0a16

Reproducible on: Windows 11
Not reproducible on: No other environment tested

Notes:

• The difference in loading times is hardware dependant
• If Reload Scene is disabled (Edit > Project settings… > Editor > Reload Scene), entering Play mode loading time for both, packed and unpacked colliders, is a lot longer