Prefab previews are dark in HDRP when using defaultHDMaterial

Reproduction steps:
1. Create a new HDRP project
2. Create a Sphere and drag it to the project window to create a Prefab out of it
3. Select the Prefab in the project window
4. Resize the Prefab preview in the inspector (you can do this by dragging the bar named "Sphere" in the bottom of the inspector up and down)

Expected result: The Prefab preview is light-gray in the project window and the inspector.
Actual result: The Prefab preview in the project window is dark. In the inspector, it flickers between dark and light-gray.

Reproducible with: 2021.3.27f1, 2022.3.1f1, 2023.1.0f1, 2023.2.0a18

Reproducible on: macOS 13.2.1, 13.4 (Intel), Windows 10

Note: might need to click outside of the GameObject selection for the preview to update

Note: this only seems to happen for defaultHDMaterial