Steps to reproduce:
1) Create an Empty scene or open attached project
2) Make sure VT is enabled, go to HDRP asset and at the bottom tweak GPU Cache Size per Format (could be something small incrementally, like 64 -> 65)
3) Observe that viewport starts to slow down, tweak Cache size more, wait a minute, observe that Viewport becomes unusable
Notes: issue is gone after a domain reload.
If you check Task Manager before tweaks, notice that Unity eats ~3% of CPU on idle, after cache tweaks it starts to eat ~12%