This is a known and documented limitation of Burst. Burst only supports constant strings as exception messages currently, as documented here: https://docs.unity3d.com/Packages/com.unity.burst@1.8/manual/csharp-language-support.html#supported-c-features-in-hpc
For more details on when string formatting is supported see: https://docs.unity3d.com/Packages/com.unity.burst@1.8/manual/csharp-string-support.html
As for debugging, you should be able to set a breakpoint at the offending code, and you'll be able to debug the managed version of the function. Alternatively you can use Debug.Log from Burst.