UnityEditor.CoreModule.pdb supplied with Unity is either corrupted or its format does not fit the specification

Reproduction steps:
1. Open a new or an existing project
2. Make sure Rider 2021.1 is selected as the External Script Editor in Preferences
3. Make sure Rider is running in Internal Mode (instructions: https://rider-support.jetbrains.com/hc/en-us/articles/207327910-How-to-run-JetBrains-Rider-in-Internal-Mode)
4. Open a new or an existing script in the project
5. Through Rider, attach the Debugger to the running Unity process

Expected result: No errors occur
Actual result: IDE error occurs with the exception: "Failed to ReadPdb from stream Index was outside the bounds of the array."

Reproducible with: Rider 3.0.3 - 3.0.6 (2021.2.0a1, 2021.2.0a17)
Not reproducible with: Rider 3.0.3 - 3.0.6 (2019.4.26f1, 2020.3.9f1, 2021.1.8f1)
Could not test with: 2018.4.35f1 (Due to the Rider package not being available on this version)