We have confirmed this issue. The size increase of the asset is fairly small, and we do not see frequently installing and uninstalling the package as a common use case. For this reason, we will be closing this issue as "Won't Fix".
Reproduction steps:
1. Open the attached project "ReproProj"
2. Go to Assets > XR > Settings in File Explorer
3. Observe the size of OpenXR Package Settings.asset
4. Remove the “Vive OpenXR Plugin - Android” in Package Manager
5. Select My Registries and install “Vive OpenXR Plugin - Android”
6. Repeat step 3
Expected result: Size did not increase
Actual result: Size increased