Calling SetParent() will inherit any scale changes from the ancestors on your object.
So either you need to make sure all ancestors have a local scale of (1,1,1) or use the SetParent() function with the argument worldPositionStays=true and manually adjust the position afterward. From the docs about worldPositionStays: "If true, the parent-relative position, scale, and rotation are modified such that the object keeps the same world space position, rotation, and scale as before."
See the documentation here: https://docs.unity3d.com/ScriptReference/Transform.SetParent.html