The Animator does a pattern-matching pass during SetupAvatarDataSet to identify the root of the animated hierarchy.
This operation does a string-based pattern matching, which scales on the number of transforms under the GameObject which holds the Animator Component.
As evidenced in this specific case, having a large number of root GameObjects under the Animator GameObject results in very long calculations.
In the case of very large hierarchies, where a significant portion of the hierarchy is not animated, it is better to move the Animator component at the root of the animated hierarchy to avoid having the Animator take into account irrelevant objects.