The genital mutilation caused a unique physical appearance too. An excerpt from the USRF article:
The manner in which the castrati appeared to their audiences can be judged from our clinical experience of eunuchoidism due to spontaneous primary hypogonadism. A tallness of stature, which was unusual in the 18th century, was commented upon by contemporary writers and was due to failure of the epiphyses to close at puberty, thereby allowing the unopposed action of growth hormone and other growth factors. There was a smooth pale skin, with, later in life, fine wrinkles around the eyes, no beard, plentiful scalp hair, a tendency to obesity, rounding of the hips, and narrowness of the shoulders; the pitch of the speaking voice was similar to that of a female.
I did find some disparities between what I read online and what was talked about in the play, so I guess take I have to everything with a grain of salt, unless I decide to make a trip to the library.