Do you not think you are stereotyping a bit klondike? Like you I am sure I can tell a gay male either by his voice, movements or some affectation, but there are many gay men who do not fit the stereotype at all and are not in the least feminine. I have worked with a number of gay men and did not realise it until told. As far as transvestites are concerned, I am sure it is much more difficult to disguise your original sex, but as you mention, it can be done. They don't all have hairy arms, knobbly knees and prominent Adam's apples.
Mike.X