Why does it matter how we know when it is obvious we can? We pick up myriads of cues about people's sex - any combination of which can be a give away. It is why people with trans identities are rarely successful in fooling people about their sex. It is a hard thing to do.