You have defended your policy pretty reasonably and admirably, but
you still cannot justify casting ALL MEN as sexually interested in the women in your group. That would be no different than if I made the claim that women were "always trying to pick up men", which would probably make women angry and justifiably so.
Would you have a problem with a whites-only group who excluded blacks because they were afraid of a stereotype? Or what about a black-only group who excluded whites because of a stereotype? Or what about a group that excluded Jewish pipe smokers because of a stereotype?
And "birds of a feather flock together" isn't what it says on your website. It says:
You will feel comfortable at LOTB. No worries about being ridiculed or called names. No worries about receiving unwanted solicitation. We do not allow men to join LOTB so please do not request membership if you are a male. All membership requests are carefully screened and we will deny those that we feel are inappropriate for our club. Those few that happen to get past our screening will eventually be found out and banned.
If I created a club like yours banning women, using that exact same language, I would be on the front page of USA Today and FoxNews as the biggest sexist in the history of sexists.