Ah never mind, I think I’ve answered my own question. Just noticed this part of the Wiki page:
It can be useful to map entrances or even areas of a toilet separately.
So I guess I should map separate entrances and put the appropriate male/female/unisex tags on the entrances. The toilet building itself would then not have these tags at all?
The accessible bathroom is ment for anyone in a wheelchair or with similarly limited mobility. That tagging should at least wheelchair=designated + unisex=yes. The male and females bathrooms are tagged normally.