How to map building that have missing levels? For instance, if a building deleted all the level 4,14,24, etc. and also level 13, then how should it be mapped?
For building:levels, it doesn’t make sense to use nominal floor number, for instance it doesn’t make sense to map a 20 floor high building as building:levels=23 jist because it skipped 4/13/14 and thus used 23/F as its highest floor. Meanwhile it doesn’t make sense to map interior facilities according to their actual level either, as users would not expect to find a 23/F feature on the 20th floor in OSM. In this case the feature would be marked as level=23 in a building:levels=20 building, and one would naturally think the feature is above the building. Is there any tags that can be used to prevent such misunderstanding?
As I understand it, building:levels is the number of levels, not the index of the highest level. In your example, level=23 in a building:levels=20 building does not bother me. For example, in France and other countries, the ground level is level 0, therefore building:levels should be 1 + the index of the highest level.
I think the level tag, when possible, should contain the level index as indicated in the building.
Based on what I understand, you use level:ref=*
I could certainly be wrong, but I think this is how it works. For example, let’s take a building that has six levels, but does not have a floor number “4”. Level 0 would be the ground floor. The rest of the floors would have the following tags:
level=1
level:ref=1
level=2
level:ref=2
level=3
level:ref=3
level=4
level:ref=5
level=5
level:ref=6
level=6
level:ref=7
Amendment to my last post, this would be the numbering for a building with 7 levels 
I see. I guess when mapping one can simply use level:ref and then add the level tag back by using filter and select all elements in the area with JOSM
Here is an example of level:ref being used:
https://openstreetmap.cz/edit/#background=Bing&level=-1&map=20.00/-104.98487/39.73926
building:levels is not an indoor mapping tag, it is simply an overall count for S3DB modelling purposes.
SIT has a special tag for missing levels, see https://wiki.openstreetmap.org/wiki/Simple_Indoor_Tagging
I would use “level” to map the level as indicated locally, with a number. You can still use level:ref for other indications (e.g. in Germany, level=0 would typically corrispond to level:ref=EG).
The whole building:levels idea is only working for buildings with the same or similar height between floors, no split level, ground around the building at the same height, no inclined levels, etc. Because otherwise it isn’t clear which is the ground level, which levels count, how can you tell whether a floor is part of the roof, etc.
This all relies on definitions, which in OSM are mostly amiss.
Did you already discover building:min_level? That’s another tag you need to understand the building:levels value.
I agree that most of the things you mentioned aren’t sufficiently specified yet, but this one is actually defined on the wiki. See the example image at Key:building:levels.
Also, levels with different height can be expressed if you use Simple Indoor Tagging’s level polygons, which can carry a height tag. (This requires creating an area element for each level, though, and is therefore quite a lot of work.)
I would not call this a definition, at most it is a hint that the lowest floor with access to the outside might be the ground floor, but this picture still leaves a lot of questions unanswered with regard to actual real life situations.
Here is in image that helps explain how level=ref is used:
https://wiki.openstreetmap.org/wiki/File:Level_ref_example.jpg
I wouldn’t call this a definition; at most, it’s a hint that the bottom floor might be the lowest floor having access to the outside, but this picture still raises many concerns about situations that might actually occur in real life.