The problem is caused by the fact that that, at the moment, the Overpass API areas are quite a bit out of sync from the rest of the OSM data (normally the area data is only lagging behind the rest for a couple hours):
To explain it a bit more detail: Overpass thinks that some short pieces of the border are actually “inside” of Poland (example: overpass turbo) – these ways have been edited some time after the Overpass-internal area of Poland was last generated. In this case, this problem can be worked around by manually filtering the border lines again and excluding any lines that are also member of the relation of the original search area: overpass-turbo.eu/s/23KJ. Ideally, this would be fixed on the Overpass API servers, though.