The boundary files that you find at geofabrik are used by themselves too as far as i know. They are not always as precise as the borders described by osm. Of course you cannot use them for towns as they describe countries.