The tiles I see are coming in fast, but that does not mean anything.

It depends on which tile server you are using (you have to check your network traffic for that).
So I’m surprised you say “no matter what site”. Did you try the other layers on osm.org? Did you try some alternative layers on http://gk.historic.place/historische_objekte/l/en/index.html ? Are they all slow?

Those layers have different providers, so are served from different hosts. If they are all slow, I assume the problem is with your computer or internet provider.