I’ve never tried something linke this but this is what I found:
openrailwaymap seems to be using leaflet, wich seems to be a javasript library for creating maps based on openstreetmap data. That sound like the thing you asked for.
There is maptiler, a vector map service. In the bottom left corner there is a “Make Your Own Map”-link, that requires a login. Maybe you don’t need to set up your own server but instead just use their options. I don’t know what exactly can be done with maptiler. Here you can find more information about them.
MapTiler’s vector tiles conform to the OpenMapTiles schema, which includes major railroad tracks from zoom level 8 and minor tracks from zoom level 10. However, it doesn’t expose the operator key at all. Alternative vector tile generators such as Planetiler, Martin, Tilemaker, and Tegola allow you to define your own tile schema.