I’ve never done anything this low level, but would a software library like GDAL be able to get the “normal” coastline ShapeFiles into a suitable SQLite database without a GUI? I think at its most basic it exists mostly as a set of command line tools.
Do you need additional information that the ShapeFiles just don’t have?
I don’t have a firm grasp on how big a set of all the coastlines is in one of the native OSM formats. The limit on the “main” Overpass instance is 1GB/day. I’m not sure if it could even handle that in one go. The recommendation for large amounts of data is usually to download a copy of the planet file (or a regional extract) and run your extract locally using something like osmosis. My gut feeling is that if you can find some way of using the existing coastline extracts that’s probably going to be easiest.