Georeferencing oblique aerial imagery

This is not strictly related to OpenStreetMap so I’m posting it in general chat, hope that’s OK. Please feel free to move it.

I am interested in learning how oblique aerial imagery could be processed and overlaid against vector map data, such as OSM.
I don’t mean orthorectified as I know that oblique imagery is not very suitable for orthorectification, rather I mean how could it be georeferenced and examined against a digital elevation model.

Does anyone have any ideas or knowledge on this front?