Howto OpenLayers and Markers with Labels?

Thanks to the Openstreetmap Wiki and Openlayers Documentation I know how to add markers in different ways and also with popups (also how to autoload the popups on loading the map).

But I like to have markers (source is a database) with “normal” labels (just some text beyond the marker icon without the need to click the icon to display the text).
I didn’t find an example on how to achieve this.

Does somebody know an easy way on how to do this?

Thanks for your help

Maybe something like this example:

Or other examples here, might be something helpful: