In my previous answer I assumed that you’d gathered the coordinates by tracking them or marking them as a waypoint in your GPS. The best thing to do in that case is to load the tracks or waypoints into JOSM and add nodes to the spots you want to mark as a city or town.
Same method works when you’ve written the coordinates down. If you’ve gathered the info from some geo source (make sure the licence is compatible with OSM) you might be better of writing a script that translates the format used by that geo source to the OSM format and upload using JOSM.
For an even better answer I really need to know in what format your POI’s are, so please elaborate.