Loading geocaches onto an OSM map

I’m looking for ideas here. I want to make a Garmin .img file based on OSM which has a stack of geocaches pre-loaded. This file can then be left on an SD card and put in caches.

Now loading all of Australia into OSM then loading a cache .gpx file is beyond what current computer’s can handle in terms of memory usage, anyone got any ideas?

Another idea would be to append data directly to the XML data, anyone know how this could be done in a way that mkgmap would know about? Or how to customise mkgmap appropriately?