I mean, can I create custom directory and make JOSM keep all data and settings within that directory, so I can easily backup or move when needed?
See this page for where JOSM stores the preferences: http://josm.openstreetmap.de/wiki/Help/ResetPreferences
As it says, you can set the preferences folder by adding -Djosm.home= to the JOSM command.
eg something like this:
java -Xmx1000m -Djosm.home=myfolder/josm-settings -jar josm-latest.jar
Thanks. It works. Unfortunately, it now opens two application windows, one for java and other for JOSM.