Migrating content from old forums

Firefishy · January 10, 2023, 1:58pm

Fixed:

Herrieman · January 10, 2023, 3:38pm

Quotes don’t always seem to be displayed correctly if the quotes consists of multiple lines; example.

Reason:

[quote=However, when having the quote-tags not on seperate lines]while the quote is multiple lines
this happens on Discourse[/quote]

Firefishy · January 11, 2023, 7:44pm

I think I have the quote issue now resolved. Going to start another test import shortly.

Reformatting of [quote] was disabled: discourse/base.rb at main · discourse/discourse · GitHub

Matija_Nalis · January 12, 2023, 1:04am

I’m not sure if I’m always trying at the wrong time or what (tried dozen times by now at different days/times), but I am always getting the Croatia category empty (i.e. with only one autogenerated “welcome” message).

I’ve seen @ezekielf comment that new and old categories are different, but that does not seem to be a problem. I.e. on @Firefishy test instance I see:

new Discourse category at: Hrvatska (Croatia) - OpenStreetMap Community, which has posts from production Discourse instance, so it is just fine
different Discourse category users: Croatia - OpenStreetMap Community which I assume should be the one having imports from old forum users: Croatia / OpenStreetMap Forum – but that one only has only one (auto-generated?) welcome message and never any imported messages from old forum?

Why does the import seem to dislike Croatia forum?

Firefishy · January 12, 2023, 2:40am

Likely just caught it at a bad time. Over the last 72 hours I’ve had a few test import runs, the full import run takes over 20 hours. I keep resetting the test site back to the December 2022 snapshot and restarting the importer.

Firefishy · January 12, 2023, 9:44pm

Test import is running again, it is at 6% completed. Import will likely finish in 22 hours from now.

vvvvv Test Import Site Link vvvvvvv

^^^^^^ Test Import Site Link ^^^^^^^

The content conversion is looking good. It isn’t a 100% conversion, but good enough considering the differences in markdown.

[code] and [quote] are now being converted correctly. Nesting seems to be working ok.

Pending:

There is only basic merging of users accounts (email based). Merging based on OpenStreetMap account UserID has not yet been implemented.
Imported categories are not merged with existing categories on community.openstreetmap.org. It is likely the categories will be manually merged after the final import.
The old forum.openstreetmap.org site and old forum content will redirect to the imported content items on community.openstreetmap.org, the import scripts has some code for creating the permalinks but it has not yet been tested and is likely to requires further work.
User Avatar image importer still needs to be implemented.
Anything critical missing?

wolfgang8 · January 14, 2023, 8:35am

Looks pretty good to me!

One thing I have noticed - I can’t find this topic in the old forum - maybe it’s hidden there?

whb · January 14, 2023, 12:06pm

It is here: Wochennotiz Nr. 265 11.8.–17.8.2015 - Deutschland (Germany) - OpenStreetMap Community Forum

Wynndale · January 14, 2023, 2:16pm

What happens to forums currently without an equivalent here? Does it depend whether people ask to set one up?

Firefishy · January 15, 2023, 2:49am

All imported content will be kept (except obvious spam). Some content may be moved as appropriate.

I think the plan is for regional categories to all be kept together.

Firefishy · January 15, 2023, 3:13am

All the test imported content users: Croatia - OpenStreetMap Community seems to be there now. Can you confirm?

chris66 · January 15, 2023, 6:07am

So eg. the old german content will be imported into the new german forum or will there be a germany-old sub-forum?

Nadjita · January 15, 2023, 6:54am

I noticed that for some users (including me), the avatar is empty, and their username has a number attached, but when people quote their postings, it’s correct:

https://forum-import-test.openstreetmap.org/t/sidewalk-vs-separater-fussweg/79858/71 and next post

Or see this screenie:

I just wonder if that’s something to worry about.

whb · January 15, 2023, 7:42am

This is because the e-mail address that was stored in the old forum does not match the e-mail address of your OSM user account. The assignment is made via the e-mail address.

I think it should help if you change or enter the e-mail address in the old forum accordingly. But not until a new, up-to-date snapshot of the old forum is used.

Nadjita · January 15, 2023, 8:02am

And why does the assignment work when a message is quoted? Sorry, if it didn’t come across, but that’s my actual question

whb · January 15, 2023, 8:13am

Because there is no assignment in this case, it’s a text-only conversion (actually none at all).

whb · January 15, 2023, 8:30am

Old forum:

[quote=Nadjita]ist highway=crossing vermutlich das bessere Tagging[/quote]als way oder als node?

Conversion to Discourse:

[quote="Nadjita"]
ist highway=crossing vermutlich das bessere Tagging
[/quote]
als way oder als node?

So no username mapping is done here, just some formatting.

Firefishy · January 16, 2023, 4:40pm

This week I am working on resolving the user account merging / mapping.

Matija_Nalis · January 18, 2023, 2:08am

Yes, data for Croatia seems to be imported now (so it was just my bad timings it seems).

On initial look it seems mostly fine so far.
I’ve only noticed that post signatures are not copied in import; is that by design or omission?

e.g. Novi službeni kanal za komunikaciju - Hrvatska (Croatia) - OpenStreetMap Community Forum vs. https://forum-import-test.openstreetmap.org/t/novi-sluzbeni-kanal-za-komunikaciju/94616

Firefishy · January 18, 2023, 3:20am

Yes, the user signatures are not currently imported. Discourse doesn’t support signatures from what I can see, but does support a User BIO. The importing of the signatures → Bio could be tricky.

~10% of the old forum users have signatures. At very least 50% are spam.

Import FluxBB User Signatures → Discourse User BIO
Ignore FluxBB User Signatures. Don’t Delay with Import

0 voters