Hello everyone,

I’m aware that a some people have raised this question in different channels and also through some consultations done in the “technical beta” we run last year.

Let’s have a space here to go through the current situation and allow everyone to share their thinking. We understand that there is not a decision that will make everyone happy, but at least we want to make sure that most people is happy and also be fair with the technical and people/time constrains to execute any solution.

I would suggest we get out of the scope of this conversation the Help OSM site, which user data base is not connected with OSM accounts and the software doesn’t have an easy way to extract the data into a discourse forum.

Let’s talk about forum.openstreemap.org since it’s the other site we know we need to retire, as explained in the next steps post.

1. Agreeing on what are we trying to solve

These are some of the asks/needs people have been sharing (please add missing ones on the comments below):

• We would like be able to reference old topics in the future.
• We would like the existing urls to point to the old topics, so we don’t break many places where they are linked.
• We would like to be able to search through old content.
• We would like to talk again about discussions that happened in the past.
• We have invested a lot of time producing content and we don’t want that to disappear.

2. Understanding what are our options and their implications

Note there is no option where old data is not preserved.

Option 1: We migrate all content from the old forums

* Note that we don’t know for sure if the existing 2016 script for migration is still working with the most recent Discourse versions.

Pros:

• All content is searchable from Discourse.
• Old content might (or might not) be linked to people’s accounts.
• People will be able to continue the conversation of old topic on the topic itself.

Cons:

• Some users might not match with Discourse users or we’ll encounter conflicts due different DB restrictions. Last tests we run encountered too many issues linking users and orphaned topics.
• Content encoding might end up different.
• URLs will be broken, since we don’t have a system to match old urls with the topics imported here.
• We’ll tremendously increase the discourse DB and maintenance in this early phase. We can’t anticipate other unknown risks.
• Search engines will have to re-index content and we might get a penalty for duplicated content.
• Time investment can vary from high to high-unknown: We’ll require from an Ops person to handle the whole process. AFAIK, currently we don’t have anyone with this kind of bandwidth.
• There is no guarantee all content and users will end up looking correctly after the migration.

Option 2: We archive the old forums into a static site

Pros:

• URLs can easily be matched for redirects to the archive: Old links will keep working.
• The process is way lighter in Ops time.
• The content is preserved as it was posted.
• The content is searchable and accessible from search engines.
• There are no compromises to Discourse DB or its maintenance.

Cons:

• New discussions here about old topics need to link the archived conversation for reference.
• Users here don’t have their old posts linked to their account.
• Search will need to be done through a search engine.

3. Questions for everyone

• What are we missing from the things we are trying to solve? If something missing, why it’s important for you?
• What’s missing from the pros and cons on the options listed?
• Are there other alternative options you can think of? What are their pros and cons?

We’ll keep this first post as a living document and it will be updated it with the discussion happening below.

This proposal will be open for input and feedback until March 29th.

After that date the @forums-governance team will take in consideration all the input and take a decision.

Please note that we all understand how important is to preserve historical content, discussions and decisions. Let’s keep a constructive tone and open mind here, understanding how everyone feels and what resources we have at our disposal to solve this issue.

Thanks!

Update: Decision made

I just checked and there are only 435 users in the forum database that don’t seem to have a numeric OSM ID associated with them - the other 33773 do have an OSM ID so could be matched to discourse users at least in theory.

Can we launch a poll?

The intention for this topic is to open a discussion, get to agreements and find a balanced proposal. The final decision makers for Discourse changes are currently described here, although we want to grow this group to be more representative.

Final decision makers will also seek general agreement and balance a solution that fits to most people in the community.

The basic issue is

Germany     322'015
Russia      266'242
Netherlands  66'060
Poland       58'439
Ukraine      16'520
...

posts in the 5 largest fora (thanks to whb for compiling the numbers) clearly for the largest a substantial part of the communities history. The questions need to be put to the communities affected, not to those that have never used the old system, don’t care or even hate it and once the initial excitement has died down are unlikely to use this platform.

Hi @SimonPoole, yes, that’s the goal here, but using the word “hate” doesn’t help to have a calmed discussion, there is no need for that.

There are a few individuals that have volunteered to engage with users from the old forums and let them know about this conversation to engage.

Why the conversation is hosted here? Because any migration will also affect also to the people using this forum, and we see this place as a nexus between forum, mailing list, and other OSM channels users.

FluxBB is a piece of software not a person, lots of people don’t like it and that is completely OK.

And yes 99% of the current problems are caused by massive scope creep. The problem to be solved was two pieces of software that need replacement, if all activities could be limited to that for say the next year or two, I assure you, more will actually get done.

No problem, it was just a suggestion.

Discourse contains a redirection mecanism to handle that. I’m not sure if it works 100% but it’s worth looking at it.

Was this including a test migration from fluxBB ?
The result of such a test may give the answer or help to choose between the options.

The volume of accounts (34k), topic (52k), posts (832k) should not be a problem by itself, or meaning that Discourse cannot handle a large volume forum (I don’t think is the case) which would be a problem in the long term.
Additional maintenance ? I don’t see the reason for it, there is just additional work at start for the migration itself (that’s my past experience of migrating our phpBB to Discourse for OSM France, see below).

Feedback about OSM France migration from phpBB to Discourse

• I did the migration a little more than a year ago.
• I used the phpBB migration script available on Discourse meta forum.
• It took me less than a day to do the migration, most of it to understand the process (I did it several times on an empty Discourse instance)
• The migration kept a part of the accounts, where the email address was usable, lost “anonymous” accounts (831 out of 2700) which have been create as disabled in Discourse (I presume they were very old accounts as nobody complained that its account was disabled).
• The old topic URL have not been maintained, nobody ever complained
• We fully replaced phpBB by Discourse on the same domain name forum.openstreetmap.fr, they never ran in parallel.
• The forum is now much more active than before the migration surely because it has been seen as software change, not a communication channel change (same content, same account, same base URL).

Yes, based on my experience I really prefer option 1.
It’s more work at start but worth it from my point of view to make sure the community migrates too.

Has anyone started posting on the OSM forum to talk to the people there, and to really engage with the OSMers who have found their community there? (since the forum software doesn’t support my username, I cannot do it)

(If you’ve done that, then I ask you post those links here . It could be a useful resource for others on this forum to refute claims that “oh you never talked to anyone”)

I have posted to the OSM talk@ list: [OSM-talk] New email capable forum for OSM: https://community.openstreetmap.org/

I personally prefer option 1.
This will make it easier to search for a topic that has already been discussed (and resolved) on a single interface. If the archives are located on another interface, no one will go and look at them (especially the newbies). As for broken URL, OSM has been around for 15 years and I’m convinced that there are already broken links, because many tools or sites have disappeared.

I would prefer option 1 to keep the content alive if and only if a mechanism can be found to preserve the links from the wiki into the forum.

It has been something between good practice and a requirement to back up changes to the wiki with links to discussions. Ripping all that context away from the Wiki would cause serious damage.

Mechanisms might either be a discourse mapping mechanism mentioned further up or a bot rewriting all forum links in the wiki.

If this should not be possible, it’s definitiely option 2 to preserve the links and the wiki context.

Yes, but AFAIK that is used to handle redirects FROM discourse URLs. Right now there is nothing to help us create and correctly map to the final discourse destination hundreds of thousands of URL redirects from fluxbb.

Also note that I’m not sure if the migration script for fluxbb is able to import into an existing discourse instance or it needs an empty one.

Additionally, let’s not underestimate the effort to migrate almost a million posts to a new DB and the potential issues that can turn out in the future.

I understand that it’s easy to say “let’s migrate even if it’s complex” but think that it’s other people’s time that will need to be invested. We don’t know if it’s going to take weeks or months to have it done the right way, and there is no guarantee we’ll find someone to commit to this if the alternative is simpler and solves most of our problems.

I would recommend to agree on what are we trying to solve first and then see if there is an option that can solve most of our problems.

This is handled by the permalinks mechanism that is built-in Discourse.

The phpBB migration scripts uses this feature:

It creates permalinks using old URL points to the new ones for categories, topics and even posts.

Before closing the door, it’s worth at least to try such a migration.
Of course it takes some people time, maybe it’s time to ask who is willing to spend some time on this ?

Guess what ? I may find some time for that

Do we know how adding almost 1M redirects affects the site performance?

I would keep recommending to check if we agree on the list of problems before jumping into any solution. Might be the case that there is an additional option we haven’t thought about that doesn’t introduce high risks and solves most of the problems.

Always keeping in mind that any solution will come with trade-offs.

First of all, it’s a bit strange for me to need to discuss about the future of the huge amount of mapping knowledge from the OSM forum in a different forum, doesn’t leave an impression, you actually want a feedback. But ok.

In my understanding it’s a must have, to migrate all the threads, which have been active in the last months, in order to keep ongoing discussions alive. This will lead to a huge amount of frustration of those users.

For all the history, I think it would be beneficial to import it as well to have a common database for searching all the previous and future knowledge.

You probably did not intend it this way, but to me, coming from a board member, this comes across as a bit of a strange (and rude) reason not to want to engage with such a large segment of the OSM community.

I’ve opened a topic on the Dutch subforum. (Anyone looking in: feel free to post in English there.)

The Mailing list discussion mentions migrating the help site content too. Is this a completely different set of discussions or meant to be in this thread too?

I think they’re completely different software?

Yes, the current plans don’t seem to include a content migration of the help site (but then the plans didn’t contain a plan for migrating the forums either), technically it would be possible but probably a bit of work.

The other aspect is that the help site contains/contained a lot of question from people that didn’t create an OSM account and by the very nature of their interaction were only there to ask the question. I don’t believe the current setup of this site would support that nor that it is even something that can be supported in a reasonable fashion in an “integrated site” (there are a lot of things to be considered, not the least legal issues).

I want to engage with people on a forum. It’s just that I can’t.