Discourse database dump - is it published?

Is it possible obtain archive for own processing/backup?

Would it be possible to start publishing it?

1 Like

You can request a read only API key on your own and extract whatever you want with it.

Getting an API key - #15 by mmd

I don’t see how an API key helps? The API methods relating to backups appear (as I would expect) to be restricted to administrators and we certainly don’t encourage bulk scraping of the API as a way of performing a dump for any of our services.

Does discourse even have a way of creating a dump that doesn’t contain personal data?

The idea was to selectively fetch those posts, which are of interest, rather than fetching the whole site, or triggering a backup.

Unfortunately, OP provided no details, why such a dump would be needed in the first place. We never provided such a dump for the old forum, or Help OSM. Wiki being the only exception here.

Even if the dump doesn’t include personal data, it might include sensitive configuration settings such as OAuth credentials.

I will not be publishing the discouse database dumps / backups. The backups created by discouse are raw SQL backups.

The dumps contain private data: Direct Messages, Email, etc.

2 Likes

It would be useful at least for proper search (Discourse breaks browser search and replacement it provides is horrible), downloadable archive of public postings would allow this.

Though I have not needed it so far, as forum is not old enough for me to try finding something.

Trigger for posting this thread was Proposed features/Require proposal announcements to be made on the new forum instead of the mailing list - OpenStreetMap Wiki (search for “self-hosted with OSM account log in”)

2 Likes

Hmm, Discourse supports mailing list mode, and I think for some (sub)categories, you could probably set up a mailing list mirror, like The rails-dev Archives is set up as a mirror for openstreetmap-website on Github.

4 Likes

By the way, it’s somewhat annoying to see that some people on that wiki page quote us out of context and assume that we don’t care about their matter („some of the replies really show just how much (little) people care about this“). It’s beyond me how you would come to such a conclusion.

Is it feasible to publish more custom dumps with only public data?

It is done with the worst possible interpretation, but note that current answer is that no database dumps will become available (I also posted reply to them there).

Thanks. We’re talking about database dumps here, but I guess we should take a step back and see what issues we’re trying to solve here, and then think again about possible solutions. I have some doubts that mailing list users will be eager to work with such a dump.

2 Likes

I am already fetching mailing list as plaintext files - this allows me to grep across them ( GitHub - matkoniecz/mailing-list-downloader: Generic downloading of mailing list archives [mailing lists][tagging] )

I would be happy to start doing it with community.openstreetmap.org if possible without highly elaborate processes

Do you have copyright clearance for that? I mean, individual comments are property of their respective authors. I’ve seen forums that contain a legal waiver that grants the operator copyleft of the text, but I don’t see anything like this here.

P.S.: I recall using the new forum like ages ago, but it created a new account for me. Is it garbage collecting accounts if you don’t log in for some time?

1 Like

That’s weird, maybe it was during the initial testing phase and users were cleaned out.