OSMF should host a Forgejo/GitLab instance

contrapunctus · 2024-05-02T21:57:37.029Z

Codeberg.org, mentioned in my first post, is exactly what you describe. But people are reluctant to move there, too. So I hoped an OSMF-run forge would have more trust and the OSM SSO would remove the need to create new accounts.

Re: mirroring - as mentioned earlier, that’s my nuclear option. If upstream developers aren’t interested in moving off GitHub, I can fork repositories to Codeberg (including issues, PRs, wiki, etc) and contribute there. I doubt they’ll care enough to merge changes, which means we have an easily-prevented split in efforts. The repositories themselves could technically be kept in sync, but the non-Git data will drift (there’s no synchronization mechanism for that, AFAIK).

ZeLonewolf · 2024-05-02T21:58:50.177Z

It seems pretty clear that OSMF doesn’t have the appetite to do this (because github exists), and you’ve managed to step off on the wrong foot by insulting people that disagree with you. And there doesn’t seem to be anyone willing to donate servers for this project.

Sooooo it’s starting to sound like Sisyphus pushing the rock up the hill, and I would agree with your assessment that you’re at an impasse.

Minh_Nguyen · 2024-05-02T23:35:42.741Z

contrapunctus:

Re: mirroring - as mentioned earlier, that’s my nuclear option. If upstream developers aren’t interested in moving off GitHub, I can fork repositories to Codeberg (including issues, PRs, wiki, etc) and contribute there. I doubt they’ll care enough to merge changes, which means we have an easily-prevented split in efforts. The repositories themselves could technically be kept in sync, but the non-Git data will drift (there’s no synchronization mechanism for that, AFAIK).

Folks around here are quite familiar with this scenario, actually:

The Next Generation of OpenStreetMap — in Python! General talk

Greetings, OSM Community Today marks the unveiling of my most ambitious project yet: OpenStreetMap-NG. Nearly a month in the making, I am super excited to bring this to light. This project is a transformative rewrite of the foundational OSM infrastructure1,2 to Python. Its goal? To simplify contributions, secure against vandalism, and modernize the API, all while remaining backwards compatible. (Some) Highlight New Features 100% Python; Move away from the complexities of Ruby, …

Unfortunately, as you might’ve guessed by now: GitHub.

NorthCrab · 2024-05-03T02:20:00.830Z

I am personally all for free git alternatives. Unfortunately, today’s reality is that GitHub has the widest contributor audience. You can compare how well different projects do across different platforms. On GitHub, you will always find more reported bug reports and more contributions - which really matters for this kind of project. For private/small projects, it makes sense to host them on alternatives. For more substantial projects, and especially those that are made by the community - GitHub is the only sensible choice today.

contrapunctus · 2024-05-03T05:06:17.685Z

…and Google Maps is “the only sensible choice today” in India. The POI, bus route, and 3D building data data is vastly more complete and detailed, and there’s real-time traffic data, PT timetables, etc. Yet, us free software advocates avoid it and spend countless hours contributing to OSM, organizing mapping parties, designing promotional material, and conducting advocacy at conferences instead. Should we have done the same in this case, and contributed to Google Maps instead?

There are times when we have to do better than swallowing the bait used by proprietary platforms, and look at the bigger picture…no matter the network effect and no matter how much the world screams at us to do the opposite.

A large project like OSM moving to a free platform would in fact be a big boost for free platforms. It would bring more contributors to these forges, and maybe even inspire other projects to consider following suit.

NorthCrab:

You can compare how well different projects do across different platforms.

If others are also applying your reasoning, and thus only small projects are moving to other hosts…they would naturally not receive too many issues and PRs on either platform. Does your comparison account for that?

I also don’t see anyone questioning the popular wisdom of “if it’s not Github, there will be fewer contributors”. It is my strong suspicion that the platform doesn’t affect contributions at all - people contribute to OSM software not because they found it on GitHub, but because they are invested in OSM in some way other the other. They would contribute no matter what platform it’s on.

SimonPoole · 2024-05-03T08:27:27.484Z

While I agree that the dev machine is not ideal for non-experimental projects, on the other hand there is a long tradition of services being run on it that are considered essential by large parts of the community. At the same time there is no concept from the ops side of things of moving such services to a different, permanent, home, or even taking over such services, so I guess it is what it is.

Further we have a volunteer that has offered to do the work to support the service, so I don’t quite see what the workload issue is supposed to be (at least not different than for anything else running there). I guess it would be nice if the OSMF provided a location to store backups, but that can be worked around too.

Back to the topic at hand: there are plenty of OSM-specific projects that are not really looking for external developer contributions, take for example the editor layer index, NSI and so on. These could easily (modulo CI) be hosted on such a service without loss of potential contributors, it could actually open things up the other way around.

The 2nd realistic use case (as I already pointed out) is to provide a secondary issue tracker for projects (likely in conjunction with a mirror of the github/gitlab repo) that can be accessed with the users OSM login. Obviously there is no guarantee that any projects will take up such an offer, but it would seem really low hanging fruit to provide.

Minh_Nguyen · 2024-05-03T15:01:14.082Z

SimonPoole:

The 2nd realistic use case (as I already pointed out) is to provide a secondary issue tracker for projects (likely in conjunction with a mirror of the github/gitlab repo) that can be accessed with the users OSM login. Obviously there is no guarantee that any projects will take up such an offer, but it would seem really low hanging fruit to provide.

I wonder if this could just be a forum category that has the Solution button enabled and maybe auto-closing threads after some period of inactivity. I’ve participated in some less tech-focused open source projects that already use Discourse as a secondary feedback channel. For example, Gramps has an intake category so that their users, who are mostly genealogists, don’t have to know their way around the project’s Mantis bug tracker. Mantis remains their primary bug tracker, but it’s up to the developers to curate it based on the most actionable feedback coming through Discourse.

I think this is already how some OSM projects are using chat platforms like Slack. If projects desire something closer to a chat system, Discourse has that capability too.

SomeoneElse · 2024-05-03T15:29:44.134Z

contrapunctus:

So I am, once again, seemingly at a dead end. If anybody has any suggestions, please let me know

Not really - you’re just waiting for some person or group of people who shares your view about Github with available time, energy and cash to consider sponsoring a server. I’ve no experience of hosting gitlab et al, but based on this and setting up similar “small one job” servers I’d expect the actual server requirement to be small - based on e.g. this maybe less than €10 per month. I’d expect the bigger committment by far to be keeping the server updated, and keeping on top of user requests and occasional misuse.

As I said back here there’s no reason this needs to be an OSMF badged service on OSMF hardware (at least not initially) as it’s a proof of concept. If there really is a demand for non-Github hosting, it may well be a success. If there isn’t, it probably won’t be.

SimonPoole · 2024-05-04T07:41:14.099Z

Minh_Nguyen:

I wonder if this could just be a forum category t

At least in my case users are free to open a topic on the forum and in the past have done that many times and they will get an answer if appropriate, however that is not a replacement for working through an user detected/experienced issue in a structured fashion. Some projects use github discussions for the unstructured bit BTW.

IMHO such a process is far too noisy for a general purpose forum and far too susceptible to noise from otherwise uninvolved bystanders. Not to mention other issues like closed threads remaining visible.

SomeoneElse · 2024-05-04T08:28:38.575Z

SimonPoole:

Minh_Nguyen:

I wonder if this could just be a forum category t

At least in my case users are free to open a topic on the forum and in the past have done that many times and they will get an answer if appropriate

(we’re veering away from Github vs alternatives but)

It doesn’t need to be a separate category - you can use tags very much like categories - you’ll notice that that list has a “new topic” button. I’ve often directed DWG correspondents that way when they want to talk about a country that doesn’t have a category.

From the other side of things, many of the OSM-related software repositories (the website, mod_tile, etc.) get “issues” raised that aren’t software issues at all, but instead “I was trying to do something with OSM data and something went wrong - help!” - those would be far better on this forum than in Github (or any equivalent).

Minh_Nguyen · 2024-05-04T16:56:21.648Z

(Yes, I know I’m gently steering us away from window shopping for Git frontends. That’s called “taking a step back”.)

SimonPoole:

At least in my case users are free to open a topic on the forum and in the past have done that many times and they will get an answer if appropriate, however that is not a replacement for working through an user detected/experienced issue in a structured fashion. Some projects use github discussions for the unstructured bit BTW.

Right, Discourse isn’t a full replacement for a bug tracker. It’s more of a solution to the problem of people relying on proprietary chat platforms than the problem of relying on a proprietary platform for software development. But many of the arguments seem to apply equally, particularly the concern you expressed about sharing personal information or having to maintain an account on an unfamiliar service. (Discourse’s UI is also much more comprehensively localized for a global audience than GitHub’s, which isn’t saying much.)

Some projects do use GitHub discussions, but that feature tries to replicate Stack Overflow post voting just like this forum’s post voting plugin, with all the attendant confusion. I wouldn’t recommend it.

lonvia · 2024-05-05T09:10:30.517Z

I think this discussion somewhat mixes reporting problems and tracking issues. Those two things may look like they are one and the same on the surface, but this is a misconception. They are different problems and need different tools.

Most users who report problems do not have the experience to use an issue tracker in a structured way that helps developers with tracking. It is not rare that the first handful of comments is spent with a back and forth trying to understand what the actual problem may be. This is a necessary part of problem reporting but really not helpful for structured issue tracking.

Personally, I somewhat prefer when people start a discussion here in the forum when they see a problem with Nominatim. It’s prefect for asking clarifying questions. There are lots of knowledgeable people here, who can help with background information on OSM or help tracking down the OSM data that causes the problem. And there are regulars here who are happy to repeat answers to the questions that get asked again and again. I can’t emphasize enough how helpful this is for me as a maintainer. So, when it comes to reporting problems, this forum is just a fine tool for it and it ticks all the boxes on privacy (as far as a public venue can do that).

Issue tracking is a different beast. It has to work for the maintainers and developers. Github currently does the job, not perfectly but well enough. Access to the CI and the ability to communicate across project boundaries are a huge win. Most of the privacy concerns don’t really apply because I do open source development only. Everything is in the open for everyone to see: code, issues, comments. That is a conscious choice. There is also no real lock-in. If you are worried that some information only lives on Github, then there is always the option to mirror/backup the important stuff.

I’m not naive. I understand that there is no such thing as a free lunch, also not on Github. What Microsoft gets out of it, are millions of lines of code they can use without paying license fees. That likely pays for itself a hundred times over. So the deal is: I get an issue tracker and CI, they get the code. I’m fine with that. It’s open source.

InsertUser · 2024-05-05T11:36:40.884Z

lonvia:

I’m not naive. I understand that there is no such thing as a free lunch, also not on Github. What Microsoft gets out of it, are millions of lines of code they can use without paying license fees.

From tech press coverage It seems they also use it heavily for their internal code repositories and some of their early improvements were driven by needs they had managing their own code and teams. They’re also said to have gone moved an “inner source” methodology for all their code which a GitHub variant would be convenient for. IMHO their acquisition was probably more about preserving tooling that they find useful than snaffling code they could already just fork as an when they needed it.

I really don’t think OSM will benefit much from shifting away and it’ll be costly in terms of manpower to run the thing unless we just hand everything over to yet another faceless entity liable to being taken over.

contrapunctus · 2024-05-05T17:14:59.546Z

While I could arrange for a server and host the forge there, I’m increasingly demotivated to do so given the responses of the OSM community (…at least, what fraction of it speaks here). It seems nobody will move their projects to a free platform even if I do provide them a perfect alternative on a platter, so my efforts will probably be wasted.

In response to the recent replies, as well as the people downvoting my replies - the ignorance, apathy, and denial of the community continues to be disappointing and depressing. I had expected better from - as I said before - a community which is otherwise so invested in privacy, freedom, attribution, and copyleft. Do better.

infrastructures740×352 51.1 KB

contrapunctus · 2024-05-06T07:44:54.547Z

lonvia:

So the deal is: I get an issue tracker and CI, they get the code. I’m fine with that. It’s open source.

Is it also “open source” to reproduce nontrivial parts of code verbatim, violating attribution and copyleft? Because that’s what’s going on with Copilot. Would the pro-GitHub section of the OSM community also be okay with the same being done for OSM data?

Anyone claiming to be in favor of open source or copyleft and still using GitHub is utterly hypocritical. The amount of support this nonsensical thought process is receiving is nothing short of disgusting.

Minh_Nguyen · 2024-05-06T08:07:57.574Z

What about those of us who release our organic, unassisted code into the public domain? Or those of us who checked the box to “consider our map edits to be in the public domain”, perhaps knowing it would amount to nothing? You may not agree with those choices, but consider the environment that permitted those choices to even be made.

The OSM community is already developing a nonzero amount of code on self-hosted or non-GitHub-hosted repositories. Aside from JOSM, you just haven’t heard about them – that’s one of the points that others are trying to make here.

woodpeck · 2024-05-06T08:37:43.000Z

Is it also “open source” to reproduce nontrivial parts of code verbatim, violating attribution and copyleft? Because that’s what’s going on with Copilot.

This is essentially the same discussion that is being had the world over with regard to ChatGPT/Midjourney and the likes scouring the web for content to learn from, and to some degree our OSM data also feeds some AI models that essentially feed off our copylefted work. I am mildly critical of those endeavours even though I can see the conundrum of how we would allow a human to do all this “learning from stuff others have made” without claiming copyleft on their brains, while not allowing a machine to do the same.

But using a self-hosted whatever instance wouldn’t protect us from third parties ingesting the stuff we publish there and - potentially - violating our license.

If GitHub is doing bad things with our code, this wouldn’t stop just because we self-host; we’d still have the same issues, plus the additional work of self-hosting.

Anyone claiming to be in favor of open source or copyleft and still using GitHub is utterly hypocritical.

I think you have a very narrow perspective. Someone could also say: “They’re abusing my stuff anyway no matter what I do, so why not get something out of it at least”.

The amount of support this nonsensical thought process is receiving is nothing short of disgusting.

The fact that so little makes sense to you could also come from you not having given the matter enough thought.

SimonPoole · 2024-05-06T08:44:19.506Z

Slightly OT.

I would point out that the jury is still out (literally :-)) on this from a legal pov.

The thing is that a copy is a copy is a copy, just as if you produced a verbatim copy of something that you “saw” once you are likely going to run in to trouble. How that maps to the specific combination of producers and users of LLMs is going to be interesting, but currently is undecided. There are many dimensions to this, just consider the US fair use doctrine that doesn’t exist elsewhere in such an expansive form.

In any case as long as the contributors to a github hosted project are aware of the situation IMHO it is their call if they want to use it or not. OSS isn’t just about licences.

Mateusz_Konieczny · 2024-05-06T17:41:31.129Z

contrapunctus:

The amount of support this nonsensical thought process is receiving is nothing short of disgusting.

It was mentioned already, but may be worth repeating: insulting others very rarely convinces them.

pnorman · 2024-05-06T21:54:53.759Z

contrapunctus:

Is it also “open source” to reproduce nontrivial parts of code verbatim, violating attribution and copyleft? Because that’s what’s going on with Copilot.

The legality of ML models that can reproduce source data is still up in the air - anyone who has an answer they’re sure of world-wide is incorrect. Regardless of which way the various courts and governments go on the matter, it doesn’t really matter for this discussion. Code for OSM projects will generally be at least mirrored to Github, so it will get picked up anyways. Companies could clone the repos and do anything copyright law and the licenses allow.