(trigger warning: serious post from a usual jokester)
I think the real point here is “we have no mechanism to tie a particular OSM account to a particular real-world identity.” I think that’s an affirmative choice rather than an oversight because we value the ability for someone to contribute anonymously via pseudonyms. But it comes at the expense of having little recourse for dealing with determined vandals.
Given enough motivation of course, any security scheme, even one with 2FA, can be broken. A sophisticated vandal such as a state actor would have no trouble spinning up SMS endpoints to provide that second factor at scale. Also, if we demand 2FA, we will absolutely lose users that can’t or won’t provide a second factor. So there are very real trade-offs.
Personally, I’m fine with the DWG just dealing with the rampant vandalism when it happens. But that’s purely because I’m generally not the one that has to do all the work of patching up after them. If the day comes when the DWG says “we can no longer keep up”, then I’d be 100% onboard with tightening up the mechanism by which people authenticate to the database, even if we lose some users unwilling to participate. Because at the end of the day, it’s better to lose some users and have a working database than to have a broken database.
As long as there are important, powerful geopolitical interests that care about what appears on map, we sit with a considerable risk to data attacks. I support any efforts to keep security controls upgraded. I’d rather get ahead of attacks rather than scrambling to react to them like we did with the recent API limit controls that were added.
I really don’t get what you’re trying to say with that image. MFA being the gate and the lack of surrounding fence is that we still support Basic Auth?
Sure, doing A and B and C in the name of security might sound similar like that humorous restaurant story, and I agree that we shouldn’t do things just because of a hypothetical security risk. But we must also recognize that OSM (sadly) is a real target for… not so nice people (@SomeoneElse or anyone else from the DWG can attest to that, though I hope it’s mostly limited to vandals using their own accounts so far).
And again, I don’t agree with mindlessly implementing every security fad you can find online, some have legitimate merit:
MFA significantly reduces the risks involved in stolen passwords (though I’m personally against requiring MFA for all users)
Removing Basic Auth (or any other scheme involving a third party having access to the password, including the OAuth Password Grant) reduces the risk of third-party applications mishandling the password or doing stuff the user did not intend
Removing “unnecessary” attack vectors (in this case Basic Auth, though you’ll, of course, disagree with the word “unnecessary” here) reduces the overall attack surface (and also helps with maintainability)
Token expiry reduces the risk of an old token being stolen and used (and even if it is used, the usage will be limited in time)
Can’t say anything about OAuth 2.1 as I’ve not familiarized myself with it.
But let’s go back to the practical matter at hand. It’s very unlikely that you’ll stop Basic Auth from being removed; the maintainers want it gone which means it will be gone. What you however can do:
Offer to add support for an authentication scheme that does not have the drawbacks of Basic Auth (i.e. not give full access to the account and be able to have an expiry and in the future work with MFA, PATs, OAuth Client Credentials grant, etc. fit those requirements)
Add support for OAuth to the library you use, or write your own, so that you barely have to think about it when running your scripts
Keep complaining about it here
I don’t really see a point to continue arguing but feel free to continue 3. if you want to vent. But if you actually want something to change then I really recommend going with option 1. or 2. If you do, I’ll be happy to help with some guidance and code review.
Another thing that curiously hasn’t been mentioned yet in this thread is the development cost of changing all apps that currently use basic auth to use Oauth2 and maintenance burden of adding and supporting Oauth2 as an authentication mechanism. Would @02JanDal (and all other people who I can’t be bothered to look up in related thread for the purpose of mentioning them in this somewhat satirical retort) be ready to write PR for all existing and future apps that attempt (or will attempt in the future) to authorize to OSM API, as well as promising that they will help maintain all those authentication libraries and associated code for the next 5-10 years including handling any security concerns exposed through this new potential attack vector?
P.S. I was under impression that @NorthCrab has already promised to deliver not only the Oauth2 (and as I understand it also PAT by his own suggestion here), but a complete rewrite of API in python (including all that functionality) as well.
It can be a bit fun (and even cheeky) to be snarky, sardonic, satirical or all of the above. Yet, I must ask here and now: can we all please realize that we are “on the same team” here (the OSM team)?
As my mother taught me: “You catch more flies with honey than you do with vinegar.” Thanks in advance.
Why haven’t you familiarized yourself with revision 2.1, if it is so simple (esp. since you already grok 2.0) ?
But to respond in more serious manner: yeah, sure. Good programmer can write Oauth2 library and implement it and debug it and validate it. Lesser programmer can find and use existing library with even less effort, provided they trust its security. The question is why should either of them be required to waste their time?
To draw a parallel: surely anybody contributing to OSM must be familiar with ODbL and CT they accepted when they signed up. How about we put up a quiz about those documents that people must pass with at least 90% score before they can register account (or continue using the API if they already have the account). The benefits with improving law compliance are obvious. Surely there cannot be any drawbacks to adding such small barriers to entry, right? I mean it would be like 5 minutes (at most 15 minutes) quiz! Much less than implementing and testing Oauth2!
I can see why people might feel that supporting both Oauth1 and Oauth2 is waste, and we should get rid of the Oauth1 that leaves hundreds of duplicate entries in my preferences which I must clean periodically to keep my sanity. But as noted, Oauth2 does not expire tokens, and it adds external code (of dubious quality and trust - until you’ve audited the code, and often even after that) and code complexity, and does not provide extra access security (i.e. one could just reuse e.g. Vespucci bearer token which usually has all rights and do everything they could do with basic auth username/password)
There are some advantages I’d concede to OAuth2:
Oauth2 for specific app can easily be revoked if you no longer use/trust some app (but so can a regular password be changed and you logged in again in apps you want to continue using - which is only mild convenience advantage at a cost of mildly lesser security)
you can grant less permissions to some apps (principle of least privilege is real security advantage, but people had that forever elsewhere with PATs without overly complex solutions like Oauth2).
buggy apps can be centrally locked out by OSM admins by force-removing their Oauth2 configs in case of emergency (but we did that in the past based on user-agents versions too, yes?)
automatic token expiration forces users to confirm what apps they continue using versus old ones they’re not using anymore, thus locking out unused apps and improving security when exploits are found in them (while one of main selling points, it is seemingly not being used on OSM)
did I miss any more?
But none of those advantages seem to calls for disabling Basic auth to me?
Oauth2 also has several disadvantages:
much more complex than PAT / username+password (which we’ll still need to supposedly support for logging in to osm.org website and to authorize those OAuth2 tokens?)
more code always means more complex (and more complex means more buggy, more bloated, harder to read and more offputting)
more code also means wider attack surface
when choosing to “save time and make life easier” by using external Oauth2 libraries, it brings its own subset of disadvantages:
self contained script is much more likely to be used (and reused) that one that require you to sudo pip install whatever first
external dependencies means now you have to set up alarms to regularly follow another source of upgrades, security alerts, CVEs etc. (or lack of updates if upstream becomes dead, so you have to find a new one and convert to it) - so you increase your maintenance workload (or more likely choose to reduce your security by continuing using old libs with security bugs since “meh it works so why bother”)
to trust some external code you need to sped as much time to audit it as to write from from scratch from specs, with disadvantage that then you cannot really be sure you’ve done good enough audit. Or you need to blindly trust, which has its own set of failure modes.
Of course. I though that huge “” was a clear sign I was on the same team and just friendly-teasing @02JanDal . See my other post for more detailed technical essay on pros and cons of OAuth2. To be clear: I don’t have problem with OAuth2, nor with removing OAuth1. It is just that removing user/password auth without firstly implementing simple replacement like PAT (as suggested by @NorthCrab) does not sound like ideal course to me.
True. Assuming that one is safe (for whatever reason) is never very wise (nobody is, ever).
Exactly that. And as far as I can tell, all that is being asked in this whole thread AFAICT is to expose such token-generator (as you’ve written as cmdline script) as a simple web form on standard OSM web interface e.g. under https://www.openstreetmap.org/account/generate_token (or whatever). It can/should use Oauth2 under it if that is preferable, sure.
So one would click on few checkboxed what permissions they wanted to grant, add description to it, click generate token button and there it is for their use (and later deletion if needed). IIUC, it should not be “rocket science” to implement that (for someone versed in RubyOnRails (or Pyhon3 for OSM-NG case), right?
P.S. it would be great if that list of tokens had a timestamp when it was created, as well as timestamp when it was last used. So should current OAuth2 authorizations; I find it huge step backwards that Oauth2 authorization table does not have at least “Issued At” column (as Oauth1 table two tabs to the left does! That was as valuable to me as the name of the app in the past!)
If there are existing suitable libraries the better developer is the one who uses them. OAuth 2.0 is better here since in most languages there is a better selection of libraries than OAuth 1.0a
Yes, if someone would, in good faith, ask me to help migrating away from Basic Auth I would do so (as I’ve previously mentioned, provided it’s in a language/framework I’m familiar with), because I’m not someone who just complains (or responds to complaints), though I won’t take on any long term maintenance responsibilities (sure, you could say that that last part makes me a hypocrite, but this is still infinitely more than anyone in “camp Basic Auth/PAT” has offered so far).
I would like to, but there’s a lot of other stuff I spend my time on. Having a 48h day has been on the top of my wishlist since I’ve been a teenager.
More seriously, note how I in pretty much every post here have advocated for using an existing library, and at no point that anyone should learn OAuth (any version) past what is needed to use the library. A good library interface would look like this, note how it does not require any knowledge of OAuth more than that you’ll have to copy two strings from the OSM website.
I second what @pnorman wrote; a good programmer would never write their own OAuth 2 library (though it is a fun exercise if you want to spend time on that, just don’t try to write your own OAuth 2 server implementation (just like you shouldn’t write your own password hashing function)). A good software engineer knows which parts to implement themselves and which to take of-the-shelf.
Now this statement is once again straight out wrong, because OAuth 2 does provide extra security over Basic Auth, some of which have already been mentioned in this thread, some even in your own post.
Your list of advantages and disadvantages has some merits (though there are also points that I find questionable), but a big part of software engineering is about trade-offs. Sure, OAuth 2 has drawbacks, but they are over-weighted by the benefits.
Again, I’m genuinely interested in what kinds of clients you all who are opposed to removing Basic Auth are writing. I would really appreciate it if you could write a short description (usecase, language, libraries used, environment, etc.) or post a link, that would help me (and everyone else) to understand your worries, or even help guide you to a solution.
I appreciate hearing (again, not learning it) that vinegar catches flies, too. (Apple cider vinegar is said to be especially effective). Speaking plainly, instead of an idiom that means “it is easier to persuade others with polite requests and a positive attitude rather than with demands, negativity and even rudeness” I’ll say (plainly and simply) “it is easier to persuade others with polite requests and a positive attitude rather than with demands, negativity and even rudeness.”
actually DOES tell me that there is some “tongue in cheek” or “some kidding going on here” attitude, but such mistakes are easy to make. It can be helpful in this forum to be direct, polite, helpful, make criticism constructive if possible, et cetera. We all know that doesn’t always happen, but most of us really do prefer honey to vinegar. Really, we do.
Now, back to our regularly scheduled programming. (A bit sweeter, we might expect).
Edit and Postscript; Timestamps are (and have been) a good idea with authorization. To remove that feature seems retrograde to me and I think many might like to see it return.
To be fair, that’s part and parcel of working with OSM. Requirements change. Moving to multipolygons-on-relations* was a whole bunch of code and head-scratching for editors to implement, but it got done.
If I were to compare the impact of that on two of my projects, I find it’s more work to keep cycle.travel in sync with the latest wiki-fuelled tag insanity, than it is to update Potlatch for things like multipolygons-on-relations and OAuth 2.
* Multipolygon relations are still a stupid idea because relations are metadata and multipolygons are geometry, but I digress. (At least until we get a popcorn emoji.)
Note that I was asking for why it is being removed, and I am not protesting against removal. Though I am also not entirely convinced that it is worth it and I am not a great fan of stuff being deprecated (though it is hard to find people liking that stuff they use got deprecated).
On the other hand if maintainers of website (and therefore ones that would be getting complaints if something goes wrong), and ones who handle it really well, think that it should be done then I am not planning to campaign against such move. Especially as I made almost no contributions to osm website and I do not know well its internal architecture.
Sure, so would I help people asking me in good faith for help migrating away from Oauth2 to PAT.
you could say that that last part makes me a hypocrite, but this is still infinitely more than anyone in “camp Basic Auth/PAT” has offered so far).
Here, I just did, so we’re equally altruistic now! But I don’t think that your comment was fair to @NorthCrab at all - dude has not only offered (see start of this thread and connect to this thread) but is actually actively working on implementing all of that (and much more). You might not think that it is a good idea or worth it (or are just not aware of it despite being mentioned several times in this thread alone already), but saying that he’s hasn’t put an effort is just not true.
I would like to, but there’s a lot of other stuff I spend my time on. Having a 48h day has been on the top of my wishlist since I’ve been a teenager.
That I agree with, and even had idea how to make it work in zero sum game - double the number of hours in a day, and to compensate, reduce the length of workweek for (about) a half. Sadly, no success with that idea yet
More seriously, note how I in pretty much every post here have advocated for using an existing library, and at no point that anyone should learn OAuth (any version) past what is needed to use the library
Well, it seemed to me differently at the time; but not dwell on it and to illustrate my main concern, allow me to paraphrase popular saying “There is no cloud, only other people computers” for this situation: “there is no simple Oauth2 - only shifting complexity, trust and maintenance to someone else (and hoping for the best)”
Now, one might claim that it is always a good idea to use overly complex solution, and then use pre-made libraries to abstract that complexity away – but I’d disagree on that “always” part. I’ll concede that doing that is sometimes indeed needed and a best solution. But not always. ObXKCD #2347.
That fact that I’ve seemingly so heavily contradicted myself in the same post should’ve been dead giveaway to reread the paragraph more carefully and notice that this “access” word before “security” was emphasized - probably for a reason. So to clarify what I meant by “no extra access security”:
situation (A) - someone has MitM you and stolen your basic auth username+password to access the api.osm.org
situation (B) - someone has MitM you and stolen your Vespucci Oauth2 bearer token for api.osm.org
My claim was: in both situations, attacker now has equal access rights to api.osm.org under your credentials, thus access security to api.osm.org between those two situations is not different.
As for other advantages and disadvantages of Oauth2, as you correctly noted, I enumerated them in that same post, so I won’t needlessly repeat them here.
Your list of advantages and disadvantages has some merits (though there are also points that I find questionable), but a big part of software engineering is about trade-offs
Thank you! What I’d like (and what I’ve hoped for, and actually actively invited people with that “did I miss any more?” bulletpoint there), is to have conversation about those points that you disagree with, or which I’ve failed to mention.
Sure, OAuth 2 has drawbacks, but they are over-weighted by the benefits.
…And after we have worked out a table of pros & cons that we can all agree on, then I’d like to proceed on discussing benefits/drawbacks ratio of Oauth2 and PAT (and other techs if there is interest, like WebAuthn and MFA). To reiterate: I’m not opposed to Oauth2 (and I welcome work done by people to improve and and get rid of Oauth1); but I’m not really keen on the the idea that Oauth2 should be sole access method.
Again, I’m genuinely interested in what kinds of clients you all who are opposed to removing Basic Auth are writing. I would really appreciate it if you could write a short description (usecase, language, libraries used, environment, etc.) or post a link, that would help me (and everyone else) to understand your worries, or even help guide you to a solution.
This post is getting longish (understatement of the year ), but since you ask again I’ll assume you’ve missed it earlier - as I’ve noted before (I’m not sure if in this thread or linked github one), most of my current use cases are simple oneliner wget | sed or similar scripts of small complexity (i.e. shell + wget/curl + grep / sed / awk + jq + xmlstarlet etc. - you get the idea). Quite often the bughunting for thing that broke, not some superb new idea that I’ve been the first on the planet to think of and written a script to do it.
The last one for example was trying to identify which of the obsolete OSM preferences introduced breakage when trying to PUT modified preferences back together (i.e. /api/0.6/user/preferences endpoint) for one of my accounts, but not other. Turns out it was preferences containing % sign put by Merkaartor app which I’ve used long time ago, which turned out to be the culprit when used with some Content-Type headers, but not others. I do not tend to save such one-time-use scripts on github for long term use, though (especially as they have hardcoded authentication tokens all over them). In hindsight, all bugs are shallow, but finding them was not trivial for me.
For more complex ones, I usually prefer to wget stuff from planet.osm.org and work with that (like my-notes or osm-torrent-related ones), although some do access api.osm.org instead (like osm-blame). But as even read-only API endpoints and even raw planet archives are increasingly likely to be auth-walled in some time in the future (search the OSM GDPR-related issues on github), so those usages might see sharp increase too.
And I hate when stuff breaks compatibility for no good reason, esp. if the stuff is not written by me, is no longer maintained upstream and is written in languages I do not really speak. (e.g. I’m more likely to just have to rewrite the whole thing in perl from scratch, than try to say learn Go and fix the issue in original code. Now if it was just changing URL to include PAT, it would obviously been significantly less painful and more time efficient)
I’m well aware of his work (indeed I’ve been quite active in his other threads, including having the first reply in the thread you linked to). And while I think his work is admirable, I think he is approaching it wrong (doing the technical before community), and though I wouldn’t mind being disproven I believe the likelihood of his project replacing the current OSM backend stack is next-to-none.
Now, if his project does get deployed alongside the Ruby port (or even eventually replaces it) I would be happy to see it support PATs. But he has (unless I missed it) never offered to do that work for anything else than his own private project (and just to be clear; he of course has no obligation to do so either).
Please point me to the place I’ve said that we should use an overly complex solution without reason so that I can highlight clearly that I miswrote that.
OAuth 2 is reasonable complex yes, and while there certainly are some parts of it that could have been made a little simpler, most of it is done as it is for a very good reason.
I’d love for us to live in a world were we can login with just our username and nothing else for the minimal amount of complexity, but sadly that’s not the world we live in.
But the access security is different. With your OAuth token (or PAT), I can at most edit data on behalf of you, handle notes on behalf of you, change your preferences, upload and read GPX traces on behalf of you, and publish diary entries in your name (and that’s only if the app those token was stolen got access to all that, most clients don’t (need to) request all possible permissions). I cannot, however, change (or even see) your email or password (possibly locking you out of your account), delete your account, etc.
And there we’re back to why Basic Auth is such a bad idea and should be removed: There is no decent way to solve that problem; once an attacker has your username+password they’re in (and not only in OSM, since most users have bad password hygine). (MFA would help, but it’s only a bandage, not a solution)
That’s fine, especially since so far no one has argued against that point (I have argued that having to use OAuth isn’t as bad as some have made it out to be and against people almost requiring PATs be implemented by unpaid volunteers, and others similarly)!
OK, I’ll ask tomhughes through GitHub if he can ignore an ugly v1 UI for a moment and imagine being able to get the atime of an inode. If being able to have that (the access time) of a timestamp (as part of an authentication protocol) doesn’t help, mmm, I think it could. It’s a deep rabbit hole of misplaced trust if it goes sideways, though that is a concern with the design of any good authentication protocol. But let’s not invent anything new, I’m imagining a potentially useful scenario where, yes, I think, there IS a “last used timestamp.” It is not stored “anywhere,” it is deep in the bowels of the OS as the inode’s atime, if I am not mistaken. What could have OAuth 2.0 done with this? Something, for sure, but it remains unknown to me what that is. Nothing, probably. Though, something, possibly.
The rest of the topic’s participants, kindly bubble back up as I pop this stack. Return.
True (never needed to even try looking for it in the documentation, weirdly), though there is an OAuth scope for it. Probably there were some plans for that which never got materialized.
But I don’t think that your comment was fair to 
), but since you ask again I’ll assume you’ve missed it earlier - as I’ve noted before (I’m not sure if in this thread or linked github one), most of my current use cases are simple oneliner