On replacing Basic Auth with OAuth 2.0

Foundation
41

localhost → 127.0.0.1 will work. IIRC, this was specifically done for JOSM. But you might be interested in Allow HTTP to localhost on list of redirect URIs for OAuth2 applications · Issue #3613 · openstreetmap/openstreetmap-website · GitHub .

  • JOSM (done by me) – Java
  • Vespucci (done by SimonPoole) – Java
  • StreetComplete (done by westnordost , IIRC) – Kotlin (could have used Java)
  • GoMap! (done by bryceco) – Swift IIRC
  • ??? (I can almost guarantee that some other editor has written their own OAuth 2 implementation)

To be fair to JOSM and Vespucci, I believe we wrote our OAuth2 implementations concurrently. I should have reached out to @SimonPoole to avoid duplication of work.

Quite frankly, the OAuth 2 authentication dance is simple. If someone cannot write a client for it given the specification, I strongly question whether or not they should be trying to write data to a live database using software they have written. Even with that caveat, most people should use a library for OAuth2. OSM even has an RFC 8414 compliant endpoint, so some libraries won’t even require configuration beyond the client id and the domain.

I assume you are talking about client credentials. The specification specifically says

The client credentials grant type MUST only be used by confidential clients.

This is not the case with any client-side application. All client side applications are not “confidential” clients. You could probably make an argument if the user has to provide their own client id and client secret though.

With that said, I would highly recommend all new OAuth 2 clients implement the extensions that are required as part of the draft OAuth 2.1 specification and treat the OSM authentication endpoints as if the were OAuth 2.1 compliant. Of note, one of the things that the draft specification removes is the implicit grant type; it was removed because it could be intercepted by a malicious application.

If, for whatever reason, you don’t feel like having your application listen to a port on localhost, (127.0.0.1), you can have the user copy/paste the redirect URL and perform the rest of the authentication dance in the application. Which is “just” network requests, but if you are talking with the OSM API, you already are making network requests.

42

More, more specifications!

We also don’t have 2FA support! I don’t understand how we haven’t all been hacked yet.

Then your device has more serious problems.

43

Actually in this case, and just in this case :grinning: , it isn’t your fault. I was a lot later (you are thinking of PMTiles).

Totally OT: the main reason besides reusing the OAuth 1 webview code, is that there are at least 3 if not more HTTP client implementations in common use on Android, so you either have to use an OAuth 2 lib that supports the client you are using or employ an abstraction layer, in the end I couldn’t be bothered.

PS: naturally the real reason is that I couldn’t be the odd one out and let myself be trumped by the other editor devs :sunglasses:

44

I think it’s quite important here to draw a distinction between clients such as iD, JOSM och Vespucci, where the user and client are distinct, and scripts run by a single person (the topic of this thread) in which case user and client are (or at least can be regarded as) the same. This fits a bit awkwardly into the OAuth specification, but in that case the user provides their own client ID and secret as per your last sentence, so I think it makes sense to use the client credential flow for that, even if it slightly deviates from the specification.

It could make sense however to only allow using the client credential flow for import accounts, both to make sure that all automated edits are following proper procedure and to safe-guard against a developer accidentally giving users access to act as the developer rather than themselves. That would however add additional burden onto the DWG (or whoever handles import accounts) in cases were an import account is not strictly needed, so might not be ideal (maybe have a switch so that you can set an import account as such yourself, with the DWG being able “elevate” those import accounts that need higher rate limits etc.?).

Right, by this I assume that you also use RFC 793 and friends to implement your own TCP stack? </sarkasm>

Again, in software we use abstractions. The OS handles TCP connections. A library that handles HTTP requests on top of that. Another library that does OAuth on top of the HTTP library. A library that does the OAuth dance using OSM endpoints, and gives you (typesafe if that kind of language) access to the API.

What you’re saying: “We don’t have security feature X, so it’s fine to keep security risk Y”

I’m reasonably confident in the security of both my laptop, desktop and servers I manage, as in I’ve done what can be reasonably expected of someone who isn’t a cyber-security expert or is guarding highly sensitive data. Yet, were it makes sense I still assume that my systems could be compromised, and take measures such as not storing passwords in clear text (only in an encrypted password manager) and storing backups in a way that they should not be writable by ransomware. Assuming that just because you have the latest security updates and don’t install any potentially dangerous software you’re safe is not very wise.

1 Like
45

You must not be paying attention then. We are CONSTANTLY dealing with sockpuppets and malicious bad actors that just keep spinning up new accounts and there’s little to nothing we can do to stop them beyond the recently implemented API limits. I’m sure @SomeoneElse and the team at the DWG would have a substantially easier time dealing with cases like the persistent vandal of Erie, PA and other persistent vagrants if we had the capability to block a particular cell phone number as a 2FA source, for example. There are many good reasons why we haven’t chosen to implement 2FA, but we are absolutely attacked on a routine and recurring basis by bad actors that would be stopped or at least severely hampered with a 2FA system that allowed us to block someone by identity.

1 Like
46
Or in other words

изображение
изображение1500×1500 402 KB

So, 2FA is just an example. I mean, here we are with suggestions for periodic token updates, OAuth 2.1 extensions, abandoning Basic Auth, etc. we are starting to play A Hacker in a restaurant.

47

(pedantry alert)

We sort of do, in that OSM supports external identity providers (like Google) that do support MFA. Naturally this only works if the email address is also protected by MFA, and not easily exploitable by other means.

(but yes, you’re correct that there’s no native MFA sign-on support on OSM.org)

2 Likes
48

Aaargh! We’ll get oodles of posts saying why SMS is insecure for MFA now!
:slight_smile:

1 Like
49

SMS is insecure for MFA.

sorry

50

Pedantry alert was well-placed in this case.

(trigger warning: serious post from a usual jokester)

I think the real point here is “we have no mechanism to tie a particular OSM account to a particular real-world identity.” I think that’s an affirmative choice rather than an oversight because we value the ability for someone to contribute anonymously via pseudonyms. But it comes at the expense of having little recourse for dealing with determined vandals.

Given enough motivation of course, any security scheme, even one with 2FA, can be broken. A sophisticated vandal such as a state actor would have no trouble spinning up SMS endpoints to provide that second factor at scale. Also, if we demand 2FA, we will absolutely lose users that can’t or won’t provide a second factor. So there are very real trade-offs.

Personally, I’m fine with the DWG just dealing with the rampant vandalism when it happens. But that’s purely because I’m generally not the one that has to do all the work of patching up after them. If the day comes when the DWG says “we can no longer keep up”, then I’d be 100% onboard with tightening up the mechanism by which people authenticate to the database, even if we lose some users unwilling to participate. Because at the end of the day, it’s better to lose some users and have a working database than to have a broken database.

As long as there are important, powerful geopolitical interests that care about what appears on map, we sit with a considerable risk to data attacks. I support any efforts to keep security controls upgraded. I’d rather get ahead of attacks rather than scrambling to react to them like we did with the recent API limit controls that were added.

1 Like
51

I really don’t get what you’re trying to say with that image. MFA being the gate and the lack of surrounding fence is that we still support Basic Auth?

Sure, doing A and B and C in the name of security might sound similar like that humorous restaurant story, and I agree that we shouldn’t do things just because of a hypothetical security risk. But we must also recognize that OSM (sadly) is a real target for… not so nice people (@SomeoneElse or anyone else from the DWG can attest to that, though I hope it’s mostly limited to vandals using their own accounts so far).

And again, I don’t agree with mindlessly implementing every security fad you can find online, some have legitimate merit:

  1. MFA significantly reduces the risks involved in stolen passwords (though I’m personally against requiring MFA for all users)
  2. Removing Basic Auth (or any other scheme involving a third party having access to the password, including the OAuth Password Grant) reduces the risk of third-party applications mishandling the password or doing stuff the user did not intend
  3. Removing “unnecessary” attack vectors (in this case Basic Auth, though you’ll, of course, disagree with the word “unnecessary” here) reduces the overall attack surface (and also helps with maintainability)
  4. Token expiry reduces the risk of an old token being stolen and used (and even if it is used, the usage will be limited in time)

Can’t say anything about OAuth 2.1 as I’ve not familiarized myself with it.

But let’s go back to the practical matter at hand. It’s very unlikely that you’ll stop Basic Auth from being removed; the maintainers want it gone which means it will be gone. What you however can do:

  1. Offer to add support for an authentication scheme that does not have the drawbacks of Basic Auth (i.e. not give full access to the account and be able to have an expiry and in the future work with MFA, PATs, OAuth Client Credentials grant, etc. fit those requirements)
  2. Add support for OAuth to the library you use, or write your own, so that you barely have to think about it when running your scripts
  3. Keep complaining about it here

I don’t really see a point to continue arguing but feel free to continue 3. if you want to vent. But if you actually want something to change then I really recommend going with option 1. or 2. If you do, I’ll be happy to help with some guidance and code review.

2 Likes
52

Another thing that curiously hasn’t been mentioned yet in this thread is the development cost of changing all apps that currently use basic auth to use Oauth2 and maintenance burden of adding and supporting Oauth2 as an authentication mechanism. Would @02JanDal (and all other people who I can’t be bothered to look up in related thread for the purpose of mentioning them in this somewhat satirical retort) be ready to write PR for all existing and future apps that attempt (or will attempt in the future) to authorize to OSM API, as well as promising that they will help maintain all those authentication libraries and associated code for the next 5-10 years including handling any security concerns exposed through this new potential attack vector?

:wink:

P.S. I was under impression that @NorthCrab has already promised to deliver not only the Oauth2 (and as I understand it also PAT by his own suggestion here), but a complete rewrite of API in python (including all that functionality) as well.

3 Likes
53

It can be a bit fun (and even cheeky) to be snarky, sardonic, satirical or all of the above. Yet, I must ask here and now: can we all please realize that we are “on the same team” here (the OSM team)?

As my mother taught me: “You catch more flies with honey than you do with vinegar.” Thanks in advance.

2 Likes
54

Why haven’t you familiarized yourself with revision 2.1, if it is so simple (esp. since you already grok 2.0) ? :wink:

But to respond in more serious manner: yeah, sure. Good programmer can write Oauth2 library and implement it and debug it and validate it. Lesser programmer can find and use existing library with even less effort, provided they trust its security. The question is why should either of them be required to waste their time?

To draw a parallel: surely anybody contributing to OSM must be familiar with ODbL and CT they accepted when they signed up. How about we put up a quiz about those documents that people must pass with at least 90% score before they can register account (or continue using the API if they already have the account). The benefits with improving law compliance are obvious. Surely there cannot be any drawbacks to adding such small barriers to entry, right? I mean it would be like 5 minutes (at most 15 minutes) quiz! Much less than implementing and testing Oauth2!

I can see why people might feel that supporting both Oauth1 and Oauth2 is waste, and we should get rid of the Oauth1 that leaves hundreds of duplicate entries in my preferences which I must clean periodically to keep my sanity. But as noted, Oauth2 does not expire tokens, and it adds external code (of dubious quality and trust - until you’ve audited the code, and often even after that) and code complexity, and does not provide extra access security (i.e. one could just reuse e.g. Vespucci bearer token which usually has all rights and do everything they could do with basic auth username/password)

There are some advantages I’d concede to OAuth2:

  • Oauth2 for specific app can easily be revoked if you no longer use/trust some app (but so can a regular password be changed and you logged in again in apps you want to continue using - which is only mild convenience advantage at a cost of mildly lesser security)
  • you can grant less permissions to some apps (principle of least privilege is real security advantage, but people had that forever elsewhere with PATs without overly complex solutions like Oauth2).
  • buggy apps can be centrally locked out by OSM admins by force-removing their Oauth2 configs in case of emergency (but we did that in the past based on user-agents versions too, yes?)
  • automatic token expiration forces users to confirm what apps they continue using versus old ones they’re not using anymore, thus locking out unused apps and improving security when exploits are found in them (while one of main selling points, it is seemingly not being used on OSM)
  • did I miss any more?

But none of those advantages seem to calls for disabling Basic auth to me?

Oauth2 also has several disadvantages:

  • much more complex than PAT / username+password (which we’ll still need to supposedly support for logging in to osm.org website and to authorize those OAuth2 tokens?)
  • more code always means more complex (and more complex means more buggy, more bloated, harder to read and more offputting)
  • more code also means wider attack surface
  • when choosing to “save time and make life easier” by using external Oauth2 libraries, it brings its own subset of disadvantages:
    • self contained script is much more likely to be used (and reused) that one that require you to sudo pip install whatever first
    • external dependencies means now you have to set up alarms to regularly follow another source of upgrades, security alerts, CVEs etc. (or lack of updates if upstream becomes dead, so you have to find a new one and convert to it) - so you increase your maintenance workload (or more likely choose to reduce your security by continuing using old libs with security bugs since “meh it works so why bother”)
    • to trust some external code you need to sped as much time to audit it as to write from from scratch from specs, with disadvantage that then you cannot really be sure you’ve done good enough audit. Or you need to blindly trust, which has its own set of failure modes.
3 Likes
55

Of course. I though that huge “:wink:” was a clear sign I was on the same team and just friendly-teasing @02JanDal . See my other post for more detailed technical essay on pros and cons of OAuth2. To be clear: I don’t have problem with OAuth2, nor with removing OAuth1. It is just that removing user/password auth without firstly implementing simple replacement like PAT (as suggested by @NorthCrab) does not sound like ideal course to me.

True. Assuming that one is safe (for whatever reason) is never very wise (nobody is, ever).

Exactly that. And as far as I can tell, all that is being asked in this whole thread AFAICT is to expose such token-generator (as you’ve written as cmdline script) as a simple web form on standard OSM web interface e.g. under https://www.openstreetmap.org/account/generate_token (or whatever). It can/should use Oauth2 under it if that is preferable, sure.

So one would click on few checkboxed what permissions they wanted to grant, add description to it, click generate token button and there it is for their use (and later deletion if needed). IIUC, it should not be “rocket science” to implement that (for someone versed in RubyOnRails (or Pyhon3 for OSM-NG case), right?

P.S. it would be great if that list of tokens had a timestamp when it was created, as well as timestamp when it was last used. So should current OAuth2 authorizations; I find it huge step backwards that Oauth2 authorization table does not have at least “Issued At” column (as Oauth1 table two tabs to the left does! That was as valuable to me as the name of the app in the past!)

1 Like
56

If there are existing suitable libraries the better developer is the one who uses them. OAuth 2.0 is better here since in most languages there is a better selection of libraries than OAuth 1.0a

2 Likes
57

Yes, if someone would, in good faith, ask me to help migrating away from Basic Auth I would do so (as I’ve previously mentioned, provided it’s in a language/framework I’m familiar with), because I’m not someone who just complains (or responds to complaints), though I won’t take on any long term maintenance responsibilities (sure, you could say that that last part makes me a hypocrite, but this is still infinitely more than anyone in “camp Basic Auth/PAT” has offered so far).

I would like to, but there’s a lot of other stuff I spend my time on. Having a 48h day has been on the top of my wishlist since I’ve been a teenager.

More seriously, note how I in pretty much every post here have advocated for using an existing library, and at no point that anyone should learn OAuth (any version) past what is needed to use the library. A good library interface would look like this, note how it does not require any knowledge of OAuth more than that you’ll have to copy two strings from the OSM website.

I second what @pnorman wrote; a good programmer would never write their own OAuth 2 library (though it is a fun exercise if you want to spend time on that, just don’t try to write your own OAuth 2 server implementation (just like you shouldn’t write your own password hashing function)). A good software engineer knows which parts to implement themselves and which to take of-the-shelf.

Now this statement is once again straight out wrong, because OAuth 2 does provide extra security over Basic Auth, some of which have already been mentioned in this thread, some even in your own post.

Your list of advantages and disadvantages has some merits (though there are also points that I find questionable), but a big part of software engineering is about trade-offs. Sure, OAuth 2 has drawbacks, but they are over-weighted by the benefits.

Again, I’m genuinely interested in what kinds of clients you all who are opposed to removing Basic Auth are writing. I would really appreciate it if you could write a short description (usecase, language, libraries used, environment, etc.) or post a link, that would help me (and everyone else) to understand your worries, or even help guide you to a solution.

58

I think this is going slightly offtopic, but it appears as if you caught most flies with vinegar AND honey :wink:

59

I appreciate hearing (again, not learning it) that vinegar catches flies, too. (Apple cider vinegar is said to be especially effective). Speaking plainly, instead of an idiom that means “it is easier to persuade others with polite requests and a positive attitude rather than with demands, negativity and even rudeness” I’ll say (plainly and simply) “it is easier to persuade others with polite requests and a positive attitude rather than with demands, negativity and even rudeness.”

actually DOES tell me that there is some “tongue in cheek” or “some kidding going on here” attitude, but such mistakes are easy to make. It can be helpful in this forum to be direct, polite, helpful, make criticism constructive if possible, et cetera. We all know that doesn’t always happen, but most of us really do prefer honey to vinegar. Really, we do.

Now, back to our regularly scheduled programming. (A bit sweeter, we might expect).

Edit and Postscript; Timestamps are (and have been) a good idea with authorization. To remove that feature seems retrograde to me and I think many might like to see it return.

60

To be fair, that’s part and parcel of working with OSM. Requirements change. Moving to multipolygons-on-relations* was a whole bunch of code and head-scratching for editors to implement, but it got done.

If I were to compare the impact of that on two of my projects, I find it’s more work to keep cycle.travel in sync with the latest wiki-fuelled tag insanity, than it is to update Potlatch for things like multipolygons-on-relations and OAuth 2.

* Multipolygon relations are still a stupid idea because relations are metadata and multipolygons are geometry, but I digress. (At least until we get a popcorn emoji.)

1 Like