Importing City of Calgary open data

Communities Canada
1

The City of Calgary publishes open map data under the OpenStreetMap-compatible Open Government Licence - City of Calgary license:

highways:

buildings:

enforcement:

recreation:

other:

You might find more here: https://data.calgary.ca/browse?sortBy=relevance&pageSize=20&limitTo=maps

We’re working on importing some of this into OpenStreetMap, you can see some of the Python scripts I wrote (with ChatGPT/GitHub Copilot) for this here: github.com/verhovsky/calgary_osm_import.

So far I’ve imported the red light camera locations (without a corresponding enforcement relation though), bus shelter outlines and fixed a bunch of street names. We’re working on importing building outlines neighborhood-by-neighborhood and hopefully one day building (parcel) addresses.

All the datasets I’ve looked at in depth have had mistakes. The Intersection Safety Camera Locations dataset specifies the direction of speed cameras in an inconsistent way and has at least one camera I couldn’t find in real life, the Street Centreline dataset has at least like 30+ wrong or misspelled street names and a few artifacts, the Building Roof Outlines dataset has really good outlines generally, but they contain tons of unnecessary nodes and the outlines are often misclassified, in a neighborhood there will be a garage or two misclassified as a residence or vice versa.

2

Can you confirm who is “we”?

OpenStreetMap generally has quite restrictive import processes - have you seen Import/Guidelines - OpenStreetMap Wiki ?

Well done checking it rather than importing blindly!

3 Likes
3

I appreciate the contributions to the map in Calgary, as otherwise things like building outlines have been very, very slowly added over the years.

(Not trying to toot my own horn too much, but I’d say the vast, vast majority of buildings in the southwest Inner City I drew in manually and manually cross-checked the addresses, which was extremely ‘inefficient’ :sweat_smile:, but I would argue has been far, far more accurate than simple imports.)

That said, I’m very glad to hear you’ve been looking at the data and attempted to cross-check it against reality first, rather than blindly importing things and presuming it’s correct—or more correct—than what’s already on the map.

From a cursory look at the “street centreline” data, I wouldn’t presume any of it is more accurate than what’s already on OSM. You’ve already caught many errors in the intersection cameras data, and thankfully left notes on the map (e.g. here, here, here, here, here and here) which I’ve tried to backfill with whatever local knowledge I have. Likewise with your attempts to clean up the names of school buildings, and remove them entirely where they’re already present on the school grounds, such as here. And you’ve paused before adding school grounds that no longer exist, which is great.

That said, some assumptions have been in error, e.g. copying the name of a school building to the school grounds where it wasn’t warranted, or “correcting” a road name incorrectly.

Looking through the data sets, I can anecdotally tell you from a cursory glance that most of them are somewhat accurate, but their spatial precision leaves something to be desired. For instance the “private water hydrants” data and “public tree locations” data seems fairly accurate in that there are hydrants and trees in the vicinity of most (but not all…) of the dots on the maps, but many are off by several metres. Same goes for the “public art locations”; I was curious whatever happened to a couple statues at Century Gardens: they’re still on the map with fixme notes added by yours truly (1, 2), but the City data still shows them in those original locations and I can tell you with 100% certainty they’re not there… :sweat_smile:

“Parcel Address Points” seem ~97% accurate, but the closer you get to Downtown the more convoluted it is. There are plenty of addresses in the data that I can tell you don’t actually exist anymore due to redevelopment, especially where townhouses and apartment buildings have been “infilling” where single-family detached dwellings used to be. Pockets of them are also blatantly non-existent, such as at the intersection of Glenmore Trail and 24th Street SE, the vicinity of which I coincidentally edited about an hour ago.

In general I would say: keep up the good work! :slight_smile: However, please do elaborate as to who “we” are, and please follow import guidelines, as per Jarek’s reply above.

4

With respect to transit routes and stops, I presume you’re not working with other editors who’ve been (re-)adding them in recent months, e.g. 1, 2, 3?

I’ve noticed the route names have been (re-)named with “/number/ /name/” formatting, which I guess allegedly jives with the ‘preferred’ PTv2 format (…?), and the stops have been imported or (re-)named with a heavily abbreviated format copied from the City data, e.g. “SB 14 ST SW @ 24 AV SW”. I rather painstakingly tried to ‘unabbreviate’ these names and follow what the voice announcements on the buses actually say (e.g. “14 Street & 24 Avenue SW”) when I added them to the map in the first place, but oh well I guess… :face_with_diagonal_mouth: :man_shrugging:

In the past I got a lot of guff from people about ‘abbreviated’ quadrants in street names (and by extension addresses), so don’t be surprised if you import the transit stops and routes without changing the names you get ornery admonishments from others. (As far as I’m concerned those people can fix it if they’re so inclined; I certainly won’t lift a finger to assist. :laughing:)

5

Howdy, I’ve been actively importing as well in various formats of data in conjunction with Knbx, we had been working in parallel independent. But have since been in correspondence after realized we were working on the same tasks. Just want to say how impressed i am with the effort Boris has put into checking, processing and building scripts to handle the data. Its very accurate once setup and very efficient to process once you learn a few tricks. It would have taken months and months if i continued at the more manual setup i was using to work with the open data. Additionally he has put together a great writeup on the wiki with resources and methodology that i think will help.

While i might be a new name to the Calgary area mapping i have been working in the Edmonton area for a few years now and wanted to help out in Calgary as well.

Primarily its been building outlines and addresses imported with guidance, its not a simple mass import but the JOSM tools do help to streamline. i haven’t really played with any of the other data yet. Just to reiterate, there simply is no blind wholesale imports. It’s not viable for these, and caution should be given for the other data sets if we tackle them too.

Some observations and responses.

Bing sat imagery is updated in the late 2023 timeframe and seems to be the best quality and accuracy. As the city’s import was done in the 2024 timeframe it is pretty easy to spot and capture and updates from existing data. To date its been focused on areas that were completely missing data or had poor quality imports (ie MS footprints or old traced data from poor quality imagery) The core communities that have been hand traced, addressed and split out have not been touched as they are the best quality geometry on the map and would be a nightmare to attempt to import data in any wholesale means. But, some sort of review would be prudent to pick and choose changes to do one on one given the data is pretty new and great quality.

Agreed on the roofline data, it does get a little goofy on extra nodes, generally i’ve been using it raw, as it hasn’t been an issue but working with redstone and cornerstone it was getting pretty ridiculous.

100% a sweep for building classification checks, there are misses, but we are also coming for a bunch of map data as “building=yes” which was useless (i am guilty of this as well). The data is limited, its easy peasy in the far flung suburbs to classify garages, detached, terraces, apartments, etc… and generally i have been doing so when i am doing the addressing imports as its part of the process.

Street centerlines, i have only used it as a basis to improve the existing data via the JOSM tools. I agree there is no value to a wholesale import as there is no consistency to the quality or accuracy of this data from the city, it is best used as a quick check, outline for new roadways and naming structure check. A note there are new roads with active completion dates that are not yet represented in imagery yet, I personally have been adding them, but can’t physically confirm, i suppose going forward I will add with a note. Generally speaking most roads are well done but there are some abysmal neighborhoods with presumably old data that are worthy of a check. It’s definitely a case by case basis for using the road data.

I agree on the general data accuracy, hydrants can be pretty good, but it needs data cleanup and shifting, addresses just aren’t worth it as you get into more commercial areas, that is going to still be hand and street level confirmed for the best results. But most of this has been the focus of previous work so its in good shape already.

Some issues: early on i was oblivious to the conflation and geometry replacement functionality so i was deleting and replacing some of the previous work in a few areas. We lost some history on a few buildings in the West and NW as a result i did early on, My apologies to anyone, i wish i knew earlier on.

I would love to see a bit more collaboration so it is more of a community effort, i get big imports are not for everyone but the tools are fantastic and there is opportunity to get addressing done for the city in fairly short order.

I think some pause and discussion before working with the other datasets is probably worthwhile, what is valuable and what is clutter is worth a chat, but there is a ton to do still on core useful for navigation stuff yet.

1 Like
6

I have “imported” the public washrooms data from Calgary’s Open Data.
Method:

  • Download the “Corporate Structures” data as geoJSON.
  • Use JOSM search to find only structures with the tag “washroom” and copy that to a new layer (and save it as CalgaryBathroomsAll.osm)
  • Go through each building in that working layer while using the OSM Carto imagery background to see if a building with the washroom icon already exists in that location. Delete from my working layer if there is already a matching building in OSM.
  • Go through each building in my working layer while using Esri imagery to find buildings that are hard to identify (due to tree cover), or possibly not there (because I know Eau Claire and Olympic Plaza are undergoing construction right now). Separate these out into CalgaryBathroomsNeedVerification.osm.
  • Go through each building in working layer and identify where there is a building existing in OSM (building=yes), but there is no amenity tag yet. Save these into a new file CalgaryBathroomsTag.osm. Add the amenity=toilet tag to the existing building in OSM. Upload to OSM.
  • Go through each building remaining and add a new building outline by tracing Esri imagery and then adding the building=yes and amenity=toilet tags. If there was an amenity=toilet node nearby, then make that node one of the building corners, copy-paste the node’s tags into the new building outline tags, remove the tags from the original node, and upload.

No data was actually copied from Calgary’s Open Data, but it was used as a reference for finding public toilet locations so that I could update tags or trace from Esri.

Changesets are:

I’m not that familiar with the forum software, so sorry if my formatting is weird or not community-norm. It also looks like I’m not allowed to upload .osm files; if anyone is interested in the above working files perhaps there is a good way of transferring them?

I’m not familiar with the “Conflate” JOSM plugin - I took a quick 2 minute look at it, but decided not to bother learning it today, since there aren’t that many bathrooms in Calgary.

1 Like
7

I realized that even if I could upload the “NeedVerification” osm file, it may violate some license.
So I’ve marked the difficult-to-map washrooms as “notes” on OSM map. You can see them as notes in my profile added today:

Hopefully someone who is close to these locations (or even myself if I go for a walk/bike in the areas in the future) will be able to update the notes or the map.