City of Edinburgh Street Lamp Import

Hi,

I am proposing to import the Street Lighting Columns dataset, sourced from The City of Edinburgh Council.

Documentation

The wiki page for my import is located here: Scotland/Import/Edinburgh/Street Lamps - OpenStreetMap Wiki

The download for the data is available here: Street Lighting Columns

You can find the latest GeoJSON file which has been translated to the OSM schema on the wiki page under Import Data > OSM Data Files

License

I have checked that this data is compatible with the ODbL.
This data is distributed under UK Open Government Licence (OGL) v3.0.

Abstract

The dataset contains 58,854 records, and after removing records with invalid coordinates, 58,819 nodes are to be imported. After converting to the OSM scheme the final GeoJSON comes in at 18.1MiB.

The data was converted to the OSM scheme through a python script adapted from the City of Coquitlam Street Lamp Import.

I plan to conduct this import in one batch using josm and the conflation plugin. There are only 180 existing street lamps that overlap with the data so any conflicts during conflation can be checked manually.

The list of tags to be imported can be found on the wiki page.
As all street lamps in Edinburgh managed by the council use led lamps so I plan to add ā€˜light:method=led’ to all lamps.
I also would appreciate comment on light:method vs lamp_type.

support=pole will also be added to all lamps as all records have feature__1=SL: Lamp Post. Additionally after checking all non-pole mounted lamps that I know of, they do not seem to be included in the data.

This import was heavily inspired by the City of Coquitlam Street Light Import by @maxwellward and is also my first import so any feedback would be greatly appreciated!

Best,
Papweer

5 Likes

I’ve shared this in the ā€œOSM Edinburghā€ signal chat. They meet about once a month in the city (usually posted on OSM Cal).

1 Like

Thanks for this! I’m glad I could be an inspiration :slight_smile: In the output GeoJSON it looks like you’re missing the base light_source=lantern tag, and support=* should probably be light:support=*

I completely missed light_source=lantern, added now.
As for light:support=* the proposal mentions using support=* and I can’t find any documentation for light:support=* anywhere.

@maxwellward what do you think about choosing a software license for your original script? I’m an Edinburgh local and am wondering about long-term re-use and adaptation of the script - if for example the format (or field values) of the street lamp dataset from Edinburgh Council changes in future. (cc @Papweer)

1 Like

Looks like I might have been wrong! I’ll have to update my script as well!

I’m happy to add a license to the original gist on GitHub. Do you think MIT would be appropriate? I don’t mind how it’s used, copied, or shared, with or without attribution, but MIT seems more broadly used and understood than something like Unlicense.

CC0 could work as well but I’m not sure if people prefer one or the other.

MIT is a widely understood and enforced license and would be a very suitable choice if you are happy with the grants it gives people (they can use the code freely and derive it) and the responsibilities it requires of them (they have to share the license text alongside derivative works).

Sorry for a delayed reply: my first thought was to suggest MIT licensing (as @gurglypipe did too); it’s straightforward and well-recognised.

For complex/involved software where there’s a risk that a codebase could be used by competing interests that might not share their changes/improvements back to users, I tend to prefer GPL or even AGPL - I don’t feel strongly that that’s worthwhile in this case, though.

Someone did an import of all (about 35,000) the streetlights in Dublin City (but not surrounding councils) and deleted the pre-existing ones (I think about 3,000).

There are a few complications:

  • Lighting is associated with a variety of organisations - council, non-council road operator, non-council housing operator, power network operators own poles, power supplier’s own light fittings, light rail catenary supports. This can make the operator=* tag complicated.
  • Some lighting is on man_made = mast, e.g. at motorway interchanges.
  • There is significant lighting mapped that isn’t on public roads - port, national parks, universities, shopping centres, sports grounds and other car parks.
  • Not all lighting is on a pole - some is ground mounted, on a building or other wall, etc.
  • The council data might not be accurate (numbered pole on wrong street; pole has been removed; supports misidentified, etc.) or precise (all poles are offset by X metres in Y direction; position is rounded off inappropriately).

Comparing all the data for a few streets before doing a full import might be useful.

  • Lighting is associated with a variety of organisations

As far as I know this dataset only includes street lighting the council is directly responsible for and keeps track of. I haven’t investigated this too much though so I may be wrong.

  • Some lighting is on man_made = mast, e.g. at motorway interchanges.
  • Not all lighting is on a pole - some is ground mounted, on a building or other wall, etc.

This data is only of pole mounted lighting as:

  • All records have feature__1=SL: Lamp Post
  • I’ve checked areas which I know of that have non-pole mounted lighting, managed by the council (e.g tunnels) and they are missing from the data.
  • I’ve also checked abnormal height values (anything over 10m and under 3m) and there seems to be nothing.
  • There is significant lighting mapped that isn’t on public roads - port, national parks, universities, shopping centres, sports grounds and other car parks.

Since I’m conflating data instead of deleting existing data this shouldn’t be an issue.

  • The council data might not be accurate
  • In terms of location I have checked many lamps and the worst error I have seen was about 2m and those cases are very rare. Most lamps are within 0.5m.
  • I’ve not seen any issues with material, though I have only been able to check a limited number of materials.
  • There are 19 lamps with a height of 0 and all other heights seem pretty accurate. Any lamp with a height of 0 has the height key removed.
  • The field with the most issues is the commission (start_date) field. It has many ā€˜Not Applicable’ or random word values that can’t be interpreted as a date. For all these poles I’ve not added any start_date key at all.
1 Like

Sounds good.

I would try to keep as much data as possible, even if it is a little fuzzy, possibly using custom tags. Adding a note to that effect would be useful.

I did have a look at the raw data, and agree it looks pretty good to me.

One thing that is interesting; the data does look to be t least 2 years out of date. But there are newer ares that have been around for longer but aren’t included in the dataset. Ideally if we are going to import this, it would be good to see if we can get the council to publish some more up to data data?

My final question is the data Street Lighting Columns it is released under ā€œOpen Government License v3.0ā€ but it then says:

ā€The data was captured by the City of Edinburgh Council against Ordnance Survey basemaps and requires the following attribution statement to acknowledge the source of information:

ā€œCopyright City of Edinburgh Council, contains Ordnance Survey data Ā© Crown copyright and database right (2023)ā€"

Is this any concern?

3 Likes

The data has already been updated once since it’s publishing, and if any further updates were to be published, they could be imported without much hassle. As for now I feel that the data is up to date enough that any differences are minor and can be updated at a later date if new data becomes available.

As for the OS license it only applies to the basemap the data is presented on as described on the Ordnance Survey wiki page. Thus it is fine to use in OSM.

1 Like

The import has been preformed!
Thanks everyone for the feedback.

1 Like

Was omitting highway=street_lamp a deliberate choice?

2 Likes

Yes, I’m using the tagging scheme from this proposal. It’s use was originally suggested here on the City of Coquitlam Import.

It doesn’t look like ID editor is correctly identifying the data as a streetlamp / lampost.

Almost no data consumers use light_source. More use street_lamp

ID currently doesn’t support light_source=lantern instead of highway=street_lamp. I have created a issue to add support in ID.