Hey everybody,
I ran across Open Infrastructure Map and viewed the power layer. It is really pretty awesome, and I have spent some time trying different things to extract the osm/osm.pdf format for all the generating stations, substations, and lines specifically.
I am finding out that to do this is harder then I thought as the Other Tags field seems to have most of the relevant information about Voltage etc.
I have the north-america file in postgre after failing to do it in windows. I was able to do it in ubuntu, but think with the default style I am not grabbing all the power elements the way I need them in the database. I have my database to auto-update, but realize I need to get a good style file to populate my database with the relevant fields. I will probably delete my original database and give it another go, but wanted to reach out to everybody to see if they have any pointers?
End goal would be mapping the transmission infrastructure in OSM to a synthetic or real grid for a certain area to do power system analysis. Or evaluate trends around infrastructure changes.
I have been looking at Key:power - OpenStreetMap Wiki and a style file https://raw.githubusercontent.com/boundlessgeo/OSM/master/mapzen_osm2pgsql.style
Trying to understand how to create my own style file for all the power elements so I grab all the elements but can view all the elements of each power element. Would I just create a style file that lists everything out in the Key:power section along with every sub field possible? Or could I use a web application to go to a specific region and grab all the power elements to create the style format? I still donât understand the style field format and trying to find a source for style formats used for different layers within OSM. Is something like that shared within the community?
I load everything in using the default style file and it has a lot of stuff in it I donât need which comes in as NULL (waterway, wetland, wood). Is there a reference for style files for the different layers available? I am not a mapping person at all and studying Electrical Engineering so sorry for all the dumb questions.
Some things I have ran across over-pass turbo)
If anybody has any python packages that are power/grid related that can take the OSM data I would be interested. I have tried a couple, but with the entire NA-OSM file it errors out and I have a more immediate need to identify generators, lines, and substations along with how everything is interconnected. The database relationship is more important the mapping of these 3 things
Python Packages - that may help - looking for others
GridTool Current: - GridToolGridTool
:Grid Finder- gridfinder uses night-time lights imagery to as an indicator of settlements/towns with grid electricity access. Then a minimum spanning tree is calculated for these connect points, using a many-to-many variant Dijkstra algorithm and using existing road networks as a cost function. Adapted from this work from Facebook. Currently gridfinder only uses road networks, but it would be trivial to add other cost parameters such as slope or terrain
GridKit - GridKitGridKit uses spatial and topological analysis to transform map objects from OpenStreetMap into a network model of the electric power system. It has been developed in the context of the SciGRID project at the Next Energy research institute, to investigate the possibility of âheuristicâ analysis to augment the route-based analysis used in SciGRID. This has been implemented as a series of scripts for the PostgreSQL database using the PostGIS spatial extensions.
The network model is intended to be used in power systems analysis applications, for example using the PYPOWER system, or the PyPSA toolkit. In general this will require the following:Careful interpretation of results (e.g. power lines and stations may have multiple voltages).Realistic information on generation and load in the system (electrical supply and demand).Reasonable, possibly location specific assumptions on the impedance of lines, depending on their internal structure. Unlike SciGRID, such assumptions have not yet been applied.Of note, PyPSA implements several methods of network simplification, which is in many cases essential for ensuring that the power flow computations remain managable.
Transnet - The Transnet project consists of a set of Python and Matlab scripts for the automatic inference of high voltage power (transmission) grids based on crowdsourced OpenStreetMap (OSM) data. Transnet yields two different models, a Common Information Model (CIM) model and a Matlab Simulink model. The latter can be used to perform load flow analysis. This manual guides you to the Transnet setup and gives several usage examples.
GridMapping: GridMappingh grid information is a prerequisite for various power system applications such as renewable energy integration, wildfire risk assessment, and infrastructure planning. However, a generalizable and scalable approach to obtain such information is still lacking. In this work, we develop a machine-learning-based framework to map both overhead and underground distribution grids using widely-available multi-modal data including street view images, road networks, and building maps. Benchmarked against the utility-owned distribution grid map in California, our framework achieves > 80% precision and recall on average in the geospatial mapping of grids. The framework developed with the California data can be transferred to Sub-Saharan Africa and maintain the same level of precision without fine-tuning, demonstrating its generalizability. Furthermore, our framework achieves a R2 of 0.63 in measuring the fraction of underground power lines at the aggregate level for estimating grid exposure to wildfires. We offer the framework as an open tool for mapping and analyzing distribution grids solely based on publicly-accessible data to support the construction and maintenance of reliable and clean energy systems around the world.
earth-osm - earth-osmearth-osm is a python package that provides an end-to-end solution to extract & standardize power infrastructure data from OpenStreetmap (OSM).
SciGrid - Open Source Mapping from OpenStreetMap âpowerâ data to Europe power grid - open source transmission network model
Matlab can parse OSM file format different packages list to try:
- 35819-openstreetmap-functions
- 108539-open-street-maps-osm-for-machine-learning
PyrosmPyrosm is a Python library for reading OpenStreetMap from Protocolbuffer Binary Format -files (*.osm.pbf) into Geopandas GeoDataFrames. Pyrosm makes it easy to extract various datasets from OpenStreetMap pbf-dumps including e.g. road networks, buildings, Points of Interest (POI), landuse, natural elements, administrative boundaries and much more. Fully customized queries are supported which makes it possible to parse any kind of data from OSM, even with more specific filters
Geodesk
If you want to analyze OSM data in Python, you can use Download an .osm.pbf file (extracts from many regions are provided for free from vendors like geofabrik, turn it into a Geographic_Object_Library (GOL), then query the data. For example, to retrieve all power plants in California and display them on a map:
import geodesk
california = geodesk.Features("california.gol")
california("na[power=plant]").map().show()