I’ve found a couple of exceptions that need to be managed. I’m working on tidying that up. WRT the Spur/Alt stuff the existing loop is handling those properly. They just get replaced by entries higher up in the dictionary.
1 Like
The 2024 state highway log (5 MB PDF) calls it a spur.
Yes, as the comment says, the log says it’s a spur but it’s signposted as an alternate route. Apparently this has been the case for some time, but Wikipedia is applying the off-the-ground rule.
I just did a quick comparison between the dict I built up from wikipedia vs the route numbers in that PDF and compared against the osm export. I added WA 2 and WA 395 to the substitution list. They’re unsigned references that are used for administration purposes that still need to be corrected.
So I did some double checking of the actual dataset to see what has a matching pattern in it. From the info in the PDF that @Carnildo linked I added two entries to this list:
swapdict = {
'WA 2': 'SR 2', #added from other list
'WA 3': 'SR 3',
'WA 4': 'SR 4',
'WA 6': 'SR 6',
'WA 7': 'SR 7',
'WA 8': 'SR 8',
'WA 9': 'SR 9',
'WA 10': 'SR 10',
'WA 11': 'SR 11',
'WA 12': 'SR 12',
'WA 14': 'SR 14',
'WA 16': 'SR 16',
'WA 17': 'SR 17',
'WA 18': 'SR 18',
'WA 19': 'SR 19',
'WA 20': 'SR 20',
'WA 21': 'SR 21',
'WA 22': 'SR 22',
'WA 23': 'SR 23',
'WA 24': 'SR 24',
'WA 25': 'SR 25',
'WA 26': 'SR 26',
'WA 27': 'SR 27',
'WA 28': 'SR 28',
'WA 30': 'SR 30',
'WA 31': 'SR 31',
'WA 35': 'SR 35',
'WA 41': 'SR 41',
'WA 92': 'SR 92',
'WA 96': 'SR 96',
'WA 99': 'SR 99',
'WA 100': 'SR 100',
'WA 102': 'SR 102',
'WA 103': 'SR 103',
'WA 104': 'SR 104',
'WA 105': 'SR 105',
'WA 106': 'SR 106',
'WA 107': 'SR 107',
'WA 108': 'SR 108',
'WA 109': 'SR 109',
'WA 110': 'SR 110',
'WA 111': 'SR 111',
'WA 112': 'SR 112',
'WA 113': 'SR 113',
'WA 115': 'SR 115',
'WA 116': 'SR 116',
'WA 117': 'SR 117',
'WA 119': 'SR 119',
'WA 120': 'SR 120',
'WA 121': 'SR 121',
'WA 122': 'SR 122',
'WA 123': 'SR 123',
'WA 124': 'SR 124',
'WA 125': 'SR 125',
'WA 126': 'SR 126',
'WA 127': 'SR 127',
'WA 128': 'SR 128',
'WA 129': 'SR 129',
'WA 131': 'SR 131',
'WA 140': 'SR 140',
'WA 141': 'SR 141',
'WA 142': 'SR 142',
'WA 143': 'SR 143',
'WA 150': 'SR 150',
'WA 151': 'SR 151',
'WA 153': 'SR 153',
'WA 155': 'SR 155',
'WA 160': 'SR 160',
'WA 161': 'SR 161',
'WA 162': 'SR 162',
'WA 163': 'SR 163',
'WA 164': 'SR 164',
'WA 165': 'SR 165',
'WA 166': 'SR 166',
'WA 167': 'SR 167',
'WA 168': 'SR 168',
'WA 169': 'SR 169',
'WA 170': 'SR 170',
'WA 171': 'SR 171',
'WA 172': 'SR 172',
'WA 173': 'SR 173',
'WA 174': 'SR 174',
'WA 181': 'SR 181',
'WA 193': 'SR 193',
'WA 194': 'SR 194',
'WA 202': 'SR 202',
'WA 203': 'SR 203',
'WA 204': 'SR 204',
'WA 206': 'SR 206',
'WA 207': 'SR 207',
'WA 209': 'SR 209',
'WA 211': 'SR 211',
'WA 213': 'SR 213',
'WA 215': 'SR 215',
'WA 220': 'SR 220',
'WA 221': 'SR 221',
'WA 223': 'SR 223',
'WA 224': 'SR 224',
'WA 225': 'SR 225',
'WA 230': 'SR 230',
'WA 231': 'SR 231',
'WA 232': 'SR 232',
'WA 237': 'SR 237',
'WA 240': 'SR 240',
'WA 241': 'SR 241',
'WA 243': 'SR 243',
'WA 251': 'SR 251',
'WA 260': 'SR 260',
'WA 261': 'SR 261',
'WA 262': 'SR 262',
'WA 263': 'SR 263',
'WA 270': 'SR 270',
'WA 271': 'SR 271',
'WA 272': 'SR 272',
'WA 274': 'SR 274',
'WA 276': 'SR 276',
'WA 278': 'SR 278',
'WA 281': 'SR 281',
'WA 282': 'SR 282',
'WA 283': 'SR 283',
'WA 285': 'SR 285',
'WA 290': 'SR 290',
'WA 291': 'SR 291',
'WA 292': 'SR 292',
'WA 294': 'SR 294',
'WA 300': 'SR 300',
'WA 302': 'SR 302',
'WA 303': 'SR 303',
'WA 304': 'SR 304',
'WA 305': 'SR 305',
'WA 306': 'SR 306',
'WA 307': 'SR 307',
'WA 308': 'SR 308',
'WA 310': 'SR 310',
'WA 311': 'SR 311',
'WA 339': 'SR 339',
'WA 395': 'SR 395', #added from other list
'WA 397': 'SR 397',
'WA 401': 'SR 401',
'WA 402': 'SR 402',
'WA 403': 'SR 403',
'WA 407': 'SR 407',
'WA 409': 'SR 409',
'WA 410': 'SR 410',
'WA 411': 'SR 411',
'WA 431': 'SR 431',
'WA 432': 'SR 432',
'WA 433': 'SR 433',
'WA 500': 'SR 500',
'WA 501': 'SR 501',
'WA 502': 'SR 502',
'WA 503': 'SR 503',
'WA 504': 'SR 504',
'WA 505': 'SR 505',
'WA 506': 'SR 506',
'WA 507': 'SR 507',
'WA 508': 'SR 508',
'WA 509': 'SR 509',
'WA 510': 'SR 510',
'WA 512': 'SR 512',
'WA 513': 'SR 513',
'WA 514': 'SR 514',
'WA 515': 'SR 515',
'WA 516': 'SR 516',
'WA 518': 'SR 518',
'WA 519': 'SR 519',
'WA 520': 'SR 520',
'WA 522': 'SR 522',
'WA 523': 'SR 523',
'WA 524': 'SR 524',
'WA 525': 'SR 525',
'WA 526': 'SR 526',
'WA 527': 'SR 527',
'WA 528': 'SR 528',
'WA 529': 'SR 529',
'WA 530': 'SR 530',
'WA 531': 'SR 531',
'WA 532': 'SR 532',
'WA 534': 'SR 534',
'WA 536': 'SR 536',
'WA 537': 'SR 537',
'WA 538': 'SR 538',
'WA 539': 'SR 539',
'WA 540': 'SR 540',
'WA 542': 'SR 542',
'WA 543': 'SR 543',
'WA 544': 'SR 544',
'WA 546': 'SR 546',
'WA 547': 'SR 547',
'WA 548': 'SR 548',
'WA 599': 'SR 599',
'WA 603': 'SR 603',
'WA 702': 'SR 702',
'WA 704': 'SR 704',
'WA 706': 'SR 706',
'WA 801': 'SR 801',
'WA 821': 'SR 821',
'WA 823': 'SR 823',
'WA 831': 'SR 831',
'WA 832': 'SR 832',
'WA 833': 'SR 833',
'WA 900': 'SR 900',
'WA 901': 'SR 901',
'WA 902': 'SR 902',
'WA 903': 'SR 903',
'WA 904': 'SR 904',
'WA 906': 'SR 906',
'WA 908': 'SR 908',
'WA 920': 'SR 920',
'WA 970': 'SR 970',
'WA 971': 'SR 971',
'WA 9 Spur': 'SR 9 Spur',
'WA 14 Spur': 'SR 14 Spur',
'WA 14 Bus.': 'SR 14 Bus.',
'WA 16 Alt.': 'SR 16 Alt.',
'WA 16 Spur': 'SR 16 Spur',
'WA 20 Spur': 'SR 20 Spur',
'WA 20 Bus.': 'SR 20 Bus.',
'WA 26 Spur': 'SR 26 Spur',
'WA 28 Spur': 'SR 28 Spur',
'WA 92 Spur': 'SR 92 Spur',
'WA 100 Spur': 'SR 100 Spur',
'WA 104 Spur': 'SR 104 Spur',
'WA 105 Spur': 'SR 105 Spur',
'WA 108 Bus.': 'SR 108 Bus.',
'WA 109 Spur': 'SR 109 Spur',
'WA 110 Spur': 'SR 110 Spur',
'WA 125 Spur': 'SR 125 Spur',
'WA 129 Spur': 'SR 129 Spur',
'WA 141 Spur': 'SR 141 Spur',
'WA 151 Spur': 'SR 151 Spur',
'WA 155 Spur': 'SR 155 Spur',
'WA 174 Spur': 'SR 174 Spur',
'WA 240 Bus.': 'SR 240 Bus.',
'WA 281 Spur': 'SR 281 Spur',
'WA 302 Spur': 'SR 302 Spur',
'WA 503 Spur': 'SR 503 Spur',
'WA 504 Spur': 'SR 504 Spur',
'WA 510 Spur': 'SR 510 Spur',
'WA 524 Spur': 'SR 524 Spur',
'WA 525 Spur': 'SR 525 Spur',
'WA 529 Spur': 'SR 529 Spur',
'WA 903 Spur': 'SR 903 Spur',
'WA 906 Spur': 'SR 906 Spur'
}
I then ran through the exported .osm data to get ways that have matching patterns which aren’t some sort of highway or a ferry route.
#find ways that have matching keys except for ones that contain specific tags
#make sure there aren't any tags that have matching values that should not be replaced
way_search_tag_exclude_list = ['highway']
key_val_pair_exclude = [['route', 'ferry']]
for way in root.iter('way'):
tags = way.findall('tag')
tag_names = [t.get('k') for t in tags]
vals = [t.get('v') for t in tags]
tag_dict = dict(zip(tag_names, vals))
if any(lv in vals for lv in swapdict.keys()): #if any strings to replace appear in the values
if not any(lk in tag_names for lk in way_search_tag_exclude_list): #exclude any that have tags from exclude list
for pair in key_val_pair_exclude: #exclude specific key values pairs
if tag_dict.get(pair[0], None) == pair[1]: #if the key/value pair matches the exclude entry then skip to next pair in loop
continue
else:
logging.info(f"{way.get('id')}") #log all tags for ways that have a matching key
logging.info(f"{tag_dict}")
The remaining ways are
- Lookouts on the national registry that actually do use the WA prefix, and should not be changed.
- Tags for destroyed or demolished highways, which should be updated to stay consistent.
Below is the log output from the above loop.
227614387
{'balcony': 'yes', 'building': 'fire_lookout', 'building:colour': 'white', 'building:design': 'L-4 cab', 'building:levels': '1', 'building:material': 'wood', 'direction': '360', 'historic': 'building', 'name': 'Mount Pilchuck Lookout', 'operator': 'Everett Mountaineers', 'ref:nhlr': 'US 62', 'ref:nhlr:state': 'WA 4', 'ref:nhlr:url': 'http://nhlr.org/lookouts/us/wa/mount-pilchuck-lookout/', 'roof:shape': 'pyramidal', 'start_date': '1938', 'tourism': 'viewpoint', 'was:emergency': 'fire_lookout', 'website': 'https://www.fs.usda.gov/recarea/mbs/recreation/recarea/?recid=17852', 'wikidata': 'Q86927081'}
235689615
{'access': 'private', 'balcony': 'yes', 'building': 'fire_lookout', 'building:colour': 'tan', 'building:design': 'R-6 cab', 'building:levels': '1', 'building:material': 'wood', 'emergency': 'fire_lookout', 'gnis:feature_id': '1515833', 'historic': 'building', 'name': 'Alpine Lookout', 'operator': 'Okanogan-Wenatchee National Forest', 'ref:nhlr': 'US 123', 'ref:nhlr:state': 'WA 11', 'ref:nhlr:url': 'http://nhlr.org/lookouts/us/wa/alpine-lookout/', 'roof:shape': 'pyramidal', 'start_date': '1975', 'wikidata': 'Q86868000'}
372549452
{'balcony': 'no', 'building': 'fire_lookout', 'building:colour': 'white', 'building:design': 'L-4 cab', 'building:levels': '1', 'building:material': 'wood', 'charge': '85 USD/night', 'dog': 'leashed', 'drinking_water': 'no', 'electricity': 'no', 'fee': 'yes', 'gnis:feature_id': '1530305', 'historic': 'building', 'name': 'Evergreen Mountain Lookout', 'openfire': 'no', 'operator': 'Mt. Baker-Snoqualmie National Forest', 'ref:nhlr': 'US 206', 'ref:nhlr:state': 'WA 25', 'ref:nhlr:url': 'http://nhlr.org/lookouts/us/wa/evergreen-mountain-lookout/', 'ref:nrhp': '87001187', 'roof:shape': 'pyramidal', 'smoking': 'no', 'start_date': '1935', 'toilets': 'yes', 'tourism': 'chalet', 'website': 'https://www.recreation.gov/camping/campgrounds/234178', 'wikidata': 'Q86925448'}
372628985
{'access': 'private', 'building': 'fire_lookout', 'building:colour': 'lightgrey', 'building:design': 'L-4 cab', 'building:levels': '1', 'building:material': 'wood', 'emergency': 'fire_lookout', 'historic': 'building', 'name': 'Sourdough Mountain Lookout', 'operator': 'North Cascades National Park', 'ref:nhlr': 'US 208', 'ref:nhlr:state': 'WA 27', 'ref:nhlr:url': 'http://nhlr.org/lookouts/us/wa/sourdough-mountain-lookout/', 'ref:nrhp': '88003449', 'roof:shape': 'pyramidal', 'start_date': '1933', 'wikidata': 'Q17184058', 'wikipedia': 'en:Sourdough Mountain Lookout'}
963101746
{'demolished:highway': 'primary', 'lanes': '4', 'maxspeed': '45 mph', 'name': 'Pacific Highway East', 'old_ref': 'US 99;PSH 1', 'postal_code': '98422', 'ref': 'WA 99'}
985409968
{'access': 'permissive', 'building': 'fire_lookout', 'building:colour': 'lightgrey', 'building:design': 'L-4 ground cabin', 'building:material': 'wood', 'ele': '2088', 'electricity': 'no', 'fee': 'no', 'fireplace': 'no', 'gnis:feature_id': '1520758', 'historic': 'yes', 'internet_access': 'no', 'name': 'Hidden Lake Lookout', 'operator': 'Friends of Hidden Lake Lookout', 'ref:nhlr': 'US 175', 'ref:nhlr:state': 'WA 22', 'ref:nhlr:url': 'http://nhlr.org/lookouts/us/wa/hidden-lake-peak-lookout/', 'ref:nrhp': '87001184', 'roof:shape': 'gabled', 'start_date': '1932', 'tourism': 'wilderness_hut', 'website': 'https://www.fs.usda.gov/recarea/mbs/recreation/hiking/recarea/?recid=17672', 'wikidata': 'Q86928307', 'wikimedia_commons': 'File:Hidden Peak Fire Lookout at sunset, with smokey haze from nearby fires.jpg'}
985737348
{'access': 'private', 'balcony': 'yes', 'building': 'fire_lookout', 'building:colour': 'tan', 'building:design': 'L-4 cab', 'building:levels': '1', 'building:material': 'wood', 'emergency': 'fire_lookout', 'gnis:feature_id': '2554698', 'historic': 'building', 'name': 'Sugarloaf Peak Lookout', 'operator': 'Okanogan-Wenatchee National Forest', 'ref:nhlr': 'US 124', 'ref:nhlr:state': 'WA 12', 'ref:nhlr:url': 'http://nhlr.org/lookouts/us/wa/sugarloaf-mountain-lookout/', 'ref:nrhp': '90001914', 'roof:shape': 'pyramidal', 'start_date': '1933', 'wikidata': 'Q107356374'}
987006141
{'access': 'permissive', 'building': 'fire_lookout', 'building:colour': 'white', 'building:design': 'L-4 ground cabin', 'building:levels': '1', 'building:material': 'wood', 'fee': 'no', 'gnis:feature_id': '1530031', 'historic': 'building', 'name': 'Winchester Mountain Lookout', 'openfire': 'no', 'operator': 'Mount Baker Club', 'ref:nhlr': 'US 93', 'ref:nhlr:state': 'WA 8', 'ref:nhlr:url': 'http://nhlr.org/lookouts/us/wa/winchester-mountain-lookout/', 'ref:nrhp': '87001188', 'roof:shape': 'pyramidal', 'start_date': '1935', 'tourism': 'wilderness_hut', 'was:emergency': 'fire_lookout', 'wikidata': 'Q8023840', 'wikipedia': 'en:Winchester Mountain Lookout'}
987022116
{'balcony': 'no', 'building': 'fire_lookout', 'building:colour': 'white', 'building:design': 'D-6 cupola', 'building:material': 'wood', 'historic': 'building', 'name': 'North Twentymile Peak Lookout', 'operator': 'Okanogan-Wenatchee National Forest', 'ref:nhlr': 'US 161', 'ref:nhlr:state': 'WA 21', 'ref:nhlr:url': 'http://nhlr.org/lookouts/us/wa/north-twentymile-peak-lookout/', 'roof:shape': 'pyramidal', 'start_date': '1923', 'was:emergency': 'fire_lookout', 'wikidata': 'Q86925536'}
987022117
{'balcony': 'yes', 'building': 'fire_lookout', 'building:design': 'L-4 cab', 'building:material': 'wood', 'height': "42'", 'historic': 'building', 'man_made': 'tower', 'name': 'North Twentymile Peak Lookout', 'operator': 'Okanogan-Wenatchee National Forest', 'ref:nhlr': 'US 161', 'ref:nhlr:state': 'WA 21', 'ref:nhlr:url': 'http://nhlr.org/lookouts/us/wa/north-twentymile-peak-lookout/', 'roof:shape': 'pyramidal', 'start_date': '1947', 'tower:construction': 'lattice', 'tower:design': 'CT-3', 'tower:height': "30'", 'tower:material': 'wood', 'tower:type': 'observation', 'was:emergency': 'fire_lookout', 'wikidata': 'Q86925536'}
987023831
{'access': 'private', 'alt_name': 'Lemanasky Lookout', 'balcony': 'yes', 'building': 'fire_lookout', 'building:colour': 'lightblue', 'building:material': 'wood', 'emergency': 'fire_lookout', 'man_made': 'tower', 'name': 'Aeneas Lookout', 'note': 'Aeneas Lookout is the last DNR lookout in active use anywhere in Washington', 'ref:nhlr': 'US 118', 'ref:nhlr:state': 'WA 10', 'ref:nhlr:url': 'http://nhlr.org/lookouts/us/wa/aeneas-lookout/', 'roof:shape': 'flat', 'tower:construction': 'lattice', 'tower:material': 'wood', 'tower:type': 'observation', 'wikidata': 'Q86867978'}
987026744
{'access': 'private', 'balcony': 'yes', 'building': 'fire_lookout', 'building:design': 'R-6 cab', 'building:levels': '1', 'building:material': 'wood', 'emergency': 'fire_lookout', 'gnis:feature_id': '1529767', 'height': "32'", 'historic': 'building', 'man_made': 'tower', 'name': 'Mount Bonaparte Lookout', 'operator': 'Okanogan-Wenatchee National Forest', 'ref:nhlr': 'US 160', 'ref:nhlr:state': 'WA 20', 'ref:nhlr:url': 'http://nhlr.org/lookouts/us/wa/mount-bonaparte-lookout/', 'roof:shape': 'pyramidal', 'start_date': '1961', 'tower:construction': 'lattice', 'tower:height': "20'", 'tower:material': 'wood', 'tower:type': 'observation'}
987582918
{'balcony': 'yes', 'building': 'fire_lookout', 'building:colour': 'lightgrey', 'building:design': 'L-4 cab', 'building:levels': '1', 'building:material': 'wood', 'height': "32'", 'historic': 'building', 'man_made': 'tower', 'name': 'Miners Ridge Lookout', 'operator': 'Mt. Baker-Snoqualmie National Forest', 'ref:nhlr': 'US 75', 'ref:nhlr:state': 'WA 6', 'ref:nhlr:url': 'http://nhlr.org/lookouts/us/wa/miners-ridge-lookout/', 'ref:nrhp': '87001183', 'roof:shape': 'pyramidal', 'start_date': '1953', 'tourism': 'viewpoint', 'tower:construction': 'lattice', 'tower:design': 'CT-3', 'tower:height': "20'", 'tower:material': 'wood', 'tower:type': 'observation', 'wikidata': 'Q86928758'}
1083002961
{'name': 'Old Highway Road', 'old_ref': 'WA 14', 'razed:highway': 'track', 'surface': 'unpaved'}
The highway lookout ways are trivial to exclude. Other than that I’m reasonably satisfied that the script shouldn’t touch anything that it shouldn’t.
2 Likes
While doing some further validation of the script output I discovered some differently formatted tags that needed to be corrected. I adjusted the strategy to use regex instead and validate against a list of the route numbers.
This improved search uncovered something odd in the log, three roads that are tagged but don’t appear on any state route list I’ve found so far. Since this is out of scope I want to ignore it for now, but that’s something that should be looked into.
The functional parts of the script:
route_numbers = ['2', '3', '4', '6', '7', '8', '9', '10', '11', '12', '14', '16', '17', '18', '19', '20', '21', '22', '23', '24', '25', '26', '27', '28', '30', '31', '35', '41', '92', '96', '99', '100', '102', '103', '104', '105', '106', '107', '108', '109', '110', '111', '112', '113', '115', '116', '117', '119', '120', '121', '122', '123', '124', '125', '126', '127', '128', '129', '131', '140', '141', '142', '143', '150', '151', '153', '155', '160', '161', '162', '163', '164', '165', '166', '167', '168', '169', '170', '171', '172', '173', '174', '181', '193', '194', '202', '203', '204', '206', '207', '209', '211', '213', '215', '220', '221', '223', '224', '225', '230', '231', '232', '237', '240', '241', '243', '251', '260', '261', '262', '263', '270', '271', '272', '274', '276', '278', '281', '282', '283', '285', '290', '291', '292', '294', '300', '302', '303', '304', '305', '306', '307', '308', '310', '311', '339', '395', '397', '401', '402', '403', '407', '409', '410', '411', '431', '432', '433', '500', '501', '502', '503', '504', '505', '506', '507', '508', '509', '510', '512', '513', '514', '515', '516', '518', '519', '520', '522', '523', '524', '525', '526', '527', '528', '529', '530', '531', '532', '534', '536', '537', '538', '539', '540', '542', '543', '544', '546', '547', '548', '599', '603', '702', '704', '706', '801', '821', '823', '831', '832', '833', '900', '901', '902', '903', '904', '906', '908', '920', '970', '971']
regex_pattern_full_tag_wa = r'WA(?:\s{0,1}|-)\d{1,3}(?!\d)'
regex_pattern_digits_only = r'\d{1,3}(?!\d)'
exclude_tags_list = ['ref:nhlr:state'] #exclude ways with these tags from being updated
exclude_key_value_pair_list = [('tourism', 'hotel'), ('tourism', 'motel')]
tree = ET.parse('export.osm')
root = tree.getroot()
way_count = sum(1 for _ in root.iter('way'))
#replacement loop
for way in tqdm(root.iter('way'), total=way_count): #iterate through ways
way_id = way.get('id')
way_tags = way.findall('tag')
tag_dict = dict(zip([t.get('k') for t in way_tags], [t.get('v') for t in way_tags]))
if any(k in v for v in tag_dict.values() for k in route_numbers): #if any of the WA tags are present in any of the tags for the way
if not any(k in tag_dict.keys() for k in exclude_tags_list): #if way does not have tags which are in the exclude list
if not any(p in tag_dict.items() for p in exclude_key_value_pair_list): #check for excluded key value pairs
for tag in way_tags: #iterate through all tags in the way
init_tag_value = tag.get('v')
new_tag_value = tag.get('v')
matching_elements = re.findall(regex_pattern_full_tag_wa, init_tag_value)
if len(matching_elements) > 0:
for e in matching_elements:
element_numbers = re.search(r'\d{1,3}(?!\d)', e).group()
if element_numbers in route_numbers: #make sure route number is valid
new_tag_value = new_tag_value.replace(e, f'SR {element_numbers}')
logging.info(f'swap,{way_id},{init_tag_value},{new_tag_value}')
tag.set('v', new_tag_value)
else:
logging.info(f'invalid_route_id,{way_id},{init_tag_value},{e}')
else:
logging.info(f'skip_exclude_pair,{way_id}')
#else:
#logging.info(f'no_matching_elements,{way_id},{init_tag_value}')
else:
logging.info(f'skip_exclude_tag,{way_id}')
else:
logging.info(f'skip_value,{way_id},{tag_dict.get("name", "")}')
tree.write('export_updated.osm', encoding='UTF-8', xml_declaration=True)
Yeah, at a glance, I couldn’t find out anything about a State Route 273. No route markers are visible in street-level imagery. It appears to be a county road, but I don’t know if Whitman County numbers its roads even internally.
I think I’m ready to go ahead and pull an updated dataset and apply this to the actual map. I’ve attached the log from the tests to this post so others can review it for issues I might’ve missed. The parameters in the log are comma separated.
tag_replacement_log.txt (551.4 KB)
I have pushed the changes through, I did have to make one edit to the script to add the ‘action’=‘modify’ tag to updated ways so JOSM would read the changes from the imported XML properly. I also discovered a quirk with how the XML library in Python handles apostrophes. The XML exported from overpass uses 'instead of just embedding ’ in the tags. So the full pipeline was to run the Python script, then do a text replacement so everything parsed properly.
Overpass query used:
[out:xml][timeout:90];
area["ISO3166-2"="US-WA"]["admin_level"="4"]->.searchArea;
(
way[~"ref"~"(^|;)WA"](area.searchArea);
);
(._;>;); out meta;/*fixed by auto repair*/
Full script used to generate the updated dataset:
import xml.etree.ElementTree as ET
import logging
import re
from tqdm import tqdm
logging.basicConfig(filename='tag_replacement.log', level=logging.INFO, format='%(message)s')
route_numbers = ['2', '3', '4', '6', '7', '8', '9', '10', '11', '12', '14', '16', '17', '18', '19', '20', '21', '22', '23', '24', '25', '26', '27', '28', '30', '31', '35', '41', '92', '96', '99', '100', '102', '103', '104', '105', '106', '107', '108', '109', '110', '111', '112', '113', '115', '116', '117', '119', '120', '121', '122', '123', '124', '125', '126', '127', '128', '129', '131', '140', '141', '142', '143', '150', '151', '153', '155', '160', '161', '162', '163', '164', '165', '166', '167', '168', '169', '170', '171', '172', '173', '174', '181', '193', '194', '202', '203', '204', '206', '207', '209', '211', '213', '215', '220', '221', '223', '224', '225', '230', '231', '232', '237', '240', '241', '243', '251', '260', '261', '262', '263', '270', '271', '272', '274', '276', '278', '281', '282', '283', '285', '290', '291', '292', '294', '300', '302', '303', '304', '305', '306', '307', '308', '310', '311', '339', '395', '397', '401', '402', '403', '407', '409', '410', '411', '431', '432', '433', '500', '501', '502', '503', '504', '505', '506', '507', '508', '509', '510', '512', '513', '514', '515', '516', '518', '519', '520', '522', '523', '524', '525', '526', '527', '528', '529', '530', '531', '532', '534', '536', '537', '538', '539', '540', '542', '543', '544', '546', '547', '548', '599', '603', '702', '704', '706', '801', '821', '823', '831', '832', '833', '900', '901', '902', '903', '904', '906', '908', '920', '970', '971']
regex_pattern_full_tag_wa = r'WA(?:\s{0,1}|-)\d{1,3}(?!\d)'
regex_pattern_digits_only = r'\d{1,3}(?!\d)'
exclude_tags_list = ['ref:nhlr:state'] #exclude ways with these tags from being updated
exclude_key_value_pair_list = [('tourism', 'hotel'), ('tourism', 'motel')]
tree = ET.parse('export.osm')
root = tree.getroot()
way_count = sum(1 for _ in root.iter('way'))
#replacement loop
for way in tqdm(root.iter('way'), total=way_count): #iterate through ways
way_id = way.get('id')
way_tags = way.findall('tag')
tag_dict = dict(zip([t.get('k') for t in way_tags], [t.get('v') for t in way_tags]))
if any(k in v for v in tag_dict.values() for k in route_numbers): #if any of the WA tags are present in any of the tags for the way
if not any(k in tag_dict.keys() for k in exclude_tags_list): #if way does not have tags which are in the exclude list
if not any(p in tag_dict.items() for p in exclude_key_value_pair_list): #check for excluded key value pairs
for tag in way_tags: #iterate through all tags in the way
init_tag_value = tag.get('v')
new_tag_value = tag.get('v')
matching_elements = re.findall(regex_pattern_full_tag_wa, init_tag_value)
if len(matching_elements) > 0:
for e in matching_elements:
element_numbers = re.search(r'\d{1,3}(?!\d)', e).group()
if element_numbers in route_numbers: #make sure route number is valid
new_tag_value = new_tag_value.replace(e, f'SR {element_numbers}')
logging.info(f'swap,{way_id},{init_tag_value},{new_tag_value}')
tag.set('v', new_tag_value)
way.set('action', 'modify')
else:
logging.info(f'invalid_route_id,{way_id},{init_tag_value},{e}')
else:
logging.info(f'skip_exclude_pair,{way_id}')
else:
logging.info(f'skip_exclude_tag,{way_id}')
else:
logging.info(f'skip_value,{way_id},{tag_dict.get("name", "")}')
tree.write('export_updated.osm', encoding='UTF-8', xml_declaration=True)
Log file from script:
tag_replacement.txt (2.2 MB)
Changesets:
1 Like
I removed the ref=WA 273 tags. The same user previously added the fake number in 2017 and got reverted in 2020 before they restored it in 2023.
I submitted another set of changes to correct the old_name tags on roads which used to have different SR designations.
Well it looks like there might still be more work to do. I just discovered this relation that uses the wrong prefix.
I was unaware of the existence of these relations. I’ve got a working overpass query that finds them.
[out:xml][timeout:90];
area["ISO3166-2"="US-WA"]["admin_level"="4"]->.searchArea;
(
relation[~"name"~"(^|;)WA"]['route'='road'](area.searchArea);
);
(._;>;); out meta;
Unless anyone can think of a reason not to change these I’m going to clean these up too.
1 Like
Go ahead and delete the name=* tag from these relations. At least it would need to be spelled out as “State Route”, but it isn’t adding much in this form.
I can add “State Route”, that’s not an issue it’s just a batch text replacement.
Update completed, I baselined the script I used to generate the changes from the script I used for the ways. I had it select only relations and specifically target the name tags. Validated with a local diff and review of the log attached to this post.
tag_relation_replacement.txt (2.5 KB)
Could you please also update the mileposts? Example:
(I don’t recall seeing any with a signed ref.)
Milestones have been updated: Changeset: 188490675 | OpenStreetMap
Once again used the previous script as a baseline and validated with a local diff and review of the attached log. tag_replacement_relation.txt (2.0 KB)
1 Like