I found that I had used the wrong hgt files from http://viewfinderpanoramas.org. I think I used the 1" instead of the 3" versions. With the correct files, it works beautifully.
Thanks for your helpful responses. The suggestion of checking the osm file with josm helped narrow it down.