By the way, your analysis indicates a correlation (based on very few data points), so it’s inaccurate to describe it as a causation. One can also observe, with as many data points, that there were more contributors immediately after the TIGER import than before it. There are many other potential factors at play, which would be a decent separate topic.
The editor-layer-index project has collected a number of high-resolution imagery sources at the state and county level that you could use to supplement NAIP. ELI is what powers the background imagery selector in iD; there’s a similar index for JOSM.
Many of the local sources are leaf-off imagery, which is great for avoiding the edge case mentioned earlier. However, this also means the conditions will be more damp in some places, changing the appearance of some surfaces.
Some of the agencies that have published these layers also publish high-resolution DEMs, though few have been added directly to ELI.
@jdalrym2, it would certainly be interesting to see what the results would look like if you a) used a much smaller training set of ways with a known to be correct surface tag, or b) used only ways assigned both a surface and smoothness tag for training?
I am pretty sure you are currently severely limited by the quality of the training data.