I tried all the steps listed but this is still happening. It started a couple days ago out of the blue. It seems other users on the Microsoft forums are having this issue as of this week: png file on my computer shows a proper thumbnail anymore. In Windows Settings > Apps > Apps & Features > Microsoft Photos > Advanced options, I clicked Reset, then rebooted again.Ran DISM /Online /Cleanup-Image /ScanHealth, which showed 'No component store corruption detected.
png app from Photos to a different app, then back to Photos (that actually made the problem worse, because now windows is assigning the files this ugly blurry default icon instead of the Photos app icon)
mp4, and other file formats seem to work just fine, it's only.