Good thread. I had the same experience installing ERR in a Lionel train set 4-4-2, which uses a very similar motor (except that the steam loco only has one of them.) One thing I've found is that the cogging seems to be VERY strong when new. It gets smoother as the loco accumulates years and miles. Perhaps the field permanent magnets lose some of their strength?
Lionel started using those walnut-sized Mabuchis circa 1979 in their DC-only sets. They first bolted one into the truck of a diesel around 1982. Later in the 1980s, K-Line followed suit. They are used in low-end train set locos right up to the present day. At some point early on, the motor case changed from a shiny gold sheen to a dull metallic grey jacket. Almost all of these motors that I've seen have a white nylon end bell, except the NASCAR series 4-4-2 steam locos. The motors in the NASCAR locos are the same length and diameter, but the end bell is yellow, and I believe there are two little orange components that look like candy drops (PTC thermistors?) on the brush terminals.
I would love to know if the motor in the NASCAR 4-4-2's is a premium, high-performance variant. I couldn't find anything definitive on the Lionel parts site. These were special commemorative items, and usually part of sets. I think the only link I found was to the generic train set 4-4-2 with no mention of an alternate part number for a different motor.
Another thought: The R/C car hobby has a lot of adherents. Installing and upgrading motors, changing gearsets, etc., is a much bigger part of that hobby than it is in 3-rail O gauge. Surely there must be an R/C car motor that would fit in these "transverse" applications, and offer higher performance than the Lionel 8008 motor and its K-Line equivalent. Following!