I respect Bob2's knowledge of brass. But these are 3-rail locomotives with as many as eight (8!) rubber tires if they're all still intact. No skidding. If you take the flywheel out, and some inveterate postwar Lionel operator hits the "DIR" button, even if the train is running slowly you'll have a bone-jarring stop, and a spectacular derailment. Even worse you could bend a driving rod, break a crank pin, or cut a burr into the brass worm wheel. Flywheels are a good thing!
Pat is a wizard, I love his conversions! But with your stock gearbox, the Pittman motor will be too slow. Although it's noisy and a little cheap, your stock motor is good for 13,000 RPM at 24 volts (which you could get back in 1990 with a Right-of-Way transformer or a prewar Type Z.) This would equal the 70 mph speed of the prototype. The slow-turning Pittman redlines at about 6800, so your loco would be lucky to hit 40 mph with that. If your current motor is a 3-pole RS-550 you could replace it with a 5-pole RS-555, but it would still have to be the high-rpm winding.
The MTH Challenger has a 12-volt, 6800 RPM Pittman motor and self-locking 16:1 worm gears. This combo is good for about 90 mph on the top end, which is faster than the prototype, and IMO faster than anyone needs to go. With 30" of loco, how long are your straightaways? In a few seconds you'll need binoculars to see the darn thing! It also has a flywheel that's really too small to be effective at the low RPMs it's turning. I had the PS1 version. Again, if a kid hit DIR while it was running at normal speed, you would get a bone-jarring stop. The old 773 was much more friendly in this regard. The Challenger was hyped as a "scale model" when it came out in 1993, but it had shiny driving wheel tires and handrails, oversized smoke box hinges, no sprung drivers, etc. Despite all the hype and the high price it was still very much a toy train, and in some ways a step backwards from the products offered by Williams and Weaver.
With 16:1 gearing, the PS1 version had to be coaxed to run consistently at anything below 10 mph. For example: pull out of the engine house (no trailing consist.) As soon as the drivers got on a curve it would slow noticeably or stall. With all that weight over the rear tires the wheels couldn't skid, and there just wasn't enough torque or stored energy in the flywheel at 800 RPM to overcome the sudden increase in friction. I gradually became disappointed and traded mine, lost a lot of money. Instead of installing a separate 30:1 gearbox like they should have, MTH (and Lionel) applied the band-aid of electronic speed control, which gets us to where we are today.
What you have is actually very good, and I encourage you to work with it. Flywheels aren't expensive. Order the largest one you can from Lionel or MTH, or try the trick I recommended with the file. Definitely add the DynaXorb and the rubber grommets, this isn't hard to do. Then come back and tell us if it worked!