MED has been spot on in everything presented. Do that. Nothing over that max anymore. You may have cooked the solenoid coil making it weak. Always start low volt on unknowns in model RR (and dc before ac.) If you don't see action by 9-10v, it is unlikely you will unless it's electronics wanting 12v.
But nobody has said they've had one apart to see if somebody hasn't slipped a (an uneeded ) small board or diode or something in the design.
If it works or tries to when tried on high voltage, but not on low, it may be the coil has a short in the windings.
But the OP also mentioned a surging without explaining them. What was surging? An attempt at working? Lamp brightness?; Jerky action? Measured Voltage?
I can't see the higher voltage causing a triggering that doesnt unlatch either; not unless there is something to latch. I.e. ???? a redundant relay or transistor that actually could have been skipped 99.9% of the time ?
18v isn't going to jump an air gap at the isolation points until they are like microns from each other.
And doesn't that yellow kline transformer use a shark fin or square wave? That would explain the lionel not working if the electronics want "wave#X".
Which older lionel lionel is failing to work? Pure sine? Or one of those little DC starter set power pacs rated in VA with an ac accessories output?.. (awful little buggers)
For fun try blowing the old transformer whistle to see if it triggers, maybe revere transformer leads and try again. (this throws a dc offset and that wave just may excite something "stuck" signal wise....Working one polarity not the other tells us there is likely something other than a coil and bulb in there) ...It is a start at an answer if MEDs help didn't solve it already.