Regarding your Amazon pre-wired LEDs, for O-gauge buildings, will 7" wires be long enough? Also, this style of LED has narrow (20 degrees) output. A crude analogy would be trying to illuminate a room with a bunch of flashlights. What's nice about traditional incandescent bulbs is they project light 360 degrees which generally makes for more uniform, smooth lighting in buildings.
As to directly driving the 12V DC pre-wired LEDs from a fixed Z-4000 AC output, even if you get the voltage right (using resistors or whatever), you will get pulsed light. The LEDs will be flashing on and off 60 times per second. Many people claim they can't "see" this flickering but it's there and can be annoying over time ... sort of like the eye-strain folks would complain about in traditional fluorescent room lighting (old 60 Hz ballasts vs. modern high-frequency electronic ballasts) or older CRT computer displays which pulsed at 60 Hz or so. To that end, look into a method that includes a bridge-rectifier as suggested by others.
Finally, have you considered using the 12V DC LED strips that come in rolls? There have been hundreds of OGR threads on using these. They end up costing about 2 cents per LED, they include the resistor so no fussing with tiny components, they are wide-angle for more uniform lighting, probably easier to mount in buildings, and so on. In practice they do end up requiring some soldering of wires so not exactly plug-n-play.
In any case, if you want to use the Z-4000 AC output to drive DC LEDs, you really need "something" in between. As you point out, if you do this "something" on an LED-by-LED basis such as provided by the Evans LEDs, it can get somewhat spendy ($2.50 per LED). Hence, much more effective to provide this "something" once. The something of course is any of dozens of AC-to-DC converter modules as suggested in earlier postings. Less than $5 to drive dozens of LEDs. So your all-in cost per LED is at least 10 times less.