... I guess the real question is can the DC negative output of the buck converter be "phased" with my transformer by connecting the negative output to common, since that would have to be the return path though the non-isolated outside rail.
How much current should I expect a relay board to draw from the control inputs? Without a schematic (and without a general understanding of how one would design an optocoupler input for a relay) I don't have any intuition about that.
Here's a diagram I posted on a previous OGR thread (which I can't find). This shows one way to do the "phasing" between the AC track voltage common and the 12V DC relay voltage. You need an electrically isolated DC power source such as using a 12V DC-output wall-wart. These are about $2 on eBay (free shipping from Asia).
As shown, tie the DC- to the AC outer rail. Using a 12V relay module with "low" level triggering, a relay is then triggered when a train wheel axle straddles the outer-rail into the insulated trigger section.
The inputs to these relay modules are simply a resistor driving the opto-isolator LED - the current requirement is a few mA to trigger the corresponding relay. As has been noted, you should be able to get a 1,2,4,8, or 16 channel relay module for less than $1 per relay on eBay (free shipping from Asia).
There are DPDT relay modules but as you pointed out, at $1 per relay (SPDT type) it's hard to resist just using two SPDT relays. Obviously you just drive both relays with the same input - and since the input current requirement to trigger the relay is small, this should not a problem.
Separately, regarding the chatter issue.
Again, another diagram I posted on a previous OGR thread on this very topic. What you can do is add a 10 cent Resistor-Capacitor filter which will demote chatter when a consist enters/leaves an insulated rail trigger section. That is, when you only have one or two axles in the trigger section you will frequently get intermittent contact and the relays respond in a fraction of a second and hence chatter on/off. For "low" level trigger, the lower diagram applies. The "trick" if you want to call it that is the DC input voltage to these relay modules do NOT direct drive the relay coil. There is a transistor buffer so that you only need a few mA to trigger the coil instead of tens of mA if driving the coil directly. This means you don't need a huge capacitor across the coil to "hold" the voltage during the brief losses of trigger voltage. If this makes sense to you I'd have to go back an find the OGR thread where I'm pretty sure I came up with some suggested values. I do recall if you try to do this with a 12V DC bare relay (no buffer), you need a capacitor in the 100's or 1000's of uF to de-chatter. I'm thinking it was something like 10's of uF. The low-value resistor (tens of Ohms or so) is used to limit inrush current to a discharged capacitor so you don't get sparking.
Update: I found this video from this OGR thread showing an 18 Ohm and 47 uF filter.
Example of 10 cent R-C filter. Note that while you can get qty 50 or qty 100 of a single resistor value for 99 cents (free shipping from Asia), if you're just getting in to all this electronic stuff I recommend a resistor assortment for $2 which gets you a assortment of values. In this case the assortment does not have an 18 Ohm resistor but it does have 22 Ohm which is effectively the same.
Separately, in re the inductive load derating issue. For any modern O-gauge engine with electronics, the track voltage does not directly attach to the inductive motor winding. You have all kinds of components in between - at minimum a bridge-rectifier since "all" engines now use DC motors. Assuming the engine has any smarts at all - such as reversing capability - there will be transistors that drive the inductive motor and protection components to clamp the wild voltage excursions during interruptions in motor drive current. These semiconductors are much more sensitive to voltage excursions than a relay contact so be assured these have been clamped before they make it back to your block relay contacts!