Right. So you are using option 2 in the Tortoise diagrams. You have two 12V DC-output adapters connected as shown. And you have re-wired the 3-terminals of the SPDT manual toggle switch to the 3-terminals of an AIU SW port.
Same "problem." That is, when you press "straight" or "diverge" the AIU relay clicks (or "pulses" to use the terminology of the day) for only 1/2 sec. I suspect that if you wired it correctly each time the Tortoise moves just few degrees (it is only getting power for 1/2 second). Then do it again in the same direction and it moves a bit more. Then maybe after a dozen presses (or whatever) the turtle finally makes it to the other side?
You still need a latching relay for each Tortoise. Again, try an Advanced OGR search with the terms suggested. Note that if you use the suggested external latching relay per the MTH website, it would be something like $40 per Tortoise!
In round numbers an off the shelf latching relay is about $10. If you don't mind some soldering or wiring you can get it down to maybe $2 per Tortoise. All this detailed in the various OGR threads.