I would buy your loco first and measure carefully. Ask a few friends to bring their largest locos and do some firsthand research! But I do think you would be safe at 6". After all, that's the spacing Lionel used for its concentric curves: O48 - O36 = 12, divided by two (for either side of the oval) = 6" between mainlines. This formula holds as you go from O48 to O60, O60 to O72, etc., although at larger radii you probably don't need 6" unless you're running articulateds and 21" passenger cars.
All that being said, I think you're wasting space and at 6" your parallel mainlines won't look very realistic. I'll venture a guess that you could get away with 5.5," and probably much less on parallel straights. The key is not to get the mainlines too close in curves, and especially when exiting out of a curve near an adjacent straight. Sometimes you can reduce the overhang with an "easement"- Use one piece of larger diameter curve to lead into the O36. Of course you have to do the same thing when you come out of the curve, and this will take up more room. But it'll pay dividends in terms of realism and reliable operation.
I like the transition era, and I'm not a fan of articulateds, so I can get my tracks closer than someone who wants to run Big Boys, 6-axle diesels, and long modern-era freight and passenger cars. YMMV.