If you put current 5ft O Scale nest to Proto-48, you probably can see the difference in track gauge and the narrower wheelsets and flanges. However, if you just use plain old O scale by itself it looks pretty good. Also the difference between Proto-48 and 5 ft O scale is not nearly as obvious as between 3 rail and 5ft.
With Proto-48, to get the full affect, you need to use narrower truck bolsters and narrower truck sideframes to show the obvious proto-48 look.
Unless you want to do a lot of conversion and scratchbuilding, you probably don't want to do Proto-48. Proto-48 is certainly more accurate in the track and and wheels, but there is not much available RTR. Also, steamers would require very significant skill to convert.
Standard O Scale requires relatively the same precision that HO does. This is more that required for 3 rail, but less than Proto-48.
As for transition from different rail heights, say 125-148 It is relatively easy. The way I do it is to use the larger rail 148 size joiner. I cut a small strip of brass that is the same width as the base of the smaller 125 rail and maybe half inch long - can be longer or shorter. For this transition a strip, 023 inches thick by probably 3/32 or .093. Width does not matter as long as less than or equal to the base of the rail. .023 thick is not so common, but .022 is usually available in the brass strip racks in hobby shops and will work fine I think. Solder this strip to the underside of the base of the smaller 125 rail to make it the same height at the joint as the code 148. Then slide the smaller rail into the rail joiner and the top should match the top of the coder 148. You may be using only one size rail joiner.