Switching layouts aren't for everyone... just like simple continuous running layouts aren't for everyone either. So it's strictly a personal decision, and inquiring here will only get you lots of different opinions. Guaranteed.
I built an HO switching layout when I was a teen, based on the Kinnikinnick Railway and Dock series of articles from Model Railroader back in the early 1970's. I enjoyed it, but admittedly there were times I missed just sitting back and watching the trains roll.
A big component that made switching layouts worthwhile was the fact that newer (for that time period) walk-around momentum throttles became generally available -- which allowed for super smooth movements of locomotives while you followed them around the layout. Does anyone remember the TAT IV throttle from Bruce Chubb??? Back in those days, you wouldn't even think of building a switching layout in O-Gauge, 'cause the transformer technology and locomotive motors that could provide smooth switching movements were non-existent.
Nowadays, Legacy and DCS systems -- along with the latest locomotive motors/gearing -- provide for ultra smooth switching movements as well as walk-around control. But standard O-Gauge coupler technology still leaves a bit to be desired, unless you're converting to Kadee-style couplers on your rolling stock.
Aside from that, it's a matter of personal choice... and whether you'll be happy with NOT being able to just sit back and watch the trains roll once and awhile. For me, that layout I build as a teen back in the 1970's was a great learning experience... enough so that I've decided I probably won't undertake a purely switching layout again. My dream O-Gauge layout would provide switching possibilities, but I gotta also have at least a dual track mainline where trains roll through the sceneryt when the feeling strikes me.
David