Interesting, but problematic. It seems that your design is a combination of "let-em-run" and an "operating" railroad. The operation side is where I see problems. If you are pulling cars from (or pushing cars into) the two longest tracks at either the coal yard or the town freight yard, you will be fouling the main line.
Suggestions: Why not run your "town spur" between the mainline and the town; then, curve it around to your coal yard. Might even put in a little passing siding at the coal yard just for run-around moves there. To fix the town yard issue, try moving the mainline completely to the outside of your table. Then, add a passing siding to the inside of the main line with an extension on the town end that becomes your branch to the coal yard. Your new, concentric passing siding becomes an arrival/departure track for the town yard, and with the branch extension through town, you now have a yard lead that doesn't foul the main line.
Chuck