Stan Fischler's classic on the NYC subways (Originally called uptown/downtown) implied that the NYWB ended at 180th street (which always puzzled me, because I could see the old bridge supports south of 180th driving on the Cross Bronx Expwy), this explains it. I did briefly glance the old Westchester Ave station taking the train from Boston to NYC, I made it a point to look for it.
The question I have is couldn't they have simply built a crossover, and used the New Haven's tracks into NYC...or did the New Haven even go to NYC? I know the current Metro North New Haven branch does, but they do so by connecting with the Harlem division I believe at Mt. Vernon, switching from overhead to third rail (I remember seeing it, I lived in the northeast Bronx for around 8 years in the late 80's-mid 90's and commuted from the Botanical Gardens station, and more then a few times we ended up heading all the way to Mt. Vernon, then turned around and got dropped off going south...). Did the New Haven do that back in the day to get to Grand Central, and the New Haven't South Bronx yard was the end of the line, literally?