I don't understand what happened, but I have to believe one factor may have been not allowing it to "cure" long enough in the environment it was to be used in. I brought the Baltic Birch and Homasote into my finished basement and it was stored flat for a few weeks before the actual build. I have always had a couple dehumidifiers running keeping the humidity below 40% at all times. When the Homasote was laid, I put lots of screws along the edges, but probably a screw a foot over the middle sections. I used no glue as I was thinking if I ever had to pull some up.
I had absolutely no buckling of any kind for the Homasote, it's as flat today as when it was laid. I do note you have significant runs with no breaks, that could be a factor in the issues with buckling.
Here's a shot of after the screws were filled before sanding and painting the surface.