Let's take it from the top.
Select the "Rectangle" tool:
Drag out your rectangle. (It is in 3-D space, but it is a 2-D object):
Still using the rectangle tool, click on the surface of the rectangle, and drag out a second one inside it:
Draw two more rectangles, inside the surface of the second one:
Switch to the "Push-Pull" tool. Click on the first rectangle. Push in a little to give the wall some thickness:
Now use "Push-Pull" to click on the window frame. Pull out to give the frame some depth:
Finally, switch to the "Select" tool (the arrow). Click a window to select it. Hit the Delete key to erase it. Repeat for the other window:
If you want to stay in 2-D, just skip the "Push-Pull" steps.
Hope this helps.