Let's take it from the top.
Select the "Rectangle" tool:

Drag out your rectangle. (It is in 3-D space, but it is a 2-D object):

Still using the rectangle tool, click on the surface of the rectangle, and drag out a second one inside it:


Draw two more rectangles, inside the surface of the second one:

Switch to the "Push-Pull" tool. Click on the first rectangle. Push in a little to give the wall some thickness:

Now use "Push-Pull" to click on the window frame. Pull out to give the frame some depth:


Finally, switch to the "Select" tool (the arrow). Click a window to select it. Hit the Delete key to erase it. Repeat for the other window:

If you want to stay in 2-D, just skip the "Push-Pull" steps.
Hope this helps.