My kids are about the same age as your kids. Our layout is on the carpet and I have a bunch of accessories that we swap out periodically. It's been nice in that I got to test out a bunch of things and see what works for them. At these ages they like to be hands on - and that can be rough.
The Oil Drum Loader is a favorite and all they have to do is put the barrels back in the slot. Dump them in to a gondola or a post war metal dump car and you shouldn't have to worry about things getting broken. The kids will play with this one all day. My kids don't put much in their mouth but oil drums are small enough to be a choking hazard.
https://www.youtube.com/watch?v=jb5CvKdXNcQ
Forklift platform - doesn't have much they can mess up. Like the Oil Drum loader, it is easy enough for the kids to put the boards back on the train car without knocking the car off the track. Hours of fun. I have an extra and would sell one. If you are interested my email is in my profile.
https://www.youtube.com/watch?v=ATidAXcWv0U
Piggy Back loader - My 5 and 3 year olds can do this, and will, all day. It doesn't get more hands on than this. The hollow trailer sides can be a little fragile if say the 1 year old is trying to remove it from the 4 year old's hand... but these are easy to put a piece of foam in to help stiffen them up.
https://www.youtube.com/watch?v=gVBEZJ8nGLg
Lionel 164 style log loaders. These are great fun, hands off, and all the kids like working the buttons or levers depending on the version you have. The buttons are more kid proof. The levers on the newer versions are a little weak. It would be nice if the spokes came back up after dumping the logs in a car. It is possible to leave the spokes out across the tracks.
The kids love the Lionel saw mill action but removing the logs from the saw mill and refilling the boards is best left to adults. Particularly if you end up with a "I'm putting the boards in - no I'm putting the boards back in" situation you will end up with something broken. Not that that ever happens here. My kids are all yes sir, yes ma'am and ok you put the boards in I'll take a turn some other time...
Ice station - they love it! It also causes all kinds of problems when the station worker stuffs the ice car with more than 5 ice cubes, when the kids are constantly removing the ice car to retrieve the ice cubes, when the tracks are shorted from kids trying to put the ice car back on after retrieving the ice cubes. Keep this one on your list for a few years down the road! My 5 year old is great with it as long as he is the only one by it. The 3 and 1 year old are great a loading it. Unloading is for adults. My kids don't put much in their mouth but ice cubes are a choking hazard.
Anything coal is a big mess. Period. It's on the track, the floor, the train cars, in the kids ears... Of course the kids love a big mess. We tried things instead of coal, larger plastic beads were ok - still everywhere. Cleaning everything well and experimenting with Nerds candy was their favorite. Fun for once in a while, super popular at family parties. Nerds all over. A hand crank coaling tower was made briefly but I've never found one. If I do, I'd probably buy it despite the mess just because it is manual and kids love that.
Culvert loader and unloader - I was lucky enough to find the old hand crank version of the culvert unloader. It is very hands on. The kids love it. There isn't much they can break other than the fence on the back is thin and culverts do not always load or unload straight which puts small hands on the fence. We have the automated culvert loader to compliment it. The kids much prefer hand cranking to pushing a button.
Gantry crane / track side crane. The kids love it. Like the coal, it is a big mess. We use plastic coated paper clips to help prevent shorting the track. At this age the kids are focused on whatever is in their head, not what is actually going on. Moving the magnet out of the path of the trains is not something they keep in mind, especially if one kid is running the train and another one is running the crane. There will be stuffs left on the track at most times. Most versions of the cranes wind the magnet up or down as the cab is rotated, this concept is lost on the children. If you have the magnet all the way up and turn the cab you either lock it up, or break the string. Running the cab in circles is fun, always circling. The magnet can get HOT. Older versions had a light to show when the magnet is on. This is most helpful for the children. The light in the cab can get hot - hot enough to melt the cab. I replaced mine with an LED. I have an extra 182 track side magnetic crane with red led in the cab that I would sell. When I find good deals I tend to buy as I can always sell it, or let more than one kid run a crane at the same time, etc.
Milk car and platform - All the kids love it. 5 year old is fine. The 3 and 1 year old can unload. Putting the milk cans back in is not for the little ones. My kids don't put much in their mouth but milk cans are definitely a choking hazard. Magnetic milk cans can work a hole through your child's insides and kill them if more than one were to be swallowed. Small magnets are generally a bad idea around kids. We lock up the cans when not actively playing with this one with adult supervision.
Cow / Horse corrals - the kids love to watch them. These items are finicky and often the animals do not want to get in/out of the cars, fall over, etc... It is only fun when it works. Kids love to just press the button and have the animals go around in the pen though - it is only problematic when you involve the stock car. Get a horse or cow sound car to run around the layout when you have the corral out and the kids lose their mind.
I have the barrel loader and unloading car but have not kid tested them yet. I am guessing they will be a hit and there isn't much to go wrong with either piece.
The barrel loader, log conveyor, saw mill and culvert loader / unloader take up a lot of straight track. The other items mentioned have a decently small footprint.