@JCitron
When switching the trains, I've noticed two or more switchers working the yard with one or two stationed at one end and a similar number at the other to accommodate the trains. This is a lot more efficient than having a single switcher run the length of the yard to go from one end to the other to handle the pushing and pulling of the trains.
Many yards have tracks at both ends, often called "drill tracks". These tracks are where the switchers operate. For most actual model railroads, it is difficult to find enough space to have a double ended fiddle yard with the drill tracks. Often the drill tracks will curve away from the yards, at a corner along the adjacent walls.