It doesn’t bother me too much, and I realize that for a lot of industries the railroad does the switching. It may not drop or pickup as it passes, it may just move a few around.
Ops me thinks not prototypical, hobby me thinks that caboose just adds a lot of extra complexity. :)