I used to love using portals for background traffic and for delivering wagons for me to sort and deliver.
I haven't used them recently, but I intend to when my route is ready for them.
What I would say, from my experience with them in the past, is to make sure you give them plenty of space to spawn and leave the portal area.
The Portal spawning gets upset if you don't.
I fell for that when I first started using them.
Just make sure to give them a long line out from the portal, without any signals or junctions getting in the way until the portal has completed the spawning process.
Maybe make the first driver command sound the horn of the train, so that you have audible confirmation that the spawning has finished and the driver commands are being processed.
See if that helps.