Whilst the pings remained quite low last night and thus tracerts were not going to show up an issue, later in the evening the connection info was showing quit a lot of packet loss, occasionally reaching 25%. That is very high. But it's difficult to say whether that is the server, or anywhere between server and sender / recipient. It would only take one person to be having a problem with their connectivity, and for them to be the only person at that moment to be trying to transmit, and the packet loss would be affected.
I've been tweaking, and will try and continue to tweak the server connection codec bandwidth use, which might affect voice quality, to see if a balance can be obtained.