If you run a manual ping to one of the game server ip without the game running do you get good numbers. If you have access to another device what happens if you ping the game servers ip at the same time the game itself is complaining.
Games so called ping times are generally not actual ping commands. They are traffic that the game sends itself that does a similar function. Problem is anything that can affect the game can affect its ability to give valid numbers. I have seen people say fixing video setting fixes ping times...it sounds crazy since the video is not sent over the network but if the game is really busy doing video stuff it may not have time to do the network stuff.
Now if you are seeing issue with a real ping command then you need to go the next step of things like tracert to try to see if it your router or something farther into the internet.
I will assume you are not using wireless, that can cause all kinds of strange issues.