[TERA PC & Console] En Masse is closing, but TERA lives on! We will continue to support TERA PC (NA) and TERA Console until service is transferred. Stay tuned for more information.
[TERA Console] The Grotto of Lost Souls update (v85) is now live! Read the patch notes here: https://bit.ly/TERACon_v85

[TERA PC] The 64-bit update (v97) is now live. Check out all the changes delivered on August 11 here: https://bit.ly/tera64_patchnotes
[TERA PC & CONSOLE] Summerfest Part 2: The Beach Bash is on from August 11 until September 1! Participate in event activities to earn tokens redeemable for costumes, consumables, mounts, and more! Details: https://bit.ly/tera_sf20

[PSA][Information] Lagging Every Day: It's NOT your fault, but it's also NOT the server's fault!

So I'm seeing a lot more people who are noticing the ping spikes in-game at a certain time period, and felt like a very large majority of them are uneducated about WHY they're lagging.

This thread came up immediately on a Google search and I took the advice from one of its members and used PingPlotter myself to find the following:

UC39rt2.png
(Updated with a 30-minute plot)

So what you see is that I have a IP address (hidden for obvious reasons), Tera's IP is 208.67.49.145 (or 208.67.49.150 if you use CMD), my ISP is Comcast, and there exists a routing service between me and Comcast and Tera called Zayo. Pay attention, this is important.

See where the packet losses are occurring and what "bars" are the farthest to the right; this is the place which is causing all this latency (lagging) problems. Let's look a little to the left. Ah-ha! The name of that hop is Zayo!

Disclaimer: This is just a 30 minute recording of the ping at 5 second intervals. Obviously it's not a good sample, but I'm continuously running PingPlotter to give you guys a better one later.

tl;dr or the super dumbed down version of this PSA:
Your ISP is fine
Your computer is fine
There's nothing wrong with Tera's Client
Tera's Servers are also fine
ZAYO'S ROUTING SERVICE IS NOT OKAY AND CAUSING ALL THIS SUFFERING AND PAIN FOR ALL OF US
«13

Comments

  • voidyvoidy ✭✭✭✭✭
    People have been talking about Zayo for months, but as far as I can tell nothing's been done to change it. At one point perhaps they tried something different, as indicated here (https://forums.enmasse.com/tera/discussion/8939/please-turn-zayo-back-on) but it just caused problems for a different group of people.
  • I cry. I cant play tera during peak time.
  • edited November 2016
    voidy wrote: »
    People have been talking about Zayo for months, but as far as I can tell nothing's been done to change it. At one point perhaps they tried something different, as indicated here (https://forums.enmasse.com/tera/discussion/8939/please-turn-zayo-back-on) but it just caused problems for a different group of people.

    Please correct me if I'm wrong, but most people who are having problems WITHOUT Zayo are those who play from outside US, from a farther distance, or use various VPNs or 3rd party services such as WTFast. I can't be the judge of whom EME should prioritize, but the above post is undeniable proof that Zayo is causing problems to perhaps hundreds of players on Tera currently. If some of the players wishing to turn on Zayo provided some plots of their connection to Tera with Zayo off, that would help their point, but all they're complaining is about higher ping which could have been caused by many other factors.
  • voidyvoidy ✭✭✭✭✭
    Seraphrose wrote: »
    voidy wrote: »
    People have been talking about Zayo for months, but as far as I can tell nothing's been done to change it. At one point perhaps they tried something different, as indicated here (https://forums.enmasse.com/tera/discussion/8939/please-turn-zayo-back-on) but it just caused problems for a different group of people.

    Please correct me if I'm wrong, but most people who are having problems WITHOUT Zayo are those who play from outside US, from a farther distance, or use various VPNs or 3rd party services such as WTFast. I can't be the judge of whom EME should prioritize, but the above post is undeniable proof that Zayo is causing problems to perhaps hundreds of players on Tera currently. If some of the players wishing to turn on Zayo provided some plots of their connection to Tera with Zayo off, that would help their point, but all they're complaining is about higher ping which could have been caused by many other factors.

    I have no way to know if you're wrong or not. The timing lines up; Zayo goes away, and a certain new demographic gets the lag that the old ones used to receive. But that doesn't necessarily mean one event caused the other. I certainly want to believe that in their case it's just other issues causing their latency so we can do away with the Zayo problem without any lasting consequences, but I doubt that will be the case.

    Hopefully their team finds some way to resolve this issue in a way that doesn't screw over anyone.
  • LancerJivaLancerJiva ✭✭✭✭✭
    That's all well in good, and I realize that it isn't the servers, but still, this is out of the players hands, and EME are the only ones that can fix it.
  • TacticalGigaTacticalGiga ✭✭
    edited November 2016
    Seraphrose wrote: »
    voidy wrote: »
    People have been talking about Zayo for months, but as far as I can tell nothing's been done to change it. At one point perhaps they tried something different, as indicated here (https://forums.enmasse.com/tera/discussion/8939/please-turn-zayo-back-on) but it just caused problems for a different group of people.

    Please correct me if I'm wrong, but most people who are having problems WITHOUT Zayo are those who play from outside US, from a farther distance, or use various VPNs or 3rd party services such as WTFast. I can't be the judge of whom EME should prioritize, but the above post is undeniable proof that Zayo is causing problems to perhaps hundreds of players on Tera currently. If some of the players wishing to turn on Zayo provided some plots of their connection to Tera with Zayo off, that would help their point, but all they're complaining is about higher ping which could have been caused by many other factors.

    That's basically what it looks like from that thread. If they want a routing service, there are many services that they can get for that. It is ridiculous to make those of us in NA suffer to the point to were the game is literally unplayable (1000+ ping spikes), to improve the ping of those who aren't even from the Area that the game is being published to.

    I don't understand how they're determining that Zayo is the solution to their problem when they aren't even providing traces or anything show what is causing their lag to begin with. Where as in our case, several of us posted clear proof that Zayo is the cause of the lag.

    I have also noted the same thing when I ran a trace from my computer (Refer to the thread below):
    https://forums.enmasse.com/tera/discussion/9284/routing-issues-can-we-get-someone-from-en-masse-to-respond#latest
  • LancerJiva wrote: »
    That's all well in good, and I realize that it isn't the servers, but still, this is out of the players hands, and EME are the only ones that can fix it.

    Again, I'm only trying to spread the correct information so people can stop screaming "Fix your Servers" and instead demand that EME talk to Zayo about this problem. Thousands of players coming on the forums to say "Tera your servers are potato" sound like whiners, but coming on the forums to provide accurate information and back that up with data seems much more intelligent, professional, and will be conveyed easily.
    That's basically what it looks like from that thread. All those complaining seem to have 100-300 ping to begin with. If they want a routing service, there are many services that they get for that. It is ridiculous to make those of us in NA suffer to the point to were the game is literally unplayable (1000+ ping spikes), to improve the ping of those who aren't even from the Area that the game is being published to.

    I share the same opinion. The latency today seems to be lingering for much longer than yesterday (which had a high congestion due to events), and while I have nothing against EU players who migrate over to NA, if they demand Zayo just so they can play more smoothly, that would be a different story.
  • LancerJivaLancerJiva ✭✭✭✭✭
    Seraphrose wrote: »
    LancerJiva wrote: »
    That's all well in good, and I realize that it isn't the servers, but still, this is out of the players hands, and EME are the only ones that can fix it.

    Again, I'm only trying to spread the correct information so people can stop screaming "Fix your Servers" and instead demand that EME talk to Zayo about this problem. Thousands of players coming on the forums to say "Tera your servers are potato" sound like whiners, but coming on the forums to provide accurate information and back that up with data seems much more intelligent, professional, and will be conveyed easily.
    That's basically what it looks like from that thread. All those complaining seem to have 100-300 ping to begin with. If they want a routing service, there are many services that they get for that. It is ridiculous to make those of us in NA suffer to the point to were the game is literally unplayable (1000+ ping spikes), to improve the ping of those who aren't even from the Area that the game is being published to.

    I share the same opinion. The latency today seems to be lingering for much longer than yesterday (which had a high congestion due to events), and while I have nothing against EU players who migrate over to NA, if they demand Zayo just so they can play more smoothly, that would be a different story.

    In a way its just easier to say "fix your servers" because its the routing path that leads to the servers thats causing the servers to be potato. Not everyone knows or goes the technical route of saying "fix Zayo" or whatever it is.

  • YamazukiYamazuki ✭✭✭✭✭
    They did turn off Zayo not long ago, and all it did was cause issues for other people. One of the main issues is NA has very poor and out dated infrastructure and it would cost all these providers money to improve it, and they aren't going to until they're forced to. Tera is not the only game to be impacted by the bad infrastructure either. Some other games and other websites have had to deal with it, and large companies like Blizzard, Riot, Google, Amazon etc tend to use some of their own systems due to the amount of money they have to minimize these problems.
  • edited November 2016
    Yamazuki wrote: »
    They did turn off Zayo not long ago, and all it did was cause issues for other people. One of the main issues is NA has very poor and out dated infrastructure and it would cost all these providers money to improve it, and they aren't going to until they're forced to. Tera is not the only game to be impacted by the bad infrastructure either. Some other games and other websites have had to deal with it, and large companies like Blizzard, Riot, Google, Amazon etc tend to use some of their own systems due to the amount of money they have to minimize these problems.

    Money is always an issue and most of us are aware of it. I'm not sure if you've read my post on the "Server Instability" thread, but the problem that people have is not necessarily the fact we have poor infrastructure or even the latency, but the fact the staff SAID they'll do something about it then do not follow up in months and refuse transparency. In fact, if EME flat out came and said "Screw you guys; we're greedy mofos and we're not going to change our system", there will be many who quit, but the rest won't be raging as much as we saw in the past few days/weeks.

    Furthermore, it's still very unclear what the negative effects were when Zayo was removed. People reported higher ping, packet loss, and disconnecting, but as mentioned previously, this can be due to multitude of other factors and the complete lack of even one person bothering to post a screenshot of their ping traces suggest they didn't find it too significant to do so.

    On a brighter note, as of 11:05 PM EST, the latency is gone. WOOHOO

    2aLLaX2.png
  • Just because this issue went away @ 10 PM CST, does not make it something worth dropping. I'm still waiting to see what EME has to say about this. I'm sure we'll get the exact same lag again tomorrow like how it was for the past week or so.
  • edited November 2016
    Just because this issue went away @ 10 PM CST, does not make it something worth dropping. I'm still waiting to see what EME has to say about this. I'm sure we'll get the exact same lag again tomorrow like how it was for the past week or so.

    Oh yes. I'm very well expecting another one of these episodes tomorrow at around the same time (7PM EST for me). I'll be bumping and bothering EME about this until someone can give me a more clear answer

    EDIT: I also have a couple of theories to WHY it's just these times. It's pretty much the worst possible scenario, and I'd rather keep it to myself until more solid proof appears
  • Seraphrose wrote: »
    EDIT: I also have a couple of theories to WHY it's just these times. It's pretty much the worst possible scenario, and I'd rather keep it to myself until more solid proof appears

    You don't need crazy theories: it's because it's primetime data consumption time in the U.S. Everyone gets home from work and starts streaming, torrenting, or whatever else they do on their Internet, and routing companies are not willing to invest the money necessary to increase capacity to handle the load. This is the same reason why Netflix had such problems with routing providers in the U.S. during primetime over the years. The only solution Netflix could do was pay the ISPs like Comcast to host nodes inside their own network, which works for a streaming company but obviously isn't possible for an MMO. And indeed, in the recent experiment where EME switched off Zayo to route people over Savvis, it just traded one set of problems for a different demographic because all the routing providers have capacity problems. Again, if a large company like Netflix couldn't resolve the routing problems without embedding hardware directly at the ISPs, a small fry like EME isn't going to be able to call them up and get any traction. (Actually they'd call their hosting provider, who would then have to call their backbone provider, who would then have to chase the issue to the faulty/over-capacity node which could well be in another state/country...)

    As for why it's gotten worse in the last few months, I suspect it's in part because the Mirai botnet of IoT devices is causing havoc everywhere and straining all the networks, even those not being directly targeted. This additional strain combined with primetime traffic is enough to cause significant congestion and packet loss. And because the devices are distributed everywhere, it's going to affect all the routing providers. And indeed, if you follow various MMOs, there's been similar issues across the board in the last month or so.

    So that's why there's no easy solution. That doesn't mean that EME should not keep trying whatever they can to keep pushing the complaints up the chain, but companies like Zayo/Savvis/Level3/whatever aren't going to spurred to action by a small niche MMO.
  • feazesherofeazeshero ✭✭✭
    edited November 2016
    Yep this is the same result I get when I run a trace route, always packet loss at zayo. Last night I think after 11 pm eastern I felt like the ping went back to normal but I didn't notice it till 1 hour after. And just because this issue went away temporarily, it's bound to happen again; no surprise there.

    What makes this weird is that this issue started sometime around the sshm patch and was very consistent at like a certain time (I think like 8-9pm eastern everyday for 1 or 2 weeks). The issue seem to return last Thursday and now it's happening pretty much throughout the day. Now whenever I play tera, I'm always constantly looking at my ping and wondering if I can play or not play that day. So sad as an elite user and pretty sure many people in the same boat.

    When I see launcher taking forever to load and not auto populate the username, then I know for sure something is wrong. There are also people who can't even get to this point; they get play button grayed out.
  • Server location is perfect, but there are so many freakin inconsistencies with the network which TERA is hosted on.
    Spiking for reason, then causing problems for certain internet provider services.
    I have no clue why I'm spiking or even touching 60-70 ping, I've played on MC servers, other game servers hosted in Chicago and have rarely/never had such inconsistencies with packet transfer.
    Needs to be seriously looked into now, it wasn't a big problem for me earlier in the year.. but now it's pretty damn bad.
Sign In or Register to comment.