What EME is doing to reduce lag in TERA [Updated 5/18]

1356720

Comments

  • I am liking this, you had a troll thread up before telling us to check our client files, drivers and ISP's blah blah.
    very happy to see you are finally acknowledging that it is your server, took a while, but it is a step in the right direction.

    Hopefully you can get this problem sorted, because the game is almost unplayable at the moment.
  • Geletron wrote: »
    DYJGAM94YD wrote: »
    Hi. I'm a south american player, brasilian. Sorry for my bad english. Have you considered a server in miami? Many games have servers in miami for us. I think this can be a improvement in our pings, and give us some chances for competitive pvp. No one suffer with lag spikes like us..

    Oi como vai. We have looked at hosting TERA servers in different places. Wonderholm was hosted in a different data center, though we had to add some additional constraints to make it work properly.

    There are some infrastructure challenges that make it difficult to host TERA in multiple locations. TERA servers like to run in the same location as the services they rely on. One side affect of that restriction is that dungeon and battleground matches may be limited so that you could only play with people from the same server. That in turn could make some of the end game content less fun.

    European here, from Denmark to be specific. Have you considered hosting TERA in a different location altogether ?
    Given everything you say you've already attempted, I can't help but think that it seems your datacenter is having issues with its network either internally or externally.
    On average, ignoring today's worse than usual lagspikes, I tend to get about 140-150ms ping on the Fey Forest server which, while playable, isn't great compared to other US based games I've played where I might normally see about 80-120ms. Of course there's the "but you're not in America" card to pull, but from what I gather it seems that even a large part of the NA playerbase is having the same kind of ping as I am, which to me is a bit worrying to say the least.

    We have considered hosting TERA in different data centers and/or different physical locations. Moving everything, however is a monumental task. It took about a year of planning and months of work to upgrade all the existing hardware while minimizing down time. The Wonderholm server taught us a lot about what it would take to build and move a single world server to a new data center and we are going to use what we learned from that experiment as we plan on how to host TERA moving forward.

    Our biggest concern at the moment is finding root causes of lag spikes people have been experiencing on different servers. It makes the play experience bad no matter what your normal ping is.
  • MargaretheMargarethe ✭✭✭✭
    Thank you for finally admitting there are issues.
    I hope a resolution is found soon.
  • Game has lagged out on MT, everyone is reporting it on discord.

    Nice fixes, nice game, nice servers.
    Nice.
  • Hi GM! Another person here who is glad the issue is being investigated. The lag has been consistent for a few patches now. I am curious as to if you guys have investigated how other games are running their servers to avoid this issue? Tera has a rather small player base and I have a feeling a lot of people would be okay with downtime if the servers could be consistent. Well, mainly just instance servers. What has changed since maybe SCHMs initial patch to now? I don't think we had as much lag back then if I can remember correctly and I am doubtful that it is the increase in player base. Lag has sadly been the main reason I haven't been able to enjoy playing the game and really look forward to it getting fixed!
  • JasonTERAJasonTERA ✭✭✭
    Basically, I and my friends are on teamspeak and/or discord when we are playing, and we communicate like constantly even if we are doing different things.

    We play TERA from various areas such as LA, Ohio, NYC, Toronto, Huston, Chicago, D.C. , etc (sounds like everywhere) , with various ISPs (TWC, fios, comcast, etc)

    Let me point out some things about lag.

    1. When I get a lag issue on my side, I always call "LAG!!" on TS/discord, and everyone else in TS/Discord also shout same thing as well. so I guess it is not area or ISP-related.

    2. Indeed, when I just stay in a town/ciry, sometimes I still have lag issue so that I cannot open bank or trade broker, or use skills. At the same time, my friends in dungeon(RMHM, HH, LK, etc) also get same issue together with me. So I guess it is not specific server(dungeon and town)- related issue.

    3. Like some other people mention before, it happens only in TERA, not any other games.

    I understand what you are doing for this issue, and why you first mention ISP and area as a possible reason. However, very apperantly, it is not because of that, but becuase of TERA itself or because of TERA server center.


    In short, every player gets the lag issue at the same moment regardless their ISP, area, in-game location
  • Avi wrote: »
    Hi GM! Another person here who is glad the issue is being investigated. The lag has been consistent for a few patches now. I am curious as to if you guys have investigated how other games are running their servers to avoid this issue? Tera has a rather small player base and I have a feeling a lot of people would be okay with downtime if the servers could be consistent. Well, mainly just instance servers. What has changed since maybe SCHMs initial patch to now? I don't think we had as much lag back then if I can remember correctly and I am doubtful that it is the increase in player base. Lag has sadly been the main reason I haven't been able to enjoy playing the game and really look forward to it getting fixed!

    We do talk to Bluehole about how TERA is hosted in other regions and members of our team have worked on large scale games from Arenanet, Microsoft, Daybreak, and other studios worldwide.

    There are almost weekly changes to TERA servers and services so pinpointing an exact change that could affect lag is challenging. One of the reasons we are asking for more information and diagnostic reports from the community is to help us both identify what areas are most prone to lag and to find better ways of identifying signs of when a server may need to be taken down for emergency maintenance.
  • Geletron wrote: »


    A: We have been trying several things to reduce lag, and we’re working to do a better job informing players what we’re doing to address the issue. In the past year we have:
    • Updated the TERA server hardware (increased CPU, increased RAM, etc)
    • Experimented with load balancing between two different ISPs from the TERA datacenter
    • Worked with our ISPs and data center representatives to find ways to improve network performance
    • Set up an experimental server to test how TERA performs in a cloud-based environment
    • Set up new monitoring software to notify us when server hardware performance decreases


    I rarely reply on threads but I thought I could give my two cents here. I play from Europe, and my average ping was 180ms, and never seen it under 164ms for the past two (maybe three) years of playing. However, about a month ago something felt odd when I logged in, and I realized it was suddenly decreased by 40-50ms. I thought it might be a mistake at first, but my ping has stayed like this for the past month (126-160ms at times, but mostly stays at 137ms). Note that my PC is the same, I do not use GPNs, and my ISP said they haven't changed anything in their service. My only conclusion is that with these improvements you made, something *did* work.

    Well, while my overall ping did decrease (50ms improvement - such wow, many amaze!), it did nothing when it comes to lag spikes, I still experience them in dungeons during events such as double/drople drop, just like many other players.
  • CatoriiCatorii ✭✭✭
    The lag has been steadily increasing since you made the switch back to Zayo, and now it's pretty unplayable as of yesterday's patch. A combination of Zayo pingspikes and servers being too weak to handle the amount of players on them makes for an awful experience.

    When the entire party is lagging so much that 10s of attacks are happening in .5s, you still want to pretend that it could be ALL of our ISPs?

    K.

    As far as I can tell, you've done nothing but downgrade the servers in the past year. I never had lag before that - we'd lag during double drops in specific dungeons, and that was it. No lag the rest of the time. The past 6-8 months have been nothing but lag, and you keep blaming it on us.
  • clfarron4clfarron4 ✭✭✭
    edited May 2017
    Geletron wrote: »

    Q: What can I do on my side?
    A: There are some common things that can cause network performance problems that you can check:
    • Make sure that you are not playing over Wi-Fi if you are having lag problems.
    • Make sure that you or other people in your house are not doing bandwidth intensive things such as streaming video, downloading large files, etc.
    • Try playing at different times to avoid primetime internet traffic from your ISP (usually 9 PM – 11 PM in your local time zone).

    Please remember that network lag is not the same as problems with your framerate. BHS is aware of problems in this area as well, but this is not an issue related to our server infrastructure.

    Here's the issue that I have with this. Everyone I know that has the same issues as me (even though we are located in different places geographically, with a few hundred miles within a U.S. State between the closest and most of the U.S continent and either the Pacific or Atlantic Oceans) is that:

    - No-one uses Wi-Fi. Like, we know it's rubbish for gaming.
    - We are the only people using the line, and not intensively using it with heavy usage programs
    - I do most of the serious runs (HMs etc) from 3pm to 6pm and midnight to 4am Fridays and Saturdays my time, which should be the quietest time locally because almost everyone is asleep.

    ====

    Unrelated to my point above, I've been looking through the history of special events hosted by EME for TERA and it would seem that since the Secrets and Shadows (Shadow's Sanguinary or Ninja) patch:

    - events that were run before transitioned were longer, slower paced events with rewards based on longevity and dedication to the event
    - events that are run after are much shorter, quicker paced with more instant rewards for burning through content

    I'm guessing that this would make the servers work harder with the newer events as people realise that you get the most by farming the crap out of everything as quickly as you can, where previously, you would just do things as normal because there was no extra gain by intensely farming more than what you normally would.

    ====

    Also, within the first 24 hours of the patch going live:

    - MT died twice
    - FF died once
    - AV died once
    - Login Servers Died
  • Geletron wrote: »
    DYJGAM94YD wrote: »
    Hi. I'm a south american player, brasilian. Sorry for my bad english. Have you considered a server in miami? Many games have servers in miami for us. I think this can be a improvement in our pings, and give us some chances for competitive pvp. No one suffer with lag spikes like us..

    Oi como vai. We have looked at hosting TERA servers in different places. Wonderholm was hosted in a different data center, though we had to add some additional constraints to make it work properly.

    There are some infrastructure challenges that make it difficult to host TERA in multiple locations. TERA servers like to run in the same location as the services they rely on. One side affect of that restriction is that dungeon and battleground matches may be limited so that you could only play with people from the same server. That in turn could make some of the end game content less fun.

    How about hosting TERA cloud-based (or your results on experimenting with it.), I assume that the game server runs up an emulator that simulates and populates the game world. Server upgrades can only do so much at some point when a game keeps growing, not every build you guys get is the same in size and logic. Have you guys looked into virtualization options as well?
  • Also Wirth's Law applies here: Software gets slower faster than hardware gets faster.
  • dingo790dingo790
    edited May 2017
    Nice login for MT is down again and has been down for over an hour now.
    Also lag has been an issue for like what, 6+ months, every patch, and you guys still haven't fixed it or implemented a solution? What are you doing? You've migrated the servers, done some tests, but on our end nothing has changed.

    Also like, with the servers being down multiple times today, you could at least make a forum post or a banner or something saying you're aware of it, instead I come onto the forums and read more dodging of actual problems and issues, rather than owning up to them. Heck you could even post on twitter if signing in to the forum was too difficult. Do you guys even know that these issues have happened? Or do you just react to community forum posts after the fact and then do nothing about the issues, as seems to be the case.

    You are constantly having the same issue Every. Single. Patch. Every time you release a patch there are server issues, the servers go down, the servers lag to the point of the game being unplayable, and yet the issue KEEPS HAPPENING. This is NOT GOOD, in case you haven't realized yet.

    Nice.
  • PeperoncinaPeperoncina ✭✭
    edited May 2017
    Hours ago,
    me and my party members were lagging in new dungeon VHHM/VSNM multiple times and now...
    Servers are all DOWN? :(

    I hope you will get a good solution EnMasse o/
  • TianaTiana ✭✭✭
    My main issue is how I can afk/explore Dragonfall with a ton of lag, then get into CS with almost none. I go from a completely dead area- no one at all even in town, to a battleground.
    This could be right before or right after. Never at 'high end' times where many are playing. That's my concern. How terrible it is to have dead zones lag on you more than a battleground.
Sign In or Register to comment.