[TERA PC & Console] En Masse is closing, but TERA lives on! We will continue to support TERA PC (NA) and TERA Console until service is transferred. Stay tuned for more information.
[TERA Console] The Grotto of Lost Souls update (v85) is now live! Read the patch notes here: https://bit.ly/TERACon_v85
[TERA PC] The 64-bit update (v97) is now live. Check out all the changes delivered on August 11 here: https://bit.ly/tera64_patchnotes
[TERA PC & CONSOLE] Summerfest Part 2: The Beach Bash is on from August 11 until September 1! Participate in event activities to earn tokens redeemable for costumes, consumables, mounts, and more! Details: https://bit.ly/tera_sf20
Server hardware lag discussion
Alright, so every thread about lag I see is getting confused and/or side tracked for the same basic reason.
There are a lot of different types of lag at play in tera, the three major ones being:
-FPS/client side lag (every time there's a ninja in my party my game is less smooth!)
-Network/isp related lag (my ping spiked for a moment, but my party did not, or my ping is higher than it used to be)
-Server hardware lags (global chat is spammed because the entire server lagged at once)
This thread is specifically asking for more information about instances of server hardware lag, where the whole server lags at once. Please try to keep the thread clear of non-contributing posts so our very time-constrained GM's don't have to sift through a lot of posts to see the ones that may contain helpful information. Basically, please just post here if you have a question that should be addressed publicly by staff/someone more knowledgeable, or to make a comprehensive list of times that entire servers have lagged at once, ideally with screenshots and as specific a time as possible.
I've been playing tera a long, long time and these server lag spikes are worse than they have been since perhaps the first iteration of nexus. After that was addressed, the 'new' nexus didn't even cause lag like we get during modern-day rally when there were far more players present. Which leads me to wonder what happened? Was tera virtualized at some point, and the spikes are related to that? was the hardware actually an upgrade in name only? Is there more strain on the server related to logging? The fact that it's only getting worse is not a good sign. On that note, I have a question.
I would like to know more about the server hardware, and how it compares to the pre-upgrade hardware. I ask because I know there's a trend of adding cores and reducing core frequency in modern hardware to save power and money while improving performance, however I know that as far as the client is concerned, the multi-threading is quite poor. This means that if the server followed that trend, performance would decline significantly but at a glance all would appear fine - for instance, my processor has 6 cores and 12 threads, but tera only shows 20% cpu use at most because it won't expand beyond 2 threads plus a little overhead. It would be easy for someone unfamiliar with tera to assume it's multi-threaded well and glance at total cpu usage, assuming there's no problem when in fact it's completely peaked on the threads it's using. That would cause lag spikes when load increases from something such as garbage collection (when the server frees up ram that is used but not needed) or a large database query (elite reset or many items dropping at once, for instance). The solution in that case would be to either improve multi-threaded performance(probably not something EME can do), or switch to processors better suited to the amount of threads the server software can actually utilize.
I suppose my basic questions are: If the hardware is an upgrade, why is the lag and related player experience worse than it's been since the free to play launch? What has been investigated so far in terms of hardware-related lag? Given that the only official answer I can find seems "it's your isp or computer," I'm not optimistic that it's a long list. Please change my opinion.
There are a lot of different types of lag at play in tera, the three major ones being:
-FPS/client side lag (every time there's a ninja in my party my game is less smooth!)
-Network/isp related lag (my ping spiked for a moment, but my party did not, or my ping is higher than it used to be)
-Server hardware lags (global chat is spammed because the entire server lagged at once)
This thread is specifically asking for more information about instances of server hardware lag, where the whole server lags at once. Please try to keep the thread clear of non-contributing posts so our very time-constrained GM's don't have to sift through a lot of posts to see the ones that may contain helpful information. Basically, please just post here if you have a question that should be addressed publicly by staff/someone more knowledgeable, or to make a comprehensive list of times that entire servers have lagged at once, ideally with screenshots and as specific a time as possible.
I've been playing tera a long, long time and these server lag spikes are worse than they have been since perhaps the first iteration of nexus. After that was addressed, the 'new' nexus didn't even cause lag like we get during modern-day rally when there were far more players present. Which leads me to wonder what happened? Was tera virtualized at some point, and the spikes are related to that? was the hardware actually an upgrade in name only? Is there more strain on the server related to logging? The fact that it's only getting worse is not a good sign. On that note, I have a question.
I would like to know more about the server hardware, and how it compares to the pre-upgrade hardware. I ask because I know there's a trend of adding cores and reducing core frequency in modern hardware to save power and money while improving performance, however I know that as far as the client is concerned, the multi-threading is quite poor. This means that if the server followed that trend, performance would decline significantly but at a glance all would appear fine - for instance, my processor has 6 cores and 12 threads, but tera only shows 20% cpu use at most because it won't expand beyond 2 threads plus a little overhead. It would be easy for someone unfamiliar with tera to assume it's multi-threaded well and glance at total cpu usage, assuming there's no problem when in fact it's completely peaked on the threads it's using. That would cause lag spikes when load increases from something such as garbage collection (when the server frees up ram that is used but not needed) or a large database query (elite reset or many items dropping at once, for instance). The solution in that case would be to either improve multi-threaded performance(probably not something EME can do), or switch to processors better suited to the amount of threads the server software can actually utilize.
I suppose my basic questions are: If the hardware is an upgrade, why is the lag and related player experience worse than it's been since the free to play launch? What has been investigated so far in terms of hardware-related lag? Given that the only official answer I can find seems "it's your isp or computer," I'm not optimistic that it's a long list. Please change my opinion.
0
Comments
- World Servers
- Instance Servers
- Database Servers
- Authentication Servers
- Store Servers
- Load Balancers
- In-Rack Networking Routers/Switches
- Service Provider inbound networking equipment
Latency will be impacted by each of these components and the interaction between them, and any hiccup in the chain could cause lag in game (or, in some cases, cascade into a server crash).
Some of the "what has been investigated" was covered in the FAQ thread but I hope they can provide more details, since the perception is certainly that whatever they're doing is not enough (since the problem isn't solved). As you allude to, it certainly doesn't help that all the other threads about this get distracted by the other two types of issues you outlined, so when EME sees a thread with a lot of people from overseas saying they're getting disconnected from the game constantly with constant 1000+ ping, that's most likely a separate issue (but contributes to the same perception).
This
I think the data they got from that test server was not enought becuase there were less players than they really need to get proper data about server stability.
I could be as stated before that the new servers are better than the previous in specs but for Tera code these new servers don't have enought muscle in the single arm Tera force them to use. If that's the case then EME servers could get the same thing as us, 20-25% of CPU usage but few core being overloaded and resulting in colapsing or lag/frezzes for most of the players.
Now, is obvious than not all players on all servers have the same experience as we saw that some times a specific server is down when other servers are on-line. However makes me think a lot how CH is alive some times when every other server is down. That makes sense when you think that now current servers can't handle many players on-line at once doing the same activities.
I think Wonderholme was a test server for cloud computing based server wasn't it? (I mean as in En Masse outsourced the processing)
The factor about low multi-threaded support is a thing to consider indeed, but if we go back a bit, I think most server processors are heavy multithread aren't they? Like, there's not many options I know of low core count and high single core performance. Of course I'm not even considering TERA running/being coded for Itanium and Pentium Xeon processors, those were single core yeah but out of the scope.
STILL, point still stands we have no idea how the server side code works and how is it optimized for new server processors.
I think this topic would benefit from my theory/conspiracy about the sudden world wide lag spurs and the last flash BAM event they made crashing us all at once even in other areas of open world, so put your tinfoil hats on read about that if you might: https://forums.enmasse.com/tera/discussion/comment/142824/#Comment_142824
Well, yeah, and it's also true that you're much more sensitive the problems in instanced content too, particularly when you're fighting bosses (or other players in PvP), since the pressure is on and you're experiencing it in real-time with other players. If you're just killing BAMs on IoD or whatever, a little lag probably won't kill you (literally or figuratively).
I do believe that there is something going on server-side as well, but it may also be true that people with already-bad ping times will notice it more acutely in instanced content. (It also doesn't help that the client itself has more trouble with optimization in instanced content, which compounds the experience even if it's an unrelated issue.)
These days Tera is becoming unplayable. I could survive at 250ms, but now that I am at 350ms, the game is pretty much unplayable. I lost more and more packets to Zayo's garbage routing service and the routing cross the globe than at the server, so I don't really think it is just the server that is having issues.