The WAH, 6/15/09 - "This has been a test of the WAR: Age of Reckoning System. . . ."

[Vegetta]

i actually tried testing on pts but guess what- once you log in there were no in-game clues /hints telling you what to do or where to go and tbh I couldn't be bothered with scouring through the herald for this information....

Basically once you log in they should have had a pop up window saying you can go here and do this then let us know what did and didnt work right.

[Andy_Mythic]

Frank you know I love guys you but:

• To say that this round of PTS testing was ineffective is doing a grave injustice to the players who were actively participating in this round of testing. We had nights of upwards of 400 people online at a time (200/side) stress testing the RRQ, C&C changes and everything else.We could not have been as successful without the Open PTS feedback, not to mention the solid 2 months that our Core Testers put in on testing new content.
• We were able to gather a valuable amount of information about the RRQ by observing how quickly players did gather resources when both opposed and when unopposed (the dreaded RvE). As much as players dislike going after undefended keeps and BO's, it does still happen (granted, that's something we're working diligently on fixing) and we needed to account for it. Additionally, there were nights that we artificially sped up the resource gain so as to encourage a realm to lock down the LotD.
• The combat and career changes were deemed to be too sweeping to be implemented without even more in depth testing. Player feedback was a part of making that decision, but ultimately not the only deciding factor. Moving forward, we are still planning on implementing some of these changes, but we're testing it further to ascertain the full impact.
• To those who state we fired our QA department, you couldn't be further from the truth. We have an extremely compentant group of QA folks who are active and vocal not just internally, but also to the players. Take a moment and check the dev tracker for BryantC or ask any of the Core Testers about us having a QA representative on hand for each and every test event. We're proud of our level of communication from the QA department and stand by it.

Bottom line is this. The PTS is a very useful tool which, due to incentivization rewards, has been very successful for 1.3. Did we catch every single bug? No, it's not possible to do so, because some bugs won't surface until you get a normal server load on them and some just, well, slip through the cracks.

We are going to continue to offer strong in game incentives for PTS participation moving forward. We believe that if you are willing to invest your time into helping us make this game better, you should have something to show for it. You've already seen some rewards with the Shroud and keep looking out as we're going to be rewarding the Imperial Griffon and Enslaved Manticore mounts for PTS testing soon.

As always, thanks to our awesome players for their continued (testing) support!

[radishlaw]

Originally Posted by Andy_Mythic

Frank you know I love guys you but:

• To say that this round of PTS testing was ineffective is doing a grave injustice to the players who were actively participating in this round of testing. We had nights of upwards of 400 people online at a time (200/side) stress testing the RRQ, C&C changes and everything else.We could not have been as successful without the Open PTS feedback, not to mention the solid 2 months that our Core Testers put in on testing new content.
• We were able to gather a valuable amount of information about the RRQ by observing how quickly players did gather resources when both opposed and when unopposed (the dreaded RvE). As much as players dislike going after undefended keeps and BO's, it does still happen (granted, that's something we're working diligently on fixing) and we needed to account for it. Additionally, there were nights that we artificially sped up the resource gain so as to encourage a realm to lock down the LotD.
• The combat and career changes were deemed to be too sweeping to be implemented without even more in depth testing. Player feedback was a part of making that decision, but ultimately not the only deciding factor. Moving forward, we are still planning on implementing some of these changes, but we're testing it further to ascertain the full impact.
• To those who state we fired our QA department, you couldn't be further from the truth. We have an extremely compentant group of QA folks who are active and vocal not just internally, but also to the players. Take a moment and check the dev tracker for BryantC or ask any of the Core Testers about us having a QA representative on hand for each and every test event. We're proud of our level of communication from the QA department and stand by it.

Bottom line is this. The PTS is a very useful tool which, due to incentivization rewards, has been very successful for 1.3. Did we catch every single bug? No, it's not possible to do so, because some bugs won't surface until you get a normal server load on them and some just, well, slip through the cracks.

We are going to continue to offer strong in game incentives for PTS participation moving forward. We believe that if you are willing to invest your time into helping us make this game better, you should have something to show for it. You've already seen some rewards with the Shroud and keep looking out as we're going to be rewarding the Imperial Griffon and Enslaved Manticore mounts for PTS testing soon.

As always, thanks to our awesome players for their continued (testing) support!

There are a few minor points I disagree with but let's just leave it at that. From the spelling mistakes you made I can see you are kind of passionate about this, and I don't really want to mar what I am trying to say here. (it's both about PTS and the game in general)

I can really see you guys working hard, but frankly, QA is not a strong point of WAR. There are many bugs that persists since release (please refer to all the bug compliations here or in the official forums - most of them didn't get fixed), and while I appreciate you and a few others taking the time to answer posts, we players really feel a lack of communication in here.
It is not for want of trying: you guys tried to clarify things as much as possible, and some of we players either work logs/screenshots and do compliations to make bug tracking as easy as possible. For example, the bug list you posted in the OF is a good way to let us know. The problem is, we really want to know if you guys have known about some of our issues. As I said, we get things that are broken since release, or at least for a very long time. In those cases, people from Mythic never addressed them directly, and that leaves a very uncertain impression. Some of these, like broken abilities, should never have the need to wait for a major patch - they should be hotfixed asap. It is similar on the PTS - we may report it, but we are never sure if you know about it unless one of the nice people at Mythic reply to it.

I am getting a bit drifted here, but I really hope you guys can list out bugs/issues, particular those related to careers and post it to the respective forums, so we know what you know, and know which to report and not - barring exploits of course. I hope Mythic can use a faster patch cycle, pushing out finished fixes instead of lumping it in a big patch. Pray believe me when I say that I appreciate bugs to be fixed as soon as possible, and too many changes at a time make it hard for both players and developers to test it out. If we have a faster patch cycle, with PTS testing a few changes at a time, I believe it would be much easier for all of us.

We are at the nine months mark since the release of WAR and it is facing severe competition from this year's mmos. I really hope Mythic can rise to the challenge and improve the game better.

[Merces]

Originally Posted by akalukz

Excellent post. The most difficult part of the PTS for me is boredom. There is rarely anything going on. And if the PTS was any indication as to what LoTD will do to live servers...then we all might as well prepare for 3 months of RvE.

LOL RVE in this game = total fail.

If the new content only brings that I am out... might as well go to a game that does PvE right.

[Kovaks]

Originally Posted by Andy_Mythic

Frank you know I love guys you but:

• To say that this round of PTS testing was ineffective is doing a grave injustice to the players who were actively participating in this round of testing. We had nights of upwards of 400 people online at a time (200/side) stress testing the RRQ, C&C changes and everything else.We could not have been as successful without the Open PTS feedback, not to mention the solid 2 months that our Core Testers put in on testing new content.
• We were able to gather a valuable amount of information about the RRQ by observing how quickly players did gather resources when both opposed and when unopposed (the dreaded RvE). As much as players dislike going after undefended keeps and BO's, it does still happen (granted, that's something we're working diligently on fixing) and we needed to account for it. Additionally, there were nights that we artificially sped up the resource gain so as to encourage a realm to lock down the LotD.
• The combat and career changes were deemed to be too sweeping to be implemented without even more in depth testing. Player feedback was a part of making that decision, but ultimately not the only deciding factor. Moving forward, we are still planning on implementing some of these changes, but we're testing it further to ascertain the full impact.
• To those who state we fired our QA department, you couldn't be further from the truth. We have an extremely compentant group of QA folks who are active and vocal not just internally, but also to the players. Take a moment and check the dev tracker for BryantC or ask any of the Core Testers about us having a QA representative on hand for each and every test event. We're proud of our level of communication from the QA department and stand by it.

Bottom line is this. The PTS is a very useful tool which, due to incentivization rewards, has been very successful for 1.3. Did we catch every single bug? No, it's not possible to do so, because some bugs won't surface until you get a normal server load on them and some just, well, slip through the cracks.

We are going to continue to offer strong in game incentives for PTS participation moving forward. We believe that if you are willing to invest your time into helping us make this game better, you should have something to show for it. You've already seen some rewards with the Shroud and keep looking out as we're going to be rewarding the Imperial Griffon and Enslaved Manticore mounts for PTS testing soon.

As always, thanks to our awesome players for their continued (testing) support!

The PTS was a joke. I logged on for a few hours during the scheduled nights and there was 100-200 destro sitting at the flight master in IC waiting for thier turn in LOTD (wich never came) or just completely lost with no idea of what to do.
There was no organization or communication by anyone from Mythic in game.
Im curious how you can say the testing was effective. Every one was just standing in IC doing absolutely nothing.

[Kazamxi]

Have you guys ever thought of using a Launchpad system for bug tracking so players can add the bugs in cluster them and triage them?

It works really well in the open source community where thousands of users help to find, identify and reproduce bugs before devs need to look at them.

EDIT: thinking about it, you couldn't do that. Some bugs can't be public knowledge as others would exploit them

[radishlaw]

Originally Posted by Kazamxi

Have you guys ever thought of using a Launchpad system for bug tracking so players can add the bugs in cluster them and triage them?

It works really well in the open source community where thousands of users help to find, identify and reproduce bugs before devs need to look at them.

EDIT: thinking about it, you couldn't do that. Some bugs can't be public knowledge as others would exploit them

I was thinking the same thing - if we can get someone to host it for us, I think we can set up some kind of "civilian bug tracker" so we can keep track of the bugs we know versus the bugs acknowledged by Mythic.
Exploitable bugs would sadly have to be excluded as you said.

[Vizyon]

Was that the same QA group that told you AMs were ready to go at release?
(6 months later a total mechanic overhaul is needed to make them a little more than just barely viable)

Is the same group that is telling you, "Oh don't worry about fixing the AM ultra fragile group heal right now. They can deal a little longer."
(took 9 months to admit the bug, and now we can wait around 'till you get to it)

If you can't understand why your words sound hollow when you praise the quality of your QA group, then you are out of touch.

Anglekhl: In my opinion, your energy and creativity is exactly where it needs to be. Please continue to keep Mythic sharp.
(uhm...and can you ask your editor to spend a bit more energy making your great work a little more reader-friendly? i.e. sections, etc. thx and thumbs up!)

Afterthought:
I just want to add that an independent news outlet with its own character is fundamental in this ongoing exchange we have with Mythic. An official forum will never be able to meet this need. In fact, it is one of the central pillars that keeps the U.S. society going, such as that is. I, for one, am thankfull WHA is spending their time doing what is desperately needed, so that I can spend my time doing what is needed from me.

[Avien]

Originally Posted by radishlaw

[snip]
We are near the one year mark of WAR [/snip]

Just a quick note because I see this popping up all the time...how are we near the one year mark? Release was on September 18th, 2008. It is now June the 17th, 2009. That's not near the one year mark, that's almost exactly 3/4 of a year. Not saying your argument is any more or less valid because of that, of course.

Originally Posted by Vizyon

(uhm...and can you ask your editor to spend a bit more energy making your great work a little more reader-friendly? i.e. sections, etc. thx and thumbs up!)

Haha, criticism noticed. Will do

(It's just that he makes so many grammar mistakes that all my time is consumed by them!

...Just kidding, he really doesn't.)

[Svengrom]

Originally Posted by Vizyon

Anglekhl: In my opinion, your energy and creativity is exactly where it needs to be. Please continue to keep Mythic sharp.
(uhm...and can you ask your editor to spend a bit more energy making your great work a little more reader-friendly? i.e. sections, etc. thx and thumbs up!)

Agree 100%, its like a breath of fresh air when i read these threads you make, it means other people other than the guys i talk to in vent in guild/ alliance feel the same way.
If we had people instead saying what a great job mythic have done then they would/ more so relax at getting stuff actualy fixed.
Cos as we know its far from fixed and with sweeping class changes inbound there is going to be alot more broken.

There are still broken class abilities after 9 months of it being live ( dont forget the year or more of beta, quote me if im wrong but i was not fortunate enough to get a beta key)and now they want to make sweeping changes but expect us to test it all for them while we pay them for the it.
And dont give any one that rubbish about the mounts, its hardly anything to lookforward to cos you MIGHT get chosen to have one AND its only fluff.

It what makes me use these forums instead of the Gestepo monitored official one where if you have any bad word to say they drag your thread out to the back ally with a bag over its head in the middle of the night and shoot it.

[Resh]

Originally Posted by Kovaks

The PTS was a joke. I logged on for a few hours during the scheduled nights and there was 100-200 destro sitting at the flight master in IC waiting for thier turn in LOTD (wich never came) or just completely lost with no idea of what to do.
There was no organization or communication by anyone from Mythic in game.
Im curious how you can say the testing was effective. Every one was just standing in IC doing absolutely nothing.

And the reason was explained on why the resource system was broken. And as for Organization and Communication from Mythic, it was as the OP stated "Hands Off" They would be around to monitor, not to hold players hands. At the same time, the PTS was up almost the whole time to include during the day. The scheduled nights were for the event that they would be present for, but the server was still up at other time periods, and players could still test and provide feedback.

Originally Posted by Vizyon

Was that the same QA group that told you AMs were ready to go at release?
(6 months later a total mechanic overhaul is needed to make them a little more than just barely viable)

Is the same group that is telling you, "Oh don't worry about fixing the AM ultra fragile group heal right now. They can deal a little longer."
(took 9 months to admit the bug, and now we can wait around 'till you get to it)

If you can't understand why your words sound hollow when you praise the quality of your QA group, then you are out of touch.

Anglekhl: In my opinion, your energy and creativity is exactly where it needs to be. Please continue to keep Mythic sharp.
(uhm...and can you ask your editor to spend a bit more energy making your great work a little more reader-friendly? i.e. sections, etc. thx and thumbs up!)

How does a QA department work? Especially before the release of the game, so that would make Beta correct, then shouldn't you be blaming the OP since he was a Beta tester and failed to bring up issues with your AM. That is what the Beta Testors were there to help with. QA is nothing without players to give them feedback, and test properly. But you still glorify the OP who as a Beta tester hosed your class by not properly testing it, and providing the proper feedback.

At the same time, you are upset, but might as well say "gimmie gimmie gimmie" to the C&C changes, when they would have made the game worse with issues that the QA department as well as the Players testing the changes were able to address and make aware in order that Mythic just didn't go on cruise control and release something that is broken. But you seem to think it would be ok to hose the game up as long as they fixed your AOE heal.

So many players have this same mindset. Always upset and always complaining. I mean honestly the OP should have wrote this article and posted it in the PTS feedback forums, or send this to one of the Devs, but instead he chose to complain about it here, thus not helping to rectify the situation.

Right now it honestly feels like Warhammer Alliance purged almost all the trolls from their forums, and need to create some sort of orchastrated drama to get their site hits up.

Sitting around complaining is not going to change anything. Players have to be proactive if they want to speed things up.

[Pancakez]

Originally Posted by Anglakhel

This is an op-ed piece. I am certainly open to hearing criticism if you think that I've stepped over a line, but perhaps you can be more clear about why you think this article is an inappropriate attack on Mythic.

It's a matter of composition. I'm not sure if you're familiar with the use of weasel words and what they imply, but I'll try to explain a little bit. Weasel words refers to statements that are used to create the illusion that what's being said is direct, clear and even supported by evidence. This is done in a few different ways, but if something is littered with vague statements, passive voice, euphamysms and subjunctive verbs something weasely is probably up.

What was beginning so many sentances by telling the reader that if they did something then this is what they experienced was meant to accomplish? Is this an opinion piece explaining your personal experiences, or is it based on some type of survey with reliable data as to the average players experience?

This is your conclusion:
The PTS is a unique and valuable tool. It's a chance for Mythic to share ideas with the players and get feedback. It's an opportunity to find issues before they make it to the live servers. However, the PTS will do more harm than good if Mythic doesn't get better control over this resource. They need to better plan testing events and focus on systems they need to collect data on. They need to realize the significant differences between the live servers and the PTS and intelligently mold the PTS to better simulate the live servers. Most importantly, Mythic needs to make changes based upon how these new systems and mechanic adjustments will affect the live server in complex situations. If they don’t, we’ll continue to see changes made to the game that haven’t been adequately tested and we’ll continue feeling like we’re paying to beta test on live servers.

How will it harm servers? What evidence have you presented that the PTS servers could harm live servers? Where in any of this writing was there anything which necessarily showed that Mythic did not have a firm control over the PTS servers? Did you have any experience with their testing and data collection process? How would Mythic give a better environment to simulate live servers, while maintaining control over testing?

The last sentence is a direct attack on Mythic and WAR as a whole. You've thrown a bunch of accusations without support and followed with an attack on how Mythic has dealt with the game. Is this meant to be constructive and helpful, or is this an outlet for anger? You're insinuating a lot of things, but do you have any support for your arguments? It's very misleading.

The only truly productive part of your article was the following:
The Sands Run Red was a nightmare test, but it didn't have to be. Getting a little "hands-on" and directing the testing would have helped to better simulate the live servers and helped to improve the quality of the feedback on the Resource Expedition Quest. A simple step like creating several testing guilds, so that the RvR Campaign could be pushed, would have enabled a far more accurate test of the Resource Expedition Quest. At the very least, Mythic could have gotten valuable test data about what would happen with the quest if one faction abandons the campaign and heads to the Land of the Dead. We didn’t even get that though. We simply saw inaction. Players that wanted to be testers were extremely frustrated. The PTS was causing more harm than good. It wasn’t gathering relevant data about the control mechanisms for Land of the Dead and it was creating bad opinions in players, which translate to poor word of mouth and decreased morale.

What is most frustrating perhaps is that Mythic had to know this would be a problem. They had hosted another PTS event to test the Expedition Resource Quest a week earlier and saw the same problems with a campaign unable to be pushed. That test generated a lot of feedback about the problems with the Expedition Resource Quest and an inability to accurately test it on the PTS. Yet, they pushed ahead with a "hands-off" test of the system without making preparations to ensure that the new test would better simulate the live servers and meaningfully test the Resource Expedition Quest.

That section was actually well written and shows what some of the problems were and how they were handled, but the conclusions drawn from this are completely illogical. You suggest that Mythic should have gotten more involved. Later on you indicate that this wasn't the first test, but it was instead the second test after they were more involved with the previous test. If they had already been involved with testing and wanted to see how testing would run with them observing, rather than being directly involved, why would they then get involved? How would them being involved make the PTS function more like a live server, where admins are not directly interacting with players and directing their activities?

What I find out of line is veering away from discussing the PTS testing and how poorly it may have gone. I've experienced PTS testing with WAR before and it was not a pleasant experience at all, but for none of the reasons you've put forth. The shortness of the overall test cycle. Not giving players access to characters with which to test the intended content. Instead of my concerns you claim, “The real problem is that the set up of the PTS for this event in no way represented the dynamics of the live servers.” That's a very bold claim and could have some serious implications, but is it substantiated? If that's the real problem why is it such a minor focus of the article? You never took a step back and tried to explain your opinion. Instead the article bolts ahead with this:

The problem is that adjustments are being made to the system when the system hasn't been tested. We never saw whether or not the system would flip control of the Land of the Dead in a reasonable time frame. What we saw was a "hands-off" PTS event that generated frustration in large segments of the testing population and seemed to stray far from the stated intent of the test.

Is the problem that the PTS doesn't represent the dynamics of live servers, or is the problem that adjustments are being made without being tested? Wait, later in the article as I referenced above you mention that in fact two tests were run on the system. Why are you claiming here that it wasn't tested? Is it possible that the tests run in fact showed what you've claimed about frustration to be true and resulted in the halving of the required resources? You keep repeating that there's this problem of emulating live servers with hands-off testing. Why is this the case? Why would hands-on testing be superior in fulfilling this goal?

That's the problem I see here. This article was littered with repeated arguments that had no support except their own repetition while taking pot shots at Mythic. This is the second of these “WAH” posts that had this format. Take a small quote and then extrapolate a problem from that before going to great lengths to flog it to death. If there was a horse it's be glue.

This sort of thing is why op-ed pieces are very rarely written by staff members. When a staff member is the writter you can't hold true the mantra of throwing facts to the wind within an opinion piece. As a staff member the reader needs to feel you have some credibility. That isn't to say that it's impossible to publish an opinion article from a staff member; Previous WAH pieces never needed a disclaimer to inform us that they were an “op-ed piece” that we had to take with a grain of salt. If I wanted salt I'd have dried pork. This article reads like a forum flame and looking at the response from Andy it's clear that it's also completely disconnected with what actually occurred on the PTS.

[Shakrah]

Originally Posted by Resh

Sitting around complaining is not going to change anything. Players have to be proactive if they want to speed things up.

Beta-testing times are over, everyone must wake up to that. The license can not hide that anymore.

6 months+ in and the balance and content is tested as described?... C'mon, that alpha tests.

Highlighting the shortfalls of the PTS openly raised my opinion of WA massively.

[kingbarbarossa]

Speaking as someone who has been a QA Tester before, 90% of the time, the fact that bugs are in the final game have nothing to do with QA. Producers ultimately make the call on whether or not a bug is going to be fixed and are in charge of assigning priority to them. I'd be willing to bet that nearly every bug we're finding now has been found and reported by a QA Tester before, because let's face it, they're not hard to find. So don't blame the QA staff cause there's bugs, it's only their responsibility to find them, not fix them.

[Vizyon]

Originally Posted by kingbarbarossa

Speaking as someone who has been a QA Tester before, 90% of the time, the fact that bugs are in the final game have nothing to do with QA....

You are right, of course. I was hasty in making my point, and I apologize to hardworking QA testers.

I trust, however, that Mythic can hear my discontent. I know they are passionate and dedicated, and I think they really do realize, deep down, that some things that are occuring are unacceptable.