3.0 Usability Testing

3.1 Filtering and Organizing Testers

As previously mentioned, we analyzed all 1066 playtesters in depth for research purposes. Furthermore, we split the playtesters into various groups based on various variables such as VR hardware, gaming experience, excitement level, player type, CPU & GPU performance etc, so we can create a stratified sample and to be able to compare the different feedback from different groups.

‍

‍

3.2 Testing with Stakeholders

From our filtered groups we found 25 steam VR playtesters and 25 oculus playtesters. They were then divided into 5 smaller comparison groups to understand how different player types with the same hardware and different hardware respond. How do players of the same hardware but different player types respond etc. With control groups established and set up, not only will we have painted a bigger picture of what we need to improve， but also a better understanding of which user personas reacted most positively or most negatively to our designs.

We will look at the player's experience from two dimensions: Were players able to successfully complete their tasks and was the experience enjoyable.

‍

‍

3.3 Testing with Oculus Developers (Professional Critique)

We had an opportunity to be invited by Oculus to test with six other VR developers, each developer responded in writing a report consisting of positive aspects, and aspects need improving, below are their synthesized responses.

‍

‍

3.4 Synthesizing Testing Feedback

After two weeks of in-depth testing from both stakeholders and professionals, four key points were identified. Flimsy sword combat, AI, and the combat system were too easy, the placeholder voice-over breaks immersion and too many technical bugs.

‍

‍

3.5 Propose Specific Redesigns (Sword Weight)

The current design of slowing the perceived sword swing and the controller was received extremely negatively by both stakeholders after multiple iterations, we realized the key is to make the physics feel good, rather than feel real. Therefore we proposed the redesign below.

Prototype showing the difference between the original and redesigned sword weight system. Players feel better sword weight because removing delay mid-swing feels as if you swung harder.

‍

‍

3.6 Propose Specific Redesigns (Combat System)

With the sword weight redesigned, the combat system was next that needed an overhaul. The fundamental flaw of the combat system was that it lacked order and rules, without rules its hard to be strategic and hard to feel a sense of good vs bad. The new system creates limitations and rules which offers challenge and a sense of mastery.

‍

‍

3.7 Redesign Validation

With our schedule of a September 22nd launch less than a month away, our developers noted that although agreeing with our proposed combat redesign, implementation would be impossible. Therefor the focus became validating the sword weight redesign, and a majority of bug fixes with another round of play testing.

Hearing that most of the comments are now geared towards the voice acting and the combat system which were both not yet implemented due to time constraints. We feel confident in moving on to creating our final deliverable. Since good design is often invisible, no negative feedback on the sword weight was concluded to being a very successful redesign.