We collaborate with production, developers, artists and designers to coordinate our test planning, test expectations, and goals for the product at each milestone. Our process includes creating test specifications for each functional area of the game (UI, Gameplay, Audio, Art, etc.) to ensure each area gets the focus it needs. Early risk identification, with solution focused escalations give our clients opportunities to address concerns before they impact the product development schedule. We remain adaptable, we expect changes, plan for it and are consistently responsive to new and high priority requests. Our process utilizes data analysis and data driven testing to ensure we meet or exceed test coverage goals. We seek to inform, through dashboards and reporting on test progress and key data points such as stability and performance that can help production, developers, artists and designers make critical decisions.
By combining the use of tools with manual and scripted testing techniques we optimize the capacity of smaller testing teams to do more. There are many automated or partially automated solutions we utilize to reduce the time impact on the test team, by spending less time manually performing build deployment, stability tracking, performance data gathering or test coverage tracking. This allows our teams to have more time to do the work that must be done manually with black box methodologies.
Modern software development and testing require accomplishing much more with less staff; one of the many ways we deliver on this goal is by getting involved as early as possible to help establish a telemetry plan that not only provides answers to common questions like Key Performance Indicators, but also to maximize the impact of efforts across test and development teams. All too often, data is gathered for the sake of gathering data. Because of this, we strive to ensure the telemetry plans we are involved with maximize impact and avoid bloat. Continuously monitoring telemetry information confirms accuracy and value while ensuring that once released the data our clients receive from end users will be actionable. We also strive to prevent situations where too much data is generated, which could cause ingestion problems or increase data storage costs for our client unnecessarily.
Gathering data through telemetry, automation or manual testing can provide unique insights into the state of your software. We utilize dashboards to present data in near real time to keep all stake holders informed of key performance indicators, bug trends, stability metrics and other key data points. Our expertise in data analysis allows us to call out potential risks and back them up with data while also recommending potential mitigation strategies and projected affects such changes may have.
Using test coverage data, we direct test coverage to areas that need it and not into areas that have enough. We make smart choices about what tasks need to be run when and how often to ensure confidence in the test coverage for all stakeholders. Responsible and flexible resource management allows us to do more with less for the benefit of our clients.
With experience and expertise in releasing game software we call out potential pit falls and possible platform requirement violations early in the project development cycle. We keep this in mind from the start when we begin reviewing design documentation or testing early prototypes of the software. When we compose our bug reports we endeavor to provide expected results that will meet platform requirements and not cause further problems down the road.
When a project reaches a milestone, it is important to ensure that the developers have met the expectations of production. We perform acceptance tests for all milestone criteria and provide feedback about how well implement each item is and how well this corresponds to the goals for the milestone. When possible, we work directly with production, design and art leads to ensure the spirit of the criteria is understood and the test team’s expectations are aligned with their goals.
Our staff develops expertise in the certification requirements for the platforms we test. We document common failures and build our test cases to ensure we avoid them. Early discovery of potential certification failing issues helps to stream line the final stages of development and reduce the likelihood of resubmission. Our goal is to give your product the best possible chance of a first pass through final certification without exceptions.
By utilizing our staff and technical solutions to meet the maximum capacity and scope for multiplayer game testing, we minimize the need for large teams while confirming the effectiveness of a title’s multiplayer functionality. We leverage automation to simulate extra players and plan bug bashes with members of other teams to fill out the roster.
By providing actionable data about game performance through reporting and dashboards, we highlight areas that need improvement through heat maps and session data. We run focused stress tests to uncover potential root causes for low performance or stability. By consistently monitoring data on all areas of the game we can help to determine when a problem may have been introduced, such as a specific build where load times or memory usage increased.
When we staff for multiple projects at one location for a client, we are flexible with staff movement between that client’s projects to ensure each project has the staff they need that day. When one project is in a lull and another project has a need for more staffing we quickly and efficiently adjust our staffing to the needs. This can be especially effective for short bursts of extra test coverage, while helping to avoid overtime, keeping costs down for our client.
While your game is in test, the testers often have the most hands-on time with the game of all parties involved. This gives a unique perspective about what is fun or may be too difficult or repetitive. We can provide various levels of feedback about the expected customer experience, call out potential exploits and provide ratings for different aspects of the game with feedback and suggestions about what could be improved. Whenever possible we collaborate with the client and their user research team to focus our feedback on their areas of concern.
While our functional test teams are focused on ensuring the product is functional in all languages, we also provide Localization translation test support with respect to audio, text, currency, and globalization. Our localization team will use a subset of the functional team’s test pass that ensures they are exposed to the entirety of game features that support localization, ensuring that all supported language translations are implemented with the same high quality.
Selective Beta releases in the retail environment can be set to initially small targeted audiences to gather data and consumer concerns, then grow larger to help stress test back end services. By utilizing telemetry metrics and reviewing bugs reported by the Beta community we can help to inform on the viability of new features or targeted changes. We assist production on planning and execution of beta releases and follow through to ensure the data gathered from the Beta is turned into actionable bugs.