Picture this, a detective sitting at their desk, a pile of evidence laid out in front of them, communicating with fellow investigators looking for clues and hoping to get a breakthrough in one of their cases, a dedicated squad helping out people out day in and day out.
No, this isn’t your local police department or that charming crime scene investigation team from the television. They’re the dedicated men and women of the Support and Quality Assurance teams working on the best racing experience, Nitro Nation Online.
For many mobile developers, the challenge of device testing can be very difficult and time consuming. Android alone supports a mindblowing 10 thousand plus pieces of device hardware and often these tests must be run manually on each device by a Quality Assurance tester.
“F2P games are actually a living service similar to an online banking service. Both require constant updates to meet customer demands, both suffer from similar problems: legacy code, over-engineering, unmanageable code and yes a flood of bugs coming with every update.” writes Nitro Nation Developer Filipp Keks.
The provided Unity Test Tools didn’t fit the team’s needs for several reasons. Mainly it didn’t allow them to drive long chains of UI interactions in a way similar to how a user would, so they decided to develop test automation framework from scratch. (You can read more about why Game Developers are afraid of Test Automation on Filipp’s blog.)
Nicknamed the Cthulhu automation system, this software can support as many devices as there are USB ports available. The first step is to make a build of the app with tests enabled, then the app is sent to the automation system, this is where Cthulhu then installs the app to all connected devices, starts the app and logs the test data, saving it to separate log files for each device for our QA testers.
“I play Nitro Nation all day every day, players would imagine I have the ultimate dream job.” Says Lead QA Engineer Anton Glotov, “But, with hundreds of different devices, operating systems and platforms that we support – it can be a real nightmare sometimes. Working with players to make a better game experience for everyone makes it all worthwhile. “
These tests are performed offline and help to decrease the need for manual testing so that the QA team can focus on other tasks. The Cthulhu project can be found on Github Github.com
Every day our teams work hand and hand with the player community to track down an assortment of bugs and issues that beset the game. During the Nitro Nation 5.0 sprint our QA team opened 651 unique bugs and eliminated 796 bugs. In the past there has been an informal tradition of rewarding players who help eliminate bugs by submitting Support Tickets.
“We have a fantastic cooperation with our players, they are great at finding and reporting bugs.” says Support Department Head, Jelena Balasova. “We as a team trust our players to share their feedback, it really motivates us to work even harder to eliminate bugs! I’m very happy to be able to compensate them for their time and efforts.”
In order to properly appreciate the time and energy our players spend helping to report bugs and issues, Nitro Nation Online will start a formal bug bounty program Starting November 14th. This pilot project will give rewards to the players who are the first to successfully help to report a previously unknown bug, (existing bugs and cases will not be included.) Depending on the level of severity:
Critical Bug – 50 Gold
A severe bug causes system failure, crashes or causes a deadlock and affects several areas or consistently prevents a function from being used without a work-around. This is most urgent and should be resolved with a Hotfix patch.
Major Bug – 15 Gold
A bug that causes significant performance issues, but does not impede game progress and usually has a work-around. So it can be resolved with the next scheduled update.
Minor Bug – 5 Gold
These are the most common type, they generally do not inhibit normal functionality of the game. These may simply be cosmetic or non-reproducible issues that are low priority.
In addition, the Support Team has committed itself to improving the speed of service for Nitro Nation Online players and improving the overall quality of communication with the community with dedicated agents overseeing all aspects of service related to Nitro Nation and working closely with the Development team.
Players looking to report bugs can submit them at the Support.Creative-Mobile.com website or discuss them more in depth on the NitroNation.net forums. The Support Team offered a few pieces of advice for players looking to submit a support ticket:
- Include your Player ID, e-mail or other details about your account
- If the issue is related to a purchase, remember to include your order # or purchase receipt
- For Race Verification and Server related issues, include the time it happened and the time zone you are in.
- Be sure to include a detailed description of the issue
- What did you see? Did you get a screenshot or a video?
- How and where it happened
- Is this a one-time issue or a reoccurring issue?
- Anything else unusual happening with the game?
- Give detailed steps about how to reproduce the issue
- Device type, operating system version and game version
- Step by step procedure from opening to the game to where the bug occurs
