New certification requirement: Application Testability
Published on Thursday, April 5, 2012 4:55:00 AM UTC in Programming
A new requirement has been established in the technical certification section for the Windows Phone Marketplace: Application Testability (5.1.4). Let's take a minute to look at the requirement in detail:
"The application must be testable when it is submitted to Windows Phone Marketplace. If it is not possible to test your application for any reason […] your application may fail this requirement."
The requirement then lists some items that apply to testability. For example, if using your application requires a user account, you need to provide test credentials the tester can use during the certification phase. Of course, the list provided there is not exclusive. Take my open source project PAARC as another example. It doesn't require a user account, but it is only really functional if you use the phone app in conjunction with a desktop application you want to control remotely. To this end, I provided a sample application and added the download link as well as detailed instructions for testers during submission. You can use the already available field for testing instructions in the certification process for this. In some cases you may even need to file a technical exception, for example if the app is only functional with additional external hardware.
One twist in this story is that you should not forget to remove, deactivate or suspend test accounts or any test infrastructure that you established for certification once the app has been accepted. To continue the sample of the certification documentation: you don't want anybody to find out how to create gift cards for free, or use a sample gift card code without paying for it, right?
In my opinion, providing sufficient information for testing, and making sure that the required infrastructure for your app (web services etc.) is available, up and running during the certification phase already, is only common sense. However, until now your app passed certification if parts of it couldn't be tested due to these issues. This will now be different and turned into the opposite: if testers are not able to access certain features of your app because required data or technical support is missing, it will fail certification.
So make sure that you keep this in mind, especially because at the moment (re-)certification of an app may take up to 7 calendar days due to the high volume of submissions – you don't want to wait twice just because you didn't provide a test account the first time, do you?