Mister Goodcat

Peter's home of all things life

Thursday, 4/5/2012 8:55 AM
by Peter Kuhn

New certification requirement: Application Testability

Thursday, 4/5/2012 8:55 AM by Peter Kuhn | 4 Comments

A new requirement has been established in the technical certification section for the Windows Phone Marketplace: Application Testability (5.1.4). Let's take a minute to look at the requirement in detail:

"The application must be testable when it is submitted to Windows Phone Marketplace. If it is not possible to test your application for any reason […] your application may fail this requirement."

The requirement then lists some items that apply to testability. For example, if using your application requires a user account, you need to provide test credentials the tester can use during the certification phase. Of course, the list provided there is not exclusive. Take my open source project PAARC as another example. It doesn't require a user account, but it is only really functional if you use the phone app in conjunction with a desktop application you want to control remotely. To this end, I provided a sample application and added the download link as well as detailed instructions for testers during submission. You can use the already available field for testing instructions in the certification process for this. In some cases you may even need to file a technical exception, for example if the app is only functional with additional external hardware.

One twist in this story is that you should not forget to remove, deactivate or suspend test accounts or any test infrastructure that you established for certification once the app has been accepted. To continue the sample of the certification documentation: you don't want anybody to find out how to create gift cards for free, or use a sample gift card code without paying for it, right?

In my opinion, providing sufficient information for testing, and making sure that the required infrastructure for your app (web services etc.) is available, up and running during the certification phase already, is only common sense. However, until now your app passed certification if parts of it couldn't be tested due to these issues. This will now be different and turned into the opposite: if testers are not able to access certain features of your app because required data or technical support is missing, it will fail certification.

So make sure that you keep this in mind, especially because at the moment (re-)certification of an app may take up to 7 calendar days due to the high volume of submissions – you don't want to wait twice just because you didn't provide a test account the first time, do you?

Comments (4) -

This should not be a big deal.  For our major enterprise app we provided them with a test account they could use and test server information.  We did this along with a long description of how to actually use the application.

I agree, it shouldn't be a big deal - but if everybody thought that way, there wouldn't be need for that new requirement. Apparently there are devs out there who think different and don't provide that information.

I got the same 'problem'..though the workaround was a bit silly.
I created a remote control app for a specific type of mediacenter hardware appliance. Since the testers didn't have that specific piece of hardware they couldn't test a thing. Communication with the appliance was done through a php-page , hosted on the device itself.
I let a friend of mine simply emulate that page and pointed the testers to that page (it basically always responded with 'succes' for any given compatible command it received) . That was all the testers needed to certify my app...

Hi Tim. Of course your workaround wasn't sufficient to fully test the app, like error conditions etc; but I believe that's already more than a lot of others would do, and it helps the testers understand what the app does and validate the majority of remaining requirements. A nice example for testability!  

Pingbacks and trackbacks (1)+

Comments are closed