Coding Error or Testing Error? July 12, 2016Posted by Peter Varhol in Publishing.
Tags: Pokemon, testing
Today the news is full of the revelation that the mobile game Pokemon Go takes full permissions if you sign in using your Google account. Niantic, the game’s developer, acknowledged this and said it was merely a coding error. Neither Niantic nor Nintendo had any plans for full access to anyone’s Google account, and apparently a patch is being prepared to fix the error.
I am copasetic with the explanation (I am not a gamer; if I were, I may feel otherwise), and I see how it can happen. But is it an error in coding, or an error in testing? I call it a testing error; here’s why.
Developers do whatever it takes in order to make an application functional. On more than one occasion in the past (I no longer code, except for fun), I have allocated too much memory, declared too many variables, and kept objects alive too long, in order to ensure that the application works as I expect it too. My job is to get the application and its features working.
Grabbing too many permissions is similar. I have seen teams that bypass security restrictions because that seems to be the only way to add the needed functionality. For example, an application may require that the user account have local admin privileges. Are these required? Probably not, but in many cases local user privileges didn’t work. Rather than diagnose why they didn’t work, the developers simply open it up completely. Their job is to get it working, after all.
I don’t know if Niantic has a formal testing program driven by professional testers, but this sort of problem is all too common. And testers need to step up and take responsibility for issues that fall into the category of access rights and security.
Granted, this isn’t functional UI testing, which even today many testers believe represents the scope of their responsibilities. Few testers look for permissions issues, and these are almost never discovered on development or testing computers, which typically have full permissions.
But they should. I was in a development lab that lost a major enterprise sale because our software required local machine admin rights in order to install. That enterprise didn’t give any staff employees local machine admin rights, and simply installing our software would have required an IT person to go around to hundreds of computers to adjust permissions.
This is the sort of thing that testing is all about. Understand your customer. And by your customer, I also mean the organizations you are selling into. Test the login process, not only to make sure it works, but also to make sure it doesn’t create a security failing.
Yes, this is a failure of testing. To define this as a coding error is misleading. A competent and curious tester should have caught this before it went out the door. If you are missing this kind of problem, it behooves you to rethink how you do your job.