Always Try to Extrapolate What Happens During Testing

A couple weeks ago I went through the effort of documenting and testing a new PIN code change on mobile devices. This PIN code change effectively made it so that users would have to meet best security requirements, such as have an expiring PIN code and not re-using old PIN codes.

Well, during testing I changed my PIN so many times that I actually forgot my PIN. I had no way of getting into my device, and had to actually remotely clear my passcode just to get in. I put this error fully on myself and forgot about it.

Weeks later, after going live in production with this change, we immediately saw an uptick of support tickets opened because users were forgetting their PIN code. They did exactly what I did, which is set a PIN code and then forget what they had set. Support teams were not ready for this issue. They didn’t know what to do to help users help themselves. I knew what to do, of course, and immediately drafted communications to support teams on how to support users.

What could have been done better? Well, for one I should have seen this coming. I even made the same mistake myself! Sometimes during testing, you forget that any mistake that happens should be extrapolated to all users. If I forgot my PIN code, you can be damn well sure that users will too. I should have prepared the documentation before even going live with the change, and made sure that support users were ready to help.

Lesson learned: if you make a mistake during testing, expect users to make the same mistake. On the same line of thinking, anything that happens during testing, try to remember that it may happen many times during a production go live, because thousands and thousands of users will be doing the same action.

Leave a Reply

Your email address will not be published. Required fields are marked *