OK, it’s not exactly a corporate doomsday scenario. Still, a recent story from the BBC is a perfect illustration of why testing real-world scenarios needs to be a part of your testing strategy. The sample size might be too small to draw any definitive conclusions about humanity but, if this group is any indication, you might want to keep an extra eye on your 11-year-old.
So, what happened? In a nutshell, some students (aged 5-15 years old) were given a standardized spelling test – on their laptops – online. Have you guessed where this is headed? Well, it appears that some of the students in primary 7 (roughly 6th grade for us Americans) were touched by the light of inspiration. They turned on their online spell checkers.
SNSA P7 spelling. Pupils asked to correct spelling of words. P7 pupils worked out if you right click on your answer, the computer tells you if it is correct! Brilliant! 😂
— George Gilchrist (@GilchristGeorge) May 16, 2018
Yes, the “hack” was that simple and had the obvious effect- misspelled words were underscored in red, à la Word. Students were able to then right-click in order to get the correct spelling. Clever, right? Interpreting the nonplussed reaction of the school system, it seems that a critical setting (browser lockdown? wifi?) was overlooked on some of the computers used to administer the tests. A Scottish government spokesperson said, “advice is being given to schools about how to disable the spellchecking function.”
The takeaway? You can’t just test in a sterile, controlled environment and expect to get a full set of meaningful results. To avoid the kind of oversight that caught out the Scottish school system, you need to test in the real world: the dirty, seedy world with 11-year-olds in it. Outside the box. Things like poor network connections, apps running in the background – or even children – might throw off your app in ways you hadn’t considered in your normal testing routine. In the 21st century, there’s no particular stigma attached to having your bug discovered by a child; kids are shockingly tech-savvy. Still, if you test everything and test often, there’s a much better chance you won’t be embarrassed by a 6th-grader!