What Happens When The Online Bot Is Believed But The Police Are Not

Kashmir Hill, writing in the New York Times, tells a tale of a father who had been asked by the family pediatrician to send in photo of his son’s infected penis. The photos helped the doctor to understand what was happening, and a successful treatment resulted. It also caused Google’s artificial intelligence bot in charge of detecting child sexual abuse to determine that the father was very likely a criminal. It cancelled all his Google accounts, including phone service, photos, email, contact lists, personal and business documents. Without the email and the phone, he couldn’t sign in to his many other accounts. Two police investigations took place and the children were registered in a database of potentially abused children.

How did the AI bot get the images? They were automatically saved “in the cloud.”

False positives can quickly do real damage to the innocent. If you have any family photos where some people are not fully clothed, it’s best not to email these to others online or let your system automatically back them up into the cloud. If your docs need images for telemedicine, use only the hospital’s secure communication system.

Marion Lincoln Lewis Lee bathing her daughter, Lucy Lee, photograph by Francis Watts Lee, 1931. Digital image from original file by Library of Congress http://hdl.loc.gov/loc.pnp/ppmsca.39446