Let me get this out of the way, first: I use AI but it isn’t called AI. It is called autocomplete. It is called spelling and grammar correction. It is called finding photos by describing them. It is forced on me by algorithms in YouTube and Spotify.
What I don’t do is allow statistical models to take over my decisions. In the last few months, I have been on the receiving end of the carnage brought about by the rush to allow statistical models to make decisions.
The first example is a contract. I was entering into a large contract for a 3 year plus term and was sent the first draft. I sent it over to a lawyer to review. I had looked over the contract to prepare for our meeting to discuss the contact, and it looked like what a contract should look like, but had some weird clauses in it. When I got a call from the lawyer, he said that he suspected it was written by ChatGPT (or another LLM/word predictor) service. It had vague language, included unrelated items, and had sections that didn’t make any sense. My lawyer called the other party’s lawyer, and they said that they had not seen the contract and would never recommend that their client sign the contact their own client had sent over.
Our lawyer had to rewrite the contact and it ended up costing me a bunch of money that was unnecessary. It was clear to me that the other party saved a bunch of money by using ChatGPT on their side, and it ended up shifting the responsibility (and cost) to us. They also ended up paying their lawyer to review the changes.
The other example is related to IT software. A few days ago, I was on a support call to discuss an issue with our software. A web page was redirecting to another web page and that page never loaded. It only affected one user and only on one machine. It was very strange. After an hour of troubleshooting, we narrowed it down to a web filter client on the machine and, after disabling it, everything worked fine. The customer went off to investigate and reported that the next day, multiple machines had the same issues across multiple users, machines, and software. It turns out that the web filtering vendor had pushed out an AI update that determines the rules for blocking traffic. It was unrelated to our software and both myself and my customer spent days going back and forth, and spent an hour on the phone, to determine that the statistical model was being used to block traffic in unexpected ways.
Just because something looks correct doesn’t mean it is. Be skeptical and verify. Test. Review.