My mom says you should read science-fiction if you want to know what the future will look like. In her 50+ years as an avid reader of the genre, she has seen things once considered preposterous materialize into reality. Her assessment of ChatGPT and other generative AI is that it is coming for us all.
I'm more of a historical fiction person myself, so my take on generative AI is a bit more measured – it is an impressive tool that will become an integral part of our everyday existence, much the same as the internet did in the 1990s and early 2000s.
About a month ago, my colleague Shabbi Khan and I were discussing how the use of this tool in the workplace will continue to expand. While in agreement that we should write a blog on the legal issues surrounding the use of generative AI (like ChatGPT) at work, we lamented that it would take quite a bit of time to accomplish. This is a challenge for anyone, but particularly for lawyers where the product you sell is an hour broken into six minute increments.
I mused that we should just ask ChatGPT to write the blog post for us. We had a little laugh. From there, this 'man vs. machine' experiment was born. I generously offered to take the task of prompting ChatGPT to write the blog post while Shabbi was tasked with drafting it the "normal" way.
Author's note: ChatGPT is a mouthful. It must have been named by a computer programmer and not a marketer. From here on, I am going to call ChatGPT "Cathy" after "Chatty Cathy," the 1960s pull-string doll that likewise was a technological marvel for its time.
The results of this experiment were unsurprising:
Efficiency: +1 Point for the Machine. My time commitment to this experiment, a mere 11 minutes (or .2 non-billable hours, if you will), paled in comparison to that of Shabbi, who spent 10 non-billable hours over 8 days researching, pondering, and then finally drafting and editing the article. To no one's surprise, Cathy wins this point.
Bias: +1 Point for the Human. I award this point to Shabbi because Cathy did not mention that she hallucinates (that is, she makes up responses sometimes when she doesn't know the answer). It's funny – one would think that this lack of self-awareness would have been the human trait, but not so in this case. Perhaps this deserves extra weight because the fact that Cathy is so confident, and "doesn't know what she doesn't know," could easily lull one who relies on her into a false sense that her result is more accurate than it actually is.
Now, Shabbi is an IP lawyer and his list of 10 issues is mostly centered on IP topics. But I forgave this in my scoring because when Shabbi and I decided to undertake this project, we both agreed that to do it fully, we would need to ask many of our colleagues across various practice areas to weigh in (e.g., labor and employment, data privacy). It was determined that doing so would slow the process down so much that ChatGPT would be outdated technology by the time we had completed the blog post. Perhaps this demonstrates that Cathy deserves another point, but she was already awarded a point in efficiency and, frankly, I feel humans need a finger on the scale at the moment.
Readability: +1 Point for the Machine. Cathy's responses were snappy, short, and easy to read. But that readability came at the expense of depth. See next bullet.
Effectiveness: +1 Point for the Human. Shabbi's article took a much deeper dive into the topics that he raised than Cathy's did. She also repeated some of the same issues more than once in slightly different ways to round out a list of ten. For this reason, Shabbi gets the point.
Winner: With man and machine each scoring two of the available four points on my totally made-up scoring system, we have a tied ballgame.
Takeaways: My takeaways from this experiment, as well as with the content of the articles themselves, is that there may be a place for using Cathy at work but the boundaries of when its use is appropriate are still being established. For now, here are some practical tips for using Cathy (and generative AI in general) to be more efficient while avoiding trouble:
- Figure out how to prompt it in a way that gives the best result.
- Use it for appropriate projects – blog posts on 'man vs. machine' style experimentation is a good example of such an appropriate project.
- Do not feed it confidential information – the user cannot control what it does with that information.
- Always verify what it gives you is accurate – trust, but verify.
- Do diligence to ensure that the response it gives you is not plagiarized.
- Include appropriate notices and disclaimers about the item being produced with ChatGPT. I expect someday this will be like a Prop65 warning – it'll be on everything and consequently barely noticed by the reader.
The bottom line is that the way everyone works is likely about to change at a breakneck speed, and no one is perfectly clear what that means from a legal perspective. The laws will play catch up to the technology. In the meantime, here are some of the legal issues that Shabbi and Cathy identified with respect to using generative AI in the workplace:
Top 10 Legal Issues of Using Generative AI at Work
Context
It is not lost on us that in this experiment we, being lawyers, used legal content as our test case – specifically asking Cathy to spot and write about the legal issues surrounding the use of generative AI. As lawyers, we are especially concerned about providing accurate content for both our client work and, as here, where our analysis is offered for general information and not considered legal advice. But accuracy is critically important in almost any field.
Thus, as noted above, even before Shabbi would post his work for public consumption, we said he would normally consult with more knowledgeable attorneys (which he didn't do here.) The same is true for anyone who relies on Cathy. The content she generates should not be used on its own, and should be checked and vetted before anyone relies on it or offers it up to someone else as authoritative. Needless to say, the content generated by this experiment is merely an illustration and is not intended to be relied upon for its substantive accuracy.
The bottom line is that the way everyone works is likely about to change at a breakneck speed, and no one is perfectly clear what that means from a legal perspective.
The content of this article is intended to provide a general guide to the subject matter. Specialist advice should be sought about your specific circumstances.
