Have you spent time with ChatGPT yet to assess its potential impact and possibilities?
I did this weekend.
The experience was impressive:
Overall, ChatGPT interacted and behaved much like a human co-worker would.
Here are my seven insights into this emerging working relationship. Change ChatGPT to a human name, like Charlie, and you’d think it was an analyst’s performance review!
I asked several times for very simple and direct citations in answer to a long series of questions. Initially, it delivered the very specific information I needed. Then it veered into more prose-based descriptions of the citations. In some instances, it would respond with a whole paragraph to explain the citation's context to me. When it gets going, ChatGPT will over-answer to seemingly appear smarter. Later, though, it returned to delivering the simple citations I requested originally.
Multiple times, I asked ChatGPT to review and analyze too much data, which caused the program to stop. After breaking the data sets into multiple groups, I asked about summaries or patterns. Invariably, it leaned on the most recent data to characterize the entire data set, which gave me a queasy feeling.
I asked for potential titles for a new book; it didn't double back to see if any of its suggested titles were used. When I asked about that, instead of checking the initial list, it offered more titles. As with a junior analyst, it tends to use a lot of words to say very little when it's struggling to respond to your request. Those first drafts are a help, but they aren't ready for prime time as they are delivered.
We went through a series of exchanges where ChatGPT tried to tell me that what I was asking for didn't exist. I provided information to suggest that it did exist. It argued some more and requested that I show more proof. When I produced the proof, it quickly backpedaled, telling me I was correct and thanking me for pointing out the error.
ChatGPT typically makes you wonder whether its answers are original or plagiarized. For one request, I asked it to pull perspectives from multiple Brainzooming blog posts without incorporating any other sources. I noticed phrases in its response that I typically never use. So, where did those phrases come from? My concern was compounded when I accidentally hit ENTER a few times before pasting in the data that I wanted ChatGPT to review. Even without the data, it would start answering the question. Where were those answers coming from???
Many people, even senior professionals, struggle to categorize big lists of comments quickly and effectively to create valuable insights. Like a smart junior analyst, ChatGPT did a nice job of quickly summarizing responses. With a closer look, categories often contained a good deal of overlap. Invariably it would include a list of outliers in its response, mentioning that all these other ad hoc comments were mentioned, too.
At the end of each interaction, it would typically express its willingness to help and invite me to submit additional requests. I applaud that eagerness!
Your experience may certainly differ in working with ChatGPT. I’m looking forward to formulating more tasks to delegate to it.
If you’re incorporating ChatGPT into your work, I’d love to hear about it.
Yes, I asked ChatGPT to review this article. I did call it Charlie for the version I submitted. One request (beyond suggesting edits) was for it to provide a perspective on whether this performance review treated Charlie fairly. Here was the response:
Based on the information provided in the article, it appears that you have provided a balanced and fair assessment of Charlie's performance. You have noted both the strengths and weaknesses of Charlie's performance, and have provided specific examples of where Charlie excelled and where it fell short.
I’d call that success for both of us! - Mike Brown