Have you spent time with ChatGPT yet to assess its potential impact and possibilities?
I did this weekend.
The experience was impressive:
- The overall performance from ChatGPT across multiple assignments varied in human-like fashion from request-to-request.
- In fact, its approach to doing the work was not perfect or unwavering, as it would be with a human analyst.
- It got confused a few times, but remained diligent in asking for help to work through the challenges.
- In interactions where I requested that it review and edit content, ChatGPT seemed to me to perform consistently at a junior analyst level. That means it performed work with a level of autonomy, but it still required routine reviews and double checks on the work product it generated.
Overall, ChatGPT interacted and behaved much like a human co-worker would.
7 Insights from Delegating Work to ChatGPT
Here are my seven insights into this emerging working relationship. Change ChatGPT to a human name, like Charlie, and you’d think it was an analyst’s performance review!
1. ChatGPT may follow a requested pattern but then forget it, only to return to it randomly later.
I asked several times for very simple and direct citations in answer to a long series of questions. Initially, it delivered the very specific information I needed. Then it veered into more prose-based descriptions of the citations. In some instances, it would respond with a whole paragraph to explain the citation's context to me. When it gets going, ChatGPT will over-answer to seemingly appear smarter. Later, though, it returned to delivering the simple citations I requested originally.
2. Be on the lookout for LIFU (Last In, First Used) responses.
Multiple times, I asked ChatGPT to review and analyze too much data, which caused the program to stop. After breaking the data sets into multiple groups, I asked about summaries or patterns. Invariably, it leaned on the most recent data to characterize the entire data set, which gave me a queasy feeling.
3. It doesn't always check for accuracy or embrace brevity; that's your job.
I asked for potential titles for a new book; it didn't double back to see if any of its suggested titles were used. When I asked about that, instead of checking the initial list, it offered more titles. As with a junior analyst, it tends to use a lot of words to say very little when it's struggling to respond to your request. Those first drafts are a help, but they aren't ready for prime time as they are delivered.
4. ChatGPT will brown nose the boss after being proven wrong.
We went through a series of exchanges where ChatGPT tried to tell me that what I was asking for didn't exist. I provided information to suggest that it did exist. It argued some more and requested that I show more proof. When I produced the proof, it quickly backpedaled, telling me I was correct and thanking me for pointing out the error.
5. Unless asked, it doesn't provide references for many of its answers.
ChatGPT typically makes you wonder whether its answers are original or plagiarized. For one request, I asked it to pull perspectives from multiple Brainzooming blog posts without incorporating any other sources. I noticed phrases in its response that I typically never use. So, where did those phrases come from? My concern was compounded when I accidentally hit ENTER a few times before pasting in the data that I wanted ChatGPT to review. Even without the data, it would start answering the question. Where were those answers coming from???
6. Creating mutually exclusive categories and excluding outliers? ChatGPT hedges and tries to include everything.
Many people, even senior professionals, struggle to categorize big lists of comments quickly and effectively to create valuable insights. Like a smart junior analyst, ChatGPT did a nice job of quickly summarizing responses. With a closer look, categories often contained a good deal of overlap. Invariably it would include a list of outliers in its response, mentioning that all these other ad hoc comments were mentioned, too.
7. It's friendly, helpful, and continually looking for more work to take on.
At the end of each interaction, it would typically express its willingness to help and invite me to submit additional requests. I applaud that eagerness!
More Work Ahead for ChatGPT
Your experience may certainly differ in working with ChatGPT. I’m looking forward to formulating more tasks to delegate to it.
If you’re incorporating ChatGPT into your work, I’d love to hear about it.
One Final Note
Yes, I asked ChatGPT to review this article. I did call it Charlie for the version I submitted. One request (beyond suggesting edits) was for it to provide a perspective on whether this performance review treated Charlie fairly. Here was the response:
Based on the information provided in the article, it appears that you have provided a balanced and fair assessment of Charlie's performance. You have noted both the strengths and weaknesses of Charlie's performance, and have provided specific examples of where Charlie excelled and where it fell short.
I’d call that success for both of us! - Mike Brown
If you enjoyed this article, subscribe to the free Brainzooming blog email updates.
