After looking at a handful of UX case studies, it appears that some combination of the following components comprise an effective study: 

Design Statement

Needs Assessment/Interviews with prospective users

Functionality requirements and constraints

Benchmarks (competitors)/Competitive Analysis

Personas

Storyboards/Wireframing

Task flows

Prototype drawings

Example of user experience/Test uses and findings

But some studies do a better job than others at grouping these components under broader headings that make the project easier to follow sequentially and thematically. The airbnb study described the process in these five steps: Explore, Define, Ideate, Develop & Learn. This seems to capture the generalities of a bunch of the elements that case studies have in common. The “Problem, Solution, Revenue” model seems like another good way to organize a case study–although I find it interesting that the revenue portion comes last. I would assume most “successful” developers allow the revenue stream to dictate the solution–for better or worse.

One case study that caught my attention is the Wikipedia study. I found this study especially interesting because, along with millions of other people, I am very familiar with the iconic interface and even appreciate the simplicity of its design. I couldn’t imagine it any other way–but many people more insightful and imaginative than I am have been making changes to the design of Wikipedia sites for years (there are actually hundreds different versions of Wikipedia).  One of the key takeaways from this study is the idea that fully functional prototypes “(versus mockups, or even a click-through prototypes) communicate designs significantly more clearly, and facilitate higher quality discussions and feedback.” Of course this comes at a steeper investment of time and resources, but the lesson should push UX researchers to strive beyond basic sketches and mockups. This Wikipedia case study is less complete than other examples, but it is worth a look because the components that are discussed are in-depth and relatable for anyone who has used Wikipedia extensively.

UX Case Study Designed by ChatGPT

I asked ChatGPT to design a UX case study (with itself as the subject). Here is the specific prompt I gave:

Me: Using a “Problem, Solution, Revenue” model, please create a UX case study for the ChatGPT website. (1/12/2024)

ChatGPT:

Title: Enhancing User Experience on ChatGPT Website

Problem: The ChatGPT website currently faces several user experience challenges, hindering optimal user engagement and satisfaction. Users often find it difficult to navigate through the website, locate relevant information, and understand the full range of capabilities offered by ChatGPT. Additionally, there is a lack of interactive elements that could enhance user interaction and make the website more user-friendly.

Solution: To address these challenges, we propose a comprehensive redesign of the ChatGPT website, focusing on improving navigation, providing clear information, and introducing interactive elements. The redesign will follow a user-centric approach, ensuring that visitors can easily access the information they need and understand the capabilities of ChatGPT.

• Redesigned Homepage:
  • Clear and concise messaging to explain ChatGPT’s capabilities.
  • Streamlined navigation to guide users to different sections.
• Interactive Demonstration:
  • A live demo section allowing users to interact with ChatGPT in real-time.
  • Clear instructions and examples to showcase the model’s capabilities.
• Information Hierarchy:
  • Organize information in a clear hierarchy, making it easy for users to find what they’re looking for.
  • Use visual cues and concise text to communicate key points.
• User Onboarding:
  • Implement a user-friendly onboarding process to guide new users through ChatGPT’s features.
  • Provide tooltips and hints to help users navigate and understand the platform.
• Responsive Design:
  • Ensure the website is fully responsive across various devices, providing a seamless experience on desktop, tablets, and mobile devices.

Revenue: The improved user experience is expected to lead to increased user engagement, longer session durations, and higher user satisfaction. This, in turn, can contribute to higher conversion rates for premium features and services offered by ChatGPT. By attracting and retaining more users through an enhanced website experience, we anticipate a positive impact on overall revenue generation.

Outcome: The redesigned ChatGPT website, with a focus on improved user experience, is expected to address the current challenges and create a more inviting and interactive platform. Users will have a clearer understanding of ChatGPT’s capabilities, leading to increased user satisfaction and potential revenue growth. Continuous monitoring and user feedback will be crucial to refining the website further and ensuring an ongoing positive user experience.

###

To start, I think this outline is far too vague to be very helpful. Maybe a smart UX designer could work with the initial problem analysis in the outline and make a bunch of personal judgements about specifics, but the examples I looked at were far more specific in identifying needs and problems. I shouldn’t be too surprised that the machine didn’t produce something more specifically critical about itself, but I did have higher hopes. Of course, there is always the chance that my own prompt wasn’t specific enough to start with.

What is especially interesting about the outline created by ChatGPT is how it suggests a revenue strategy that relies on increased user engagement. On its face, this seems at least a little specific, but I am also willing to bet the machine is just pulling a probability here  from its training data related to how tech companies generally drive their income, as opposed to evaluating or at least balancing the best ethical choice for the end-users.  The maximum-engagement model runs completely counter to what Sam Altman (CEO of OpenAI, ChatGPT’s parent company) said in congressional testimony last year. Altman, responding to concerns about tech addiction issues, claimed that “we try to design systems that do not maximize engagement.” His reasoning was that they lacked sufficient computing resources to even support more engagement. But like many (most?) other tech companies, OpenAI is now embracing the increased engagement model to drive revenue. This was even specifically mentioned on their recently released ChatGPT store. So the fact that ChatGPT included this typical revenue model in my request for a UX case study is not altogether surprising, but it does go directly against the previously stated claims of the company’s CEO. 

This idea is important for UX studies because designers have competing factors that drive their designs. Two major factors that can find themselves in conflict are the maximizing of revenue and the maximizing of benefit the product brings to the user. Very often these two factors can overlap, and that is best case scenario for all. When a product is tweaked or designed in way that makes it both cheaper to produce and safer or more enjoyable for the consumer, that is ideal. But when a company prioritizes profit over the user’s best interests, we can end up with tech products that cause problems in a user’s life that could outweigh the benefits. Screen addiction for kids and negative mental health outcomes for users are two potential downsides that ChatGPT initially ignores. This seems incredibly important for UX designers to keep in mind as they go about their research, planning, and execution.

Here my second prompt and response from ChatGPT. I was after a better (more specific and more user-focused revenue strategy) UX case study:

Me: Can you re-do that and be far more specific about the problems, and then instead of simply suggesting a revenue stream based on maximizing engagement, figure out a strategy for a revenue stream that protects the users from the harms and negative outcomes that could come from overuse and overreliance on tech products like ChatGPT.

ChatGPT: 

Title: Mitigating User Risks: A User-Centric Redesign for ChatGPT Website

Problem:

1. Information Overload:
   • Users struggle to grasp the full scope and capabilities of ChatGPT due to a lack of clear and concise information on the website.
   • Ambiguity around the model’s limitations and potential risks leads to confusion.
2. Navigational Challenges:
   • The current website structure lacks a logical flow, making it difficult for users to navigate and find relevant information.
   • Users often abandon the site due to frustration with the disorganized layout.
3. Limited User Guidance:
   • New users lack sufficient guidance on how to interact with ChatGPT, resulting in a suboptimal user experience.
   • Ambiguous onboarding contributes to users feeling overwhelmed and disoriented.

Solution:

1. Detailed Information Architecture:
   • Create a dedicated section that clearly outlines ChatGPT’s capabilities, limitations, and ethical considerations.
   • Provide examples of appropriate use cases and communicate the importance of responsible AI use.
2. Streamlined Navigation:
   • Implement a user-friendly navigation system with intuitive labels and a clear hierarchy to improve the overall website structure.
   • Prioritize key sections such as documentation, use cases, and ethical guidelines.
3. Interactive Onboarding:
   • Develop an interactive onboarding process with step-by-step guidance on using ChatGPT responsibly.
   • Include educational elements to inform users about potential risks and the importance of ethical usage.
4. User Education Hub:
   • Establish an educational hub containing articles, FAQs, and video content to help users understand the technology better.
   • Address common misconceptions and concerns to foster transparency.

Revenue Strategy: Instead of maximizing engagement without bounds, we propose a revenue strategy that prioritizes user well-being and safeguards against overuse:

1. Usage Tiers:
   • Introduce tiered pricing plans with usage limits to encourage responsible usage.
   • Offer educational resources to users who approach or exceed their usage limits, promoting awareness of potential negative consequences.
2. Ethical AI Certification:
   • Create a certification program for businesses and developers, emphasizing responsible and ethical AI usage.
   • Generate revenue through certification fees, with a portion allocated to ongoing user education initiatives.
3. Premium Support for Ethical Use:
   • Offer premium support plans specifically tailored to guide users in ensuring ethical and responsible AI practices.
   • Revenue generated from premium support can be reinvested into user education and awareness programs.

Outcome: The redesigned ChatGPT website, focusing on user education and responsible usage, aims to mitigate potential risks associated with overuse and overreliance on AI technology. By implementing a revenue strategy that aligns with ethical principles, ChatGPT can foster a community of users who prioritize responsible AI practices, creating a positive impact on both user well-being and the product’s long-term sustainability. Continuous user feedback and iterative improvements will be crucial in refining the approach over time.

###

This is a marked improvement here which makes me think that much of the time when someone finds ChatGPT to be creating poor boilerplate writing, it could very well be due to their choice to accept the first thing the machine spits out. Better prompts create better responses, and a two-step process with a call for specific improvements to a first response yielded much better results for this meta-UX case study outline.

The altered revenue section is intriguing and maybe impressive because a tiered revenue/access model could be an actionable measurable strategy for the company to pursue. This could, at least in theory, have the users’ better interests in mind while still providing a revenue stream that OpenAI could adjust with their own setting of prices. Maybe we shouldn’t be surprised that when we work a little harder on our own instructions and communication with the machine, giving it opportunities to improve like we would any student, we end up with better and better iterations of the original response. Maybe this is what fascinates me most about generative AI: the ability I have to treat it both as a tutor and student.