Introduction

As someone who has never designed a questionnaire before, I was and still am very apprehensive about designing one. I spent some time thinking and researching the right type of questions to be put forth to the participants to elicit the right answers.
So, for this week’s assignment, I developed questionnaires for the ToDo application from the previous assignment. The questionnaires were designed to gather feedback from two different types of users – business professionals and students. I did so because this would help me with my final project. I used these personas with the Todo App in mind, which I will use as my final project. I divided the process into two stages – user research and usability testing. 

UX Research

As a UX researcher aiming to identify user needs and understand their problems, it’s crucial to frame questions that delve deep into users’ experiences, pain points, and challenges.

• Start with Open-ended Questions: Begin the interview or survey with broad, open-ended questions to encourage participants to share their experiences freely.
• Explore Specific Pain Points: Once the conversation flows, dig deeper into specific areas where users encounter difficulties or frustrations.
• Probe for Context and Details: Follow up on users’ responses with probing questions to gather more context and detail about their problems. This helps you understand the underlying reasons behind their challenges.
• Ask About Workarounds or Solutions: Inquire about any workarounds users have developed to cope with the identified problems. This can provide insights into potential solutions or improvements.
• Probe for Unmet Needs: Explore areas where users feel their needs need to be adequately addressed by the current product or service. This can uncover opportunities for innovation and improvement.
• Encourage Stories and Examples: Invite participants to share anecdotes or real-life examples that illustrate their challenges and frustrations. This provides a richer context and helps you empathize with their experiences.

In the Context of the ToDo App, some of the questions are shown below:

Usability

The questions may vary depending on the type of usability testing being carried out.

5-second test

• First Impression: These questions capture participants’ initial reactions and perceptions of the product or interface within the first few seconds of exposure.
• Clarity and Memorability: These questions assess how well participants remember and understand key information presented in the interface.

Walk-around and Exploration Tests

• Exploration Behavior: These questions aim to understand how participants navigate through the product or interface and which features they explore.
• Discoverability: These questions assess how easily participants can find specific features or information within the product.

Essential Tasks Test

• Task Completion: These questions evaluate participants’ success in completing essential tasks within the product.
• Efficiency and Effectiveness: These questions assess participants’ efficiency in completing tasks and overall satisfaction with the process.

Surveys/Questionnaires

• Demographic and Background: These questions gather information about participants’ demographics, experience, and usage patterns.
• Experience and Satisfaction: These questions assess participants’ overall experience with the product and their satisfaction level.
• Feedback and Suggestions: These questions invite participants to provide feedback and suggestions for improving the product.

In the Context of the ToDo App, some of the questions are shown below:

Conclusion

In my learning journey as a UX researcher, I realized that questionnaires are valuable tools for gathering systematic user feedback, which can inform the design, development, and optimization of user experiences. The questions should be clear, unbiased, relevant, and suitable for the participants.
As a UX researcher, does this blog help you with insights in drafting your questionnaires?
( I hope I am learning to frame some questions here 🙂 )

References 

Pervaiz, Muhammad Ahsan. “UX Design Process Simplified from User Research to Usability.” Packt Publishing, January 2020, https://learning.oreilly.com/course/ux-design-process/9781800208797/.

ChapGPT 3.5 prompt. (February 2024).

• For user research and usability testing, what would be the criteria for asking good questions in one-on-one interviews, group discussions, and surveys?
• As a UX researcher conducting research to identify user needs, how would I ask questions to understand user problems for a busy professional and a student persona for a ToDo app?
• As a UX researcher conducting usability testing and conducting – 5-second tests, walk-around and exploration tests, essential tasks tests, and surveys/questionnaires, how would I ask questions for a specific to busy professional and a student persona for the Todo app, elaborate more?

Introduction

This usability testing is intended to assess the user experience of a mock-up of a Todo application. The testing aims to pinpoint any possible usability issues and gather feedback on the overall usability of the application. The ToDo app being tested is a mockup of a task management application that allows users to create and organize their tasks intuitively. The application provides task creation, removal, priority adjustment, and task completion tracking options. The testing involved participants to complete predefined tasks using the application. Their feedback was collected to improve the usability of the application further.

Methodology

As part of the usability testing for the ToDo app, I have created a methodology that outlines the process we will use to conduct the testing. To begin with, I have created a mock-up of the app as part of the previous assignment, which I will use during the testing. The mock-up includes all of the app’s essential features and is designed to replicate the actual app as closely as possible.

Our methodology involves using the think-aloud technique during the testing. This technique requires participants to verbalize their thoughts and actions as they navigate the app. I will also be capturing audio and video of the testing using Screen-pal to analyze the user’s behavior, which will help us identify areas of the app that need improvement. Before the usability testing, the author has taken verbal consent from the participants. The usability testing consent and instructions script are in the appendix section.

To test the app’s usability, I selected the following two participants with varying levels of experience:
Participant 1
Age: 13
Gender: Female
Occupation: Middle schooler
Experience with ToDo Apps: Beginner
Frequency of ToDo App Usage: Rarely
Preferred Platform: Web
Additional Notes: Participant 1 rarely uses ToDo apps and relies mainly on the paper version of a Todo task organizer.

Participant 2
Age: 19
Gender: Male
Occupation: College student
Experience with ToDo Apps: Intermediate
Frequency of ToDo App Usage: Daily
Preferred Platform: Web/Mobile
Additional Notes: Participant 2 uses ToDo apps daily and relies mainly on Google Keep to organize day-to-day work.

During the testing, I will monitor the participants’ performance and collect data on how long it takes them to complete each task, what difficulties they encounter, and how they feel about the app’s overall usability. This data will help the author identify the areas of the app that need improvement and make the necessary changes to enhance the app’s usability.

Observations

Usability Testing

Post-Usability testing feedback

Observations	Participant 1	Participant 2
Rating (1-10 scale)	8	8.5
Improvements	Instead of the Edit button, clicking on the task should allow you to edit the task The background should be more colorful Make the size of the up and down buttons smaller, or use arrows that can be used instead	Was easy to use and good for younger kids The app was basic, without a background or colors/pictures to make it more appealing Sorting by day and time would be useful
Screenshots

Recommendations

Based on the identified improvement, here are some actionable recommendations/feedback to improve the application’s usability, functionality, and user experience:

1. Add a theme or background: To make the application more appealing, consider adding a theme or background that aligns with the genre of the participant. This will make the application more visually engaging and help users feel more connected to the platform.
2. Improve the edit functionality: Rather than relying on an edit button, the edit functionality can be improved by allowing users to click on the task to edit it. This will save users time and streamline the editing process.
3. Prioritize functionality improvement: The prioritization functionality is useful but can be further improved by adding up/down arrows instead of buttons. Additionally, consider adding the ability to add a day/time for prioritization as another means for users to sort and manage their tasks effectively.

Learnings

I personally interacted with the participants during my testing process to help them record their experiences using Screen Pal. This involved setting up a face-to-face meeting to ensure the participant’s experience was accurately captured. However, if one wishes to use Screen Pal offline and still work with the participant, one must use a video application such as Zoom or Teams.

Usability testing may seem simple, but it requires a lot of practice to develop open-ended questions that don’t bias the participant towards specific app features. I made a conscious effort to avoid using app-specific terms such as “button” or “tab” in some places, but in other cases, I had to use these terms to ensure that the participant understood what was being referred to.

In a more realistic usability testing scenario, I would select participants based on the persona for the application. This would involve ensuring that the participants are representative of the target audience for the application and have the relevant characteristics that would make them the ideal users of the app.

Overall, usability testing requires attention to detail and precision to ensure that the results are accurate and reliable.

Appendix

Usability Testing Consent and Instructions 

Introduction and Purpose

Hello! Thank you for joining me today. My name is Afshan Naaz, and I am in my role as a UX Researcher from Georgia State University. Today, I’m here to learn more about how people use an app to track or record their daily tasks and chores.

Voluntary Participation

Before proceeding, I want to emphasize that your participation in this usability test is voluntary. You are not obligated to continue if you feel uncomfortable or decide it’s not for you. Your decision to participate or withdraw will not have any consequences whatsoever. ScreenPal will record your interactions with the application throughout the test. ScreenPal captures both your screen activity and any audio commentary you provide. This allows us to closely analyze your interactions and better understand your thought process as you navigate the app.
Are you comfortable moving forward under these terms?

Overview of Procedures

My role here is to listen to your thoughts and experiences as you use the app. Your feedback will help me understand what works well and what could be improved. There are no right or wrong answers – we’re simply interested in your honest opinions. By no means is this test for you, but rather, it is for the app.

So, feel free to explore the app as you normally would, and don’t hesitate to share any thoughts or questions you have along the way. Your input is incredibly valuable and will directly influence how we shape the future of this app. As you use the app, please speak aloud so I can understand your thought process.

Next, I’ll present you with a series of tasks to complete using the application. These tasks are designed to simulate real-world scenarios and will help us understand how intuitive and user-friendly the application is. You’ll be asked to create a new task, set a reminder, and mark tasks as complete.
Please feel free to ask any questions or seek clarification at any point during the test. Your feedback is invaluable in helping us improve the usability of the application.

The app can be accessed using the following URL – https://afshannaaz-todo-app.w3spaces.com/todo.html

Confidentiality

I assure you that all information shared during this usability test will be kept confidential. Your privacy is of utmost importance to us. Any data collected, including your interactions with the application and feedback, will be used solely for research purposes and will not be shared with anyone outside our research team.

Instructions

Imagine you are a student or professional with a busy schedule, and you’ve decided to use an app to help you stay organized. Today, we’ll test this app to see how well it meets your needs.

• Task 1: How would you go about making a note of all the tasks or chores you need to perform:
• Task 2: If you have to make changes to the existing tasks or chores, how would you do about:
• Task 3: How would you keep track of completed tasks or chores:
• Task 4: How would you organize these tasks?
• Task 5: Can you search for a task or chore?
• Task 6: Does the app provide another way to record the tasks or chores?

Closing Questions

• Share your overall impression of the app’s interface and design.
• How would you rate this application on a scale of 1-10?
• Were you able to complete the tasks easily?
• Did you face any challenges?
• Is there anything specific you liked or disliked about the Todo app?

Introduction

Screen recording is a tool used to capture and record a user’s interactions with a digital interface, such as a website or mobile application. This tool provides valuable insights into user behaviors, preferences, and pain points. By recording user sessions, UX Researchers can analyze user behavior in real time and gain a better understanding of how users interact with their product. Participants can also use screen recording to share their feedback, concerns, and suggestions, which can help improve the overall user experience. Overall, screen recording is a highly valuable tool that provides deep and comprehensive insights into user interactions and behaviors.

Approach

For the software evaluation, I used the free version of Screencast-O-Matic as it was compatible with my Mac. I aimed to evaluate the software for two personas: the UX Researcher and the usability/UX research participant. I captured features that will be used by these two personas, as well as common features that will be useful.

Screencast-O-Matic

Screencast-O-Matic has changed its name to ScreenPal. Screencast-O-Matic is a screen capture tool. This tool records a designated area of your screen as well as audio and webcam input. You can use this tool to record narrated presentations or demonstrations and produce the recordings as videos for your courses. The software offers a variety of features, including the ability to record full-screen or specific regions of the screen, add annotations and effects, and export videos in a variety of formats. Screencast-O-Matic also has a built-in video editor that allows users to trim, cut, and merge videos. Screencast-O-Matic doesn’t have a watermark when you download it for free. But it limits you to 15 minutes of video time only. Users can increase the time limit by buying the full/premium version of the software. A limited video editor is another limitation of Screencast-O-Matic’s free version.

This is the screen shot of my screen’s recording. I was able to record specific sections of my screen as well as capture full-screen recordings. It records anything that is within the white and black dotted frame. 

Also, I was easily able to cut and remove parts from anywhere in the videos. 

Additionally, features like blurring the video, highlighting the text, etc, can be used. 

Also, I was able to make the content more engaging and entertaining by using different sound effects from Screen-O-Matic’s sound library.

Findings

The following feature table can be used to compare other screen and voice recording products and make informed decisions based on their features and capabilities. I also used ChatGPT to get the list of important features used by UX Researchers, Usability/Research participants, and common to both.

Common Features

Features	Review	Comments
OS accessibility		Available on Windows, Mac, Chromebook, iOS, and Android; and Browsers like Chrome, Edge, Safari, and Firefox
Free		Limited features are available in the freemium version, also offered in three different versions: Deluxe, Premier, and Max.
One Button launch		Has the capability to launch a recorder or take screenshots
Export and Sharing option with lightweight output		Available to be downloaded in various formats like MP4, AVI, FLV, and GIF formats in different resolutions
Help and support		Only available through email
Tutorials		How-to’s, training, and tutorials are available to get started
Positive industry reputation		Overall, the rating of the app was 4.3-4.5 out of 5

UX researcher

Features	Review	Comments
Annotation and Marking Features or Tagable Output		Basic annotation features may be available, allowing researchers to add text and shapes during editing
Playback and Editing Capabilities		Provides the capability to basic editing capabilities like cut and copy
Collaboration Features		Couldn’t find the capability to collaborate with multiple researchers, allowing team members to share insights and observations seamlessly

Usability/Research participant

Features	Review	Comments
Minimal Disturbance		Minimal impact on the participant’s experience during recording
Remote Testing Friendliness		Suitable for basic remote testing scenarios where participants record their screen interactions.

Conclusion

Although the recording tool is helpful, its freemium version offers limited functionality. A comparison of feature tables of various recording tools can provide better insights and help in making the right choice.

Field Notes

Field notes are new to me. They are typically used by professionals such as engineers and scientists to track important information as they work. In the context of user research, field notes can be used to note observations and collect data during a project. During the Usability Testing class for the Todo mock, I compiled the following field notes for various activities we did in the class:

Five-second test

Observations

We asked 12 class participants to evaluate the effectiveness of a simple Todo mock. Their first thoughts about the usage were as follows:

• It is probably something for adding tasks
• Used for an organized list of tasks
• Making a note of todo items

Several aspects of the mock were liked, and specific features were highlighted, including:

• Simple to use
• Minimalist (too minimalistic)

Some of the recommendations for improvements included:

• A title describing the functionality would be useful
• The light color complete button confusing
• A check box for completed tasks would be nice

Inference

The 5-second test gave us the effectiveness of the Todo app, its messaging and overall appeal. Based on the initial feedback, the UX researcher can look at improvements.

Heuristics

Definition – A heuristic analysis aims to identify usability problems and issues that may affect the user experience and hinder the overall usability before carrying out the usability testing.

The following guidelines are known as heuristics and were first introduced by Jakob Nielsen:

• Visibility of system status
• Match between the system and the real world
• User control and freedom
• Consistency and standards
• Error prevention
• Recognition rather than recall
• Flexibility and efficiency of use
• Aesthetic and minimalist design
• Help users recognize, diagnose, and recover from errors
• Help and documentation

The survey results showed mixed bags of yes and no responses for various criteria for the Todo mock. 

These are the findings of my Heuristic survey done in class.

Reflection

Pre-Test: I use a to-do list very often. I use it every week to add my to-do things. This helps me stay organized and gives me a sense of accomplishment when I check off the items on my list. 

Post-Test: The usability testing taught me how to add and sort the tasks. I use post-it to make my to-do list more often than using an app. After the usability class, I am inclined and confident about using it on my phone. 

The field notes came in very handy while reiterating the entire process. 

Design mockup Todo list 2.0

A set of recommended heuristics and improvements were identified during a five-second test. To implement them, a mockup was created with the help of Microsoft’s GitHub Co-pilot (similar to ChatGPT or Brad), a code completion tool. The following is a sample interaction with the co-pilot prompt for adding the strikethrough functionality for the Todo app. Unfortunately, the entire history cannot be exported from the browser.

The resulting application includes a range of features, such as displaying tasks as a list of checkboxes, indicating completed tasks with a strikethrough, and editing or deleting tasks as needed. Moreover, users can clear all tasks at once or search for specific tasks using a search bar. Additionally, users can change the order of tasks by moving them up or down on the list or adding new tasks through audio input.

https://afshannaaz-todo-app.w3spaces.com/todo.html

The mockup was designed using HTML, JavaScript, and CSS and is currently hosted on W3Schools. By providing users with a range of features that optimize task management and productivity, this application has the potential to enhance workplace performance and efficiency.

Design thinking is an approach to problem-solving that focuses on understanding the user’s needs and redefining problems to develop innovative solutions. It involves a series of steps to address complex challenges and create user-centric solutions, including product design, service design, and business strategy.

Proposed Case Study Outline

For the case study I plan to do, the outline will follow the ‘Design Thinking’ principles and comprise five stages, each with unique activities to achieve specific goals.

Empathize: In this stage, the focus is on understanding the needs, pain points, and motivations of the end-users. The goal is to put the user at the center of the design process and generate empathy towards their experiences. The activities in this stage include:

• User Research
• Interviews with users 
• Competitive Analysis

Define: Based on the insights gained from the previous stage, this stage’s goal is to define the problem or challenge clearly. The activities in this stage include:

• Articulate user needs, insights, and pain points
• Formulate the problem
• Create Personas
• Building Empathy Map

Ideate: This stage is about generating creative solutions to the defined problem. The focus is on generating as many ideas as possible and selecting the most suitable ones for further development. 

• Create solutions
• Come up with User Journeys
• Building the Information Architecture

Prototype: This stage involves developing tangible representations of the selected ideas. The primary goal is to test the ideas and gather feedback to improve the solutions. The activities in this stage include:

• Create Paper prototype
• Build Wireframes
• Recommend style guide
• Create final design

Test: In this stage, the emphasis is gathering user feedback on the prototypes developed in the previous stage. The goal is to refine and improve the solutions based on the feedback received. The activities in this stage include:

• Conduct usability testing
• Collect feedback
• Iterate on the design and make necessary adjustments.
• Implement the iterated design

The case studies that were assigned for review were all unique, but one particular case study that is aligned with my thinking is Muskan Attri’s Haven Self-Care Meditation App. It describes the process in the five similar steps above.

Interesting Case Study

In his case study, Arvind describes the problem and possible solutions for the users. The solution is explained step by step to make the user journey easy. Qualitative and Quantitative Research has been supplemented with interview questions, answers, and pie charts. It provides a robust foundation for user-centered problem-solving. The Competitive Analysis taken into account by Arvind helps to understand the product’s market positioning comprehensively and identify opportunities for refinement and innovation. 

In the ideate stage, the information architectures component helps create frameworks or mind maps that visualize relationships between concepts and potential solutions. This allows designers to determine how users will navigate through a system, thus ensuring a seamless and user-friendly experience.

Next, the Wireframe feature is an interesting add-on that allows designers to establish the information hierarchy by indicating the placement and prioritization of different elements on the screen. The clean and simple home screen design is what the user wants so that they can easily locate the options and place an order without any hassle. The search page is simple yet very effective. It has features like Search History, trending searches, offers, and promotions to make the shopping experience easy and enjoyable. One feature that I found interesting was that the important details of the product are displayed along with user ratings. 

Arvind’s Case Study is a good example of a UX Case Study. It has offered pertinent information I think I will consider as I start to research and work on my UX Case Study.