Datalab 14: Conceptualizing a Wireframe Prototype

Today we're going to cover user testing as a means to prototype our application. We're first going to review our vertical slice prototypes with our peers by means of a user test. Then we're going to have a Wizzard of Oz workshop and user test to prototype the AI component of our application!

Learning Objectives:

After this module, you'll be able to run and conduct:

User testing;
Wizzard of Oz user testing;

Table of contents:

Stand-up: 0.5 hours
Q&A:
User Testing: 1 hour
Wizzard of Oz Testing: 3 hours
In-Class Discussion: 0.5 hours
Work on creative crief: 2 hours

Questions or issues?

If you have any questions or issues regarding the course material, please first ask your peers or ask us if you can't figure it out together!

Good luck!

0) Stand-up

We start by hosting a stand-up. Form groups of ~ 5 and run on-another through the following points:

What progress have you made up since last datalab?
What progress do you anticipate to make today?
What impediments are you facing or expecting?
With what could you use help or support?

Open your worklog and plan your day informed by the stand-up and today's schedule

1) Q & A

We start by briefly reflecting on what we learned about interaction and interface design. Do you have any questions? Now is the time to ask them!

2) User testing

All products should include some form of user testing, whether they are digital or not, interactive or non-interactive: as a designer you generally know too much about the product and technology relating to your own project. You cannot simply pretend you don't know these things; that's not how your brain works, and therefore we perform user tests to create a ecologically valid setup to test our designs!

The most basic form of testing your design is simply showing it to a participant and having them ‘conceptually' interact with the application. This can be a generic prototype, like you can do with your wireframe, or a more designer guided one; usually non-interactive, where you are controlling the prototype instead of the participant.

Open your HCAI design document and create a section called ‘User Test Datalab 14'. Create a bullet point section for logging all feedback points of your peers! Log feedback regardless of your considerations, refuting the feedback comes at a later stage and should not be a filter while transscribing the feedback.

Run you classmate through your vertical slice and explain him along the way what will happen for each given condition and user story. Collect any feedback he might have! This is the crucial part of user testing because you will need to know what might work and what might not work in order to iterate on your design: make the best possible application.

Save your HCAI design document to Github and note what your did; and your evidence, in your work- and learning-log

3) Wizzard of Oz Prototyping

Watch the following video first!

Bla https://pair.withgoogle.com/guidebook/workshop

So basically, Dorothy and her friends go to see the Great and Powerful Wizard of Oz, and find out that he's a fraud with no real magic. When using the Wizard of Oz prototyping method, we try to create the illusion of a functional product, to allow for testing before producing the final version. This way, we can test the interaction before investing the time and money to truly make it functional. With respect to AI this is especially useful because a big challenge with machine learning systems is prototyping. If the whole value of your product is that it uses unique user data to tailor an experience to her, you can't just prototype that up real quick and have it feel anywhere near authentic. Also, if you wait to have a fully built ML system in place to test the design, it will likely be too late to change it in any meaningful way after testing.

Essentially, Wizard of Oz studies have participants interact with what they ‘believe' to be an autonomous system, but which is actually being controlled by a human (usually a teammate). Having a teammate imitate an ML system's actions like chat responses, suggesting people the participant should call, or movies suggestions can simulate interacting with an "intelligent" system. These interactions are essential to guiding the design because when participants can earnestly engage with what they perceive to be an AI, they will naturally tend to form a mental model of the system and adjust their behavior according to those models. Observing their adaptations and second-order interactions with the system are hugely valuable to informing its design.

So, let's start coming up with our own Wizzard of Oz experiment!

Here's a quick example walkthrough of how to create a Wizzard of Ozz (Paper) Prototype!

Open your HCAI design document and create a section called ‘Wizzard of Ozz Experiment'

Conceptualize the test using your user stories and knowning the functionality of your algorithm.
Create the necessary images, videos, animations and elements to do the test. (if you need to print materials, just ask the lecturers)
Run the test and observe the user's actions while making the system react to those actions by triggering the different responses the system should give at that moment in the interaction. It's best if the ‘Wizzard' hides from view but that's not always possible.
Take notes of what works and what does not work while doing the test.
Ask participants about their impression of the system and the design. Take notes.

Save your HCAI design document to Github and note what your did; and your evidence, in your work- and learning-log

Note: A Wizzard of Ozz Prototype can be executed in a variety of ways, this time we chose to do Paper Prototyping because it's most accesible and rapid way to prototype a design.

4) In-Class Discussion

At 14:30, there's a meeting you're encouraged to take part in to ask questions and to discuss our progress and reflect on today activities.

5) Work on Creative brief

Implement your findings in your wireframe prototype!

Up Next!

Next week, we will continue building our Interface Design skills and work on our wireframe prototype!