Datalab 15: Introduction to A/B-Testing

Yesterday we made a deployable prototype of our vertical slice for user testing. Today, we'll have a lecture on A/B-testing after which we're going to we're going to test it with our peers. If we have time left, we'll spend it by iterating on our wireframe prototype based on the collected feedback!

Learning Objectives:

After this module, you'll:

be able to design and run A/B-tests;

Table of contents:

Stand-up: 0.5 hours
Q&A: .5 hours
A/B Testing Lecture: 1 hour
A/B Testing on deployed prototype: 2 hours
Iterate on Wireframe Prototype: Remainder of the day

Questions or issues?

If you have any questions or issues regarding the course material, please first ask your peers or ask us if you can't figure it out together!

Good luck!

0) Stand-up

We start by hosting a stand-up. Form groups of ~ 5 and run on-another through the following points:

What progress have you made up since last datalab?
What progress do you anticipate to make today?
What impediments are you facing or expecting?
With what could you use help or support?

Open your worklog and plan your day informed by the stand-up and today's schedule

1) Q & A

We start by briefly reflecting on what we learned about user feedback and user testing. Do you have any questions? Now is the time to ask them!

2) A/B Testing Lecture

This lecture will be delivered live but you can preview the slide here!

You can watch back the lecture over here.

3) A/B Testing on deployed prototype

Yesterday you made the version of your vertical slice as you envisioned it. But now, we're going to create a second version of you vertical slice which is slightly different then the one you created yesterday. First test your prototype with a classmate and identify points of improvement for you veritical slice to subject to A/B-testing, implement that change in the vertical slice of you wireframe prototype and deploy that version as well for user testing.

If you don't know what to change, ask the lecturers to look together with you for anything which might serve as a possible improvement. Then, run through the steps above.

Done with you new deployed version for user testing? Open your HCAI design document and create a section called ‘A/B Test Datalab 15'. Create a bullet point section for logging all feedback point!

Let several of your classmates; or passer-by's (anyone you fancy apart from the lecturers) test your vertical slice and note along the way what/if different interactions between occur. Every other intersting feedback point should be noted too but in a seperate bullet point list called: "Non A/B-test feedback:". This is the crucial part of user testing because you will need to know what might work and what might not work in order to iterate on your design: make the best possible application.

Save your HCAI design document to Github and note what your did; and your evidence, in your work- and learning-log

4) Iterate on Wireframe Prototype

Now, based on the feedback you collected today; start improving on your design in your HCAI design document. If you have time left, you can try to immediately implement these design changes into you wireframe prototype!

5) In-Class Discussion

At 16:30, there's a meeting you're encouraged to take part in to ask questions and to discuss our progress and reflect on today activities.

Up Next!

Tomorrow, we will cover the Development integration of AI algorithms!

− Interaction Design: beyond human-computer interaction
− UX Fundamentals for Non-UX Professionals : User Experience Principles for Managers, Writers, Designers, and Developers