Simple, quick tests you can run to inform how you build your idea
When building something new there are no instructions or guides on whether or how your idea will work. You need to create that guide yourself and you can either do that through guesswork (đ±) or by quickly checking and testing as you go (đ).
‘Lean’ experimentation is all about learning if your idea is right for your audience. It’s not just about tweaking your idea. The aim is to decide whether to ‘pivot, persevere or proceed’ – change what youâre doing, carry on learning more, or move forward with the idea because you have enough information. You’re learning your way forward through multiple, quick experiments.
This type of testing increases your confidence and likelihood of success because:
- You learn more about your idea and if there are problems or options you have not identified.
- You can learn about your idea from the perspective of the people who itâs for.
- Your audience can answer any questions better than you can!
- You can widen the range of opinions you have on the idea.
- You can keep team bias at bay by checking in with others.
For effective testing, you should always start with a clear, specific and measurable question. Use that to decide how you might test. A great research question helps you choose what test to run, keeps you focused as you make decisions, and makes analysis of results simple and easy.
Once you have that question, you can find a corresponding method to start answering it. Below are 2 charts to help you with identifying a good method to use for your test.
First, decide what type of test will best answer your questions
- If you want to learn more about your audience, the people this idea is for, that is a âMarketâ test.Â
- If you already have an idea and you you want to learn more about your idea or the thing you want to deliver, that is a âProductâ test.
- Look for a âGenerativeâ test if you want to find out what your options are
- Look for an âEvaluativeâ test if you want to learn which of your options is best
Here is an example from a previous team who wanted to test a new Christmas fundraising idea.
First, they mapped their assumptions:Â
Then they ran 3 tests to learn more about some of their assumptions before they launched the campaign. They also ran other tests while the campaign was live.
Based on the assumptions above the team decided to complete the following 3 tests before they launched.Â
They each test these assumptions:
- We assume the messages in the films are emotive and clear enough
- The Campaign shows the need and the solution
- Assume we have chosen the correct campaign
- We are assuming that people will feel that their money has gone to a good cause
1. Comprehension test the key messages and later the video and advert content
What will we do? Ask a handful of people to read/watch and then explain them back to you in their own words to check they understood and/or took away the message that you expected them to.
What questions will we learn about?
- Are the messages in the campaign clear enough? (Assumption 1)
- Do people understand the need and the solution? (Assumption 2)Â
- If the âcorrectâ campaign is clear, emotive and demonstrates that the money goes to a good cause, is that true for the content we have so far? (Assumptions 1, 2, 3 and 4)
How long will it take? As little as 10 minutes per person, or approximately 1 hour total. You can allow a bit more time if youâd like to get additional feedback but if you want to keep it short, itâs fine to use staff who havenât been involved with the work so far.
What materials do we need for the test? To run this remotely, the easiest option is to use video calling and share your screen with the participant to show them either the advert content, the whole advert, or just the video.
How many people are needed to run the test? You only need 1 person, but as many as 6 (1 per participant) can do it to save time.
How many people do we need to test with? Iâd recommend 6 given the short time window and the high importance of the tone and messaging for this campaign.
What metrics will we measure? You will measure the number of people who recognise the key messages and themes in your advert/video.
Before you start testing, you should agree on what you want people to understand from the content. The simplest way is to agree the key messages and/or some key themes or words youâd like people to use.
What will it tell us? After you have done the tests, check if the participant used the same words or messages in their description. If they do, congratulations, your content is clear enough.Â
If they donât, we consider that a failure because they did not recognise the intent, even if they can explain the content in a way that makes some sense.
What will we do with the information? Even if only 1 person doesnât pass the test, pay attention to what they missed and consider if you can make that clearer.
2. Preference test options for Christmassy images/animations/etc. for the video and images/content for the adverts.
What will we do? Ask a handful of people to choose between 2 or more options you provide.
What questions will we learn about?
- Which of the options are most emotive and have the clearest connection to the campaign? (Assumption 1)
- Which options best show the need of the campaign? (Assumption 2)
- If the âcorrectâ campaign is appealing and attractive, which images or graphics appeal to the most people? (Assumption 3)
How long will it take? As above, this can be very short, as little as 5 minutes.Â
What materials do we need for the test?
As above, to do this remotely, sharing screens is the easiest approach. Youâll need clear pictures of the options – this works best for visual options like images or graphics – to share with the participant.
You may need different questions for different materials. For example if you are choosing between 2 images then you might ask âwhich of these do you think is more emotive?â, if you are choosing christmas-themed visual effects for the video you might ask âWhich of these do you like better?â or âWhich of these is most cheerful/sombre/etc.â.
How many people are needed to run the test? Again, 1 person can do this or you can divide it up to save time.
How many people do we need to test with? 6 is a good rule of thumb again here.
What metrics will we measure? The number of people who choose each option you present.
What will it tell us? Which option people like the best.
What will we do with the information? Decide between a couple of options based on peopleâs preferences.
3. A UX Word Grid test for the videos when complete or nearly complete
What will we do? Show the videos to a few people unconnected to the campaign planning and then ask them to choose words from a list or grid to describe the video. Obviously youâd be looking for things like ârelatableâ, âemotiveâ, etc.
What questions will we learn about?
- Are people easily able to identify words? If they are not, is it because they are not clear on the messages? (Assumption 1)
- Do people pick up on the intended tone of the campaign, specifically noting words like âemotiveâ and ârelatableâ? (Assumption 1)
- If the âcorrectâ campaign is clear, emotive and demonstrates that the money goes to a good cause, is that reflected in the words people use? (Assumptions 1, 2, 3 and 4)
How long will it take?
You will need the time for them to watch the video in full plus a minimum of 5 minutes for the test. You mentioned having a short video for social media and a longer one for the web. For this Iâd recommend asking them to watch the short video, choose their words from the grid, then watch the longer video and choose new words (obviously they might be the same words and thatâs fine).
You can also invite people to add their own words if they feel they are missing.
What materials do we need for the test?
Remote screen sharing is a good idea again. Start by asking them to watch the video, then show them the word grid – donât let them see the grid while they watch the video.
Itâs best to create a custom word grid, I have sent you an example so you can use that as a base and switch out less appropriate words and add your own for this campaign. For example, âemotiveâ and ârelatableâ came up a lot so those would be good to include.Â
Also agree what question you want to ask with the word grid. Some examples are âhow did the video make you feel?â, âhow did the video make you feel about carers?â, âhow would you describe the video to a friend?â.
How many people are needed to run the test? Again, 1 person could do this but if you show the 2 videos then Iâd recommend 2 or 3 people splitting the work to save time.
How many people do we need to test with? Normally, Iâd like to get a higher volume of responses for this type of test but because youâre tight on time, Iâd stick to 6 people again.
What metrics will we measure? Which words people use to describe the video most often.
What will it tell us? Whether the reaction to the content is negative or positive, whether your content comes across how you wanted and expected it to, and if there are areas you could improve on.
What will we do with the information? What will we do with the information? Make adjustments to the content to improve on the negative areas or feel confident that your content has the right tone and is unlikely to prevent the campaignâs success.
