Moderated user testing revealed that the Pay As You Go journey had significant friction across multiple touchpoints, making it harder than it needed to be for customers to find the right product. I restructured the journey, drove a company-level homepage change over stakeholder resistance, and set up Adobe Analytics dashboards to track the impact of every change. Within 25 days of launch, daily PAYG SIM orders were up by 10.
What The Research Showed
I ran moderated testing with around 20 participants, writing the test script, moderating the sessions, and analysing the results myself. The picture was clear quickly. Only 40% of users found PAYG pages reliably. When they did arrive, 70% couldn’t tell the difference between a SIM and an add-on. On the free SIM page, only 4 out of 10 users spotted the tab they needed.
The filter data told a similar story. Filters were hidden by default. Only 6.5% of users opened them, and of those, just 7.4% went on to purchase. When we showed participants the filters open during testing, engagement went up immediately. The filters weren’t the problem. Nobody could find them.
Approach
Give PAYG a proper homepage presence
The biggest structural change was splitting the homepage’s last tile to promote both Pay Monthly and PAYG side by side. Stakeholders pushed back. This was one of the most commercially sensitive areas of the site. The PM, who had a UX background himself, backed the research and gave the go-ahead. The results justified the call.
Make the filters visible by default
Moving the filters from hidden to permanently visible sounds like a small change. The impact wasn’t small. Filter usage went from 6.5% to 18% within three months, and 23% of users who used the filters went on to purchase. The feature was always there. It just needed to be seen.
Restructure the PAYG SIM journey
I redesigned the SIM Only pages to make the distinction between SIMs and add-ons clear, surfaced the free SIM option more prominently, and brought the pages into closer visual consistency with the rest of the site. Clarity in testing went from 30% to 90% after the changes.
Set up tracking before and after
I hadn’t used Adobe Analytics before this project. I worked with the analytics team to learn it and set up dashboards to measure the impact of each change, meaning we could see results within days of each release rather than waiting for a quarterly review.
Month
Filter usage
Used and purchased
January (pre-change)
6.5%
7.4%
February
15.1%
16.8%
March
18.1%
23.3%
Within 25 days of going live, the team was seeing 10 additional PAYG SIM orders per day.
By March, PAYG hit its strongest performance since August 2017, with 1,317 orders at 6.8% of total volume, up 470 on February and 283 on January.
“The work the team have put in on this project is clear to see. The improvements in the online journey will make a big difference in how we serve our customers and how they spend with us on the PAYG line of business.”
Chris Moseley, Head of Trading, Three UK
Reflection
The homepage tile change was the most contested decision in the project and probably the one with the biggest impact. Getting it through required a PM willing to back research over instinct, and results specific enough to be hard to argue with. That combination of clear evidence, a PM willing to back it, and not backing down is what turned a reasonable proposal into an actual change.
The filter finding is also worth holding onto. Sometimes the right answer isn’t a new feature or a redesign. It’s making an existing tool visible. That’s easy to miss if you go into a project looking for things to build rather than things to unblock.