Designing for policy change

The project

A change to the Scottish Rate of Income Tax meant Pension Scheme administrators needed a way of checking a member’s residency status.

A multidisciplinary team were stood up to design and create this service, I was responsible for the user research on the project in it’s Alpha phase through to Private Beta.

Methods
Remote usability testing
Interviews
Observations

Phase
Alpha – Private Beta 

Page in service
Page in service

Problem

In December 2017, the Scottish government voted to change the income tax bands. Pension scheme administrators now needed a way to easily identify a member’s residency status so that the right tax relief can be applied.

As a Pension Scheme Administrator, I need to be able to obtain the residency status for new scheme members, so that I can claim the correct rate of tax relief against their pension contributions and avoid over / under payments occurring against their tax accounts. 

– Epic user need

Research objectives 

Before this service could go live we needed to understand:

  • If our users would understand what the service was and who it was for
  • If our users understood the residency results provided
  • If users were able successfully to get through authentication 
  • If the service was meeting the needs of different Pension Scheme administrators.

What I did

Rounds of prototype testing to test the end to end journey before go live. To understand how users would expect to find the service and if they understood the guidance before starting.

Observational research with pension scheme administrators using the live service for the first time. To understand what users had to do prior to using the service and whether they were able to successfully use the service using real data to identify their members residency status. 

Retrospective telephone interviews with users who had used the service multiple times. To understand behaviours in how users were using the service and what problems occurred in service and what users them did to resolve them. 

Project research plan timeline
Research plan timeline

End to end prototype testing

To understand how easily our users could complete the service’s end-to-end journey I tested how users would expect to find the service and how they would get through government’s verification to be able to access it, designers in the team also helped create dummy results to be able to test user’s understanding of the results.

Our users were spread out across the country, I conducted remote sessions using Skype to screen share we could observe users using the prototype. I set up an observation room so the team could all observe making notes of what they saw on post it notes, after each interview we discussed the feedback and at the the round of testing had a team affinity sort.

Findings after multiple rounds of testing

Testing highlighted problems in the service.

  • There was still a lack of clarity what tax year users will see depending on when they access the service.
  • Users were uncomfortable having to use a personal mobile device to access the service, concerns were also raised because multiple people would need access to the service. (2 step verification)
  • In the result file there was still some doubt among some users which column related to which tax year. (Due to lack of headers)
  • Certain guidance such as CSV format, when the results are available, how long they’ll be available and restrictions around how many files they can uploaded was not acknowledged on guidance page which lead to questions once they’d entered the service.
  • A couple of users wanted to know when the results would be ready to collect to stop them having to unnecessarily  checking.

Improving the service

Content on guidance page was revisited to see how we can make it more concise ensuring users understood all the information they need before entering the service. We also explore ways of explaining what tax year the user will see at different times of the year to ensure they understand they when should be accessing the service.

Our research was shared with the team working on the Government Gateway service. 

End to end prototype testing

To understand how easily our users could complete the service’s end-to-end journey I tested how users would expect to find the service and how they would get through government’s verification to be able to access it, designers in the team also helped create dummy results to be able to test user’s understanding of the results.

Our users were spread out across the country, I conducted remote sessions using Skype to screen share we could observe users using the prototype. I set up an observation room so the team could all observe making notes of what they saw on post it notes, after each interview we discussed the feedback and at the the round of testing had a team affinity sort.

Testing the service in live

Once in Private Beta we were able to understand how users would use the service with their real data.

We combined analytics, interviews and observational research to understand user behaviours and the problems users were having with the service.

We learnt that the single search functionality wasn’t used in the way that we’d expected, small Pension schemes with less than 10 members were using the bulk search as still a lot quicker for them. We found that the single search was instead used when there was error in the data returned.

The biggest issue that we identified in research was the download feature not working. 14 of the first 21 users that used the service received an error that prevented them from downloading the file. Through the interviews we understood that our users used a variety of different spreadsheet applications and different versions which caused the service to not be able to read the file.

Outcome of this research

Since this research:

  • The service passed it’s GDS Private Beta service allowing it go live on GOV.UK.
  • Improvements and iterations have been made to the live service.
  • Documented what we had learn and new user needs established.