Our approach to Welsh language content and testing

29 Jan 2024 12:26 PM

Blog posted by: Nia Campbell, 25 January 2024 – Categories: Our services, usability testing, welsh translation.

For the past year, the Modernising Lasting Powers of Attorney (MLPA) team has considered its approach to the Welsh language. Here they share what they’ve learnt from testing Welsh content, and challenges they need to solve in the run-up to public beta.

If you’ve worked on a Welsh language product or service, you probably know that translation tends to be one of the last steps in the process. English content is often sent in bulk to translation teams, after the important design work has been done.

Yet when the design work is done solely in English, the needs of Welsh speakers and the mechanics of the language itself are not always considered. This can lead to problems further down the line. The classic example is that ‘yes’ and ‘no’ answers in Welsh depend on the question, so there’s no one-size-fits-all translation here. (See other examples in HM Land Registry’s guide to creating a Welsh language service.)

While translators do their best, without enough context of products and services, translations can be overly formal or even different in meaning. 

As research from the Centre for Digital Public Services (CDPS) shows, the results can be that people are put off from trying or unable to use services in Welsh.

Testing Welsh language content

Since the summer, we’ve been fortunate to work with the HMCTS Welsh Language Unit to translate our content from English to Welsh. 

To make sure that the Welsh version provided a consistent experience across the service for Welsh language users, we decided to test it. We did not want to assume that a direct translation of English content would meet their needs. 

In our first round of Welsh testing, we focused on the content itself, rather than the functionality of the service. We tested for: 

• Readability and comprehension – could users understand the content and what they needed to do? Were there unfamiliar words or ones that did not make sense? At what points would they toggle between languages and why?
• Consistency – was the language consistent across the Welsh content? And did it seem consistent with the English when they toggled? 
• Tone – how would users describe the tone? A common complaint is that Welsh is often written too formally, which can make it difficult to understand.
• Expectations – once a user toggles between languages, do they expect the service to continue in that language? If they’re using the service in Welsh, which language do they expect their LPA to be in?

Piloting bilingual usability testing

One of the benefits of having a Welsh speaker on the team was that we could pilot running our test sessions bilingually. This meant we could give users the choice of which language they’d prefer to speak.

As one user explained, this was helpful because it meant they could read the Welsh content and share their thoughts without needing to switch to speaking another language. 

During sessions, our Welsh-speaking team member took detailed notes and interpreted for the rest of the team when users spoke in Welsh.

We learnt a lot and even defined some best practices for the team going forward: 

• Make all comms bilingual – it may sound obvious, but if you’re recruiting Welsh speakers, it’s good practice to make sure that all of your comms are in Welsh. This includes sign-up forms, consent forms, and emails.
• Provide reassurance when recruiting users – not all Welsh speakers are confident in using the language. We found that we needed to reassure some people that we were testing the content, not their Welsh.
• Have a test run – it can be daunting to work with a language you don’t understand, and a test run can help with this. Try to get anyone involved in research together with at least 2 Welsh speakers so you can hear how the language sounds, and get a feel for working with an interpreter.
• Give more time - when you run a session in 2 languages, you’ll need to account for the time it takes for interpreting. We typically ran 90-minute sessions, but often found we did not get through all of the journey. 

‘Trio editing’ with a translator

The feedback on our Welsh content was overwhelmingly positive. But there were instances where words and phrases were unfamiliar, or the tone changed. 

To act on feedback, we adapted an approach we’d seen used by content designer, Adrián Ortega and the team at CDPS called trio writing. This technique involved a content designer, translator, and subject expert designing content together in English and Welsh at the same time. 

We worked through feedback directly with the Welsh Language Unit, reviewing the Welsh and English content side-by-side. 

Not only did it save us time going back and forth in Word documents and emails, but it meant that we could provide context, discuss each comment, and decide whether any changes affected the English content too. 

What else we’ve been doing

Translating and testing content are not the only things we’ve been up to on the Welsh side of the service. We’ve also: 

• trialled a new tool called Weblate to help us manage and maintain content in both languages
• started mapping a process for translation work that involves working more closely with the translation unit 
• set up a fortnightly chat for people across government working on or interested in designing Welsh products and services 

Translation challenges as we go to public beta

While having a Welsh-speaking content designer embedded in the team has been beneficial, we still have some challenges: 

• Testing Welsh content after the English is signed off - our English content is rigorously reviewed and signed-off by our policy and legal teams. But when we test the Welsh version of the content, it can have implications for the signed-off English version too. For example, where content is simpler to understand in Welsh than in English. If we make further changes in English, we’ll need to assess if it needs input from policy again. There are also challenges around policy and legal colleagues not being able to sign off Welsh language content directly if they cannot understand the language. 
• Keeping both languages in sync - in private beta, designing content in English and then translating it works. But once the service is live, the content in both languages will need to be kept in sync. This might mean that the English version cannot be updated until it’s translated and reviewed. This can take time if the Welsh Language Unit are at capacity. 
• Welsh language provisions – when we no longer have a dedicated Welsh speaker embedded in the team, how can we continue to provide parity of languages in user research and usability testing? 

If you’re facing similar challenges and would like an invite to the fortnightly chat about designing Welsh products and services, contact Nia at nia.campbell@digital.justice.gov.uk.