1.The project background

Siam Commercial Bank introduced a finance app tailored for business account holders, enabling them to manage their finances through a dedicated account or device. In addition to standard transactions, the app offered specialised features such as approval processes and a to-do list designed for business functions.

After being in the market for approximately a year, the product team aimed to assess the app's usability and gather customer feedback to identify areas for improvement. The discovery team initiated a stakeholder workshop to gather input from the operational team, which helped define the research objectives, questions, and usability testing tasks.

The identified objectives are as follows:

1. Understand the context of use and consumer behaviour when utilising the product.

2. Identify usability issues encountered during app usage.

3. Benchmark the app against other business banking solutions available in the market.

These objectives guided the research efforts aimed at enhancing the app's functionality, user experience, and overall competitiveness in the business banking sector.

2. The methodology

The PLAN and the SETUP

In the user research sessions planned by the discovery team, two main activities were scheduled: user interviews and usability test tasks.

The METRIC

The usability test tasks were designed to measure success through task completion rates and the System Usability Scale (SUS) rating activity.

The LOCATION

As part of the research room setup, we found a small office room and installed two cameras. These cameras were used to capture participants' facial expressions and hand movements, while simultaneously recording their interactions on the screens.

This setup allowed for comprehensive observation and analysis during the research sessions.

3. Data collection and analysis

Prepare - Test - Debrief

Sessions were scheduled every other day to allow enough time for data analysis. The team utilised the User Acceptance Testing (UAT) version as a prototype, with team members alternating roles between moderator and notetaker.

At the end of each day, we conducted a review of the note-taking forms together, followed by an end-of-day briefing session. The following day was dedicated to summarising key insights and planning for the next round of research sessions.

Challenge that comes with COVID-19

Due to the COVID-19 situation in early 2021, one of the challenges we faced was the requirement for everyone to wear masks during the sessions. This hindered the team's ability to analyse participants' facial expressions effectively.

As a workaround, we prioritised capturing participants' remarks and voice tone to gauge their expressions while they interacted with the prototype. This approach helped us gather valuable feedback despite the limitations imposed by mask-wearing.

An initial insight that guided analysis

The user interview sessions revealed two primary working processes among users: individuals who are the sole users of the app, and those who manage accounts used by both makers and approvers.

Understanding these distinct user roles prepared us for analysing usability issues, as we could assess how participant roles influenced their experience and task relevance. This insight was crucial in identifying specific usability challenges and tailoring solutions to improve user interaction with the app.

4. The findings and outcome

89.7% complete rate

The usability test revealed an 89.7% completion rate, with slightly more issues reported by non-active users compared to active users. Importantly, none of the tasks caused a significant number of users to abandon the process, leading stakeholders to deem the overall usability acceptable.

Approach to addressing usability issues

With a System Usability Scale (SUS) score slightly above the average at 70, we prioritised resolving high-severity issues by adding them directly to the product backlog. For example, we identified that transaction schedulers, who require manager approval, hesitated to proceed when they encountered the notification, “This transaction will be effective immediately.” This observation led the team to formulate a problem statement, revealing an opportunity to redesign the message to better align with user needs.

For issues rated as medium and low severity, we engaged in further discussions to explore potential solutions. This process allowed for careful evaluation and solution design for each issue before determining development priority and allocating resources. The goal was to maximize the impact of our efforts on usability and user satisfaction by ensuring resources were directed toward changes with the greatest potential to enhance the user experience.

A competitor analysis for product strategy

Our qualitative data analysis has highlighted several key opportunities for product enhancement based on competitor comparisons. These include streamlining transaction processes to reduce steps, increasing user flexibility with daily transaction limits, and improving the production of transaction evidence—an essential artefact for business users—through more efficient and user-friendly methods.

5. My personal takeaways

1) Effectively analysing usability results by classifying severity

Analysing usability results and classifying issues by severity was crucial for prioritising improvements. By categorising issues into critical, major, and minor, I could focus on the most pressing usability challenges first and recommend solutions that would have the greatest impact on the user experience.

2) Challenges of conducting on-site Usability testing during COVID-19

Conducting on-site usability testing during the COVID-19 pandemic introduced significant challenges, particularly in ensuring safety protocols were followed while maintaining the quality of the test. Participant reluctance to engage in person due to health concerns slowed recruitment efforts, and the need for social distancing impacted the natural flow of interactions during the test. This experience reinforced the need to be flexible with research methods and consider alternative options like remote testing when in-person testing becomes difficult or impractical.

3) The importance of alignment with stakeholders before research

Early alignment with stakeholders was essential in setting clear expectations and defining the scope of the research. By discussing goals, key performance indicators (KPIs), and success metrics upfront, we ensured that the usability testing was focused and actionable. This also helped avoid misaligned assumptions and ensured that the results were framed in a way that could directly inform business decisions, making the findings more impactful and relevant to all involved parties.