ASSET LOSS INTERVENTION
Tackling risky cyber behaviours through measurable interventions
Quick Facts
1
Background
A government defence department wanted to improve personnel's cyber behaviours - particularly the loss of ID cards - through the delivery of behavioural interventions. A service design approach was needed to ensure interventions were scalable and could be deployed easily across different bases.
2
Skill Areas
-
User Research
-
Data analysis
-
Behavioural science (COM-B)
-
Ideation facilitation
-
Stakeholder management
-
Service Design/Transformation
-
Creative strategy
-
Project management
-
Supplier briefing and purchase order organising
3
Output
A fully designed and delivered Intervention package​ that saw a 66% reduction in the loss of personnel ID cards on the base chosen for intervention through:
​
-
Eye-catching signage
-
Inter-squadron competition to encourage positive behaviours
-
Gamified poster observation
-
Email communications
​
The Details
THE PROBLEM
A government defence department wanted to improve personnel's cyber behaviours with behavioural interventions that were trialled, measured and scalable for other bases to implement easily.
As part of a wider Cyber Resilience Programme, this government defence department wanted to deliver interventions that would tackle risky cyber behaviours being displayed by personnel. Their objective was 'to demonstrably reduce cyber security risk across the department by establishing new, and reinforcing existing, positive behaviours through targeted and measurable interventions.'
One of the risky cyber behaviours displayed by personnel was their 'failure to adequately protect access credentials from being lost - particularly ID cards' which was a big problem because this was the most frequent reported asset loss type. I led the design and delivery of an intervention package that would go onto evidence an improvement in this behaviour while using a service design approach to make this intervention scalable for other bases to run.
HIGH LEVEL TIMELINE
1 year+ project, 2023
MAKE OF THE TEAM
-
Engagement Lead
-
Project managers
-
Design delivery leads​
-
Delivery associates
KEY GOAL
Design and deliver scalable behavioural interventions that would tackle risky cyber behaviours.
MY ROLE
I was the Design delivery lead for the asset loss intervention in charge of understanding current behaviours around this risk, evaluating the most effective intervention package, driving the design of creative assets that would aid this intervention and running it on a base so the results could be measured.
My role in this project was end to end. I worked with client stakeholders to understand the risky behaviour that was being observed by personnel and I analysed incident reporting data to verify their assumptions.
The data showed that physical ID card loss was the most reported asset loss and therefore would benefit from a behavioural intervention. I worked to deliver this messaging to client stakeholders while quantifying the risk of this behaviour as the client initially didn't believe that this was as risky as other cyber behaviours. Following their buy-in I began to use the behavioural intervention methodology (by Pavitt, 2020) to develop and evaluate a series of intervention concepts that could be trialled on the base that needed the intervention most.
​​
Client stakeholders were taken along the journey on intervention concepts and how it would be implemented on base as senior endorsement was crucial in delivering this and really changing personnel behaviour. As assets were delivered to aid this intervention package, I also worked with graphic designers and product suppliers to brief them on messaging and relevant branding.
​
When the intervention package was ready, I also led the set up and running of this intervention on base over the course of 6 months. This was the length of the intervention trial so that enough evidence could be gathered on the effectiveness of the intervention. Once the intervention had been delivered I concluded this project with a detailed report on the approach and results of this intervention, a step-by-step playbook on how this intervention could be ran on other bases and a presentation to the programme on lessons learned from this intervention.
​
UNDERSTANDING THE TARGET AUDIENCE
In order to change behaviour, it was important to understand the current risky behaviour being displayed by personnel and their motivations for behaviour change.
Once the incident reporting data had confirmed a quantitative view of the asset loss risky behaviour and the base that was struggling with this the most, the next thing to do was to understand the behaviours and motivations of this group further. I ran a round of qualitative interviews with personnel that had consented to speaking more about this behaviour and what was causing them to lose their ID cards. There is a strong discipline culture in the military so in order to encourage people to put themselves forward for these discussions it was important to provide reassurance that this was not a disciplinary activity, it was purely to help improve this behaviour moving forward.
As expected with the culture of the department, uptake of interviews were minimal so this activity was supported by qualitative surveys that were sent out and included a range of closed and open-ended questions. The surveys were anonymised and had a much larger uptake within base. These results were analysed as it provided context for why personnel were losing their ID cards, we learnt that many of them were storing their ID cards in their wallets because they were not provided lanyards when they received their card. Due to this, if their wallet was lost or stolen their ID card, along with their other belongings, would be lost. This was an insight that was interesting to us because the policy for this department had stated that ID cards should be visible on personnel while on base at all times so ID cards being stored in wallets was not ideal.
​
Ethnographic research was also done on the chosen base for this intervention and this further supported the insight on the number of personnel that didn't have their ID card visible while on base.
Learning about the culture and motivations of personnel helped deliver targeted interventions
Personnel were highly motivated by competition amongst peers and senior recognition. Discovering this was important because motivation is a driver of behaviour change. It was therefore key to including these motivations in the Intervention package that was delivered on base.
BREAKING DOWN THE PROCESS
The COM-B model of behaviour change was used to design and develop an intervention package that would influence the behaviour of personnel positively so that the risk of losing their defence ID cards was reduced.
When I joined the project, the team had delivered a prioritised list of risky cyber behaviours that were being observed by the defence department. This deliverable was a strong piece of work that mapped out a list of 15 risky cyber behaviours that should be targeted with interventions throughout the project's timeline.
​
The first thing I did was understand the programme's vision on the positive cyber behaviours they wanted to see across the department. I then interrogated cyber incident reporting data to understand what risky cyber behaviours were being displayed frequently. From the data I could see that the loss of ID cards was a large portion of incidents reported with this accounting for 69% of reports. In order to suggest to client stakeholders that this was a behaviour that required an intervention, I needed to quantity the effect of these losses. I did this through a series of stakeholder conversations to understand the process of replacing a lost card, focusing on the time and financial costs of this process. All of this data was consolidated to create a strong case for why this asset loss type required an intervention.
​
Following the buy-in from stakeholders on delivering an intervention that tackled this problem, I needed to understand what was driving this risky behaviour. Qualitative research was carried out through interviews, surveys and ethnography which provided context to the incident reporting data that had been previously analysed. This context from personnel was brought to my team and stakeholders and this informed an ideation session that I facilitated to encourage blue-sky thinking on potential interventions.
​
The COM-B model states that in order for a behaviour to take place people need to have the capacity, opportunity and motivation to demonstrate said behaviour. I used insights gathered from research to pull out personnel's capacity, opportunity and motivations currently when it comes to ID card loss and how this would need to change in order for us to see an improved behaviour. This fed into the ideation session along with the NUF (new, useful and feasible) framework to prioritise intervention concepts.
​
The intervention concepts were mapping out, evaluated and packaged up so that there were multiple touch points for personnel to interact with our intervention.
The final package included:
-
Informative signage in high traffic areas
-
Email communications that would be distributed by Station executives
-
An inter-squadron competition that rewarded visible ID cards as lanyards were provided as part of the intervention
-
A gamified poster observation that took the form of a scavenger hunt and rewarded personnel that read through an intranet article.
​
This intervention package was presented to client stakeholders and following their sign off I led the design of the content and assets that would form this intervention. This included working with product suppliers and graphic designers to deliver the right branding, messaging look and feel so that we could accumulate as much traction to the campaign as possible on base and see a changed behaviour.
​
I worked with stakeholders on the chosen base to set up and run the intervention in-person and virtually. A baseline was captured for the behaviour before the intervention and post-intervention launch this was compared to the incident reports we were seeing. The intervention trial ran for 6 months and the results of the intervention was captured along with guidance on how this intervention could be ran at other bases.
PROVIDING A SPACE TO IDEATE
THE RESULTS
The intervention that I designed and delivered saw a 66% reduction in reports of ID card loss during the trial period compared to the number of reports pre-interventions. This was a huge success because incident reporting data for the two years prior saw barely any improvement.
Ahead of the intervention running, it was agreed that four metrics would be used to measure the effectiveness of the intervention.
​
-
Incident Reporting data - this metric would show how many reports were being made for ID card losses. These reports were received on a monthly basis and had high data quality because all personnel need to report their ID card as lost before they can progress with the replacement process.
-
A Self Reporting survey - this metric would gauge whether personnel noticed the interventions taking place as well as their general thoughts and feelings on the activities.
-
Engagement metrics - The intranet article that was part of the gamified poster concept provided engagement metrics. It showed how many views the article had received along with likes and dislikes. By the end of the trial there were 40,000+ views on the article.
-
Observation of positive behaviour on base - It was clear from initial analysis that personnel didn't tend to have their ID cards visible on base. This was a behaviour we wanted to change as having an ID card visible aligns with policy so we encouraged inter-squadron competition by giving out different colour lanyards to each squadron and tracking which colours we had seen the most in-person.
​
Overall we saw an improvement in behaviour during the trial period. A key requirement for our intervention package was to ensure it provided positive reinforcement so personnel didn't feel like they were receiving disciplinary action. Understanding the culture of personnel on base and leveraging this led to positive engagement with the interventions.
SOME OF THE KEY POINTS THAT LED TO THIS PROJECT'S SUCCESS:
Effective Storytelling
The client stakeholders had a preconceived idea on which behaviours required interventions and the priority. Data showed that an intervention on ID card loss was necessary. To deliver this messaging and get client sign off on this approach, effective storytelling was needed on the value of going with this approach.
Regular Stakeholder Touch points
Touch points were needed to bring stakeholders in on the process. These intervention activities cost money so keeping them close to what is being spent where and why meant that they had an easier time communicating that up.
Effective Stakeholder Management
There were a number of stakeholders that were involved in delivering this intervention from programme stakeholders to external suppliers. Keeping a hand on all stakeholders and the communications with each was a crucial part of this project's success.
Adaptability
The initial external supplier that was briefed on the intended messaging, look and feel of some of the intervention assets was unable to meet requirements. Adaptability was necessary to onboard an alternative supplier that delivered assets that the client was happy with.
Leveraging Behavioural Science
Keeping behavioural science methodology at the forefront of intervention design was crucial. It's easy to assume what would make you change a behaviour but this doesn't apply to everyone. Ennsuring that the capacity, opportunity and motivation (COM-B) is there for people will increase the likelihood of changed behaviour.
TITLE OF THE CALLOUT BLOCK
CHALLENGES & LESSONS LEARNED
My role in this project was end to end. While there were some challenges, being able to see the results of the work I led was rewarding.
Being on a year long project engagement comes with a variety of challenges as well as valuable lessons. Some to call out in particular are:
​
-
Managing stakeholder expectations: Stakeholders often have varying expectations on the project pace, outcomes and impact. Keeping them updated and aligned on progress required constant communication and diplomacy, especially when delays or changes occur.
-
Risk of scope creep: This work encouraged creativity to get attention from personnel on base. This excited our stakeholders and without strong boundaries and regular check ins, there was a risk of scope creep that could've affected initial objectives, timelines, budgets and stress levels.
-
The value of incremental progress and milestones: Breaking the project down into clear, management milestones helped with keeping the team and I focused and motivated. These milestones provided an opportunity to celebrate progress and ensured that any deviations from the original plan were addressed early.
FEEDBACK FROM TEAM MEMBERS THAT WORKED WITH ME
Leonie Lewis, Engagement Lead
Lucia built very strong relationships with stakeholders who were extremely happy with the results of the trial and provided very positive feedback about her involvement and the trial as a whole.
Paul Bailey, Project Manager
Lucia's expertise in design has been particularly impressive and has proved to be highly valuable for her role. Her ability to showcase these skills through mock ups made work move quicker and produced to a higher standard.
Hannah Jenkins, Client
She ensured continual engagement with the trial site, which set the foundations for lasting change and gave the site the motivation to continue the intervention package with little assistance following the trial.