What is ADM?

In detail

• What is automated decision-making about people?
• What is profiling?
• What is AI and how does it relate to ADM and profiling?
• What are the benefits of ADM?
• What are the risks?

What is automated decision-making about people?

Automated decision-making (ADM) is where you use personal information to make a significant decision about someone using solely automated processing, including profiling. 

ADM systems can range in sophistication. They do not need to involve complex algorithms, artificial intelligence (AI), or other types of advanced processing to potentially come into the scope of ADM.

The UK GDPR sets out specific requirements that apply to ADM. To understand if the ADM provisions apply to your processing, you should ask yourself whether:

• you are using a system that is making a decision (or decisions) about a person;
• the decision is a significant decision (meaning the decision has legal or similarly significant effects); and
• the decision is solely automated (meaning there is no meaningful human involvement). 

If the answer to all the above is ‘yes’, you must: 

• ensure your processing complies with the ADM provisions; and
• put the required safeguards in place. 

To determine whether the ADM provisions apply, you should consider these three aspects in tandem so you don’t mistakenly think the ADM provisions do not apply when they actually do. This is because if you misidentify what the actual decision is or what the significant effect was, it is easier to make a mistake about whether there was meaningful human involvement at the right point. 

What is profiling?

ADM often involves profiling. The ADM provisions in the UK GDPR specifically refer to profiling because it can be part of, or all of, the automated processing that you use to base decisions on. This means that if you carry out ADM, you must consider the extent to which this involves profiling.

Profiling is where you analyse, evaluate or predict aspects of someone’s personality, behaviour, characteristics, interests, or habits.

The UK GDPR says profiling is:

“any form of automated processing of personal data consisting of the use of personal data to evaluate certain personal aspects relating to a natural person, in particular to analyse or predict aspects concerning that natural person's performance at work, economic situation, health, personal preferences, interests, reliability, behaviour, location or movements.”

You might obtain personal information directly from people, or from a variety of different internal and external sources, such as:

• people’s searches on your app or website;
• people’s online and offline buying habits; and
• people’s physical location or movements (eg from their mobile devices, if your app collects this information). 

You might analyse this information to divide people into different groups, segments or categories. This analysis often identifies correlations between different behaviours and characteristics to create profiles about people that can relate to specific affinity groups. This profiling activity can create new personal information as a result of the analysis.

You might use profiling to: 

• analyse people’s preferences;
• make predictions about their behaviour;
• make decisions about them.

Profiling can use algorithmic systems that find correlations between different features or attributes. An algorithm is a sequence of instructions or set of rules designed to complete a task or solve a problem. It’s possible to use profiling algorithms to make a wide range of predictions. For example, whether someone is likely to buy an item based on their past behaviour, or to control access to a service. Profiling algorithms increasingly involve AI systems, and in particular machine learning. 

Examples of profiling include (but are not limited to):

• analysing personal information, in particular on a large scale (eg through algorithmic systems, AI or machine-learning);
• identifying associations to build links between different behaviours and attributes;
• creating profiles that you apply to people; or
• predicting people’s behaviour based on their assigned profiles. 

Less obvious forms of profiling involve drawing inferences from apparently unrelated aspects of people’s behaviour.

What is AI and how does it relate to ADM and profiling?

AI is a term that relates to a variety of machine learning (ML) techniques that learn from patterns in past data to perform a variety of tasks (eg predictions, data generation, classifications). A range of processing activities are involved in ML training to build models or their deployment (eg collecting data, training a model, creating data). 

Some ADM may involve ML. For example, producing a credit score may involve the use of ML techniques, such as decision-trees.

AI systems can play a wide variety of roles, from decision-support to triaging to classifying or retrieving information. This means they can be involved at different stages of your decision-making process and to different degrees. If you use AI, you must identify whether the ADM provisions apply.

However, not all AI-related processing necessarily constitutes ADM. For example, using a generative AI application to summarise a meeting is unlikely to lead to a legal, or similarly significant effect, in most cases, so this isn’t in scope of the ADM provisions. 

You can also use AI and ML for profiling because they can analyse large amounts of data and predict people’s behaviour or interests. Content recommendations used widely on social media platforms tend to be built on AI or ML systems. 

What are the benefits of ADM?

ADM can be very useful for both organisations and people in many sectors, including healthcare, education, financial services and marketing. It can lead to quicker decisions, particularly when you need to analyse a large volume of information.

It can also lead to more consistent or standardised decisions by ensuring that they are made based on criteria that you can audit and improve. This reduces the variability that may be exercised by human decision-makers. 

One of the key benefits of ADM for organisations is that it can help scale the business. Start-ups or small-to-medium-sized enterprises looking to expand their reach can more easily manage larger operations by using automation, as it reduces the barriers to entry. This can lead to better and more competitive offerings for people.

You can also use ADM to provide more personalised, tailored services that people engage with better.

You can use it to improve internal operations as well, such as risk management and compliance programmes. In a financial services context, it could mean automatically flagging and reporting transactions or data to relevant authorities, where required.

The UK GDPR recognises the potential for innovation by enabling you to carry out ADM in a wide range of circumstances, provided you implement safeguards.  This allows you to adopt automated processes with greater confidence and efficiency, and encourages responsible innovation. For example, you can scale automation faster to deliver more timely, accurate and personalised outcomes for people.

What are the risks?

While ADM can offer substantial benefits, it also presents a range of risks. These can be both process and outcome related. We set some of these out here, although this is not exhaustive. You should consider these and any other relevant risks when thinking about whether your use of ADM is fair and lawful. Where your use of ADM is likely to result in a high risk to people, you must carry out a data protection impact assessment (DPIA). You must include details in your DPIA of the mitigations you intend to put in place to manage these risks.

Lack of understanding or awareness of the processing 

The technical processes involved in your ADM can be complex, especially if you use advanced techniques like ML. This can make it challenging for people to understand how you make decisions that affect them. It can also be challenging for you to explain or justify those decisions.

Profiling is often invisible to people and, where ADM is based on profiling, they may not expect you to use their personal information in this way. They might not understand how it works or how it can affect them. 

Discrimination or unfair outcomes

One key risk to mitigate is the potential for bias and discrimination. Algorithmic systems may reflect historical inequalities or societal biases, depending on the training data. Using them to carry out ADM may replicate or even amplify these biases. This can result in unfair outcomes for people. 

ADM also carries risks of over-generalisation and stereotyping when you base your decisions on profiling, involving things like broad classifications or historical data. This is because individual people may be unfairly grouped or judged based on characteristics that do not accurately reflect their particular circumstances. 

There’s no guarantee that what may be representative of a large part of the population is automatically going to be the same at the individual level. And the ADM provisions relate to the decisions you take about individual people.

This means that when you assess things like risks of bias or discrimination, you should take a more people-focused approach that considers their circumstances. 

There are also risks about the statistical accuracy and reliability of automated outputs. Errors in automated processes, such as out-of-date data,  can scale quickly, affecting large numbers of people. Due to the predictive nature of profiling, there will always be a margin of error. Therefore, you should weigh up the risks of using the results.

Vulnerability to ADM 

People who are already in situations where they are at risk may face heightened risks in the context of ADM. People in financially precarious situations, those with certain disabilities, or children, may be less able to understand or challenge decisions and be more susceptible to harm if outcomes are inaccurate or biased. 

The UK GDPR specifically highlights that children deserve additional protection, particularly when their personal information is used for marketing or creating online profiles. The UK GDPR says:

“Children merit specific protection with regard to their personal data, as they may be less aware of the risks, consequences and safeguards concerned and their rights in relation to the processing of personal data. Such specific protection should, in particular, apply to the use of personal data of children for the purposes of marketing or creating personality or user profiles….”

If you wish to carry out ADM using children’s personal information, with the intention of influencing their choices or behaviour, you must: 

• consider what impact those choices or behaviours may have upon the child; and
• decide whether this amounts to a significant effect. 

A processing activity can have a significant effect on children and not adults. For example, behavioural advertising might raise particular concerns where children are involved. This is because they are more susceptible to influence and may not fully understand the commercial motives behind such use of their information. You should also consider related risks, such as the potential for behavioural profiling to create excessive nudging or addictive patterns, which can negatively impact mental health and overall well-being.

Considering the fact that children merit specific protection is also one of our secondary duties. We will be taking children’s interests further into account as part of developing our code of practice on AI and ADM and our broader strategic priorities.