What else do we need to consider?

In detail

• Do we have to do a data protection impact assessment (DPIA)?
• Do we need to make any other changes to our systems?

Do we have to do a data protection impact assessment (DPIA)?

A DPIA is a tool that helps you assess the risks your processing poses to people’s rights and freedoms, and identify ways to address those risks. 

The UK GDPR says you must do a DPIA if your processing involves:

“a systematic and extensive evaluation of personal aspects relating to natural persons which is based on automated processing, including profiling, and on which decisions are based that produce legal effects concerning the natural person or similarly significantly affect the natural person” 

This means you must do a DPIA if you carry out systematic and extensive ADM or other significant decisions based on profiling.  

You should do a DPIA for any ADM as this is very likely to be a type of processing that can result in high risk. Even though ADM is not “systematic and extensive” by default, it is highly likely that a lot of organisations use it in this way because of the efficiencies they expect from the scale of the processing. 

So, you must carry out a DPIA to identify what the risks of your context are and put in place appropriate measures to mitigate them. You must include  processes that ensure people receive clear, specific information about the processing and the rights available to them. 

A DPIA can help you decide whether or not the ADM provisions are likely to apply to your intended processing in the first place.

They are a good way to meet your accountability obligations by showing how you have: 

• considered the risks involved in any profiling or ADM; and
• put procedures in place to mitigate those risks and comply with the UK GDPR requirements. 

Do we need to make any other changes to our systems?

You should have mechanisms in place to diagnose any quality issues or errors and a process to document how you intend to resolve them. 

You should ensure that these mechanisms also allow you to check your systems are working as intended and highlight any inaccuracies or bias.

Recital 71 says you should: 

• use appropriate mathematical or statistical procedures for profiling; and
• implement technical and organisational measures appropriate to ensure, in particular, that factors which result in inaccuracies in personal information are corrected and the risk of errors is minimised.

To achieve this, think about how you can:

• introduce quality checks on the results from your systems by manually reviewing a sample of automated decisions, to identify any bias or discriminatory effects or both;
• take corrective action, such as adjusting thresholds, re-training or updating the model, or adding additional human oversight steps;
• delete any special category data that your system may receive or infer before profiling if you do not require it;
• identify appropriate retention policies for the information you use and keep these under review;
• implement suitable security measures such as access controls and encryption; and
• audit your machine-learning systems to check for decision-making rationale and consistency. 

It is important to note that monitoring aspects of a system’s performance, such as its statistical accuracy, is not human intervention. This is because there is not a significant decision made about a specific person.