Author: admin

Last month, we hosted a panel event on the use of AI in Financial Services in conjunction with UK Fintech Week. Partner and leading AI expert Emma Wright led the panel in a varied and interesting discussion about the current state of play as well as outlooks on the future of AI in the financial services industry. Here we touch on four key themes that came out of the discussion in a bit more detail.

1. High Risk AI Systems

The AI Act takes a broadly risk-based approach to the monitoring and governance of AI systems, with those deemed to be the highest risk banned completely, and those deemed to pose the least risk having the fewest additional compliance requirements.

Fintechs and other financial institutions should start reviewing and classifying the AI systems that they use ahead of the AI Act coming into force, as those categorised as “high risk” attract the most significant compliance burden. Whilst a lot of these requirements will not be unfamiliar to financial institutions used to dealing with a heavy regulatory load, there are requirements unique to the AI Act, such as informing the provider or distributor of risks and incidents, and keeping and monitoring automatically generated logs. Furthermore, for those companies operating within the financial services sector but falling outside of the scope of a lot of the existing regulation and oversight, a lot of this will be new.

Those fintechs and other financial institutions operating in or providing services relating to assessing creditworthiness as well as pricing and risk assessments in life and health insurance are most likely to be utilising high risk AI systems. In addition, all companies should be mindful of deploying AI systems intended to be used in the recruitment or candidate selection process, including to place targeted job adverts, to evaluate candidates, and to analyse and filter job applications, which may also be considered high risk.

2. Garbage in, garbage out

Whilst the AI Act has a focus on the use case(s) of the relevant AI system when classifying risk, another point that was raised by all our panellists is that the use and the output of any AI system is only as good as the input to that AI system, or in other words the data that has been used for its training and validation. An AI system can have an incredibly sophisticated algorithm, but if the input data is poor quality then the output will likely be poor quality as well.

Data can be poor quality in a number of different ways, from inaccurate and inconsistent, to outdated, irrelevant for the particular use, and incomplete. All of these factors can significantly impact the output of AI systems in different ways leading to incorrect results, bias and discrimination. If the AI system is further trained on the results it produces then this will serve to compound the problem.

This is obviously not unique to the financial service sector, but due to the existing significant regulatory framework applicable to the sector, the size and importance of the sector, and the potential consequences of poor data input and therefore output, it is particularly prescient.

3. Consumers

A key feature of a lot of the financial services industry is the vast number of consumers accessing and using key services every day and consequently the volume of consumer data that is stored and generated. For regulated firms, the FCA’s Consumer Duty is relevant when it comes to deploying AI systems. For example, the duty requires firms to take into account the different needs of their customer base, including vulnerable customers as well as those with protected characteristics. “One size fits all” AI systems will not necessarily allow for compliance with this, so firms will need to think carefully about how their targeted use of AI can help them to achieve this nuance, and this also links to the issue of data quality discussed above.

When it comes to consumer personal data, firms are by now used to the requirements of data protection legislation. Fintechs and other financial institutions will need to think about how these requirements manifest when deploying AI systems – in particular, transparency obligations to consumers in terms of how their personal data is being used and for what purpose, allowing consumers to exercise their data subject rights, ensuring that impact assessments are completed where appropriate, and aligning the use of AI with the rules relating to decisions made by so-called “black box” algorithms.

4. Operational Resilience

As AI systems develop and mature, the likelihood is that such systems will become more embedded within the functions of the financial services sector, which will in turn become more reliant on such systems and their creators and distributors.

The general trend with other IT products, services and systems, such as cloud, has been a movement away from developing in-house capabilities towards outsourcing to third parties, a trend which we are likely to see replicated with AI. In 2021, the Financial Policy Committee concluded that “the increasing reliance on a small number of [cloud and other critical providers] for vital services could increase financial stability risks in the absence of greater direct regulatory oversight of the resilience of the services they provide”, a statement which could equally be made in respect of the use of AI.

With outsourcing in financial services expected to increase, and a concern from regulators that such outsourcing could be concentrated within the hands of a few big players in the market, a focus on operational resilience not only in the financial services sector but also the outsourced providers is crucial. This is already playing out in a number of new regulatory proposals and legislation coming into effect, including the Digital Operational Resilience Act as well as the Critical Third Party monitoring regime in the UK. Fintechs and financial institutions deploying and using AI systems as well as those entities developing and distributing AI systems will need to be cognisant of these requirements.

In order to receive updates from Harbottle & Lewis, and to receive invites for our remaining AI event series, please sign up here.

More than 22 million viewers have watched dark comedy-drama Baby Reindeer since it launched on Netflix on 11 April 2024. Much of the show’s tension, and no doubt success, derives from the fact that it is based on a true story. The show follows Donny, a stand-up comedian, who is stalked by an obsessive middle-aged woman called Martha, and subjected to severe sexual abuse by his mentor, a successful older writer called Darrien. The show has been written by Richard Gadd, who plays Donny, and dramatises the real-life trauma suffered by Gadd at the hands of his stalker and abuser.

Gadd has been open in interviews about the fact that the events depicted are based on his own experiences. Details have been changed in the show to protect the identities of his stalker and abuser but the weeks since its release have seen frenzied speculation online about who these individuals could be. The speculation became so bad that last week Gadd took to Instagram to try to calm things down, saying that people he loves and admires were “unfairly getting caught up in speculation. Please don’t speculate on who any of the real-life people could be. That’s not the point of our show.” Despite this, the woman who allegedly stalked Gadd has now identified herself, challenging the show’s depiction of her and claiming that she is considering legal action.

Jigsaw identification

Even where names are changed to avoid identifying real-life individuals, there is a risk that different, accessible pieces of information can be pieced together to reveal a person’s identity. This process is known as jigsaw identification and is defined as “the ability to identify someone by using two or more different pieces of information from two or more sources – especially when the person’s identity is meant to be secret for legal reasons.” In the case of Baby Reindeer, for example, viewers have been trying to piece together the project Gadd was working on at the time of the alleged assault by his mentor to work out the alleged abuser’s identity. This has led to several names in the screen-writing world being circulated online with potentially damaging implications. In the case of the woman on whom Martha is based, it seems she has been identified partly from the use of real emails and social media posts sent by her to Gadd and referred to in the show.

Defamation and privacy

For any production based on real-life events there will be a risk that the depiction of the individuals may be defamatory if they can be identified. A person may have a claim in defamation if they can show that they were identifiable as the person in question, and that serious harm has been caused to their reputation as a result. Of course, the programme-makers may have a number of defences available to any defamation claim, including truth, honest opinion and/or public interest.

Similarly, a person may have a claim in privacy if they can show they were identifiable and that the portrayal of them breached their right to privacy. In such a case the programme-makers would need to show that the right to freedom of expression and/or public interest outweighed the individual’s right to privacy. In relation to both defamation and privacy issues, whether any claim is likely to succeed will turn on the facts of the case.

Mitigating the risk

There are several ways to mitigate the risk of jigsaw identification where there are concerns around defamation and/or privacy issues. In addition to changing the person’s name, it may also be worth changing their age, the chronology of events, the location where the events took place and the date the events are set. Consideration should be given to how an individual might be identifiable from other information in the public domain and ultimately a balance must be struck between telling an effective story and mitigating against the risk of a potential defamation claim.

In light of two recent ICO enforcement cases, Anita Bapat, Nadia Ahmed, Grace Tang and Chenelle Olaiya explain the host of legal factors to take into account when tracking employees.

The recent enforcement action in February 2024 given to Serco highlights the dangers of implementing employee monitoring unlawfully. In Serco’s case, the use of facial recognition technology and fingerprint scanning for monitoring attendance of more than 2,000 employees was found to be done in breach of data protection laws (notably as less intrusive tools could have been used for the same purpose).

When an employer carries out any form of monitoring, they will most certainly be processing employee personal data and monitoring must be compliant with data protection laws. This will be the UK General Data Protection Regulation (GDPR) and the Data Protection Act 2018. Businesses must have a clearly defined purpose for monitoring employees and rely on a lawful basis such as contract performance, legal obligations or legitimate business interests. The monitoring of staff may inadvertently collect special category data (eg, biometric data from attendance fingerprint scans), which are subject to extra protection under the GDPR that requires additional justification. Consent is unlikely to be useful here because of the power imbalance in the employer-employee relationship and employers will need to undertake additional compliance steps.

The monitoring must also comply with the data protection principles, which include:

• To demonstrate accountability, a data protection impact assessment should be undertaken to mitigate the risks involved given the monitoring is likely to result in a high risk to employees.
• To be transparent, employers must make sure employees are aware of the nature, extent and reasons for monitoring in a way that they would understand.
• To limit the use of the monitoring data to a particular purpose, that data should not be used for any other purposes; eg, data collected for attendance monitoring shouldn’t be used for diversity statistics.

The UK’s data protection regulator, the Information Commissioner’s Office (ICO), has produced guidance on employee monitoring, with and some key points for employers conducting monitoring including:

• Consult employees before implementing and use the least intrusive method.
• Working from home has a higher expectation of privacy.
• Automated decision making is a warning area, with human oversight especially important.
• Covert monitoring is only justifiable in exceptional circumstances.

Companies should also carry out due diligence of any third-party service provider that is used to conduct the employee monitoring on its behalf and assess their compliance with the UK GDPR. This includes making sure the contract with the service provider has the required data protection clauses such as confidentiality, mechanisms to deal with sub-processors and appropriate technical and organisational measures.

Organisations should be cautious when transferring employee personal data outside of the UK, which is restricted under the UK GDPR unless an employer safeguards the personal data with a mechanism such as an adequacy decision, the UK international data transfer agreement or standard contractual clauses.

Recent enforcement action makes clear that breaches of data protection laws for collecting and processing employee personal data can be just as severe as customer data. As well as the Serco decision, in 2022, the ICO fined Interserve £4.4m for failing to secure personal data leading to a cyber attack, which affected the personal data of up to 113,000 Interserve employees.

As technology permits greater employee monitoring tools to be used in the workplace, such as for better productivity, security and compliance, the use of such tools must be undertaken with care. Proper consideration of the privacy impact to employees and compliance with data protection laws should be considered at the outset. With proper planning and compliance steps, the benefits of such tools can be fully recognised.

During its latest plenary, the European Data Protection Board (EDPB) issued its highly anticipated opinion following a request by the Dutch, Norwegian & Hamburg Data Protection Authorities (DPAs). The opinion addresses the validity of consent to process personal data for the purposes of behavioural advertising in the context of ‘consent or pay’ models deployed by large online platforms.

By way of background, Meta has used “consent” under Article 6(1)(a) GDPR as the legal basis for processing users’ personal data, for example for advertising. Under the GDPR, consent would have to be specific, informed, unambiguous and freely given. Meta was concerned that giving users such a yes or no option would limit their options. As such, Meta argued that it was part of the user contract to show ads, using Article 6(1)(b) GDPR. This was ruled unlawful by the Court of Justice of the European Union (CJEU) and the EDPB in 2023. Meta then implemented  the ‘pay or consent’ model by asking users to consent or pay a fee to access its platforms.

The EDPB was asked to issue its opinion under Article 64(2) GDPR to address the validity of consent in the context of ‘consent or pay’ models also in view of the Court of Justice of the European Union’s Bundeskartellamt judgment (C-252/21). The DPAs originally referred the ‘pay or consent’ question to the EDPB in January 2024. The DPAs sought EDPB clarity on the matter after Meta first sought to implement the model for its Facebook and Instagram platforms after the body issued a binding ruling in November 2023 banning Meta’s targeted advertising practices across the EEA, which was preceded by a €390 million fine issued to Meta by Ireland’s DPC over its targeted advertising practices earlier last year.

In its press release on the opinion, the EDPB stated that, “In most cases, it will not be possible for (large online platforms) to comply with the requirements for valid consent, if they confront users only with a choice between consenting to processing of personal data for behavioural advertising purposes or paying a fee.” The EDPB opinion applies to companies designated as very large online platforms under the Digital Services Act. However, the EDPB said it would issue further guidance later this year on pay or consent for smaller platforms.

The decision is non-binding and either Ireland’s Data Protection Commission will apply the opinion or the DPAs that referred the matter to the EDPB will issue their final decision on the legality of ‘pay or consent’ models in the context of the GDPR as they further investigate Meta, which had first sought to implement such a model.

The EDPB’s opinion means that controllers relying on consent need to evaluate the imbalance of power between the individual and the controller and confirm if it is in fact freely given. The factors to be assessed include the position of the very large online platforms in the market, the extent to which the individual relies on the service and the main audience of the service.

If you would like to keep up to date on the latest in data protection, please get in touch to subscribe to our newsletter, The Data Download.

Link to the EDPB’s opinion: https://www.edpb.europa.eu/news/news/2024/edpb-consent-or-pay-models-should-offer-real-choice_en