5 of 6

18 September 2023

Financial data – 5 of 6 Insights

Transforming how data is collected from the UK financial sector

Daniel Hirschfield looks at the joint data transformation programme, which is being led by the UK's financial regulators to transform how data is collected from the UK financial sector.


Daniel Hirschfield

Senior Counsel – Knowledge

Read More

It is often said that the Bank of England (Bank), the Prudential Regulation Authority (PRA), and the Financial Conduct Authority (FCA), are as much data regulators as they are financial regulators.  In order to properly discharge their statutory objectives, supervise firms adequately and make appropriate market interventions, each regulator is reliant on collecting significant amounts of data from firms and the markets in which they operate.  Data will also often inform how policy judgements are made and the final form of regulatory change projects.

The FCA's work with firms on digital regulatory reporting (initiated in 2018) and the Bank's data collection review (carried out in 2020) demonstrated that data collection (that is the process of collecting data from firms) gives rise to many challenges for the regulators and for the many thousands of firms that are regulated.  The regulators acknowledge that one of the most difficult areas of regulation is reporting and recognise that the data they receive is not always of the highest quality.  At the same time, as technology and business models have evolved, the data needs of the regulators have changed.

Joint transformation programme

In July 2021, the Bank and the FCA launched a joint transformation programme (Programme) together with industry to address these issues and deliver improvements to data collection over a period of 10 years.

The Programme's objective is to ensure that "regulators get the data they need to fulfil their mission, at the lowest possible cost to industry."  It identified three core measures as being essential to delivering this vision:

  • Data standards: common data standards should be defined and adopted that identify and describe data consistently throughout the financial sector. These common standards should be open and accessible for use by all who require them.
  • Reporting instructions: reporting instructions should be modernised by improving how they are written, interpreted and implemented. There are many ways this can be achieved including establishing better Q&A processes or potentially rewriting instructions as code.
  • Reporting: reporting should be integrated so that a more streamlined and efficient approach to data collection is developed. This includes making data collection more consistent across domains, sectors and jurisdictions, and designing each step in the data collection process with the end-to-end process in mind.


The Programme is modelled on the Government Digital Service's industry-standard Service Design Approach and is divided into phases with each phase consisting of a number of use cases.  A use case is defined as a "a collection, set of collections or aspect of a data collection."  The use cases then enter a 'discovery and design stage', in which issues and design solutions are explored.  Finally, they pass through an 'implementation stage' (also referred to as Beta stage). 


To ensure effective collaboration with industry, the Bank and the FCA have created a governance and delivery framework for the Programme. There are two delivery groups: a Core Delivery Team and the Advisory Group.  The Programme is overseen by the Reporting Transformation Committee and the Data Standards Committee (DSC), each of which meets monthly.

Reporting and Data Standards Transformation Board

The regulators have also established a Reporting and Data Standards Transformation Board. The Board meets every four months and is intended to serve as a forum for discussing issues of common concern regarding reporting and the development of data standards with the aim of ensuring better reporting.  The Board is empowered to look at issues and initiatives which go outside the remit of the Programme.

Phase One

Phase One began in July 2021 with three use cases in scope:

  • Quarterly statistical derivatives return (Form DQ): the aim of this use case is to improve the design of a return submitted by approximately 20 firms with liabilities over £10 billion that summarises marked-to-market valuations of derivative positions at the end of each quarter. This data is one of a number of sources used by the ONS to determine national balance of payments statistics.
  • Commercial real estate database: the focus here is to enhance both the quality and coverage of commercial real estate data available to the PRA and the Bank’s directorate for Financial Stability Strategy & Risk.
  • Financial resilience survey: during the pandemic the FCA introduced a quarterly financial resilience survey of FCA authorised firms. This use case will look at how to improve data quality, improve transparency about how the data is analysed, and look to transitioning the survey to a regulatory data capture that can be collected using the FCA's data collection platform, RegData.

Phase One findings and recommendations

The Phase One discovery and design stage finished at the end of March 2022.  Because of a lack of resource, it was decided that work on the commercial real estate database would be folded into Phase Two.  The delivery teams identified a series of themes, which cut across the use cases and which appeared to have similar root causes.  These include:

  • Users experienced challenges accessing the right information, including the context and rationale for the collection of the data.
  • Firms faced difficulties in understanding and interpreting requirements.
  • Firms and regulators had issues in providing and receiving feedback on data quality.
  • Firms did not properly understand the impact that the data has on the regulators (for instance, they did not appreciate how data is used by regulated to improve decision-making).
  • Concerns that there were multiple collections of identical or similar data.

The combined impact of these issues for the regulators and firms that are regulated is significant.  It means that time is wasted, the quality of data is poorer and the data is less useful for regulators.  It is also a source frustration and stress for those collecting and using the data.

In July 2022, the Bank published the Phase One recommendations of the Programme together with the regulators' response.

The Programme made seven recommendations, which the Bank and the FCA agreed to accept in principle, and are structured as "short term", "medium term", and "long term"  relating to the Form DQ and the Financial resilience survey use cases.  A summary of the recommendations and the regulators' responses can be found here

The initial implementation stage began in September 2022 and ran to approximately Q2 2023.  The Bank has made significant changes to the design of the landing page for Form DQ as part of a redesign of the landing page for statistical reporting; further work is required to streamline reporting instructions.  For the Financial resilience survey use case, the FCA is introducing a new financial resilience report for firms, which is due to go live in Q1 2024.

Phase Two use cases

There are four use cases in Phase Two:

  • Incident, Outsourcing and Third Party reporting: maintaining the operational resilience of the financial sector is critical for the Bank, the PRA and the FCA. However, this is an area where the regulators do not have the quality of data that they require. The use case will explore the design of new reporting policies relating to incident, outsourcing and third party reporting and consider how best to deliver 'event' driven collections i.e. the occurrence of a given event triggers a new report.
  • Commercial Real Estate database: the work undertaken during Phase One showed that the current data that the Bank is receiving is not fit for purpose. Completing the design of an integrated commercial real estate collection is considered critical for delivering other use cases in the future such as a consolidated loan collection.
  • Review Prudential Data Collection: prudential data, which comprises high level capital, liquidity, balance sheet and profitability calculations, remains some of the most important data the regulators receive. However, it is particularly costly for firms to produce these metrics. The aim of this use case is therefore to improve the collection of prudential data looking first at the prudential data collections from FCA authorised firms, which will then guide future work for PRA authorised firms.
  • Retail Banking Business Model Data: the FCA collects detailed product-level financial data in relation to a wide range of retail banking products and segments. This data is needed to support the FCA's competition objective and is made available to a number of other stakeholders across the FCA. Currently the data is collected on only an ad-hoc basis and has not been designed to meet the requirements of the various data users. The FCA intends to transition this collection to a regular collection and transform it into an integrated collection that reduces the burden on firms. It is hoped that as a result of this integration exercise, other business model data can be standardised.

Phase Two update

The Bank and the FCA provided updates on Phase Two in March 2023 and in July 2023. They expect to release use case recommendations and the regulators' response later this year.

Data Standards Review

During the Phase One, Ernst and Young (EY) were commissioned by the DSC to identify what conditions were required to achieve common data standards. The review involved a public consultation, interviews with stakeholders and research and analysis.  The DSC reviewed EY's final report, which was published in April 2023, and issued five recommendations for the Bank and the FCA.  The Bank and the FCA, in turn, published their response in August 2023, endorsing the majority of the recommendations.  In summary, they have agreed that:

  • DSC: the DSC should be renamed the Industry Data Standards Committee (IDSC).They will work with the current DSC and the IDSC to agree updated Terms of Reference, which are expected to apply from Q1 2024.
  • Taxonomy: a common taxonomy of financial data should be created for regulatory and other data collections by the Bank, PRA and FCA.
  • Metrics: they will work with the IDSC as part of the next phase of the Programme, to advise on developing a framework to measure the costs and benefits in connection with developing and adopting data standards and to measure the level of adoption.
  • Roadmap: the regulators will publish a roadmap for the simplification and standardisation of regulatory and other data collection by the Bank, PRA and FCA. As part of this, the regulators will create a library of UK reporting elements. This will assist in identifying anomalies in standards adoption.

Prior to making any commitments on the timeframe for implementing these regulatory action points, the Bank and the FCA have said that they will need to consider how this workstream will be funded and resourced.  An update will be given in the next phase of the Programme in Q1 2024.

Back to

Global Data Hub

Go to Global Data Hub main hub