Our Rule Review Framework

Our Rule Review Framework is one part of our policy-making cycle. This cycle starts with us horizon-scanning and identifying instances of harm in financial services markets. For example, where markets are working poorly and not providing sufficient benefit to users. We gather information from a range of sources to identify this actual or potential harm. These include day-to-day supervisory contact with firms, calls to us from consumers, analysing market intelligence and undertaking research, as well as our ongoing engagement with stakeholders. We do not act to remedy every potential harm. We consider the potential costs and benefits involved and our objectives.

Once we have identified actual or potential harm, we aim to diagnose its cause, extent and potential development. To do this, we must decide whether we have enough information to assess the issue or if we need to carry out further work to better understand it. We have a range of diagnostic tools to support this including market studies, calls for inputs and discussion papers. We can also analyse individual firms or several firms simultaneously, for example through our multi-firm work and thematic reviews. We aim to do this in a way that is cost-effective for us and for firms.

When we understand the potential harm, we then consider how best to mitigate or avoid that risk of harm. To do this we consider our regulatory tools and make a judgement about which could be used to remedy the harm. Rulemaking is just one of our possible tools. Other remedies include publishing guidance or other communications to firms or their customers or encouraging industry to act voluntarily to address a problem. We can vary or remove firms’ permissions to carry out certain activities. We can use our authorisations approach to control firms’ and individuals’ entry into the market. We also have our enforcement remedies where we have found breaches of our rules.

Reviewing how effective our intervention has been

Reviewing the effectiveness of our chosen remedies helps us to make better decisions, and to add more public value, in the future. There is always the risk that regulatory interventions, including rulemaking, have unintended effects or don’t work as well as we expected. External factors and changes in the market may affect how well a remedy, such as a rule, is working. Understanding whether our remedies are working as we intended and measuring their impact where possible, helps us understand if our intervention has been proportionate to the outcomes achieved. If we find that the problems originally identified in a market are still occurring and our remedies have not had the intended effect, or had an unintended effect, we consider our next steps and whether to take further action.

Reviewing our rules is also an important vehicle for delivering our objectives. This Framework advances our strategic objective to make markets function well by ensuring our rules are having their intended effect on the market. It also sets out how, during a review, we will consider whether specific Handbook rules are also advancing one or more of our operational objectives. By ensuring the rules in our Handbook are working as intended and undertaking reviews in response to evidence suggesting this may not be the case, our Framework also supports our secondary objective: to facilitate the international competitiveness of the UK economy and its growth in the medium to long term.

This Framework applies to all our rules, which are found in our Handbook. Where we monitor or review rules it will normally be helpful for us to consider the policy introduced by the ‘package’ of rules. So we may not monitor or review an individual rule in isolation, but instead consider groups of rules introduced through a policy intervention.

Our Handbook is detailed and many of our rules inter-relate. We need to keep this in mind to ensure that we do not review rules in isolation as it could cause problems if we decided, after review, that a rule is not working as intended and needs to be changed.

The duty in the Financial Services and Markets Act 2023 to keep our rules under review does not apply to any materials that are not rules. However, when we review a rule, we can choose to review our guidance and related materials to see if these are working well.

We can also use monitoring and the 3 types of review in this Framework to assess the effectiveness of other types of regulatory interventions that we make.

Some of our rules are in areas of responsibility shared with the Prudential Regulation Authority (PRA) and the Payment Systems Regulator (PSR). Our rules may also be determined or influenced by the same legislation or codes affecting other regulators, for example the Companies Act 2006 and the Corporate Governance Code. 

These regulators have their own review processes and may also be directed by the Treasury to review specific rules or, in the case of the PSR, directions. In developing this Framework, we engaged closely with these other regulators to ensure a consistent and complementary approach. While our respective frameworks are not identical, and allow for operational and strategic differences, we are closely aligned and do not envisage there being an issue with potential differences in respective approaches.

Where we propose to review a rule that is in an area shared with another regulator, or where we are directed by the Treasury to do so, we will work with that regulator to agree the best way to carry out the review. We have formal memoranda of understanding with the PRA and the PSR which set out the framework for how we co-ordinate and co-operate in these kinds of scenarios. 

We also work together with other regulators as members of the Financial Services Regulatory Initiatives Forum to produce the Regulatory Initiatives Grid. This sets out the regulatory pipeline, including initiatives that the regulators are working on together. This helps us to understand each other’s plans, including upcoming reviews and, where appropriate, to coordinate.

As a general principle, when we add new rules to our Handbook, we will collect data and monitor their effectiveness in a systematic way. There are some situations where we may choose not to actively monitor new rules and we explain this during that rule’s consultation process. We will generally monitor key metrics unless it is not feasible to do so, where collecting data and intelligence would be disproportionate for us or stakeholders or where it is not otherwise an effective use of our resources. For example, we may decide not to monitor where the new rule relates to a minor policy or rule change with minimal impact.

Where we do plan to monitor, we will usually set our intended outcomes when we develop the policy and rules. We may develop a causal chain setting out how we expect these outcomes will be achieved. We will then decide the key metrics we will monitor and the data required as a result. We know the potential burden that regulatory data requests can have, and we will endeavour to use existing data for our monitoring purposes as much as possible and take a proportionate approach where we do request data from firms. This Framework sits alongside our existing data governance policies and approach and is not intended to replace them.

For ultimate outcomes that take a long time to materialise, we may choose to monitor intermediate outcomes as leading indicators of progress towards our ultimate outcomes.

Where we identify potential unintended consequences of the rule, we may also plan to monitor these.

Once the rule is in force, we will start collecting and monitoring data on how well it is meeting, or is on track to meet, its intended outcomes. We will also consider relevant evidence from stakeholders. The appropriate timeframe for monitoring will depend on the specific circumstances for the rule.

For a minority of our rules, such as where we expect the impact to be particularly significant, we may plan at the consultation stage to conduct a specific review in the future. This is particularly likely when we believe that an impact evaluation may be appropriate. To ensure that we have the right data for this kind of review, it is best we plan for it when we are developing and consulting on the policy and rules. This Framework does not change our existing approach to maintaining a planned pipeline of annual impact evaluations.

If, while monitoring the data, including any stakeholder feedback, there is a suggestion of a potential problem with how the rule is working, we will consider whether to do an evidence assessment. An evidence assessment aims to collate any data which indicates whether the key intended outcomes of a rule or policy intervention are being, or are on course to be, met. Our policy teams use their professional judgement to decide whether to propose to complete an evidence assessment based on a number of criteria. These include the nature of the issue identified, our strategic priorities, our existing commitments, our strategic and operational objectives and our available resources.

The possible outcomes of the evidence assessment include starting the process to consider varying or revoking the rule, where the evidence suggests this is necessary and appropriate. Alternatively, we could choose to return to monitoring the rule, issue clarification or guidance on how it operates to improve stakeholders’ understanding or plan a more in-depth review.

Where our rules flow from international standards, we will not generally depart from these without a compelling justification in line with our objectives. We actively contribute to developing and implementing international standards in global standard setting bodies. These support the management of cross-border risks to financial stability, market integrity and general confidence in the global financial system.

They also support the development of common approaches with counterparts and form the basis of our UK approach where this is appropriate for UK markets and UK consumers. So we would need to carefully consider the effect that changing these rules may have if this were the outcome of a review.

The data we get from proactively monitoring our rules is one possible prompt for conducting a rule review. There may be existing rules in our Handbook that we have not been systematically monitoring data for. Additionally, we may have implemented new rules and chosen not to monitor them as we did not consider it feasible to do so. In these instances, where we have a rule without this type of monitoring data, we may still consider conducting a review where:

1. We have other evidence that the rule is not working as intended. Separate to the proactive data monitoring outlined above, we may have been collecting information involving a rule, for example through our supervisory work, which we can use to assess how the rule is working. In other cases, stakeholders might give us evidence about how the rule is working, which we will consider carefully alongside all other available evidence.
2. There have been substantive changes in circumstances, including market developments or the introduction of other rules, which affect how the rule is functioning and we want to understand the overall effect - positive or negative - that the rule is having on the market.

As with conducting a review based on monitoring data, we decide whether or not to do an evidence assessment based on a number of criteria, including our strategic priorities, existing commitments and resources. We will also consider how adequate the available data is and the likelihood of improving any evidence gaps in a proportionate way.

Once we have decided to do a review, the stages will broadly follow the same process for all rules regardless of when they entered the Handbook.

The Treasury can also require us to review a rule (see Government-directed reviews).

We set the outcomes we are seeking to achieve for new rules and policies so that we can assess whether they are working as intended once they have been implemented. We generally include these outcomes in our consultations and policy statements.

Our statutory objectives shape and guide our policy development and rule proposals. We generally explain the outcomes we want to achieve by referring to how they advance one or more of our statutory objectives.

In monitoring our rules, we need to differentiate between outputs and outcomes. Outputs stem from our activities. Outcomes are what is achieved because of the outputs we have delivered. The examples in Table 1 illustrate the difference between outputs and outcomes. 

Table 1: Examples of outputs and outcomes

More broadly, we monitor progress against our key areas of focus, as set out in our published strategy and annual work programme. We report on our performance against our annual work programme in our Annual Report.

In many cases, the outcomes and metrics for our new rules will align with our organisational strategic outcomes and metrics set out as part of our strategy. We may therefore also use those metrics to assess whether there may be an issue with our rules.

We may use a causal chain to explain how we believe an intervention, such as new rules, will work by setting out the steps between the outputs and the outcomes we want to achieve. We may set out a causal chain in the Cost Benefit Analysis (CBA), which is the document which assesses the costs and benefits of our proposals.  

Causal chains will inevitably include some assumptions, for example about changes in behaviour that may result from our intervention. However, they are a useful tool to help us understand how we will bring about the change we want to see.

We can monitor outcomes, including lead indicators or intermediate outcomes, to help us assess whether we made the correct assumptions in the causal chain. To do this, we identify a set of metrics for each outcome to help us establish whether the rule is working as intended.

We need to examine and frame these metrics carefully to avoid misinterpreting movements and to account for predictable variations that happen each calendar year (‘seasonality’). For some rules, we may find it useful to measure outputs as well as or instead of outcomes (see the difference between outputs and outcomes).

Evidence for our metrics can come from different sources of data. We collect a lot of data and have a variety of sources to help us understand how our rules are working, including our authorisation, supervision and enforcement work.

For proactive monitoring, where possible we will use data that we already collect or have access to. We know that collecting data from industry can be resource-intensive for firms to provide and for us to analyse. When we do request new data, we will follow our existing data governance procedures and be as proportionate as possible.   

The following are examples of possible sources of data for our metrics:

1. Stakeholder feedback gathered through, for example, roundtables, focus groups and other forums. We also use evidence proactively provided by stakeholders to us via our feedback tool about how the rules are working.
2. Our ongoing engagement with stakeholders’ representatives. For example through trade bodies, industry led bodies and industry associations, consumer organisations and civil society.
3. Research, such as surveys like the FCA’s Financial Lives Survey and Practitioner Panel Survey and insights from behavioural analysis, including into consumer experience.
4. Feedback from our statutory Panels and advisory committees.
5. Our supervisory work, including ongoing contact with firms and our multi-firm reviews.
6. Information provided by firms through regulatory returns and reporting.
7. Complaints data and data from the Financial Ombudsman Service and other regulators.
8. Information from our enforcement work and primary and secondary market oversight work.
9. Market intelligence, including from wider monitoring of the market, our market studies and our thematic reviews.
10. Parliamentary feedback including MPs’ letters, Parliamentary debates, and Select Committee inquiries.
11. Third party data, such as the media, market research firms and financial data providers.
12. How other bodies, such as authorities in other jurisdictions, have implemented similar measures, and reviews of these. 

These data sources are also relevant when we decide to do a review and want to explore how the rule is working in more detail. At that stage, we may also need to request further information from firms or to do further consumer research.

As part of the CBA, or earlier analysis before implementing rules, we typically establish a baseline showing the state of the market before our intervention. We also consider how the market would have changed over time without our intervention (the counterfactual). Ideally, we will collect data for the baseline for several time periods (months/years) before implementation to understand underlying trends or seasonality in the data. However, often this may be unfeasible or disproportionate.

Once the rule has been implemented, if we plan to monitor the policy, we will collect or monitor data to assess progress. Ideally, we will collect and monitor at regular intervals and compare this ongoing data against the baseline or counterfactual. The appropriate frequency will depend on the circumstances of the rule and availability of meaningful data. For example, we may collect some quarterly, some annually, and some at ad hoc or longer intervals. We will consider proportionality, weighing the value of the data against any cost for firms to provide the data and the costs of collection.

If during this monitoring there are signs that the rule is not working as we intended, we may decide to conduct a review of the rule (see Types of review and how and when we undertake them). 

While the data and information we monitor indicates that we are on track to achieve the intended outcomes of the intervention, we continue to undertake the monitoring for the pre-agreed length of time.

The duration and frequency of monitoring will depend on the relevant rule which we may have explained in the rule’s Consultation Paper or Policy Statement. Once we are content that a rule is working as intended, we can treat it as an existing rule and stop the proactive monitoring. Even where monitoring has stopped, where necessary we will still respond to any other evidence, including from our stakeholder feedback tool, that it may need to be reviewed.

We are committed to ensuring there are clear and appropriate channels for our stakeholders to share feedback on specific rules. So we have developed a feedback tool for stakeholders to give feedback on any rule in our Handbook. This form allows anyone with any evidence suggesting that our rules are not working as intended to share this with us.

It is important that stakeholders providing feedback set out, with supporting evidence:

• what is not working
• why they believe it is not working
• the effect of the rule not working as intended

This will help us to understand what is happening and, where appropriate, to prioritise a rule review.

We will consider any feedback we get and, where we agree there may be a need for a review, we will build this into our organisational planning and prioritisation processes (see How we prioritise rules for review). We cannot commit to undertaking a rule review in response to every piece of feedback. We also cannot commit to responding to all submissions via our stakeholder feedback tool.

Separate to the feedback tool, the firms and individuals we regulate can provide evidence on a rule’s effectiveness via their supervisory contact.

We will continue to consider other options for feedback including potentially having a feedback option embedded in our online Handbook, so stakeholders can easily feedback on specific rules.

An evidence assessment is a process of collecting and analysing available intelligence and information, which allows us to provide an assessment of whether the rule: 

• is on track to achieve its original outcomes or objectives, for example as set out in the causal chain at the policy implementation stage 
• has resulted in any implementation issues or unintended consequences 
• can be improved, or may be improved with more evidence, to better meet our intended outcomes  

We may consider doing an evidence assessment where monitoring suggests that the rule is not working as intended and we need to better understand the underlying reasons for this. Our focus is to assess if the key changes we expected in the policy’s causal chain have occurred, and the reasons for these changes, without necessarily isolating the exact causal effect of the intervention. An evidence assessment focuses on the early or intermediate part of the causal chain, to understand effects without providing exact quantitative estimates.

For existing rules where we did not set out a causal chain at implementation, it may be difficult for us to establish outcomes against which we can measure whether the rule is working well. In these cases, we will set out to understand the overall effect - positive or negative - that the rule has on the market. We will also consider whether there have been substantive changes in circumstances, such as market developments or the introduction of other rules, which alter how the rule is working.

An evidence assessment is designed to be informative and credible, while being less resource-intensive. It is also flexible, allowing us to design a review that is appropriate for the relevant rule and what the data shows may need to be addressed.

In most cases, we will use existing intelligence we already have, particularly data we have collected through any monitoring or our supervisory work. We may supplement this with qualitative information, such as evidence from stakeholders or their representatives, including trade associations, consumer groups and our statutory Panels. This enables us to take a quicker, more agile approach to assessing a rule or policy.

We will undertake evidence assessments internally, building on our expertise.

A post implementation review should assess whether the implementation of a rule has been successful, as measured by key changes, outcomes and discussions with stakeholders.

We expect this process will require data and stakeholder engagement beyond that collected during the monitoring process. We may use external parties to help us with parts of the review. This means a post implementation review is likely to require more time and resource than an evidence assessment.

Typically, and in line with Government practice, we use a post implementation review to establish whether, and to what extent, the rule:

• has achieved its original objectives as set out in the consultation paper or policy statement
• has been implemented and complied with as intended
• has resulted in any unintended effects
• has objectives which are still valid
• is still required and remains the best option for achieving those objectives
• can be improved to reduce the burden on business and its overall costs

A post implementation review typically seeks to identify areas of a rule implementation and delivery process which worked well, areas which could be improved upon and how external factors have influenced the context of the delivery. As such, we expect to use the outcomes of a post implementation review to guide our decisions on whether we need to change rules or provide further guidance, as well as influencing the implementation and delivery of future interventions. 

Our staff typically undertake post implementation reviews, often with input from independent experts, especially for larger reviews.

An impact evaluation is our most rigorous tool for assessing the impact of our interventions. This type of review focuses on using causal methods to quantify impacts. We typically plan for these in advance, at policy development stage, to ensure we collect the correct data.

We will make sure that we have a feasible plan for evaluation before implementing rules. A good design will allow us to analyse the causal impact of our rules, separating this from other changes in the market. This fundamentally relies on establishing a plausible counterfactual to measure what may have happened had we not intervened. In some cases, the design may also allow us to make statements about why the rule has had certain effects or the mechanisms through which there has been an impact.

However, if a market has subsequently experienced a range of external shocks, there have been multiple interventions or we cannot gather data in the form intended, it can be very difficult or impossible to establish the impact of a particular intervention (see the Annex for more discussion of challenges for impact evaluations and how we may address them).

Impact evaluations are generally the most quantitative form of review but may draw on a variety of evidence, including qualitative tools. In conducting this form of review, we know that it may not be easy to quantify all impacts.

These evaluations are a demanding form of review in terms of the data requirements and analytical resource needed. We can only undertake them when sufficient time has passed to observe the full effects of a rule. Undertaking impact evaluations requires us to use a significant amount of our resources and often to make ad hoc data requests from firms. We consider value for money in our work and that data requests can create a cost for firms. We will undertake them only when it is proportionate, for a subset of our interventions. We also consider the best ways to collect the information to undertake the evaluation and balance the rigour gained against the costs this would involve.

As a guide, where we are planning to do an impact evaluation of a rule, we normally wait until about 3 to 5 years after implementing the rule. The exact timing will depend on various factors, such as the detail of the rules, the relevant market and the scale of change. However, if the rule has a high cost or is addressing a severe harm, we might review it sooner to ensure it has been implemented as planned and is addressing the harm.

It is also important that we can ensure the robustness and credibility of our impact evaluations where we make claims about the specific impact of our interventions. We will do this by either commissioning them externally or, where carried out by our staff, ensuring they are peer reviewed by external experts. We expect to publish most, if not all, of our evaluations, so they will also be subject to external scrutiny.