Multi-firm review of algorithmic trading controls: high-level observations

3.1.1 Self-assessment and validation

The quality of self-assessment documents and the overall self-assessment process have improved since our 2018 review. Most firms had a better understanding of the requirements and governance frameworks have matured.

The level of information and detail in the self-assessments varied widely across firms. Generally, larger firms had the most comprehensive approach to the self-assessment, while medium-sized and small firms applied proportionality in their approach.

Good practice   

Some firms had their self-assessments reviewed by external auditors. This often resulted in recommendations and tracked actions for firms to complete, to further strengthen their compliance.

Room for improvement   

At some firms, we found more detail was required in certain areas of the self-assessment and identified deficiencies needed to be addressed more efficiently. This included out of date policies and unclear processes and documentation which indicated a lack of formal governance and accountability. In addition, key policy documentation was not linked or referenced in some self-assessments.

In some cases, firms did not address certain elements of RTS 6 at all in their self-assessments, such as IT outsourcing and Compliance training. It is important that firms fully assess their compliance with the requirements of RTS 6.

3.1.2 Role of the compliance function

Firms are required to make sure that compliance staff have at least a general understanding of how the firm’s algorithms operate.

Overall, technical knowledge of algorithmic trading among compliance staff and the level of oversight and challenge provided by compliance varied from firm to firm. Some firms relied more on their risk function to drive the RTS 6 self-assessment process and oversight/control. In most cases, governance structures were proportionate to the nature, scale and complexity of firms’ business models.

Good practice   

In some firms, compliance staff had very strong technical knowledge and provided strong challenge to algorithmic trading processes. Some firms had a systematic and formalised compliance monitoring plan that directly addressed compliance with RTS 6.

Room for improvement   

The compliance functions of some firms did not have as strong technical knowledge of algorithmic trading. This meant the ability of compliance staff to challenge trading behaviours was limited. The compliance function in some firms was also less involved in key algorithmic trading processes.

3.1.3 Algorithmic inventories

Firms should maintain a comprehensive inventory of algorithmic trading strategies and systems. We found that most firms maintained complete algorithmic inventories that captured the key details of each trading algorithm.

In many firms, algorithms were created, developed and deployed globally but were subject to local (UK) control requirements and approval processes.

Good practice   

Many firms maintained a very detailed inventory which included a qualitative description of each algorithm’s objective, ie its intended behaviour. There was a clear indication of who owned and who was approved to operate each algorithm and the markets on which each algorithm was used. In some cases, the inventory also included specific risk parameters that were applied to each algorithm. All this information formed part of a comprehensive algorithmic inventory, which provided useful management information.

Room for improvement   

In some cases, the algorithmic inventory did not specify the individuals who were approved to operate the algorithm.

3.1.4 Deployment of algorithms (including material changes)

It is important that firms have clear, formalised procedures for deploying trading algorithms and the management of material changes to those algorithms.

We found that most firms had formalised, documented deployment procedures, which set out clear accountability for the development, testing and deployment of algorithms. We also found that most firms had a clear and consistent definition of what constitutes a material change.

Good practice   

Some firms had robust governance procedures for deploying algorithms. In many cases, algorithms required approval from a wide range of business areas before being deployed. In cases where an algorithm was being deployed to a new market for the first time, many firms carried out a much deeper review, with elevated scrutiny applied to the algorithm. In addition, some firms had strong communication procedures around algorithm deployment, making sure that all material releases are communicated in a timely manner to all relevant parties.

In some firms, dialogue takes place between the compliance function and developers of an algorithm during the deployment process. Significant challenge was often provided by the compliance function on the functionality of the algorithm and how it would behave on the market.

Most firms had clearly documented the definition of a material change to an algorithm and had formal processes to identify those changes consistently across the business. Many firms provided continuous training for relevant staff on the material changes policy. Some firms required all changes to algorithms, regardless of materiality, to be approved by a Senior Management Function (SMF). Some firms also have processes to identify if changes were not identified as material changes when they should have been (false negatives).

Room for improvement

Some firms had out of date policies, or unclear procedures for testing and deploying algorithms. In some cases, firms had not clearly documented the definition of a material change, nor was any Management Information (MI) provided to the board regarding deployments.

In some cases, firms that used third party algorithms did not have a good technical understanding of how those algorithms were developed.

3.2.1 Conformance testing

Conformance testing is an important element of algorithmic development and approval. Firms are required to test the conformance of their algorithms with the system of the trading venue or direct market access provider, as per Article 6 of RTS 6.

We found that most firms were compliant with Article 6 of RTS 6 and carried out the required conformance testing. Some firms noted that conformance testing procedures differed significantly from venue to venue.

Good practice   

Policies and procedures of some firms clearly defined the scenarios in which conformance testing was required. Some firms proactively identified upcoming algorithmic changes and events that required conformance testing.

Some firms had robust conformance testing procedures, in some cases carrying out many more tests than required by the venue.

Room for improvement

Some firms, however, had poorly defined conformance testing procedures. This sometimes resulted in poor record keeping practices.

3.2.2 Simulation testing

Firms must maintain testing processes to identify potential issues before deployment and make sure the algorithm behaves as intended, does not contribute to disorderly trading and behaves effectively under stressed market conditions.

Many firms take a holistic approach to preventing disorderly trading behaviour of their algorithms. A key element is testing algorithms in a simulated testing environment. Along with risk controls and continuous monitoring, it is often an important way firms make sure their algorithms do not contribute to disorderly trading conditions.

The approaches firms took to simulation testing varied, in particular in how they subjected algorithms to stress. Some firms simulated theoretical trading scenarios, others used historical periods of market stress, while others used a combination of both theoretical and observed stress periods.

Most firms, however, deployed proprietary algorithms and carried out simulation testing in proprietary testing environments. However, firms who used third-party-provided algorithms relied on the simulation testing of their vendor.

Good practice  

It was clear that simulation testing of algorithms was a critical element of certain firms’ operations. Some firms dedicated significant resources and expertise to making sure that simulation testing was as robust as possible and included a wide variety of stress scenarios. For some firms, the use of simulation testing was not limited to the deployment of new algorithms or material changes. Rather, simulation testing was a frequent procedure in some firms.

When selecting historical market data against which to test an algorithm, some firms proactively selected periods of data that contained higher levels of stress. This helped to reduce the risk of the algorithm behaving in an unintended manner in a stressed market. As new stressed market events occurred, some firms proactively and swiftly updated their simulated testing data to make sure that algorithms were tested using the most up-to-date example of a stressed market.

Room for improvement

Simulation testing carried out by some firms lacked sophistication or did not appear to consider a wide range of market scenarios. Similarly, some firms lacked formally documented testing policies and procedures, even though testing was taking place. Many firms had strong pre-trade and post-trade controls in place on their algorithms. However, it is important that firms ensure that algorithms are tested appropriately before deployment, to make sure they do not contribute to disorderly trading conditions and continue to work effectively in stressed market conditions.

In some firms, there was a focus on operational effectiveness, and conduct risks were more thoroughly considered during post-trade surveillance, rather than during testing. All firms should continue to review their testing techniques and ensure that conduct risks are considered throughout the development and testing process. It is also essential that firms’ testing capabilities keep pace with the ever-increasing speed and complexity of their own algorithms, financial markets and technological advancements including cross-asset testing, where relevant.

3.2.3 Controlled deployment of algorithms

Firms must have adequate processes to make sure algorithms are deployed in an appropriate and controlled manner.

We found that firms took a conservative approach to deploying algorithms. Algorithms were deployed to the live environment in a slow, phased manner with significant scrutiny.

Good practice   

Firms submitted small pilot trades to the live environment to test the functionality of the algorithm. These pilot trades were heavily scrutinised to make sure the algorithm behaved as expected.

Many firms deployed their existing algorithms to new markets. In such instances, even when algorithms had been well-established and used for many years in other markets, these algorithms were subjected to the same level of review and approval as newly developed algorithms.

Some firms had very robust governance procedures surrounding algorithmic deployment, with each stage of deployment documented and with clear evidence of approval by senior individuals.

Room for improvement

Some firms lacked formal documented procedures for the deployment of algorithms. This was often accompanied by unclear ownership of key elements of the algorithmic deployment process.

3.3.1 Pre/post-trade controls

Firms must have adequate and robust pre/post trade controls, set at appropriate levels, to identify and reduce trading risks and control trading activity.

We found that all firms had adequate pre-trade controls in place. Many firms relied on a strong suite of pre-trade controls, in combination with robust testing of algorithms, as part of a holistic approach to preventing algorithms from contributing to disorderly trading.

Good practice     

Firms had a clearly defined suite of pre-trade controls applied to their algorithmic trading. In most cases, these controls were calibrated according to the type of algorithm being used and the asset class being traded.

Many firms carry out pre-trade controls at an internal server level. This meant that orders could not leave the firm’s internal gateway if a pre-trade control was breached.

Room for improvement

In certain cases, ownership of pre-trade and post-trade controls was poorly defined and not documented. Firms’ policies and procedures must clearly define the individuals with responsibility for managing pre-trade and post-trade controls.

In some cases, compliance staff had a lack of oversight of pre-trade and post-trade controls. This resulted in certain compliance staff having a weak understanding of the controls and how they functioned.

It is important that all firms continuously review their pre-trade and post-trade controls, as well as the governance procedures and documentation.

3.4.1 Surveillance systems and governance and oversight

Firms must consider the potential impact of their algorithmic trading on market integrity, monitor for potential conduct issues, and reduce market abuse risks.

We found that many firms used their own, in-house developed surveillance systems to identify and monitor market abuse risks. In many cases, in-house systems provided some additional efficiencies for firms, as they were easier to link to downstream systems.

Firms used their Market Abuse Risk Assessments (MARAs) to define the scope of their surveillance systems. All firms conducted regular reviews of the MARA.

Good practice   

Many firms had customised their surveillance systems to the type of trading they carried out. Some scoped their surveillance systems to monitor activity across different asset classes and trading venues. Many firms had a good awareness of the specific market abuse risks that applied to their activities. Market abuse alert logic calibration was an important consideration for all firms and was discussed regularly by relevant internal committees.

Firms had efficient and effective procedures for dealing with market abuse alerts. Most firms had clear escalation policies and formalised governance procedures to make sure alerts were investigated thoroughly and the correct action taken. In some cases, firms randomly sampled closed alerts for additional review and challenge.

Room for improvement

In certain cases, firms had not done enough to update or invest in their market surveillance systems. This meant their surveillance was not developing commensurately with the nature, scale and complexity of their trading activities.

Some firms did not have formalised procedures or governance structures around market abuse alert investigation. This often resulted in alerts taking longer to be investigated and closed. In some cases, we also found that market abuse alert investigation and closure generated significant resourcing pressure, with a small number of staff being responsible for a significant volume of alerts.