Insights & Analysis

Regulation: markets’ friend, data management’s foe?

15th June, 2023|Staff Writer

Data & Technology

Derivatives markets participants have been -figuratively - under the regulatory cosh for years and particularly since the 2008 global crash, and subsequent regulatory focus on ensuring fair and effective markets. Several regulations directly impact derivatives data management and reporting, including EMIR, SFTR (ESMA’s Securities Financing Transactions Regulation), MiFIR and the Dodd-Frank Wall Street Reform and Consumer Protection Act.

Derivatives markets participants have been -figuratively - under the regulatory cosh for years and particularly since the 2008 global crash, and subsequent regulatory focus on ensuring fair and effective markets. Several regulations directly impact derivatives data management and reporting, including EMIR, SFTR (ESMA’s Securities Financing Transactions Regulation), MiFIR and the Dodd-Frank Wall Street Reform and Consumer Protection Act.

This latter US regulation directly impacts derivatives data management and reporting, requiring swaps markets participants to both report data to repositories AND to make certain data, such as aggregate data on transactions and positions, available publicly. Further, the US CFTC has finalised rules to improve swap data reporting and approved changes that further streamline oversight of swap data reporting by seeking to improve the accuracy of data reported to - and by - data repositories.

At the same time, ISDA has developed ‘regulation future-proofed’ reporting standards for derivatives data at all stages of the transaction lifecycle, from trade execution to settlement. This Common Domain Model (CDM) is also intended to improve consistency in, and the accuracy of, data reported to regulators.


“While all markets regulations and industry initiatives are designed to protect markets – and those participating in them – through increasing transparency and reducing risks in derivatives markets activity, they present an ever-changing and growing challenge to those on the data management side of the business.”


Looking at EMIR (European Market Infrastructure Regulation), this first came into being in 2012 (implemented 2014) as part of the European Union's response to the global financial crisis, and with the noble agenda of increasing transparency and reducing risks in derivatives markets specifically. In essence, EMIR required (and requires) ALL derivative trades to be reported to trade repositories, and moreover, that they be reported in a standardised format.

More data, more rules, more reporting complexity

Not only has this led to a massive increase in the volume and complexity of data that market participants – sell and buy sides – have to capture and manage, it also required a concomitant and significant investment in firms’ internal data management infrastructures, and data quality and reporting tools, particularly with respect to achieving consistent data definitions and ‘mapping’ data to different operating systems.

EMIR imposes stringent ‘rules’ around data reporting accuracy and timeliness, with trades having to be reported within very strict timeframes, at risk of severe financial (and other) penalties. Post-Brexit, FCA (the UK regulator) implemented ‘UK EMIR’ which largely mirrors EMIR regulation and obligations, and has more recently concluded an industry consultation (that commenced in November 2021) to align the UK’s derivatives reporting framework with international guidance issued by CPMI-IOSCO[i] “to ensure a more globally consistent dataset” with respect to “mandatory delegated reporting obligations, counterparty notifications, registration and reconciliation processes and the use of XML schemas and global identifiers”.


“UK EMIR obligations – mandated for implementation by 30 September 2024 –directly impact all counterparties within the scope of regulatory reporting requirements - including third party service providers reporting on behalf of counterparties.”


EMIR Refit and UK EMIR – a little over a year to the implementation deadline

At the same time, original EMIR regulation is undergoing its own ‘refit’ that mandate significant changes to current reporting obligations: the implementation dates for the EMIR Refit for European (EEA) based firms is 29 April 2024 (in Europe) and 30 September 2024 (UK). The key changes – and additional data management challenges -include:

  • The total number of reportable fields will grow to 203, including 89 new fields, withdrawal of 15 fields, and updates to the name, definition and format of how to report a field. This is more than double MiFIR’s reporting obligations (and bigger than SFTR reporting requirements).

  • EMIR and UK EMIR reporting will be in the form of ISO 20022-standard XML schemas and submissions to facilitate more consistent reporting to different TRs (trade repositories) in and across different jurisdictions. This is, by itself, an enormous change and obligation, hence the September 2024 implementation deadline.

  • Unique Product (UPI) and Trade Identifiers (UTI) have been an ongoing challenge and struggle for market participants, pre-dating EMIR and other markets regulation. Currently EMIR reporting on UPIs accepts random combinations of ISINs, CFI codes (Classification of Financial instruments) or no UPI at all, leading to inconsistency and confusion. Under the ‘Refit’, standardised UPIs will be mandated for regulatory reporting, another enormous potential pain point for derivatives trading firms since these will have to be requested and allocated in time to meet T+1 reporting obligations – another resource and cost burden.

  • Unique Trade Identifiers are more ‘bedded in’ to regulatory reporting workflow than UPIs. New ‘refit’ rules, however, propose changes to UTI ‘generation’ practices and processes that introduce more prescriptive steps to determine which counterparty to each bilateral trade generates which UTI in different reporting situations.

It is, on the one hand, difficult to argue against regulation and industry best practices that are intended, collectively, to improve market transparency, reduce trading risk and engender more harmonised and global industry reporting.


“There is no question that a more harmonised derivatives dataset would mitigate systemic risks and financial stability monitoring. From the perspective of beleaguered data management teams in today’s financial firms, however, it is a relentless battle to keep up with regulatory and customer-driven demands around accurate and timely data delivery.”


The need for more and better collaboration and communication

This has resulted in far greater collaboration and communication in and between market participants, and with regulators, and has also led to greater engagement with third party data and data management service providers with a specialised focus on normalising and/or ‘harmonising’ data derived from multiple sources (and needed in multiple destinations and applications).

The impact of EMIR (and UK EMIR) on market and reference data management is expected to continue to evolve in the coming years, as regulators continue to focus on greater transparency and less risks in financial trading.

New technologies such as machine-learning and AI may support more effective data management through greater automation of data validation and reconciliation workflows, and identification of data patterns and trends that can be used to improve risk management and compliance.

At the same time, new cloud-based data management solutions may offer benefits in terms of the cost and complexity of managing large volumes of data, particularly with respect to scale and flexibility benefits that go way beyond inhouse infrastructure capabilities.

Nonetheless, the quality, accuracy and timeliness of data will continue to require more and greater investment in data governance and control processes, the development of data quality frameworks and implementation of data validation and reconciliation tools.

Real-time data validation and reconciliation, alongside robust data governance and control processes, are essential weapons in any data management team’s armoury. This presents opportunities for more innovation and collaboration, particularly with specialist and focused third-party providers that de facto must remain ahead of evolving regulatory obligations and completely up to speed with other data management developments and trends in this rapidly-changing environment.

Contact us to find out how we can help with your data management efficiency and regulatory reporting.