Data Cloud.png

Up, up and away – managing data in the Cloud

In the world of finance, market and reference data management is a critical function supporting trading, risk management, regulatory compliance and enterprise and customer reporting. In essence it is the efficient collection, storage, processing and distribution of accurate and timely data to support all participants and activities in financial transaction lifecycles.

In addition to informing investment and trading decisions, in today’s global financial markets, efficient data management is mission critical to satisfying evolving and growing regulatory compliance obligations. Different geographies, asset classes and market participants (e.g. wholesale, retail and consumer) may be subject to different reporting mandates and obligations. At the same time, beyond traditional financial institutions like banks and exchanges there is an increasing number of new trading platforms and channels generating transaction data and myriad new ‘end destinations’ consuming it.

“Continuing technological innovation has moved the data management dial exponentially from what was once a predominantly manual, error-prone function, to facilitate much more automated data distribution and collection processes between trading counterparties and end users.”

Most recently, the advent (and proliferation) of cloud-based infrastructure has been embraced across the board (to varying extents) by most financial market participants and service providers to support increasing volumes and complexity of financial markets data. Cloud technologies offer a number of benefits to beleaguered data management teams:

Unlimited storage and processing capacity

The primary attraction of cloud-based solutions for data management – storage and processing – are that ‘clouds’ offer pretty much unlimited storage capacity. Great news in and of itself, this is particularly true with respect to mitigating the high costs and physical resource burden of having to build and manage storage infrastructure (or shared storage facilities) inhouse.


Cloud technologies allow financial institutions to efficiently scale up – and indeed, down - their data management infrastructure according to changing needs: clouds can handle almost infinite volume capacities during peak periods caused by market volatility or unusually high trading volumes. As such, market participants no longer have to gear up expensive data management infrastructure to accommodate maximum capacities that it may only rarely need.


Cloud-based solutions are more flexible than traditional on-premise solutions, since it facilitates secure and compliant data access for any party authorised to ‘consume’ it (front, middle, back office and compliance functions), at any time, on any device and from any location. This supports remote working, access inclusivity (in less established markets) and industry collaboration, all vital in today’s global, and distributed, business environment.

Cost efficiencies

Cloud-based solutions can be more cost-effective than traditional on-premises solutions, since they eliminate the need for ongoing investment in expensive hardware and ithe newest, shiniest connectivity and processing software.

Cloud services are typically available on the basis of ‘as a service’ subscription models, meaning that means financial markets participants pay for storage and processing purely on consumption, rather than the traditional ‘tiered’ pricing models that generally benefit only the biggest ‘consumers’.

Conversely, where business process resilience and service redundancy, risk models and compliance obligations militate against putting all eggs into a single cloud basket, there will be higher costs associated with engaging with multiple providers.

Clouds on the horizon?

Of course, while cloud-based solutions offer many benefits, they also bring their own potential challenges and risks, including:


Cloud-based solutions require financial institutions to trust data to a third-party provider, which can raise security concerns. To mitigate these concerns, cloud providers offer robust security measures such as encryption, access controls, and monitoring.


Financial institutions must comply with many and varied regulatory obligations in and across all of the jurisdictions in which they operate, including stringent data privacy laws, different in different countries, and affecting both the distribution and consumption of data ‘across borders’.

Data Privacy

Cloud providers may store data in different locations around the world, raising potential concerns around data ‘sovereignty’ and privacy. All financial markets participants engaging in cloud activity must manage the complexity of storing and processing data in compliance with all applicable laws, regulations and standards of industry best practice.

Service denial

While it may seem improbable that the Amazons and Googles of this world would ‘fail’, or that they might decide to withdraw from their commercial cloud strategies, improbable things can – and do – happen. To offset this potential if unlikely risk, global markets-driving financial institutions will likely want and need to ‘spread the cloud love’ across multiple service providers.

Ahead in the clouds? Financial markets adoption

Nasdaq and LSEG are two examples of technology-led global trading environments to trumpet their adoption of – and enthusiasm for – all things cloud. An early adopter (since 2008), Nasdaq is undoubtedly a first mover and a self-styled ‘cloud evangelist’, starting its cloud journey initially with ‘foundational’ data storage and T+1 reporting and billing workflows, and moving ‘systematically to the matching engine’ and cloud-facilitated ultra-low latency trading. One year in to its multi-year partnership with its chosen provider, Amazon (AWS), it has already migrated one of its US options markets, MRX, to the cloud (December 2022) and is committed to ‘building the next generation of cloud-enabled infrastructure for the world’s capital markets’.

Slightly later to the party, in December 2022 the UK’s LSEG announced its own, ten-year, formal partnership with Microsoft to develop ‘intuitive’ next-generation data, analytics and modelling solutions via Azure, AI and Teams. (A notable element of this partnership is Microsoft’s investment in LSEG with the acquisition of c.4% of LSEG’s publicly quoted shares). Describing the partnership as a ‘step change’ in the financial markets value chain, LSEG asserts that its enhanced and fully interoperable Workspace will integrate communications channels seamlessly, securely and in full compliance.

“All financial industry participants are already leveraging cloud technologies to a lesser or greater degree. ETD has itself operated exclusively in ‘the Cloud’ for 5+ years, storing its derivatives reference data in Microsoft Cloud (Azure), securely and compliantly, across multiple geographies, and maintaining large volumes of reference data history in the cloud for regulatory reporting purposes”.

JPMorgan Chase on the sell side and BlackRock on the buy side are more examples of financial entities that are using cloud-based technologies to improve data management strategies and processes. In JPM’s case, the bank migrated its market data platform to the cloud to handle the much larger volumes of data it now generates, consumes and distributes, and to offer faster access to the data to end destination ‘consumers’.

BlackRock, the major global investment management firm, has built a cloud-based ‘data lake’ (an interesting mashup of geological events) that also facilitates more efficient storage and processing of large volumes of market and reference data, from multiple sources, including trading venues and third-party service providers.

While formal collaborations with cloud service providers, as with Nasdaq and LSEG, must be accepted on the one hand as forward-thinking and reflective of the modern world’s shift to all things ‘digital’ with respect, particularly, to ‘virtual’ connectivity and communication technologies, they are both long term investments (literally and figuratively). As such, it is too soon to tell how things will pan out over the life of these ambitious agreements, or indeed, to anticipate what shiny, new technological innovation might come along in the interim that could leapfrog ‘the Cloud’ as a data storage and processing solution.

A promising outlook

There is no question that cloud technologies are already transforming data management processes and workflows, offering scalable, flexible and more cost-effective ‘cloud as a service’ consumption-based pricing models.  Beyond internal enterprise efficiencies with respect to handling huge and growing volumes of data in and between multiple sources and destinations, the Nasdaq and LSEG examples also illustrate the potential to gain a competitive edge with innovative software and services that enable seamless data interoperability between multiple cloud-based applications such as data analytics, machine-learning and AI tools.

It's not all Cloud Nine, however. While most market participants are embracing cloud technologies to a greater or lesser degree, the recent (January 2023) ION cyber event reminds us all that there is no room for complacency in financial markets. In consultation with market practitioners, markets regulators, oversight authorities and industry bodies from the Bank of England to Basel, IOSCO to FMSB, , will continue to keep the Cloud in their (ahem) sights.

We will be at FIA's IDX in London from 19-21st June at stand 33 - so if you are attending, come by and have a chat about how we can help with your regulatory data challenges or contact us.


Related Insights

  1. Article
    In the continuing absence of single, global identification and transaction messaging standards, the challenges of managing reference data are many and varied, since every data point may be subject to multiple interpretations - and indeed uses - in different contexts.
  2. Article
    It is a truth universally acknowledged that faulty reference data is, today, one of the key contributors to regulatory reporting failures.
  3. Article
    Reference data plays an increasingly critical role in the smooth running of global financial markets, touching on every part of the transaction lifecycle.