In their presentation posted at InfoQ systems and data architects Ben Stopford, Farzad Pezeshkpour and Mark Atwell show how RBS leveraged new technologies in their architectures while facing difficult challenges such as regulation, competition and tighter budgets. They also need to cope with stringent technical challenges including efficiency and scalability.
In high frequency trading an unavailable system causes significant costs, even if the unavailability only lasts for a second. As Stopford explains, a full garbage collection might lead to such a break which is why full GC needs to be avoided under all circumstances. More technical details about pauses in the CMS (Concurrent Mark Sweep) garbage collection being used can be found in Alexey Ragozin's blog.
Farzad Pezeshkpour talked about handling business risk in the financial domain. He talked about using multi-step Monte Carlo simulations, which require large amounts of storage and computations, to handle business risk - all those interested in the mathematical foundation may read an article on how to apply such simulations to credit risk.
But how can IT handle the issue of storage volumes and efficient computations? A standard Grid architecture with application servers connected to directors and brokers which themselves are connected to the grid computers was the base for further investigation. Such an environment must deal with terabytes of storage and requirements such as speed, robustness, scalability, interoperability with Linux and Windows servers, as well as the need for an adequate support infrastructure. Among the possible options are databases, storage appliances, and distributed filesystems. The software engineers decided for using the hadoop HDFS distributed file system. In their architecture approach they collocated grid engines and HDFS nodes using 10 GB Ethernet for inter-node communication, after experimenting with various architecture alternatives.
As Pezeshkpour mentioned, there'll be further optimizations in the future with the innovations addressing all aspects, "methodologies, architecture & engineering."
The copying of data lies at the root of many of the bank’s problems. By supplying a single real-time view that all systems can interface with we remove the need for reconciliation and promote the concept of truly shared services.
Two factors make this difficult to solve: Throughput and low latency. The base element of RBS's ODC approach is the Java-based in-memory data grid solution Oracle Coherence database. RBS needed to add joins to recompose different slices of data and handle object versioning. Stopford and his team introduced various patterns to add these joins while avoiding performance and memory penalties.
They implemented replication to increase performance while at the same time applying the Connected Replication Pattern that only replicates used data. This strategy reduces replication necessities to one-tenth and depends on tracking data usage. ODC consists of two tiers, an in-memory database tier on top and a sharded in-memory database below. Both tiers together enable RBS to meet their desired quality attributes.
Mark Atwell presented an approach for data virtualization that prevents the YAS (Yet Another System) syndrome by introducing a strategic DAL (Data Access Layer) instead. The RBS architects created a uniform layer that offers a unified database view. On top of this layer virtual transformations take place that adapt the data in accordance with what the accessing applications expect. As Atwell explains, the approach reveals benefits such as improvements in distributed joins & query optimizations. But there are also some liabilities such as "Pesky physics still get in the way!" but also the necessity to create adapters for some "unusual" data sources.
As the RBS show cases illustrate, software engineering for large systems is not only about technology but also about how to leverage the right technologies efficiently and effectively within an appropriate architecture.
I'm wondering how an Erlang based system might have done here. Since GC happens per process in Erlang, is it possible that it can also be used to solve this problem?
Sure, it's a decent idea. Certainly a large number of firms stick with C/C++ for this very reason. However when the rest of the stack is Java the one language thing has many benefits and the GC problems are obviously solvable.
It'd be interesting to know if dispatching work to multiple consumers (or conflating to one) done in the Manhattan processor would be easier to solve in Erlang. I don't know enough to answer that one I'm afraid.