MBH Bank successfully transformed its enterprise logging infrastructure by migrating from on-premises OpenShift Container Platform (OCP) to a hybrid AWS cloud solution, achieving significant cost reductions while maintaining operational excellence. Working closely with AWS Advanced Consulting Partner TC2, MBH Bank implemented a two-phase approach: first, establishing a proof-of-concept with AWS CloudWatch Logs, then optimising costs by migrating to AWS Managed Streaming for Kafka (MSK). 

The project demonstrates the power of AWS’s comprehensive logging and analytics services, delivering a 70% cost reduction in log processing while improving scalability, reliability, and analytics capabilities. This transformation showcases how traditional financial institutions can leverage AWS’s cloud-native services to modernize their infrastructure while maintaining strict security and compliance requirements. 

Key achievements are the following: 70% reduction in log processing costs, seamless integration between on-premises and AWS environments, enhanced log analytics capabilities with Amazon OpenSearch, improved operational visibility and monitoring and scalable architecture supporting future growth.

Business Challenge 

MBH Bank faced significant challenges with its existing on-premises logging infrastructure. As a leading financial institution, they generated massive volumes of log data from their OpenShift Container Platform, requiring centralized management, analysis, and long-term archival capabilities. 

Technical, operational and compliance requirements have been collected. For example, an increase in log size, that is, exponentially growing log volumes from containerized applications. Additionally, there was the challenge of managing and analysing distributed logs across multiple environments, as well as a requirement for near real-time log analysis and alerting. From the cost side, there was a need for cost-effective long-term log storage and processing. In addition to all this, there were regulatory requirements for log retention and auditability. 

The bank required a solution that could handle its current 1 Gb/s Direct Connect bandwidth while providing room for future growth, all while maintaining the highest security standards expected in the financial services industry. 

Initial Solution 

Working with TC2, MBH Bank implemented a comprehensive proof-of-concept leveraging AWS’s native logging services. The initial architecture demonstrated the feasibility of hybrid cloud logging while establishing the foundation for future optimizations. 

The initial solution utilized AWS CloudWatch Logs as the central log aggregation service, with Amazon OpenSearch providing advanced analytics capabilities. The architecture included the network infrastructure (Direct Connect, VPC Endpoints, Network Firewall), as well as the data processing pipeline (CloudWatch Logs, AWS Lambda, Amazon SQS, Amazon OpenSearch, Amazon S3). 

The initial architecture successfully demonstrated the seamless integration between on-premises OpenShift and AWS services, the reliable log delivery and processing at scale, the advanced analytics capabilities through OpenSearch, and the operational improvements in log visibility and management, taking into account compliance with financial services regulatory requirements. 

Evolved Solution Architecture 

Following the successful initail phase, MBH Bank and TC2 identified a critical optimization opportunity. While the CloudWatch Logs solution proved technically sound, the massive volume of log data resulted in unexpectedly high operational costs because of the CloudWatch Logs ingestion costs scaled linearly with data volume, high-frequency log generation from containerised applications, and substantial costs for log storage and processing. 

The high costs threatened the long-term viability of the cloud logging solution, requiring immediate architectural optimisation while maintaining all existing functionality and performance characteristics. 

TC2’s cloud architects designed an innovative cost-optimised solution utilising AWS Managed Streaming for Kafka (MSK) as the primary data ingestion layer, dramatically reducing costs while enhancing scalability and reliability. 

The evolved solution replaced expensive CloudWatch Logs ingestion with a sophisticated streaming architecture based on Amazon Managed Streaming for Apache Kafka, which led to a high-throughput, low-cost log ingestion solution. 

The process involves a new, advanced data flow: the on-premise OpenShift logs stream directly to AWS MSK topics, then a Lambda consumer processes MSK messages, and Kinesis Data Firehose delivers the processed logs to S3 with automated compression and partitioning and lifecycle policies for cost-optimized storage.  

 Conclusion 

The successful transformation of MBH Bank’s logging infrastructure demonstrates the tremendous value of AWS’s comprehensive cloud services when implemented by experienced partners like TC2. This project demonstrates how traditional financial institutions can utilise cloud-native technologies to achieve substantial cost savings while enhancing operational capabilities. 

This case study serves as a blueprint for other financial institutions seeking to modernize their infrastructure while maintaining the highest standards of security, compliance, and operational excellence. The partnership between MBH Bank and TC2, powered by AWS’s industry-leading cloud services, has created a foundation for continued innovation and growth in the digital banking era.