Headquartered in McLean, Virginia, Capital One offers a broad array of financial products and services to consumers, small businesses and commercial clients in the U.S., Canada, and the UK.
I have joined Capital One in October 2015 as a Data Engineering Manager working for Enterprise Data Services, which is a horizontal division that provides technology support across all LOBs in the organization. Below are some of the notable projects I have been part of.
Machine Learning Platform Development
- Duration: December 2016 - Current
- Capacity: Sr. Data Engineering Manager
- Division: Analytics Modernization
Reporting to Sr. Director, I am leading a team of excellent data & software engineers to build a cutting edge language and framework agnostic machine learning and predictive model development and execution platform in AWS that provides all the modern bells and whistles for model developers and data scientists while reducing the time it takes for a model to get to production. The end goal of the platform is to onboard all existing models in "legacy" platforms such as SAS, Teradata, and Oracle.
Key Accomplishments
-
v 1.0 of model development product and model execution platform have been released.
-
All packages used are open source, so no software licensing cost involved
-
These products use different cutting edge technologies such as Docker, AWS ECS, EC2, EMR, RDS, Redshift, terraform, Ansible, and Cloudera Express to spin up transient computing for model development and execution
-
By combining DevOps, Cloud and distributed computing solutions, 70% reduction in the time it takes to get a model from development to production
-
By using transient computing and containerization, cost of executing models is reduced already by 60%
-
Time spent by Data Science and Quants community for setting up infrastructure, establishing connectivity, instead of model development is now reduced by 75%.
Modernize Coporate Strategy Data Pipelines
- Duration: October 2015 - December 2016
- Capacity: Data Engineering Manager
- Division: Enterprise Data Systems - Corporate Strategy
I was recruited to manage Data Engineering operations for corporate strategy, supporting and enabling upper management to manage data driven strategical and tactical directions for entire organization.
As Data Engineering Manager, my role is responsible for providing data engineering roadmap to upper management, design, develop and maintain Bigdata, data science and decision support solutions through a team of data engineers including offshore vendors.
Key Accomplishments:
-
Reduced strategical and tactical decision-making time regarding product offerings for upper management by 75%, by providing an ability to analyze, mine and model in-house and competitor data, by implementing brand new scalable and fault tolerant BigData Data Lake for corporate strategy.
-
5 million dollars per year in software licensing costs by implementing this data lake on open-source BigData technologies such as Hadoop, Yarn, Python, Spark, and Luigi.
-
Under the process of migrating corporate strategy Data Lake into AWS cloud platform to further reduce costs by 50%, and also provide elasticity for the end user analysis and mining needs.
-
Helping peers by guiding them to move their applications from traditional architecture to BigData architecture.
-
Leading teams on innovation days to invent new applications for a wide variety of use cases.
-
Helped reduce overall costs with an intelligent incorporation of open source technologies and an ideal combination of on-site and offshore developers.