Capital One HR Data Management : Sr. Data Engineer Resume Example Capital One Financial Corp

CAPITAL ONE HR DATA MANAGEMENT : SR. DATA ENGINEER

Professional Summary

I am an experienced and AWS certified Data Engineer with 5+ years of experience in various dimensions of Big-Data and Cloud Technologies, developing data pipelines and providing ETL and Data warehouse solutions to process and manipulate large datasets.

Experience in building ETL solutions on a defined architecture based on Cloud and Big Data applications.
Experience in delivering and supporting large scale data-management and migration projects.
Thorough knowledge of using the right set of tools and technologies for providing solutions to process large sets of structured and semi-structured data.
Experience in supervising and coordinating a team of 5 members for delivering daily operational and delivery issues. (US and India).
Experience in working in Banking-Finance and HealthCare domain.

Technical Skills

Hadoop Ecosystem : HDFS, YARN, Map-Reduce, Flume, Sqoop, Spark, HIVE, Kafka
AWS Services : EMR, S3, AMI, Redshift, Lambda Function, SNS
ETL : Teradata Utilities, Hydrograph(Capital One In-House Product)
Programming Skills : Java, Scala, Python

Database : MySQL, Oracle DB, Snowflake, PostgreSQL, Redshift.
Build/CI Tools : Maven, Jenkins, Terraform
Scheduling Tools : AROW
Experience in UNIX operating system and shell-scripting along with CI/CD pipeline , monitoring and alert systems.

Work History

Capital One HR Data Management : Sr. Data Engineer02/2019 to Current
Capital One Financial Corp – Richmond, VA

Migrated terabytes of data from the legacy data warehouse into the cloud environment in an incremental format.
Developed Data-Pipelines using Apache Spark and other ETL solutions to get data from various sources like Workday and EFG to the central warehouse.
Developed generic code modules and defined strategy to extract data from different vendors and build ETL logic on top of them to feed the data to the central data warehouse.
Developed AWS Lambda Server less architecture for various tasks like EC2 shutdown during business off-hours, EMR setup and S3 watcher.
Performed Data Pre-processing, Data Profiling, Data Modeling and created frameworks using Python framework for validation and automation of regular checks.
Involved in daily operational activities in order to troubleshoot ad-hoc production and data issues and enhancement of infrastructure in the space of Big Data and AWS cloud to provide better solutions to delegate the existing issues.
Built end-to-end automation using shell-scripting, AWS SNS, AROW and Pager Duty.
Used SQL Window functions to build Presentation Layer on Base tables to implement complex logic for business users.

Auto Loan Data Migration : Data Engineer02/2018 to 01/2019
Capital One Financial Corp – PLANO, Texas

Worked on migrating applications from internal data center to AWS instances by setting Security groups, Elastic Load Balancers and AMIs, Auto – scaling to design cost effective, fault tolerant systems.
Designed scripts to unload data from on-prem infrastructure and migrate it to cloud (drop box solution) and perform transformation a per data model changes between the databases.
Developed and enhanced code for implementing required data quality check before sending into the cloud based data warehouse.
Designed Cloud Infrastructure (EC2/S3/RDS) by configuring and deploying various AWS services in production and non-production environment.
Developed framework using shell-script , Scala and Apache Spark 2.1, Amazon SDK for moving the data from D1 (data after Data Quality Check) to Snowflake on Amazon cloud which was leveraged by various other data migration teams.
Worked closely with the infrastructure team to make sure horizontal incremental of EC2 instances based on the load and available memory.
Performed data analysis and data profiling using advanced SQL queries.
Involved in developing CICD (Continuous Integration and Continuous Delivery) using Jenkins.

Technical Assistant08/2016 to 12/2017
University Of Nebraska Omaha – Omaha, NE

Conceptual understanding of Map-Reduce programming paradigm and implementation of the same into algorithms such as K – means clustering and Breadth-First Search and computation of Euclidean distance between co-ordinates
Comparative study to understand the performance of several Data Streaming platforms like Spark, Flink and Storm which helps in understanding which platform is better suited for which purpose
Performed sentiment-analysis of live Tweets on popular motels to understand the business trend among the competitors in the market

Software Developer07/2012 to 08/2015
Tata Consultancy Services – Kolkata, West Bengal

Developed web-services to integrate various healthcare devices into one central IOT infrastructure to maintain patient information
Engaged in UI/UX development of Mobile-Healthcare Application system
Engaged in designing and implementing various test-cases for database connectivity and fix of errors and bugs in aforementioned application

Education

Master of Science: Computer Science12/2017University of Nebraska At Omaha- Omaha, NE

Bachelor of Technology: Computer Science And Engineering07/2012West Bengal University Of Technology- Kolkata, India

Resume Overview

Companies Worked For:

Capital One Financial Corp
University Of Nebraska Omaha
Tata Consultancy Services

School Attended

University of Nebraska At Omaha
West Bengal University Of Technology

Job Titles Held:

Capital One HR Data Management : Sr. Data Engineer
Auto Loan Data Migration : Data Engineer
Technical Assistant
Software Developer

Degrees

Master of Science : Computer Science 12/2017
Bachelor of Technology : Computer Science And Engineering 07/2012

Capital One HR Data Management : Sr. Data Engineer Resume Example

Resume Score: 80%

DISCLAIMER

Resume Overview

Create a job alert for [job role title] at [location].

Similar Resumes