SEARCH

Data Engineer Resume Example

Resume Score: 70%

Love this resume?Build Your Own Now
DATA ENGINEER
Skills
  • HDFS, MapReduce, Hive, Impala, Pig Latin, Sqoop, Oozie, YARN, Avro, Parquet,Kafka
  • Apache Spark : Spark Core, Dataframes, Spark SQL, Spark Streaming, Scala
  • Programming Languages : C, C++, JAVA EE, SQL, PL/SQL
  • Automation Tools : Python, Shell Scripting, BASH, Autosys
  • ETL and Reporting tools : IBM Datastage, IBM Cognos , Crystal Reports
  • Management Tools : Cloudera Manager, Hortonworks Ambari, Maven, SVN
Professional Summary
Professional Big Data Engineer with 6 years of industry experience including around 2.5 years of experience in Big Data technologies. Expertise in Hadoop/Spark development experience, automation tools and E2E life cycle of software design process. Outstanding communication skills, dedicated to maintain up-to-date IT skills and industry knowledge.
Work History
01/2015
Data EngineerCloudwick Technologies – Newark, CA

06/2015
Data EngineerBank of America – Charlotte, NC
  • Migrated the existing data from Mainframes/Teradata/SQL Server to Hadoop and perform ETL operations on it.
  • Designed and Implemented Sqoop incremental imports, delta imports on tables without primary keys and dates from Teradata and appends directly into Hive Warehouse.
  • Converted Pig scripts/components of the ETL process (transformations) to Spark API.
  • Worked with Avro and Parquet file formats and used various compression techniques to leverage the storage in HDFS.
  • Used the Mainframe SerDe's and Avro SerDe's for serialization and de-serialization in hive to parse the contents.
  • Written various shell scripts for DI and other error handling and mailing systems (DUSTc).
  • Designed and developed ETL workflow using Oozie and automated them using Autosys.
  • Worked on on-call production issues - scrubbing, resolve hive query issues, workaround for defects within SLA duration.

01/2015 to 06/2015
Big Data Engineer Cloudwick Technologies
  • Developed custom UDF's for pig scripts and hive queries to implement business logic/complex analysis on the data.
  • Reduced the latency of spark jobs by tweaking the spark configurations and following other performance and Optimization techniques.
  • Responsible for loading unstructured and semi-structured data into Hadoop by creating static and dynamic partitions.
  • Used Sqoop to import/export data from various RDBMS (Teradata, Netezza, Oracle) to Hadoop cluster and vice versa.
  • Configured the Oozie workflows to manage independent jobs and to automate Shell, Pig, Hive and Sqoop etc.
  • jobs.

09/2010 to 07/2013
Software EngineerComputer Science Corporation MEPZ GAP – TN
  • Datastage 8.5, Serena, Unix, TOAD, Oracle 10G Sep'10 to Nov '12.
  • Translating the business requirements into Datastage jobs.
  • Tuned the long running jobs by improving the query performance and reduced the running time of the Datastage jobs.
  • Worked with XML transformer and other complex transformations.
  • Writing shell scripts to trigger the Data stage jobs.
  • Scheduling the Datastage jobs using the ESP Scheduler.
  • Pfizer - Cognos 10, TOAD, Unix Nov'12 to Jul '13.
  • Develop and validate Reports and Dashborads.
  • Scheduling Reports and grant permissions to the reports only to the appropriate users.
  • Performed Report and Model Migration.

Education
Dec 2014
Master of Science: Computer Science Engineering
University of North Carolina - Charlotte

June 2010
Undergraduate: Electronics and Communications Engineering
Jawaharlal Nehru Technological University -
Electronics and Communication
Certifications

HortonWorks Certified Java Developer for Apache Hadoop

CCDH-Cloudera Certified Developer for Apache Hadoop

O'Rielly Developer Certification for Apache Spark

Build Your Own Now

DISCLAIMER

Resumes, and other information uploaded or provided by the user, are considered User Content governed by our Terms & Conditions. As such, it is not owned by us, and it is the user who retains ownership over such content.

Resume Overview

Companies Worked For:

  • Cloudwick Technologies
  • Bank of America
  • Computer Science Corporation MEPZ GAP

School Attended

  • University of North Carolina
  • Jawaharlal Nehru Technological University

Job Titles Held:

  • Data Engineer
  • Big Data Engineer
  • Software Engineer

Degrees

  • Master of Science : Computer Science Engineering
    Undergraduate : Electronics and Communications Engineering

Similar Resumes

View All
Data-Engineer-resume-sample

Data Engineer

Cognizant Technology Solutions

Waukegan, Illinois