Data Engineer II

Spark, Hadoop, and Python are the skills required for the role. It will be hybrid, 3 days a week in office. Please Focus on local resources or candidates within a commutable distance to Arlington

Interview: F2F

Assessment: Glider – Data Engineering (Advanced)

Overview

seeking an experienced Data Engineer II to join the Analytics Solutions Team. This team partners with stakeholders to extract data from the Data Warehouse, transform and optimize it, and build reporting/dashboard solutions that deliver actionable insights to clients.

The ideal candidate will have strong hands-on experience with Spark, Hadoop, Python, and SQL, along with a background building scalable data pipelines and ETL solutions in large data environments.

Work Schedule

Monday – Friday

9:00 AM – 5:00 PM ET

Hybrid schedule (3 days onsite per week)

No travel required

Interview Process

Two rounds minimum

Hiring Manager

Team Interview

In-person interviews preferred

Top Required Skills

✅ Apache Spark (PySpark, Spark SQL, Spark Streaming)

✅ Hadoop Ecosystem (HDFS/Ozone, Hive, YARN)

✅ Python

Level Required: Advanced

Key Responsibilities

Design, develop, and support enterprise-scale ETL processes and data pipelines.

Build scalable and efficient data processing solutions that ensure timely delivery of business-critical data.

Develop and optimize big data workflows using Apache Spark and Hadoop technologies.

Partner with Data Engineers, Analysts, and business stakeholders to deliver high-quality data solutions.

Troubleshoot data issues and implement solutions that maintain data integrity and quality.

Support dashboard and reporting initiatives by delivering clean, transformed datasets.

Utilize SQL and database technologies to improve data processing performance and efficiency.

Participate in automation initiatives to streamline operational and data engineering processes.

Follow engineering best practices including code reviews, version control, testing, and data validation.

Ensure compliance with Mastercard internal policies and industry regulations.

Required Qualifications

Experience as a Data Engineer or similar data-focused role.

Strong SQL development and query optimization experience.

Hands-on experience with

Apache Spark (PySpark, Spark SQL, Spark Streaming)

Hadoop ecosystem (HDFS/Ozone, Hive, YARN)

Python

Experience building and maintaining ETL pipelines.

Understanding of data modeling and database design principles.

Ability to troubleshoot and resolve complex data issues independently.

Experience validating and testing data for quality and consistency.

Strong written and verbal communication skills.

Bachelor's degree in Engineering, Mathematics, Finance, Business, Computer Science, or a related field (or equivalent practical experience).

Additional Details

Excellent opportunity to work on large-scale data platforms supporting global clients.

Target Profile: Data Engineers with strong Spark/Hadoop backgrounds who have built enterprise data pipelines and are comfortable working in a large-scale analytics environment.