Senior Software Engineer - Automation Engineer (Spark / Kafka / NiFi)

Mastercard is hiring a Senior Software Engineer - Automation Engineer (Spark / Kafka / NiFi) in Pune.

Our Purpose

Mastercard powers economies and empowers people in 200+ countries and territories worldwide. Together with our customers, we’re helping build a sustainable economy where everyone can prosper. We support a wide range of digital payments choices, making transactions secure, simple, smart and accessible. Our technology and innovation, partnerships and networks combine to deliver a unique set of products and services that help people, businesses and governments realize their greatest potential.

Title and Summary

Senior Software Engineer - Automation Engineer (Spark / Kafka / NiFi)

Job Posting Title: Senior Software Engineer - Automation Engineer (Spark / Kafka / NiFi) ________________________________________ Who is Mastercard? Mastercard is a global technology company in the payments industry. Our mission is to connect and power an inclusive, digital economy that benefits everyone, everywhere by making transactions safe, simple, smart, and accessible. Using secure data and networks, partnerships and passion, our innovations and solutions help individuals, financial institutions, governments, and businesses realize their greatest potential. Our culture is guided by the Mastercard Way—own it, simplify it, sense of urgency, thoughtful risk taking, unlock potential, and be inclusive. ________________________________________ Job Summary:

We are looking for a highly skilled Senior QA Engineer / SDET with deep expertise in testing large scale batch and real time data platforms built on Apache Spark, Kafka, and Apache NiFi. This role focuses on validating streaming heavy architectures, ensuring data correctness, latency, fault tolerance, and reliability across complex distributed systems leveraging object storage platforms such as Apache Ozone and Ceph.

The ideal candidate is passionate about data quality, test automation, observability, and production readiness for mission critical big data systems.

# Key Responsibilities

• Analyze business, regulatory, and technical requirements to design risk based and audit compliant test scenarios.

• Develop, maintain, and execute test plans, test cases, and test data with appropriate documentation and traceability to requirements.

• Perform functional, regression, and system integration testing in accordance with enterprise quality standards and release controls.

• Log, track, and manage defects in JIRA, ensuring clear reproduction steps, impact assessment, and lifecycle traceability.

• Validate JSON payloads, cloud storage data, and Apache NiFi data flows to ensure data accuracy, completeness, and integrity.

• Collaborate with development teams to support timely defect resolution and adherence to quality gates.

• Participate in design reviews, test plan reviews, and technical documentation

• Participate in Agile ceremonies, providing clear and accurate test execution status, risk assessments, and quality metrics.

• Test Kafka based streaming pipelines, including topic partitioning, offsets, throughput, and ordering guarantees
• Verify event replay and reprocessing flows using Kafka offsets and Spark checkpoints
• Test Apache NiFi flows for ingestion, routing, back pressure handling, error paths, and retry semantics
• Execute performance, load, and soak testing for Spark batch and streaming workloads
• Collaborate with engineering teams to validate partitioning strategies, resource tuning, and scalability assumptions

• Build and maintain test automation frameworks for data validation, streaming verification, and regression testing
• Automate validation using SQL, PySpark, and Python based test harnesses
• Integrate automated tests into CI/CD pipelines for data applications

About Mastercard

See the company's official careers page for full details, then apply using the button below.