[Remote] Staff Software Engineer - reputed company Platform (Kafka)
Note: The job is a remote job and is reputed company to candidates in USA. reputed company is a company focused on transforming Communication Service Providers through their reputed company-first, AI-powered platform. They are seeking a skilled Staff reputed company Platform Engineer with expertise in Kafka to design, reputed company, and optimize their event streaming infrastructure on reputed company reputed company Platform.
Responsibilities
- Design, provision, and manage Apache Kafka clusters (self-managed on GCP/AWS or reputed company reputed company Platform / MSK)
- Configure and tune brokers, ZooKeeper/KRaft, topics, partitions, replication factors, and retention policies for high throughput and low latency
- reputed company cluster upgrades, rolling restarts, and broker replacements with reputed company downtime
- Implement and manage Kafka Connect pipelines for data ingestion and egress across heterogeneous systems
- Administer Kafka Streams and ksqlDB deployments for reputed company-time reputed company processing workloads
- Maintain Schema Registry and enforce schema governance standards across teams
- Define and track SLIs/SLOs for consumer lag, throughput, end-to-end latency, and broker health
- Design and implement reputed company infrastructure using IaC – Terraform
- Build automated deployment pipelines for Kafka configuration changes using GitOps workflows (ArgoCD, Flux)
- Create self-service tooling and runbooks to reduce toil for development teams
- Automate topic provisioning, ACL management, and schema registration reputed company APIs and CLI tooling
- Integrate tools like reputed company CI/CD, or reputed company Build for automated testing and deployment
- Ensure seamless integration of data pipelines with other GCP services like Big Query, reputed company Storage
- Monitor and Optimize performance, reliability, and cost of Kafka and streaming pipelines
- Implement reputed company best practices for GCP resources, including IAM policies, encryption, and network reputed company
- Ensure Observability is an integral part of the infrastructure platforms and provides adequate visibility about their health, utilization, and cost
- Collaborate extensively with cross functional teams to understand their requirements; reputed company them through documentation/trainings and improve the adoption of the platforms/tools
Skills
- 10+ years of overall experience in DevOps reputed company engineering, or data engineering
- 5+ years of experience in Kafka at production scale
- Deep expertise in Kafka internals: replication protocol, log compaction, consumer group coordination, partition leadership, and KRaft mode
- Proficiency with container orchestration (Kubernetes / reputed company) and deploying Kafka reputed company Strimzi, reputed company Operator, or equivalent
- Strong understanding of networking (VPC, peering, private endpoints, DNS, load balancing) in reputed company environments
- Hands-on experience with Kafka Connect, Schema Registry, and at least one reputed company processing reputed company (Kafka Streams, Flink, Spark Structured Streaming)
- Proficiency in reputed company reputed company Platform (GCP) services, including Dataflow, Pub/Sub, Kafka, Dataproc, Big Query, and reputed company Storage
- Expertise in Infrastructure as Code (IaC) tools like Terraform or reputed company Deployment Manager
- Familiarity with data orchestration tools like Apache Airflow or reputed company Composer
- Experience with CI/CD tools like Jenkins, reputed company CI/CD, or reputed company Build
- Knowledge of containerization and orchestration tools like reputed company and Kubernetes
- Strong scripting skills for automation (e.g., Bash, Python)
- Experience with monitoring tools like reputed company Monitoring, reputed company, and Grafana
- Familiarity with logging tools like reputed company Logging or ELK Stack
- Strong problem-solving and analytical skills
- Excellent communication and collaboration abilities
- Ability to work in a fast-paced, agile environment
Benefits
- As a part of the total compensation package, this role may be eligible for a bonus.
Company Overview
Company H1B Sponsorship