Big Data Developer
Job title: Big Data Developer in USA at ACS Consultancy Services
Company: ACS Consultancy Services
Job description: Job Title: Big Data DeveloperLocation: RemoteWe are currently seeking candidates who meet the following qualifications. Responsibilities:
- Design and develop scalable data pipelines
- Work with structured, semi-structured, and unstructured data
- Implement data ingestion and processing using Hadoop, Spark, and Kafka
- Write optimized Hive, HQL, and Spark SQL queries
- Build data APIs and interfaces for analytics teams
- Monitor data pipelines for performance and reliability
- Maintain documentation and data lineage tracking
- Bachelor's in Computer Science, Engineering, or similar
- Experience in Big Data development
- Proficiency in Java/Scala/Python for data processing
- Expertise in Spark, Hive, HDFS, Sqoop, Flume, or Impala
- Experience with Kafka or other real-time data tools
- Strong SQL and NoSQL database skills
- Experience in workflow tools (Airflow, Oozie, NiFi)
- Cloud experience (AWS EMR, Azure Synapse, GCP BigQuery)
- Experience in building data lakes and data warehouses
- Knowledge of data security and compliance (GDPR, HIPAA)
- Experience in Delta Lake, Iceberg, or Hudi
- Understanding of CI/CD for data pipelines
- Federal Experience is a plus.
- Required Security clearance.