Data Engineer

• Design and implement scalable ETL/ELT pipelines.
  
• Automate data workflows and ensure fault-tolerant execution.
  
• Support real-time data ingestion and streaming.
  
• The ability to design, implement, and optimize large-scale data and analytics solutions on Data Engineering
  
• Development of scripts using Python for loading, extracting, and transforming data.
  
• Assist with production issues in data warehouses like loading data, transformations, and translations
  
• Set up and manage data lakes and warehouses.
  
• Optimize data storage for performance and cost.
  
• Ensure high-performance data access during AI/ML training.
  
• Implement data security best practices (encryption, access control).
  
• Ensure data governance, compliance, and privacy.
  

Requirements

• 3+ years of experience in data engineering, ETL/ELT pipeline development, and data management.
  
• Proven hands-on experience with large-scale data processing, real-time data streaming, and data warehousing.
  
• Strong understanding of data modeling, data architecture, and database design.
  
• Hands-on experience with data partitioning, indexing, and query optimization.
  
• Proficiency in Python, Java, or Scala for data manipulation and automation.
  
• Experience with Apache Hadoop, Spark, or similar big data frameworks on AWS
  
• Understanding of enterprise data management concepts (Data Ingestion, Data Lake, Data Warehouse, Data Engineering, Data Sharing, Data Applications)
  
• Working knowledge of software engineering best practices
  
• Able to add unit test cases of the developed workload, pipelines & workflow.
  
• Should be able to develop and deploy the CI/CD pipelines
  
  

• Hands on experience working on Glue, Apache4
  
• Experience in GenAI tools and framework
  
• Implementing AWS services in a variety of distributed computing, enterprise environments
  
• Proficiency with at least one the languages such as C++, Java, Scala or Python
  
• Should know database concepts and working knowledge of SQL, data modelling.
  

Benefits

OnebyZero is an equal opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all employees.