Data Engineer

Purpose

• As a Data Engineer, you design, develop, and maintain data pipelines and systems. Ensure efficient data processing, storage, and integration aligned with business goals. Collaborate with cross-functional teams to optimize data workflows and enhance data quality and performance.

Responsibilities

• Lead and participate in the design, development, and maintenance of data engineering solutions.
• Develop and implement robust ETL (Extract, Transform, Load) processes to ensure efficient data integration and transformation.
• Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and provide appropriate data solutions.
• Design and build scalable data pipelines to support data analytics and machine learning applications.
• Ensure data quality and integrity through the implementation of data validation and cleansing processes.
• Optimize data storage and retrieval processes for performance and scalability.
• Monitor and troubleshoot data engineering workflows to ensure reliability and performance.
• Provide technical guidance and mentorship to junior data engineers.
• Stay updated with the latest trends and best practices in data engineering and incorporate them into the organization's data strategy.

Education

• Minimum Bachelor's degree (or eqivalent of) in computer science, information technology, or a related field is required. Certifications in relevant data engineering tools and technologies such as AWS Certified Big Data Specialty, Google Professional Data Engineer, or Microsoft Certified: Azure Data Engineer Associate are highly desirable. Proficiency in English is essential for effective communication with team members, stakeholders, and external vendors.

Experience

• Minimum of 3 years of experience in data engineering, data management, or a related field.
• Proven experience in designing, developing, and maintaining data pipelines and ETL processes.
• Strong technical background with hands-on experience in working with large-scale data architectures, including data lakes, data warehouses, and real-time data processing.
• Demonstrated ability to develop and implement data engineering best practices and procedures to ensure data quality, reliability, and scalability.
• Experience with database design, and data integration using tools such as Apache Spark, Hadoop, and SQL. Data modeling experience is desired.
• Proficiency in using data engineering tools and platforms such as AWS, Azure, Google Cloud Platform, and experience in deploying and managing cloud-based data solutions.
• Strong programming skills in languages such as Python, Java, or Scala, with a focus on data processing and manipulation.
• Good communication skills, with the ability to work effectively with other team members and stakeholders.
• Relevant technology stack certifications i.e. AWS, Azure
• Attention to detail, problem-solving skills are desired for root cause analysis and addressing it with the appropriate solution.

Functional Competencies