Loading, please wait...

data engineer

Posted on December 19, 2024 by Employer details Panacea Direct Inc

Job details

We are seeking a skilled *Cloud Data Engineer* to join our team. The ideal candidate will have a strong background in *Python*, *PySpark*, *SQL*, and *data warehousing* for enterprise-level systems. This role requires technical expertise in data engineering and the ability to collaborate effectively with business users and analysts to deliver scalable solutions. *Key Responsibilities:* * Design, develop, and maintain data pipelines and workflows using *Python*, *PySpark*, and *Azure Data Factory (ADF)*. * Implement and optimize *Snowflake*-based data warehousing solutions for enterprise-scale systems. * Collaborate with business users and analysts to understand requirements and deliver robust data solutions. * Write and optimize complex *SQL queries* for data extraction, transformation, and analysis. * Ensure data quality, integrity, and compliance with organizational and industry standards. * Develop and maintain documentation for data pipelines, workflows, and architecture. * Troubleshoot and resolve performance and data-related issues in a timely manner. *Required Skills and Qualifications:* * Proficient in *Snowflake* with experience designing and implementing data warehousing solutions. * Strong expertise in *SQL*, *Python*, and *PySpark* for data engineering tasks. * Hands-on experience with *Azure Data Factory (ADF)* for building ETL/ELT pipelines. * Solid understanding of *data warehousing concepts* and best practices. * Proven ability to work with business users and translate requirements into technical solutions. * Excellent problem-solving skills and attention to detail. * Strong communication and collaboration skills. *Preferred Qualifications:* * Experience with *Azure Synapse Analytics*, *Databricks*, or similar cloud data platforms. * Knowledge of *CI/CD pipelines* and *DevOps* practices for data engineering. * Familiarity with *big data technologies* and distributed computing frameworks. * Certification in Snowflake, Azure, or relevant data engineering technologies. Job Type: Full-time Pay: Up to $65.00 per hour Expected hours: 40 per week Benefits: * Dental care Schedule: * 8 hour shift * Monday to Friday