Loading, please wait...

cloud administrator

Title posted on indeed.com - Hadoop Administrator

Posted on December 05, 2024 by Employer details Confiar Services

Job details

Job Title: *Hadoop Administrator* *Location:* Toronto, Canada (2 Days Onsite Per Week) *Job Type:* Contract *Salary:* Competitive (Depending on experience) *Job Description:* We are seeking a skilled *Hadoop Administrator* to join our team in Toronto. This role involves the management, configuration, and optimization of the Hadoop ecosystem in a large-scale data environment. The ideal candidate will have hands-on experience with Hadoop clusters and strong skills in managing and troubleshooting the Hadoop ecosystem, including HDFS, YARN, and various related tools such as Hive, HBase, and Pig. As a *Hadoop Admin*, you will work closely with data engineers, data scientists, and other IT teams to ensure the optimal performance of our big data infrastructure. This position is hybrid, with 2 days required onsite per week in Toronto, with the remaining work being done remotely. *Responsibilities:* * *Hadoop Cluster Management*: * Install, configure, and maintain Hadoop clusters (HDFS, YARN, MapReduce). * Troubleshoot and resolve issues in the Hadoop environment. * Monitor and optimize Hadoop cluster performance to ensure high availability and fault tolerance. * *System Administration*: * Manage and monitor Hadoop ecosystem components (e.g., Hive, HBase, Spark, Pig, Zookeeper). * Manage security and access controls for Hadoop (Kerberos, Ranger, etc.). * Perform backups and disaster recovery planning. * *Performance Tuning*: * Tune Hadoop clusters for optimal performance and scalability. * Work with developers and engineers to optimize big data workloads. * *Automation and Scripting*: * Write and maintain scripts (e.g., Bash, Python) for automation and job scheduling. * Automate Hadoop maintenance tasks, including cluster health checks and data backups. * *Collaboration & Support*: * Work closely with data engineers, DevOps, and other teams to support data pipelines and analytics platforms. * Provide support for data ingest, ETL processes, and batch processing jobs. * Document procedures and best practices. * *Upgrade and Patching*: * Plan and execute Hadoop upgrades and patches without causing system downtime. * Maintain software versioning and ensure compatibility with different components. * *Capacity Planning and Monitoring*: * Plan for future hardware and software requirements based on anticipated data growth. * Implement monitoring tools and dashboards for real-time cluster performance monitoring. *Qualifications:* * *Education*: * Bachelor's degree in Computer Science, Information Technology, or a related field. Equivalent work experience will also be considered. * *Experience*: * 3+ years of hands-on experience as a Hadoop Administrator in large-scale environments. * Strong knowledge of *Hadoop ecosystem* (HDFS, YARN, MapReduce, Hive, HBase, Pig, etc.). * Experience with *Apache Spark* for data processing is highly preferred. * Familiarity with *Big Data frameworks* such as *Kafka*, *Flink*, *Cassandra* is a plus. * Experience with *Hadoop cluster management tools* (e.g., Cloudera, Hortonworks, or MapR). * Experience with *containerization* and *orchestration* tools like *Docker* and *Kubernetes* is a plus. * *Skills*: * Strong understanding of *Linux/Unix-based systems* and shell scripting. * Experience with *cloud platforms* (AWS, Azure, GCP) for managing Hadoop clusters. * Proficiency in *Java* or *Python* for writing scripts or tools. * Familiarity with *monitoring and alerting tools* like *Nagios*, *Ganglia*, or *Ambari*. * Ability to troubleshoot system issues and perform root cause analysis. * Strong communication skills and ability to collaborate with cross-functional teams. * *Certifications*: * Certifications in *Hadoop*, *Cloudera*, or related Big Data technologies are a plus. Job Type: Full-time Pay: $75.00-$80.00 per hour Benefits: * Dental care Schedule: * 8 hour shift