Senior Hadoop Big Data Engineer
Contract Senior Hadoop Big Data Engineer– 30,000 45,000 AED Monthly (£6,500 - £9,500 per / month) – Dubai
Long term contract
Key Skills – Hadoop, Cloudera, Azure, Couch case, Sqoop, Kafka, MDM, Data LakesAre you looking to gain commercial exposure to an international client? An opportunity to work for one of the most prestigious organisations in the world based in Dubai has arisen for an Oracle / Hadoop Big Data Engineer
A Big Data engineer builds large-scale data processing systems, is an expert in data modelling & processing solutions and should be able to work with the latest Big Data and NoSQL database technologies. Because we work on the cutting edge of a lot of technologies, we need someone who is a creative problem solver, resourceful in getting things done, and productive working independently or collaboratively.
Key Skills / Responsibilities
• 10+ years of overall experience, with minimum 5 years in Data Lake related technologies mainly Hadoop and Hadoop-ecosystem (HDP, Cloudera)
• Experience working in Cloud environments, especially in Microsoft Azure
• Experience in Microsoft Azure’s Big Data Services like Azure HDInsight, Azure Data Lake Store (ADLS), Azure Data Factory (ADF)
• Experience working with NoSQL stores like Couch base, Cosmos DB
• Responsible for building, testing, deploying, troubleshooting, administering and maintaining Data Lake environments with ADLS/Azure Blob Storage as storage
• Should be able to develop and support data import/export processes with ADF, Sqoop, Kafka, etc.
• Should be able to support with Data and MDM governance and also with reference data management
• Monitor and control Enterprise Data Lake to support scale out architectures.
• Help engineers leverage infrastructure, application and platform features and functionality of Azure.
• Work closely with engineers to help them with automate deployments and configuration management of Enterprise Data Lake.
• Ensure high availability of Data Lake components with efficient cluster deployments and resiliency planning.
• In depth knowledge of security for Enterprise Data Lake for authentication, authorization, data privacy and data access mechanisms with Enterprise AD/Azure AD/LDAP.
• Should be able to support integration mechanisms with Enterprise Data Lake.
• Should be able to plan and exhibit disaster recovery of Data Lake components and data.
• Should be aware of Data Catalog management components like Azure Data Catalog / Cloudera Navigator for data traceability and lineage.
• Excellent written and verbal communication skills.We are keen to build strong relationships with IT consultants based on providing a transparent and supportive service. Now in our 18th year we pay promptly and are a stable contracting partner in good financial health. Deerfoot IT: Est. 1997. REC member. ISO certified. *Each time we send a CV to a recruiting client we donate £1 to The Born Free Foundation (charity no. 1070906). We never send your CV without your email authorisation