Aligne ETRM Data Engineer - Azure - Global Energy Co.
A World Leading Energy Company have an exciting opportunity for an ETRM Data Engineer with Azure experience to join their Energy Trading team.
They are building a Data Warehouse that will ingest data primarily from the Aligne ETRM system into an MS-Azure data lake and then present dashboards to the Market Risk team and Traders.
The system will eventually provide all Power, Gas and Emissions desks (about 300 people) a single source of the truth for all their reporting needs. We are currently looking for a Data Engineer to join the project team of about 12 analysts and developers.
About the role:
- To work alongside the Tech Lead to build and unit test Golden Source data pipelines, data models in data lakes, data warehouse and semantic marts.
Work in development sprint teams under the direction of the Tech Lead/Scrum Master
Be responsible for development and unit testing of user stories that have been assigned to them
Be responsible for developing robust data pipelines and data models
Use Azure technologies for development eg Azure Data Factory, Azure Databricks, Azure Data Lake, Azure Synapse, Azure Analysis Service, SQL server
Support the creation and maintenance of the analytics infrastructure
Build API endpoints exposing data to other services
Inform the Tech Lead/Scrum Master of any risks, issues or blockers
Participate in relevant sprint meetings eg daily stand-ups, retrospectives and sprint planning meetings
Maintain the required status on the chosen technology eg Jira
Provide SME input on technology and systems
Skills & experience required:
Microsoft Azure cloud services - Azure Databricks, Spark on HDInsight,
Event Hubs, Azure Stream Analytics, Azure Data Factory(v1 & v2), Azure Blob Storage,
Azure Data lake Store(ADLS Gen1/Gen2), Azure Functions, Azure DevOps, SQL DW, Polybase AzCopy, AdlCopy
Good communication skills