Scala development and design using Scala 2.10+ or Python
Hadoop, Spark/Pyspark, Hive, YARN
Data Modelling: A good data engineer should be able to design, implement and maintain data models that can support the organization's data storage and analysis needs, and demonstrate good use of associated tools
Knowledge of operating systems such as windows and linux / unix
Experience with Jira, Confluence, GitHub or other similar Agile Scrum technologies and automated deployment tools (Ansible & Jenkins)
Knowledge and experience of test methodologies, developing test plans and execution of test cases (smoke/unit/integration/performance testing)
Scheduling Pipelines using Control M/Airflow
Should have:
Work effectively within a pod structure, often unaided
Attend daily scrums, meeting or exceeding sprint targets
Providing L3 support for developed and deployed assets quickly and effectively, responding to tickets efficiently to minimise MTTR (Mean Time To Repair)