Need Sr Data Engineer (Direct Client) - Remote

Hi Associate,

Hope you are doing good.

I have urgent requirement kindly let me know if you have any resource available.

Role: Sr Data Engineer

Location: Remote

Duration: 3 Months Contract to hire

Job Description

We are seeking a highly motivated Senior Data Engineer to join our Vendor data product team. As a Senior Data Engineer you will be responsible for creating products using data from both internal and external vendor data sources to help realize goals of attracting customers through more accurate pricing, greater customer retention, and an increase in underwriting profitability. The right candidate will develop well-designed, testable, and efficient data ingestion, data enrichment, and data transformation solutions using best software development practices. You should be an analytical thinker, a self-learner, and comfortable supporting the needs of multiple projects. This role is a part of the Data Movement team of Data, Security & Infrastructure (DSI) in our Technology Solutions organization.

In This Role, You Will

• Identify, design, and implement internal process improvements, automating manual processes, optimizing data delivery for greater scalability
• Build the processes required for optimal extraction, transformation, and loading of data using a variety of languages and technologies such as Scala, Python, Kafka, Azure Data Factory, Fivetran/HVR, dbt, and Databricks
• Collaborate with stakeholders including the Product, Data Engineering, and Agile Delivery teams in an agile environment to assist and resolve data-related issues and support data delivery needs
• Work with other data platform or data domain teams to design, develop, test, implement, and support technical solutions in full-stack development tools and technologies
• Perform unit tests and conduct reviews with other team members to make sure code is rigorously designed, elegantly coded, and effectively tuned for performance
• Share your passion for staying on top of tech trends, experimenting with and learning new technologies, participating in internal & external technology communities, and mentoring other members of the engineering community
  
  

Experience & Skills

• At least 2 years of experience with designing, developing, implementing, and maintaining solutions for Big Data or data warehouse system
• At least 2 Years of experience working in a cloud environment such as Azure, AWS or other private or public cloud
• Experience performing root cause analysis on internal and external data and processes to answer business questions and identify opportunities for improvement
• Strong analytical skills related to working with unstructured datasets
• Good experience with bringing data into a centralized data repository or manipulating the available data to build additional data sets for Analytics and Reporting purposes.
• Experienced with maintaining data quality throughout the lifecycle of the data.
• Experience with Data Modeling, source to target mapping, automated testing frameworks, CI/CD pipelines and task automation using scripting
• Experienced with working in Agile environment and end to end automation
• Developing new and enhancing existing data processing (Data Ingest, Data Transformation, Data Store, Data Management, Data Quality) components
• Strong working knowledge of SQL and the ability to write, debug and optimize SQL queries and ETL jobs to reduce the execution window or reduce resource utilization
• Data Engineering experience focused on batch and real-time data pipelines development, Data processing/data transformation using ETL/ELT tools, SnowPipe, dbt, or Databricks
• Experience with Cloud Data Warehouse solutions experience (Snowflake, Azure DW, Redshift or similar technology in other private or public clouds).
• Complete software development lifecycle experience including design, documentation, implementation, testing, and deployment
  
  

Basic Qualifications

• Bachelor's Degree in a computer-related field or equivalent professional experience required
• At least 2 years of experience in data engineering using open-source technology stack along with cloud computing (AWS, Microsoft Azure, Google Cloud)
• At least 2 years of experience with designing, developing, implementing, and maintaining solutions for data ingestion and transformation projects with dbt, SnowPipe, or DataBricks
• At least 2 years of advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with cloud databases
  
  

Preferred Qualifications

• 3+ years of experience with dbt, SnowPipe, or DataBricks
• 3+ years of experience working on real-time data and streaming applications (Spark Streaming or Kafka)
• 3+ years of experience working with Cloud Data Warehouse solutions (i.e., Snowflake, Synapse, Redshift)
• 3+ years of experience with Agile engineering practices
  
  

Rahul.B

Team Lead

SPAR Information Systems

Email: rahul@sparinfosys.com