Consultant

Tasks

Good communication and presentation skills.

Strong knowledge of Azure components: Azure Data Lake, Azure Data Factory, Azure SQL, Azure Databricks.

Data Pipeline Development:

Design, build, and maintain efficient, reusable, and reliable data pipelines.
Ensure data flow between different systems and databases is seamless and error-free.

ETL Processes:

Develop and maintain ETL (Extract, Transform, Load) processes to aggregate data from various sources.
Ensure data quality and integrity throughout the ETL process.

Performance Optimization:

Optimize data pipelines and queries for performance.
Identify and resolve bottlenecks in data processing workflows.

Qualifications

Experience: 2 years or above.

Qualification: B.Tech, MTech.

Must have strong knowledge and hands-on experience in Apache Spark and Python programming, working with Delta Tables. Experience in Databricks is a must.

Strong SQL skills: Data modeling, developing SQL stored procedures, functions, dynamic SQL queries, joins.

Must be aware of development of components for data fetching from APIs.

Strong knowledge of data warehousing concepts.

Hands-on experience in ingesting data from various data sources and data types and file types.

Good knowledge of development lifecycle, best practices, and coding standards. Should be able to help team members and review their code.

Good knowledge of Azure DevOps and understanding of build and release pipelines.