Azure Data Engineer – Business Intelligence Operations Lead Position: Senior Manager Job Responsibilities: 1. Collaborating with stakeholders and CXOs, understanding source data, and translating new business requirements into technical solutions using Databricks / Azure Synapse and Azure Data Factory. 2. Designing, implementing, and managing end-to-end ETL/ELT processes for MIS automations, new system integrations, Streaming data ingestion using Delta Live tables 3. Datalake governance – Access Control, CI / CD governance 4. Change management – technically review of existing Data pipelines, SQL & Python code, for potential issues and deliver change request, through planning and due approval of Change Request through Change Approval Borad 5. Estimation and delivery of adhoc and scheduled data pipelines processing high volume structured and unstructured data. 6. Setup Proactive ETL Monitoring and troubleshooting ETL issues would be active parts the role. 7. Ensuring compliance with data governance and data security/masking standards for data delivery 8. Ensure developments follow standard coding patterns, are fully documented for audit submissions. Competencies for the job 1) Proven experience with big data solution design and development in Databricks, notebooks & schema design, development, best practice and notebooks Azure Dev Ops / CI-CD Pipelines 2) Hands On in Python PySpark, Spark SQL, Delta Live + Kafka; Azure SQL DB, Azure Data Factory, Azure DataBricks, Azure Synapse, Azure Data Lake, Delta, Pyspark, Python, Logic Apps, Azure DevOps, CI/CD implementation, Power BI / QlikSense, Blob Storage, ADLS, Azure Key Vault, ETL, SSIS 3) Experience in Query Development, Performance Tuning and loading data to Databricks SQL DW 4) Experience in data ingestion into ADLS, Azure Blob Storage, Azure Logic Apps 5) Practical knowledge of data warehouse implementation – denormalization, conformance, fact, dimensions Desired Experience: 1) 8 to 10 years of experience in Azure Data & Analytics core skills. 2) Experience of Senior Databricks IC / Lead role 3) Excellent communication skills in written and verbal English 4) Knowledge of core functions of a large NBFC/Bank – Sales, Credit, Collections, Finance and Accounts