Data Engineer (Fabric)

Sucafina

1. Data Platform Engineering (Fabric)

• Build and manage Lakehouses (Delta Lake) and Fabric Data Warehouses.
• Develop Data Pipelines and Dataflows Gen2 for batch and near-real-time ingestion.
• Create and optimize Notebook-based transformations (PySpark/SQL) and SQL stored procedures for DW workloads.
• Implement medallion architecture (bronze/silver/gold) for scalable curation.
• Publish certified semantic models and Power BI datasets aligned to business domains.

2. Performance & Reliability

• Optimize storage/compute in OneLake (file formats, partitioning, z-ordering).
• Tune Spark and SQL workloads (caching strategies, concurrency, workload isolation).
• Implement robust retry, alerting, and monitoring (Fabric Monitoring Hub, Metrics app).
• Conduct end-to-end pipeline performance testing and scalability assessments.

3. Governance, Security & Compliance

• Enforce data governance with sensitivity labels, row-level/column-level security, and workspace roles.
• Manage item-level permissions (Lakehouse tables, DW schemas, datasets) and Managed Identities for sources.
• Apply data quality rules, lineage, and documentation (Descriptions, Tags, Owner metadata; Purview if applicable).
• Ensure compliance with organizational standards (PII handling, audit, retention).

4. DevOps & Lifecycle Management

• Use Fabric Git integration and Deployment Pipelines for CI/CD across dev/test/prod.
• Parameterize pipelines and environments; externalize configuration and secrets (Key Vault).
• Implement automated testing for data transformations and schemas.
• Drive release management, change control, and rollback strategies.

5. Collaboration & Stakeholder Engagement

• Partner with analytics engineers and BI teams to design star schemas, semantic models, and DAX measures.
• Work with data source owners for SLAs, schema change management, and contracts.
• Translate business requirements into technical designs and document architecture decisions.
• Provide knowledge transfer, best practices, and support to data consumers.