Practical CI/CD with Terraform, Fabric CLI and fabric-cicd

· updated · post terraform cli microsoft-fabric

meeting data architecture

TL;DR

Automating Microsoft Fabric deployments with Fabric CLI and fabric-cicd

If your Microsoft Fabric deployment process still involves screenshots, checklists, and “did you remember to update that connection?” messages, this one’s for you.

The Fabric CLI and the fabric-cicd library give you a code-first, automatable way to manage Fabric workspaces, artifacts, and environment promotions—without hand-rolling calls against half a dozen APIs or relying purely on the UI.

This article walks through:

Audience: Data engineers, analytics engineers, platform/DevOps teams, and Fabric admins who like things repeatable.

The problem: Fabric without automation

Typical anti-patterns you’ll recognize:

This does not:

You want:

That’s where Fabric CLI and fabric-cicd come in.

Fabric CLI in a nutshell

Fabric CLI (fab) is a command-line interface for Microsoft Fabric that treats Fabric like a file system and makes it scriptable.

Key ideas:

Why you should care:

Install:

pip install --upgrade ms-fabric-cli
fab auth login
fab ls

fabric-cicd in a nutshell

fabric-cicd is a Python library for code-first CI/CD with Microsoft Fabric.

Its job:

Core expectations (from the project docs):

Supported item types (selected examples, evolving over time):

Why it exists:

Install:

pip install fabric-cicd

Why use Fabric CLI and fabric-cicd together?

Short version: Fabric CLI is your control surface; fabric-cicd is your deployment engine.

Together they let you:

Benefits:

Now let’s make this concrete.

Practical example: DEV → PROD with GitHub Actions

Goal:

We’ll cover:

Example repository structure

Imagine a repo like this:

/.
├─ fab_workspace/
│  ├─ Notebooks/
│  │  ├─ IngestSales.Notebook
│  │  └─ TransformSales.Notebook
│  ├─ DataPipelines/
│  │  └─ SalesPipeline.DataPipeline
│  ├─ Lakehouse/
│  │  └─ SalesLakehouse.Lakehouse
│  ├─ Reports/
│  │  └─ ExecutiveSales.Report
│  └─ parameter.yml
└─ deploy_fabric.py

Example parameter.yml

This file lets you map environment keys like DEV, PPE, PROD to different values.

find_replace:
  # Replace a dev Lakehouse ID used in notebooks with environment-specific IDs.
  - find_value: "11111111-1111-1111-1111-111111111111"  # DEV Lakehouse ID
    replace_value:
      DEV: "11111111-1111-1111-1111-111111111111"
      PPE: "22222222-2222-2222-2222-222222222222"
      PROD: "33333333-3333-3333-3333-333333333333"
    item_type: "Notebook"

key_value_replace:
  # Replace a JSON property that stores environment names in pipelines.
  - find_key: $.variables[?(@.name=="Environment")].value
    replace_value:
      DEV: "DEV"
      PPE: "PPE"
      PROD: "PROD"

spark_pool:
  # Example Spark pool differences between PPE and PROD.
  - instance_pool_id: "dev-pool-instance-id"
    replace_value:
      PPE:
        type: "Capacity"
        name: "PPE-SparkPool"
      PROD:
        type: "Capacity"
        name: "PROD-SparkPool"

Notes:

Python deployment script (deploy_fabric.py)

This script:

from fabric_cicd import (
    FabricWorkspace,
    publish_all_items,
    unpublish_all_orphan_items,
)


def get_workspace_config(env: str) -> dict:
    # In reality, read from env vars or a config file
    config_map = {
        "DEV": {
            "workspace_id": "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa",
        },
        "PPE": {
            "workspace_id": "bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb",
        },
        "PROD": {
            "workspace_id": "cccccccc-cccc-cccc-cccc-cccccccccccc",
        },
    }

    if env not in config_map:
        raise ValueError(f"Unsupported environment: {env}")

    return config_map[env]


def main() -> None:
    import os
    import sys

    env = os.environ.get("FABRIC_ENV", "DEV")
    repo_dir = os.environ.get("FABRIC_REPO_DIR", "./fab_workspace")
    delete_orphans = os.environ.get("DELETE_ORPHANS", "false").lower() == "true"

    cfg = get_workspace_config(env)

    workspace = FabricWorkspace(
        workspace_id=cfg["workspace_id"],
        environment=env,
        repository_directory=repo_dir,
        item_type_in_scope=[
            "Notebook",
            "DataPipeline",
            "Lakehouse",
            "Report",
            "SemanticModel",
        ],
    )

    print(f"Deploying to environment={env}, workspace={cfg['workspace_id']}")

    publish_all_items(workspace)

    if delete_orphans:
        print("Unpublishing orphan items not found in repo...")
        unpublish_all_orphan_items(workspace)

    print("Deployment completed successfully.")


if __name__ == "__main__":
    try:
        main()
    except Exception as exc:  # noqa: BLE001
        print(f"Deployment failed: {exc}")
        sys.exit(1)

Key points:

GitHub Actions pipeline using Fabric CLI + fabric-cicd

Now let’s tie it together.

What this job will do:

Create .github/workflows/fabric-deploy-prod.yml:

name: Deploy Fabric to PROD

on:
  push:
    branches:
      - main

jobs:
  deploy-fabric-prod:
    runs-on: ubuntu-latest

    permissions:
      id-token: write
      contents: read

    env:
      FABRIC_ENV: PROD
      FABRIC_REPO_DIR: ./fab_workspace
      DELETE_ORPHANS: "true"

    steps:
      - name: Checkout
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.11"

      - name: Install dependencies
        run: |
          pip install --upgrade ms-fabric-cli fabric-cicd

      - name: Azure login (Service Principal)
        uses: azure/login@v2
        with:
          client-id: ${{ secrets.AZURE_CLIENT_ID }}
          tenant-id: ${{ secrets.AZURE_TENANT_ID }}
          client-secret: ${{ secrets.AZURE_CLIENT_SECRET }}

      - name: Fabric CLI Auth using Service Principal
        run: |
          fab auth login \
            -u "${{ secrets.AZURE_CLIENT_ID }}" \
            -p "${{ secrets.AZURE_CLIENT_SECRET }}" \
            --tenant "${{ secrets.AZURE_TENANT_ID }}"

      - name: Sanity check - list Fabric workspaces
        run: |
          fab ls

      - name: Deploy to PROD workspace using fabric-cicd
        run: |
          python deploy_fabric.py

Notes:

When should you adopt this pattern?

You should strongly consider Fabric CLI + fabric-cicd if:

You might stick to built-in UI-only flows if:

But in most serious setups: code-first wins quickly.

Practical tips and gotchas

A few opinionated best practices:

Wrap-up

Fabric CLI and fabric-cicd give you:

If your Fabric workloads are becoming critical, they deserve real CI/CD.

If you tell me your current setup (Git provider, CI system, how you structure Fabric workspaces), I can tailor this example into a drop-in template for your environment.