Fabric Data Pipelines

Last modified by Dimitri Rupp on 2026/06/11 09:30

On this page

Fabric Data Pipelines

Fabric Data Pipelines

What: Strukturierte Sequenz von Schritten, die Daten von einem System ins andere bewegt — transformiert, validiert, anreichert.

Overview

A data pipeline is a structured sequence of steps that moves data from one system to another — often transforming, validating, or enriching it along the way.

In Microsoft Fabric, a data pipeline typically:

Connects to a source system (database, API, file store)
Extracts data using queries or connectors
Optionally transforms the data
Loads it into a target (Lakehouse, Warehouse, etc.)
Can be scheduled or triggered by events / other pipelines

Types of pipelines

Based on the purpose, data pipelines can be classified as:

Type	Function
Ingestion pipeline	Brings raw data into a central platform or storage layer (Data Lake / Warehouse / Lakehouse). Often involves connecting to source systems, extracting data, and landing it as-is (Bronze).
Orchestration pipeline	Coordinates downstream activities — data transformation, validation, enrichment, or triggering analytics workflows. May involve chaining multiple pipelines or notebooks. Also called execution pipelines in some contexts.

flowchart LR
    subgraph SRC[Sources]
        S1[(BMD DWH)]
        S2[(Rimo DWH)]
        S3[(H3A DWH)]
    end

    subgraph INGEST[Ingestion pipelines]
        I1[getBMDDWH_BMD]
        I2[getRimoDWH]
        I3[geth3aDWH]
    end

    subgraph BRONZE[Bronze Lakehouses]
        B1[(bmd_bronze_lh)]
        B2[(rimo_bronze_lh)]
        B3[(h3a_bronze_lh)]
    end

    subgraph ORCH[Orchestration pipelines]
        O1[0445_Daily_bmd
04:35]
        O2[0335_Daily_rimo_new
03:35]
        O3[0400_Daily_h3a
04:00]
    end

    S1 --> I1 --> B1
    S2 --> I2 --> B2
    S3 --> I3 --> B3
    O1 --> I1
    O2 --> I2
    O3 --> I3
    O1 -.->|notebooks| GOLD[spl_gold_lh]
    O2 -.->|notebooks| GOLD
    O3 -.->|notebooks| GOLD

    classDef src fill:#e3f2fd,stroke:#1976d2,stroke-width:1px
    classDef ing fill:#fff3cd,stroke:#b85c00,stroke-width:1px
    classDef brz fill:#f4a261,stroke:#b85c00,stroke-width:1px,color:#000
    classDef orc fill:#bbdefb,stroke:#1976d2,stroke-width:1px
    classDef gold fill:#ffd54f,stroke:#b45f06,stroke-width:1px,color:#000

    class S1,S2,S3 src
    class I1,I2,I3 ing
    class B1,B2,B3 brz
    class O1,O2,O3 orc
    class GOLD gold

List of all data pipelines in telefabric

Shows all the pipelines across the Fabric and Fabric_Dev workspaces under telefabric capacity. The Location column indicates Workspace name / {folder name}.

Name of pipeline	Type	📂 Location	Purpose
0335_Daily_rimo_new	Orchestration	Fabric/00_Execution	Executes all data-movement activities of Rimo tables from on-premise DWH to the gold layer. Includes ingestion pipeline and notebooks.
0400_Daily_h3a	Orchestration	Fabric/00_Execution	Executes all data-movement activities of h3a tables from on-premise DWH to the gold layer. Includes ingestion pipeline and notebooks.
0445_Daily_bmd	Orchestration	Fabric/00_Execution	Executes all data-movement activities of bmd tables from on-premise DWH to the gold layer. Includes ingestion pipeline and notebooks.
getRimoDWH	Ingestion	Fabric/01_Dataingestion	Copy tables from Rimo DWH to rimo_bronze_lh
geth3aDWH	Ingestion	Fabric/01_Dataingestion	Copy tables from h3a DWH to h3a_bronze_lh
getBMDDWH_BMD	Ingestion	Fabric/01_Dataingestion	Copy tables from BMD DWH to bmd_bronze_lh
test	-	Fabric/archived_items	Pipeline for learning and testing
dp_rimo_dev	-	Fabric_Dev	Data pipeline for developmental activities

Naming pattern

HHMM_Daily_<source> — Orchestration pipeline, daily run at the indicated time (04:35 = 04:35 AM)
get<Source>DWH — Ingestion pipeline for that source

Change log

Date	Description	Person
25.11.2025	Changed the geth3aDWH pipeline documentation	Sreejith

Fabric Data Pipelines

Fabric Data Pipelines

Overview

Types of pipelines

List of all data pipelines in telefabric

Naming pattern

Change log

Related pages

Verwandte Themen