Skip to Content

Wiki source code of Bronze Lakehouse

Version 2.1 by Dimitri Rupp on 2026/06/11 09:16
Show last authors
1 {{box cssClass="floatinginfobox" title="**On this page**"}}
2 {{toc/}}
3 {{/box}}
4
5 = Bronze Lakehouse =
6
7 {{info}}
8 **Medallion-Stage:** //Bronze — Raw Ingestion Layer//
9 **Quality goal:** As close to the source as possible
10 {{/info}}
11
12 == Overview ==
13
14 The **Bronze Lakehouse** layer contains data from various source systems with the data quality matching as close to the source as possible. Data is stored in **distinct Lakehouses depending on the source system**. Depending on the source, the orchestration method used to get data into these Lakehouses can be a (% style="background:#fff3cd;padding:1px 4px;border-radius:3px" %)Data Pipeline(%%), (% style="background:#fff3cd;padding:1px 4px;border-radius:3px" %)Gen2 Dataflow(%%) or a (% style="background:#fff3cd;padding:1px 4px;border-radius:3px" %)Synapse Notebook(%%).
15
16 == Source-to-Bronze Flow ==
17
18 The diagram below shows which source system feeds which dedicated Bronze Lakehouse, and which orchestration method is used.
19
20 {{mermaid}}
21 flowchart LR
22 subgraph SRC["**Source systems**"]
23 direction TB
24 S1[BMD<br/>ERP]
25 S2[Rimo DWH<br/>SPL Software]
26 S3[Board<br/>Planning DB]
27 S4[SharePoint<br/>Lists & Files]
28 S5[H3A<br/>Database]
29 end
30
31 subgraph BRZ["**Bronze Lakehouses**"]
32 direction TB
33 L1[(BMD Bronze)]
34 L2[(Rimo Bronze)]
35 L3[(Board Bronze)]
36 L4[(SharePoint Bronze)]
37 L5[(H3A Bronze)]
38 end
39
40 S1 -->|Data Pipeline| L1
41 S2 -->|Data Pipeline| L2
42 S3 -->|Gen2 Dataflow| L3
43 S4 -->|Gen2 Dataflow / Notebook| L4
44 S5 -->|Data Pipeline| L5
45
46 BRZ --> SIL[Silver Lakehouse<br/>cleansed & conformed]
47
48 classDef src fill:#e3f2fd,stroke:#1976d2,stroke-width:1px
49 classDef brz fill:#f4a261,stroke:#b85c00,stroke-width:1px,color:#000
50 classDef sil fill:#c0c0c0,stroke:#555,stroke-width:1px
51
52 class S1,S2,S3,S4,S5 src
53 class L1,L2,L3,L4,L5 brz
54 class SIL sil
55 {{/mermaid}}
56
57 == The five Bronze Lakehouses ==
58
59 === 🗄️ BMD ===
60
61 Data from the **BMD** database — SPL Tele's ERP system. Hosts financial postings, customer and vendor master data, and accounting transactions.
62
63 → [[BMD Bronze Lakehouse>>doc:IT-Wiki.01-Data.Ingestion.Bronze-Lakehouse.BMD-Bronze-Lakehouse.WebHome]]
64
65 === 🗄️ Rimo ===
66
67 Data from the **Rimo DWH** — SPL Tele's own software product. Provides project, resource and operational data used by Power Users.
68
69 → [[Rimo Lakehouse>>doc:IT-Wiki.01-Data.Ingestion.Bronze-Lakehouse.Rimo-Lakehouse.WebHome]]
70
71 === 🗄️ Board ===
72
73 Data from the **Board** database — used for planning and controlling. Loaded via Gen2 Dataflow.
74
75 → [[Board Bronze Lakehouse>>doc:IT-Wiki.01-Data.Ingestion.Bronze-Lakehouse.Board-Bronze-Lakehouse.WebHome]]
76
77 === 🗄️ SharePoint ===
78
79 Data from **SharePoint sites** — typically lists, libraries and Excel files maintained by business teams. Used to bring decentralised data into the Lakehouse without a dedicated source database.
80
81 → [[SharePoint Bronze Lakehouse>>doc:IT-Wiki.01-Data.Ingestion.Bronze-Lakehouse.SharePoint-Bronze-Lakehouse.WebHome]]
82
83 === 🗄️ H3A ===
84
85 Data from the **H3A** database — third-party operational system. Loaded via Data Pipeline.
86
87 → [[H3A Bronze Lakehouse>>doc:IT-Wiki.01-Data.Ingestion.Bronze-Lakehouse.H3A-Bronze-Lakehouse.WebHome]]
88
89 == Orchestration methods ==
90
91 Three methods are in use, chosen per source system:
92
93 |=Method|=Typical use|=Strength
94 |**Data Pipeline**|Database sources (BMD, Rimo, H3A)|Native Fabric orchestration; copy + transform; parameterizable
95 |**Gen2 Dataflow**|Sources reachable via Power Query (Board, SharePoint Lists)|Low-code; reuses existing Power Query knowledge
96 |**Synapse Notebook**|Complex transformations, file parsing, custom logic|Full PySpark / Python control; ideal for non-trivial SharePoint payloads
97
98 == Design principles ==
99
100 * **One Lakehouse per source system** — keeps Bronze stable and the lineage obvious.
101 * **Schema close to source** — no joins, no enrichment in Bronze; only raw or near-raw tables.
102 * **Idempotent loads** — pipelines and dataflows can be re-run without breaking history.
103 * **Append where possible, full-load where cheaper** — chosen per table based on volume and change pattern.
104
105 == Where Bronze fits in the bigger picture ==
106
107 {{info}}
108 **Bronze → Silver → Gold.** Bronze is **only** the raw landing zone. Cleansing, conforming and business-friendly modelling happens in the Silver and Gold Lakehouses.
109 {{/info}}
110
111 * [[Fabric Lakehouse Hub>>doc:IT-Wiki.01-Data.Processing.Lakehouse.WebHome]]
112 * [[Microsoft Fabric Datenarchitektur im SPL>>doc:IT-Wiki.01-Data.Microsoft-Fabric-Datenarchitektur-im-SPL.WebHome]]
113 * [[Silver Lakehouse>>doc:IT-Wiki.01-Data.Processing.Lakehouse.Fabric-Silver-Lakehouse.WebHome]]
114 * [[Gold Lakehouse>>doc:IT-Wiki.01-Data.Processing.Lakehouse.Fabric-Gold-Lakehouse.WebHome]]
115
116 ----
117
118 {{info}}
119 **Migration footer**
120 **Source:** [[Bronze Lakehouse (SharePoint)>>url:https://spltele.sharepoint.com/sites/WikiPowerBI/SitePages/Bronze-Lakehouse.aspx]]
121 **Original author:** Bukkerji Sreejith — Sreejith.Bukkerji@spl-tele.com
122 **Original last-modified:** 2025-10-17
123 **Migration date:** 2026-05-24
124 **Migrated by:** Dimitri Rupp via ClaudeAI
125 **Notes:** EN-Original preserved; decorative Hubble stock images omitted; Mermaid source-to-Bronze flow added; orchestration table extracted from prose; cross-links to all 5 sub-Lakehouses + Silver/Gold/Architektur.
126 {{/info}}
127
128
129 ----
130
131 == Verwandte Themen ==
132
133 //Kuratiert — kann direkt im Editor ergänzt werden://
134
135 * [[Fabric Data Pipelines>>doc:IT-Wiki.01-Data.Ingestion.Fabric-Data-Pipelines.WebHome]]
136 * [[Fabric Dataflows>>doc:IT-Wiki.01-Data.Ingestion.Fabric-Dataflows.WebHome]]
137 * [[Fabric Excel Ingestion>>doc:IT-Wiki.01-Data.Ingestion.Fabric-Excel-Ingestion.WebHome]]
138
139 {{include reference="IT-Wiki.Makros.Verwandte-Themen.WebHome"/}}
140
141 ----
142 **Status:** Migriert — Team-Review ausstehend · **Owner:** //(festlegen)// · **Letzter Review:** 2026-06-11