Wiki source code of Rimo Lakehouse

Version 2.1 by Dimitri Rupp on 2026/06/11 09:16

Show last authors
1 {{box cssClass="floatinginfobox" title="**On this page**"}}
2 {{toc/}}
3 {{/box}}
4
5 = Rimo Lakehouse =
6
7 {{info}}
8 **Source system:** Rimo — SPL Tele's own software product (DWH-Layer)
9 **Stage:** Bronze (raw landing zone)
10 **Owner:** Fabric team (restricted access)
11 {{/info}}
12
13 == Overview ==
14
15 |=(% style="width:80px" %)📋 **What**|Tables from the **Rimo Data Warehouse**. Acts as the **landing zone** for all Rimo-related data. No transformations are carried out here.
16 |=(% style="width:80px" %)🔄 **How**|Data is copied from the Rimo DWH via **Fabric Data Pipelines**.
17 |=(% style="width:80px" %)🔐 **Who**|Access is **restricted to the Fabric team**.
18
19 == How data is ingested ==
20
21 Data from the Rimo source system is ingested into the (% style="font-family:monospace" %)rimo_bronze_lh(%%) Lakehouse, following SPL's bronze-layer convention of preserving raw data with minimal intervention. As with all bronze Lakehouses, **no transformations are applied during ingestion** — ensuring that the data reflects the source system as closely as possible.
22
23 However, due to **type incompatibilities between the source schema and Lakehouse constraints**, certain columns may be stored using alternate data types. These adjustments are system-driven and occur by default to maintain ingestion continuity.
24
25 To ensure pipeline resilience, the orchestration process is designed to **skip non-compatible rows** rather than fail the entire ingestion job. This prevents a single malformed record from disrupting ingestion across multiple tables.
26
27 Whenever a row is skipped due to schema or type mismatches, the event is **logged in a dedicated folder** within the (% style="font-family:monospace" %)fabric-logs(%%) directory of the (% style="font-family:monospace" %)**northlakepmo2**(%%) Azure Data Lake Storage account. These logs provide traceability for data quality issues and support downstream reconciliation and monitoring.
28
29 {{mermaid}}
30 flowchart LR
31 RIMO[(Rimo DWH<br/>source system)] -->|Fabric Data Pipeline| LH[(rimo_bronze_lh<br/>Bronze Lakehouse)]
32 LH -->|consumed by| SIL[Silver Lakehouse]
33 LH -.->|skipped rows| LOGS[(northlakepmo2<br/>fabric-logs)]
34
35 classDef src fill:#e3f2fd,stroke:#1976d2,stroke-width:1px
36 classDef brz fill:#f4a261,stroke:#b85c00,stroke-width:1px,color:#000
37 classDef sil fill:#c0c0c0,stroke:#555,stroke-width:1px
38 classDef log fill:#fff3cd,stroke:#b85c00,stroke-width:1px
39
40 class RIMO src
41 class LH brz
42 class SIL sil
43 class LOGS log
44 {{/mermaid}}
45
46 == Naming convention ==
47
48 When the (% style="font-family:monospace" %)rimo_bronze_lh(%%) Lakehouse was provisioned, Microsoft Fabric had not yet introduced support for **custom schemas within Lakehouses**. At that time, all tables were automatically assigned to the default (% style="font-family:monospace" %)**dbo schema**(%%), regardless of their source schema in the Rimo Data Warehouse.
49
50 As a result, tables originating from Rimo's (% style="font-family:monospace" %)**public**(%%), (% style="font-family:monospace" %)**dim**(%%), and **custom schemas** were all ingested into the Lakehouse under the (% style="font-family:monospace" %)dbo(%%) schema. To preserve schema lineage and avoid ambiguity, a **prefix-based naming convention** was adopted.
51
52 For example:
53
54 * (% style="font-family:monospace" %)public_sdmsalesorder(%%)
55 * (% style="font-family:monospace" %)dim_sdmperson_historic(%%)
56 * (% style="font-family:monospace" %)custom_sdmperson_fte(%%)
57
58 This approach ensures that:
59
60 * Source-schema context is **retained** despite the schema flattening.
61 * Governance and reconciliation processes remain **schema-aware**.
62
63 {{warning}}
64 **Hinweis zu Tabellen-Namen:** Einige Eintraege weichen vom (% style="font-family:monospace" %)sdm*(%%)-Muster ab (z.B. (% style="font-family:monospace" %)sdibsssalesordergroup(%%)). Diese sind **1:1 wie im Quellsystem** uebernommen und werden erst nach Ruecksprache mit dem Fabric-Team angepasst.
65 {{/warning}}
66
67 == Schema overview ==
68
69 The 94 tables in (% style="font-family:monospace" %)rimo_bronze_lh(%%) come from three origin schemas:
70
71 |=Prefix|=Origin schema|=Count (≈)|=Typical content
72 |(% style="font-family:monospace" %)custom_*(%%)|Rimo (% style="font-family:monospace" %)custom(%%) schema|2|Customer-specific calculations (FTE, progress logic)
73 |(% style="font-family:monospace" %)dim_*(%%)|Rimo (% style="font-family:monospace" %)dim(%%) schema|1|Historic dimension (person history)
74 |(% style="font-family:monospace" %)public_*(%%)|Rimo (% style="font-family:monospace" %)public(%%) schema|91|Operational core: SDM/BSS, persons, projects, materials, travel, finance, storage, time tracking
75
76 == List of tables in rimo_bronze_lh ==
77
78 |=Source table in Rimo|=Table name in rimo_bronze_lh
79 |(% style="font-family:monospace" %)custom.sdmmilestoneitem_progress_manuell_diff_newlogic(%%)|(% style="font-family:monospace" %)custom_sdmmilestoneitem_progress_manuell_diff_newlogic(%%)
80 |(% style="font-family:monospace" %)custom.sdmperson_fte(%%)|(% style="font-family:monospace" %)custom_sdmperson_fte(%%)
81 |(% style="font-family:monospace" %)dim.sdmperson_historic(%%)|(% style="font-family:monospace" %)dim_sdmperson_historic(%%)
82 |(% style="font-family:monospace" %)public.otbpersonalstorage(%%)|(% style="font-family:monospace" %)public_otbpersonalstorage(%%)
83 |(% style="font-family:monospace" %)public.otbrepairstorage(%%)|(% style="font-family:monospace" %)public_otbrepairstorage(%%)
84 |(% style="font-family:monospace" %)public.otbstoragedepot(%%)|(% style="font-family:monospace" %)public_otbstoragedepot(%%)
85 |(% style="font-family:monospace" %)public.otbstoragespace(%%)|(% style="font-family:monospace" %)public_otbstoragespace(%%)
86 |(% style="font-family:monospace" %)public.sdmactivity(%%)|(% style="font-family:monospace" %)public_sdmactivity(%%)
87 |(% style="font-family:monospace" %)public.sdmaddresslocation(%%)|(% style="font-family:monospace" %)public_sdmaddresslocation(%%)
88 |(% style="font-family:monospace" %)public.sdmchangerequestcm(%%)|(% style="font-family:monospace" %)public_sdmchangerequestcm(%%)
89 |(% style="font-family:monospace" %)public.sdmcompany(%%)|(% style="font-family:monospace" %)public_sdmcompany(%%)
90 |(% style="font-family:monospace" %)public.sdmcreditor(%%)|(% style="font-family:monospace" %)public_sdmcreditor(%%)
91 |(% style="font-family:monospace" %)public.sdmcurrency(%%)|(% style="font-family:monospace" %)public_sdmcurrency(%%)
92 |(% style="font-family:monospace" %)public.sdmcustomerpurchaseorder(%%)|(% style="font-family:monospace" %)public_sdmcustomerpurchaseorder(%%)
93 |(% style="font-family:monospace" %)public.sdmcustomersla(%%)|(% style="font-family:monospace" %)public_sdmcustomersla(%%)
94 |(% style="font-family:monospace" %)public.sdmdebitor(%%)|(% style="font-family:monospace" %)public_sdmdebitor(%%)
95 |(% style="font-family:monospace" %)public.sdmdynamictenantattributedescription(%%)|(% style="font-family:monospace" %)public_sdmdynamictenantattributedescription(%%)
96 |(% style="font-family:monospace" %)public.sdmfault(%%)|(% style="font-family:monospace" %)public_sdmfault(%%)
97 |(% style="font-family:monospace" %)public.sdmfaultteam(%%)|(% style="font-family:monospace" %)public_sdmfaultteam(%%)
98 |(% style="font-family:monospace" %)public.sdmhotellocation(%%)|(% style="font-family:monospace" %)public_sdmhotellocation(%%)
99 |(% style="font-family:monospace" %)public.sdminboundstockorder(%%)|(% style="font-family:monospace" %)public_sdminboundstockorder(%%)
100 |(% style="font-family:monospace" %)public.sdmincident(%%)|(% style="font-family:monospace" %)public_sdmincident(%%)
101 |(% style="font-family:monospace" %)public.sdminvoice(%%)|(% style="font-family:monospace" %)public_sdminvoice(%%)
102 |(% style="font-family:monospace" %)public.sdminvoiceapproval(%%)|(% style="font-family:monospace" %)public_sdminvoiceapproval(%%)
103 |(% style="font-family:monospace" %)public.sdminvoiceassignment(%%)|(% style="font-family:monospace" %)public_sdminvoiceassignment(%%)
104 |(% style="font-family:monospace" %)public.sdmlistelement(%%)|(% style="font-family:monospace" %)public_sdmlistelement(%%)
105 |(% style="font-family:monospace" %)public.sdmlogisticscanitem(%%)|(% style="font-family:monospace" %)public_sdmlogisticscanitem(%%)
106 |(% style="font-family:monospace" %)public.sdmmasteritem(%%)|(% style="font-family:monospace" %)public_sdmmasteritem(%%)
107 |(% style="font-family:monospace" %)public.sdmmasteritemcategory(%%)|(% style="font-family:monospace" %)public_sdmmasteritemcategory(%%)
108 |(% style="font-family:monospace" %)public.sdmmastertool(%%)|(% style="font-family:monospace" %)public_sdmmastertool(%%)
109 |(% style="font-family:monospace" %)public.sdmmaterial(%%)|(% style="font-family:monospace" %)public_sdmmaterial(%%)
110 |(% style="font-family:monospace" %)public.sdmmaterialchangeorder(%%)|(% style="font-family:monospace" %)public_sdmmaterialchangeorder(%%)
111 |(% style="font-family:monospace" %)public.sdmmanufacturer(%%)|(% style="font-family:monospace" %)public_sdmmanufacturer(%%)
112 |(% style="font-family:monospace" %)public.sdmmilestone(%%)|(% style="font-family:monospace" %)public_sdmmilestone(%%)
113 |(% style="font-family:monospace" %)public.sdmmilestoneitem(%%)|(% style="font-family:monospace" %)public_sdmmilestoneitem(%%)
114 |(% style="font-family:monospace" %)public.sdmmonth(%%)|(% style="font-family:monospace" %)public_sdmmonth(%%)
115 |(% style="font-family:monospace" %)public.sdmofferitempoolmg(%%)|(% style="font-family:monospace" %)public_sdmofferitempoolmg(%%)
116 |(% style="font-family:monospace" %)public.sdmofferitempoolsg1(%%)|(% style="font-family:monospace" %)public_sdmofferitempoolsg1(%%)
117 |(% style="font-family:monospace" %)public.sdmofferitempoolsg2(%%)|(% style="font-family:monospace" %)public_sdmofferitempoolsg2(%%)
118 |(% style="font-family:monospace" %)public.sdmofferitemservice(%%)|(% style="font-family:monospace" %)public_sdmofferitemservice(%%)
119 |(% style="font-family:monospace" %)public.sdmofferitemservicetemplate(%%)|(% style="font-family:monospace" %)public_sdmofferitemservicetemplate(%%)
120 |(% style="font-family:monospace" %)public.sdmoffdutyentry(%%)|(% style="font-family:monospace" %)public_sdmoffdutyentry(%%)
121 |(% style="font-family:monospace" %)public.sdmondutyentry(%%)|(% style="font-family:monospace" %)public_sdmondutyentry(%%)
122 |(% style="font-family:monospace" %)public.sdmpaymentterm(%%)|(% style="font-family:monospace" %)public_sdmpaymentterm(%%)
123 |(% style="font-family:monospace" %)public.sdmperson(%%)|(% style="font-family:monospace" %)public_sdmperson(%%)
124 |(% style="font-family:monospace" %)public.sdmprocesscontainer(%%)|(% style="font-family:monospace" %)public_sdmprocesscontainer(%%)
125 |(% style="font-family:monospace" %)public.sdmproductivizationentry(%%)|(% style="font-family:monospace" %)public_sdmproductivizationentry(%%)
126 |(% style="font-family:monospace" %)public.sdmproductivizationmonth(%%)|(% style="font-family:monospace" %)public_sdmproductivizationmonth(%%)
127 |(% style="font-family:monospace" %)public.sdmproductticket(%%)|(% style="font-family:monospace" %)public_sdmproductticket(%%)
128 |(% style="font-family:monospace" %)public.sdmprojectpurchaseorder(%%)|(% style="font-family:monospace" %)public_sdmprojectpurchaseorder(%%)
129 |(% style="font-family:monospace" %)public.sdmprojectpurchaseorderitem(%%)|(% style="font-family:monospace" %)public_sdmprojectpurchaseorderitem(%%)
130 |(% style="font-family:monospace" %)public.sdmprojectpurchaseorderitemtemplate(%%)|(% style="font-family:monospace" %)public_sdmprojectpurchaseorderitemtemplate(%%)
131 |(% style="font-family:monospace" %)public.sdmprovisioncalculationitem(%%)|(% style="font-family:monospace" %)public_sdmprovisioncalculationitem(%%)
132 |(% style="font-family:monospace" %)public.sdmresourcelink(%%)|(% style="font-family:monospace" %)public_sdmresourcelink(%%)
133 |(% style="font-family:monospace" %)public.sdmrootcause(%%)|(% style="font-family:monospace" %)public_sdmrootcause(%%)
134 |(% style="font-family:monospace" %)public.sdmsalaryabsenttype(%%)|(% style="font-family:monospace" %)public_sdmsalaryabsenttype(%%)
135 |(% style="font-family:monospace" %)public.sdmsalarynegativetype(%%)|(% style="font-family:monospace" %)public_sdmsalarynegativetype(%%)
136 |(% style="font-family:monospace" %)public.sdmsalarytype(%%)|(% style="font-family:monospace" %)public_sdmsalarytype(%%)
137 |(% style="font-family:monospace" %)public.sdmsalesinvoice(%%)|(% style="font-family:monospace" %)public_sdmsalesinvoice(%%)
138 |(% style="font-family:monospace" %)public.sdmsalesorder(%%)|(% style="font-family:monospace" %)public_sdmsalesorder(%%)
139 |(% style="font-family:monospace" %)public.sdibsssalesordergroup(%%)|(% style="font-family:monospace" %)public_sdibsssalesordergroup(%%)
140 |(% style="font-family:monospace" %)public.sdmserviceitempool(%%)|(% style="font-family:monospace" %)public_sdmserviceitempool(%%)
141 |(% style="font-family:monospace" %)public.sdmserviceorder(%%)|(% style="font-family:monospace" %)public_sdmserviceorder(%%)
142 |(% style="font-family:monospace" %)public.sdmslareason(%%)|(% style="font-family:monospace" %)public_sdmslareason(%%)
143 |(% style="font-family:monospace" %)public.sdmsubticket(%%)|(% style="font-family:monospace" %)public_sdmsubticket(%%)
144 |(% style="font-family:monospace" %)public.sdmteam(%%)|(% style="font-family:monospace" %)public_sdmteam(%%)
145 |(% style="font-family:monospace" %)public.sdmtool(%%)|(% style="font-family:monospace" %)public_sdmtool(%%)
146 |(% style="font-family:monospace" %)public.sdmtravelflightitem(%%)|(% style="font-family:monospace" %)public_sdmtravelflightitem(%%)
147 |(% style="font-family:monospace" %)public.sdmtravelaccommodationitem(%%)|(% style="font-family:monospace" %)public_sdmtravelaccommodationitem(%%)
148 |(% style="font-family:monospace" %)public.sdmtravelcaritem(%%)|(% style="font-family:monospace" %)public_sdmtravelcaritem(%%)
149 |(% style="font-family:monospace" %)public.sdmtravelorder(%%)|(% style="font-family:monospace" %)public_sdmtravelorder(%%)
150 |(% style="font-family:monospace" %)public.sdmtravelreceipt(%%)|(% style="font-family:monospace" %)public_sdmtravelreceipt(%%)
151 |(% style="font-family:monospace" %)public.sdmtravelrequest(%%)|(% style="font-family:monospace" %)public_sdmtravelrequest(%%)
152 |(% style="font-family:monospace" %)public.sdmtraveltrainitem(%%)|(% style="font-family:monospace" %)public_sdmtraveltrainitem(%%)
153 |(% style="font-family:monospace" %)public.sdmwtrproject(%%)|(% style="font-family:monospace" %)public_sdmwtrproject(%%)
154 |(% style="font-family:monospace" %)public.sdmwtrprojectgroup(%%)|(% style="font-family:monospace" %)public_sdmwtrprojectgroup(%%)
155 |(% style="font-family:monospace" %)public.sdmworkorder(%%)|(% style="font-family:monospace" %)public_sdmworkorder(%%)
156 |(% style="font-family:monospace" %)public.sdmworkpackagetype(%%)|(% style="font-family:monospace" %)public_sdmworkpackagetype(%%)
157 |(% style="font-family:monospace" %)public.splallowanceentry(%%)|(% style="font-family:monospace" %)public_splallowanceentry(%%)
158 |(% style="font-family:monospace" %)public.splworkentry(%%)|(% style="font-family:monospace" %)public_splworkentry(%%)
159 |(% style="font-family:monospace" %)public.sdmbsscartitem(%%)|(% style="font-family:monospace" %)public_sdmbsscartitem(%%)
160 |(% style="font-family:monospace" %)public.sdmbssdemandorder(%%)|(% style="font-family:monospace" %)public_sdmbssdemandorder(%%)
161 |(% style="font-family:monospace" %)public.sdmbssmanualaccountingorder(%%)|(% style="font-family:monospace" %)public_sdmbssmanualaccountingorder(%%)
162 |(% style="font-family:monospace" %)public.sdmbssmaterialaccountingorder(%%)|(% style="font-family:monospace" %)public_sdmbssmaterialaccountingorder(%%)
163 |(% style="font-family:monospace" %)public.sdmbssoutboundstockorder(%%)|(% style="font-family:monospace" %)public_sdmbssoutboundstockorder(%%)
164 |(% style="font-family:monospace" %)public.sdmbssprojectcontrollingitem(%%)|(% style="font-family:monospace" %)public_sdmbssprojectcontrollingitem(%%)
165 |(% style="font-family:monospace" %)public.sdmbssreturnorder(%%)|(% style="font-family:monospace" %)public_sdmbssreturnorder(%%)
166 |(% style="font-family:monospace" %)public.sdmbssstockquerymanager(%%)|(% style="font-family:monospace" %)public_sdmbssstockquerymanager(%%)
167 |(% style="font-family:monospace" %)public.sdmbsstravelaccountingorder(%%)|(% style="font-family:monospace" %)public_sdmbsstravelaccountingorder(%%)
168 |(% style="font-family:monospace" %)public.sdmbssworkpackage(%%)|(% style="font-family:monospace" %)public_sdmbssworkpackage(%%)
169 |(% style="font-family:monospace" %)public.sdmbsswpmilestone(%%)|(% style="font-family:monospace" %)public_sdmbsswpmilestone(%%)
170 |(% style="font-family:monospace" %)public.sdmstatus(%%)|(% style="font-family:monospace" %)public_sdmstatus(%%)
171 |(% style="font-family:monospace" %)public.sdmstandardproductioncostgroup(%%)|(% style="font-family:monospace" %)public_sdmstandardproductioncostgroup(%%)
172 |(% style="font-family:monospace" %)public.sdmstandardproductioncost(%%)|(% style="font-family:monospace" %)public_sdmstandardproductioncost(%%)
173
174 == Operational notes ==
175
176 * **No transformations:** Tables in Bronze stay as close to source as Fabric allows. Cleansing happens in Silver.
177 * **Resilience:** Single non-compatible rows are **skipped, not failed**. The job keeps running.
178 * **Traceability:** Skipped rows are logged in (% style="font-family:monospace" %)northlakepmo2 → fabric-logs(%%) for downstream reconciliation.
179 * **Access:** Restricted to the Fabric team. Power-BI-Reports consume from Silver/Gold, not directly from Bronze.
180
181 == Related pages ==
182
183 * [[Bronze Lakehouse (Hub)>>doc:IT-Wiki.01-Data.Ingestion.Bronze-Lakehouse.WebHome]]
184 * [[Rimo Daten fuer Power User>>doc:WikiPowerBI.Rimo-Daten-fuer-Power-User.WebHome]]
185 * [[Fabric Lakehouse Hub>>doc:IT-Wiki.01-Data.Processing.Lakehouse.WebHome]]
186 * [[Microsoft Fabric Datenarchitektur im SPL>>doc:IT-Wiki.01-Data.Microsoft-Fabric-Datenarchitektur-im-SPL.WebHome]]
187 * [[Silver Lakehouse>>doc:IT-Wiki.01-Data.Processing.Lakehouse.Fabric-Silver-Lakehouse.WebHome]]
188 * [[Gold Lakehouse>>doc:IT-Wiki.01-Data.Processing.Lakehouse.Fabric-Gold-Lakehouse.WebHome]]
189
190 ----
191
192 {{info}}
193 **Migration footer**
194 **Source:** [[Rimo Lakehouse (SharePoint)>>url:https://spltele.sharepoint.com/sites/WikiPowerBI/SitePages/Rimo-Lakehouse.aspx]]
195 **Original author:** Bukkerji Sreejith — Sreejith.Bukkerji@spl-tele.com
196 **Original last-modified:** 2025-10-09
197 **Migration date:** 2026-05-24
198 **Migrated by:** Dimitri Rupp via ClaudeAI
199 **Notes:** EN-Original preserved; 3-card-WHAT-HOW-WHO layout converted to summary table; 3 long paragraphs about ingestion + naming retained 1:1; Mermaid ingestion diagram (incl. fabric-logs skipped-row path) added; 94 table mappings extracted and presented as monospaced table; schema-prefix overview added for navigation.
200 {{/info}}
201
202
203 ----
204
205 == Verwandte Themen ==
206
207 //Kuratiert — kann direkt im Editor ergänzt werden://
208
209 * [[Bronze Lakehouse>>doc:IT-Wiki.01-Data.Ingestion.Bronze-Lakehouse.WebHome]]
210 * [[Fabric Data Pipelines>>doc:IT-Wiki.01-Data.Ingestion.Fabric-Data-Pipelines.WebHome]]
211 * [[Rimo-DWH-Info>>doc:IT-Wiki.01-Data.Creation.Rimo-DWH-Info.WebHome]]
212
213 {{include reference="IT-Wiki.Makros.Verwandte-Themen.WebHome"/}}
214
215 ----
216 **Status:** Migriert — Team-Review ausstehend · **Owner:** //(festlegen)// · **Letzter Review:** 2026-06-11