Financial Institution
Successful implementation in a subsidiary of the largest bank in Russia, the bank of the Republic of Belarus (Top 5).
Tools used:
- Informatica Axon
- Informatica EDC
We also considered tools:
- Manta
- Alteryx (Semanta)
Description of the project: Creation of interconnected subsystems that ensure the collection, maintenance of updating and dissemination of knowledge about the bank’s data flows.
The entire knowledge base contains information from the following levels:
- Glossary
- Logical data model
- Physical data model
The unique feature of the project was that the subsystem responsible for the Glossary and part of the logical data model assumed synchronization between the parent company and subsidiary banks (Belarus and Kazakhstan).
Metadata was collected from the following systems:
- OracleDB. Collection of metadata of the structure of database objects of source systems. Attribute composition of tables and views and their relationships.
- Teradata DB (Enterprise Data Warehouse) Stage – the area for loading incremental data of source systems.
- Informatica PDC. Collecting metadata of data loading and transformation flows.
- Bteq Scripts. Collection of metadata of data processing flows. Parsing of SQL queries was performed with the collection of attribute dependencies.
- Oracle OBIEE. Bank marketing segmentation and reporting system. Attribute collection of metadata on all layers (Physical, business, presentational). Communication with data sources.
- Hadoop. Data lake + RTDM (Real Time Decision Making System). Collection of object structure metadata. Attribute composition of tables and views and their relationships. Data was loaded from the following components: HDFS, Hive, Hbase
Project duration: 6 months. The duration of the project is due to the large amount of preparatory work performed. The development of DWH/BigData systems and environment systems was carried out by our team. When developing the above systems, attention was paid to the possible implementation of Data Governance systems. In particular, they were developed and actively used, and compliance with the development requirements was monitored, in which the rules for naming objects were also prescribed.
KPMG + Telecom
Description of the project: Building dependency lines of EDW objects. Using publicly available free tools, the collection of dependencies between database objects and the processes of loading and transforming data was organized.
Tools used: Teradata SQL Parser. Free Web platforms for the formation of visualizations
Metadata was collected from the following systems:
- Teradata DB (Enterprise Data Warehouse). Collection of object structure metadata. Attribute composition of tables and views and their relationships. Parsing SQL queries of stored procedures.
- Informatica PDC. Collecting metadata of data loading and transformation flows.
Project duration: 3 months.
An example of dynamic visualization using free online resources:
