Responsibilities
- Data Source Inventory and Mapping
- Identify and document data sources, producing a catalogue of all data sources, including databases, to make a complete data requirement set for data ingestion and modelling
- Create data maps with visual representations of data flows and relationships between different data sources
- Maintain a data inventory to reflect changes in data sources and structures
- Data Gap Analysis
- Assess data availability by evaluating existing data sources to determine if they meet the business and reporting needs
- Identify data gaps by highlighting areas where data is missing or insufficient
- Recommend solutions or methods to fill data gaps, such as data collection initiatives or integrating new data sources
- Compile findings into a comprehensive report for stakeholders
- Data Model Diagrams
- Design Data Models - logical and physical data models to represent data structures and relationships
- Use modelling tools like ERD (Entity-Relationship Diagrams) to visualise data models
- Collaborate with business, database administrators, and developers to ensure data models align with business requirements
- Maintain detailed documentation of data models for reference and future use
- ETL Design Document
- Design Extract, Transform, Load (ETL) processes to move data from source systems to data warehouses
- Define transformation rules that specify how data should be cleaned, transformed, and loaded
- Create and maintain ETL documentation, including data mappings, transformation rules, and load schedules
- Define checks, validations, and reconciliations to ensure data integrity throughout the ETL process