Operating Model and Data Migration

Introduction

This section provides details on how to execute an operating model and data migration between Collibra DGC environments. Every organization have its own requirements (corporate policies, standards, security requirements, etc.) and vision on how infrastructure setup should look like. Most of the time, as a minimum, customers want to have two instances – Development and Production. But it’s an absolutely common situation to have more instances and use them for different purposes and on different steps of the implementation, e.g. – Sandbox, Playground, Development, Testing, UAT, Production, etc.

So if the customer decided to use more than one instance in the implementation we have to remember and have to remind the customer that the Data Governance Use Case Implementation process is different from SDLC. That’s why we have our own specific approach on how to design, develop, test, and deploy DG use cases.

When we talk about Data Governance it’s very critical to control the information about who created the asset, how it had been changed over time, who did these changes, why, and what was the purpose. This information will help end-users understand the data better, find people responsible for the changes, and trust the data. So we can’t create and review the content on one instance (Development) and then simply transfer everything to another (Production) since we will lose the history of changes. All content needs to be created on the Production and later reviewed and approved by Stewards, Subject Matter Experts, and other members of the Data Governance process.

This is one of the reasons why we don’t have in the tool the possibility to migrate community or domain from one instance to another. Of course, it’s possible to use export/import feature but we do not recommend to do this and there are a lot of reasons for this:

  • you will lose the history of the changes for all assets
  • if you recreate these assets you will lose all ongoing workflow tasks that could be started on these assets
  • all assets, domains, and communities after reimport will have new UUIDs
  • some of these assets could have assigned users on the asset level and you will lose them as well (you can not export/import responsibilities)

But when we talk about Operating Model objects (like asset type, attribute type, status, domain type, etc.) there is no reason why we can’t migrate them from one instance to another. Even more, if we migrate them with the same UUID it will help during the integration, workflow development, UI Customisation, etc. in the future.

Below you will find the recommended approach to how to migrate the Operating Model and Data between instances

Environments and their purposes

Sandbox (Training)

There will be several Sandboxes created for different business needs:

  • Playground – the place where users will have an option to play with the tool – create dummy data, see how main DGC functionality works and what is the impact of the changes, etc. Can/will be restored to the Prod state by request (TBD).
  • Boot camps/Hackathons – an environment that will be created/restored from the Production data and modified if it needed for the specific training needs. After the training, it can be restored back to the Production state for the next training or removed.

Every Sandbox should have at least 1, better 2 contact persons who are responsible for the content and the usage.

User permissions and roles will be defined individually for every case but for training purposes, users could have more permissions than on Production.

Development (Testing)

  • Workflow development – test custom workflows to be sure that they will not create/change/remove all existing content
  • Integration development, e.g. bulk data import tests – check the amount of the data imported, quality of this data, and avoid problems with the instance performance that end users can feel during the testing period.
  • Manual data import (from excel) – test data import and results
  • Support design and testing of new use cases before bringing it to Production. Several use cases can be in development at the same time
  • Design and development of the Traceability diagrams, Dashboards, UI customizations.
  • After development and first test configuration of the operating model (assignments, roles, workflows, and views) will be migrated to Acceptance

Acceptance (UAT)

An environment that will be used for the number of tests before applying final changes to the Production. To be sure that on Production everything will work as it should be and none of the existing data will be accidentally removed/changed all tests should be executed on real data. For this purpose, before making tests all data and customizations from Production will be restored to the Acceptance environment. List of tests:

  • Operating Model Migration testing – ensure that migration of the DGC Operating Model works. All objects migrated, nothing has been broken or removed/lost – check assignments (asset types, attribute types, relation types, statuses, domain types, articulation score, validation rules), community/domain structure, global and resource roles, workflow configuration and visual components (dashboards, views, filter, traceability diagrams)
  • Data Import test – ensure that data import works and data imported correctly
  • Non-functional testing – UI customization, German localization, performance
  • Acceptance testing – acceptance of the implemented use using real-life scenarios (with workflow functionality)
  • End-to-End testing – series of tests ensure that nothing had been lost/broken during migration
  • After all tests on Acceptance instance configuration of the operating model (assignments, roles, workflows, and views) will be migrated to Production

Production

  • Productive usage of Data Governance Center by all users

Service requirements for Production are:

  • Sustainable/resistant solution
  • Solution with very good response times
  • Usage with „On-the-Go“ and mobile access
  • Supported with 7×24 hour

Legend

Zone 1 – testing zone with access to the instances with testing data

Zone 2 – production zone with access to the instances with production data

OM Migration – using the new Migration functionality to migrate Assignments (Asset Types, Attributes Types, Relation Types, etc.) with Roles (Global and Resource), Workflows and Visualisation components (Dashboards, Traceabilities, Search Filters, etc.)

Use case implementation steps

Operating Model Migration

What can be migrated:

  • Asset Types and Assignments
    • Domain Assignments
      • Domain Types
    • Validation Rules
    • Attribute Assignments
      • Attributes Types
      • Relations Types
      • Complex Relation Types
        • Attributes Types
    • Statuses
    • Articulation Rules
    • Data Quality Rules
  • Table Views
  • Dashboards
  • Diagram Views
  • Search Filters
  • Workflow Definitions
  • Global Roles
  • Resource Roles

What will be migrated partially in case if these objects have been used in search filters, dashboards, workflow configuration, scopes:

  • Communities
  • Domains
  • Assets

What can’t be migrated:

  • Asset Articulation Scores (should be checked)
  • Validation Rules (will be migrated if they are assigned to asset types)
  • Users and User Groups
  • Workflow Tasks
  • Activities
  • Attachments
  • Comments
  • Responsibilities

Prerequisites

  • For successful Operation Model migration between instances (Development > Acceptance > Production), the Collibra Platform version should be the same.
  • The migration process initiator should have Sysadmin global role permissions on the current and new Collibra environments to be able to access the Settings menu option.
  • The release note will be prepared for each migration and will clearly describe what has been changed in the Operating Model configuration, what should be migrated, and which result we are expecting to get after migration on the target e8nvironment.
  • Skill sets – Collibra Sysadmin

You have to login to comment.