Workflows: Design and Implementation

Introduction

This document describes the coding and naming conventions for developing workflows in the Collibra Platform. These conventions described are guidelines and although it is desirable to use them it is not meant that they are applied in such a rigorous way that they restraint a smooth development process. In some cases, the guidelines are hardly recommended, in this case, this is mentioned in the text. (e.g. API version)

Best Practices and Coding Conventions

Improve User experience

User Tasks

Name of the (user) task

The choice of the name of the user task is very important since this will be the title of the form. A good user task name will describe the content of the user task in the minimum amount of words (very specific). Try to keep the user task names unique in the flow.

In addition, it is a good practice to give other tasks (e.g. script tasks, call tasks) a meaningful name that contributes to the activities of those tasks. This will have no impact on the functionality of the workflow but will be important for the developer and the user to understand the flow.  (Workflow diagram can be viewed by end-user with appropriate rights by clicking on “Other” -> “View in Workflow Diagram” button of the tasks)

Examples of bad and valid user task names

  • ‘init values’: this says nothing to the user – no added value
  • Fill in a meaningful definition’ or ‘Describe the risk level’ or ‘Assign the issue

B. Form Field labels

  • Sentence case (but asset types/concepts are capitalized)
  • Use a default value for dynamicRadiobox
  • In case a field is mandatory, put a red asterisk next to the form name (add <font color=”red”>*</font>)
  • Form Files(Property Descriptions) should be easy to read, brief, and meaningful. , eg ‘Name’, not ‘How would you like to call this business process?’

Example

C. Use generic form documentation.

It is important to take into consideration the “User Task” Documentation Property (as shown in the picture below). User Task Documentation property is visible to the user as the main text of the task in the user interface before it is opened.

After the user task is opened, it will appear as a description below the User Task header.

In the other task types, this could serve a good place to store the description of what this specific task does.

Use helptext property to add meaningful information. In case the attribute name is clear enough,

it’s better to skip the helptext. The helptext should help the user to fill in the correct value.

Examples of bad and valid helptext case

  • ‘Definition’ – attribute: Definition of the asset
  • ‘Calculation Detail’ – attribute: Description of the different calculations carried on a calculated term (summary). If you have additional documentation with the calculation detail, include the link to that documentation.

Same set of buttons for all forms

  • Back: go back to the previous form
  • Next: go to the next form

Some examples of optional (action) buttons that can be used (depends of the use cases of the project)

  • Save: Save the current status (flush into DB)
  • FinishFinish current workflow
  • Delete: Delete created resource (e.g. Asset, domain)
  • Submit: Submit the changed you did
  • Cancel: Stop (cancel) the workflow

Example

 

 .    

 

 

Give the names to arrows to add even more context/add annotations

To make the workflow clearer for other developers, you can give the flow names, see below.

Furthermore, you can use annotations to add more general information to the workflow

 

Avoid duplicate button id’s: Eg instead of using meaningless <back> as ‘button Id’ better to use <form_name>_<back>

Example of a user form called ‘roles’

Be consequent in using terminology in the documentation/display names of the workflow. If you use ‘asset’ in the first form don’t use the term ‘entity’ in the next form (if both attributes having the same characteristic)

Form architecture / general remarks

  • Add subtitles to break up long forms, and to offer a clear grouping of attributes. (font-size:14px; color:#44494D; font-weight:bold;) (see “1. General Information” field on the screenshot)
  • Where possible, consider splitting certain forms into multiple screens/steps. (see “1. General Information” and “2. Data Flow” fields on the screenshot)
  • Give users a clear indication of which step they are in. e.g. ‘Create new Data Demand (1/2)’
  • Where you do not believe ‘rich text’ input is needed, use the plain text (no text editor) input to make the forms feel less complex (see Demand Description field on the screenshot).

Numbering the subtitles will allow you to add additional information in your business process documentation, and will give your users a better sense of where in the process they are.

 Process Documentation

Rather than supplying external documentation to complete the workflows, you can use assets to document your process. For example, the ‘Articulating a Data Demand’ asset could contain information on what a Data Demand is, who can create a new Data Demand, and how to start it, as well as describing the different sections and steps users will encounter when completing a Data Demand. For each of the sections, you can supply additional information on what content is expected, and you can clarify what to do if there are different options for completing a section. You can use relations to show what the next or previous step in the process is.

The start screen of the workflow can then link to this documentation, so people can get more information on the process before they start. (see screenshot above). 

Error handling

Add meaningful error handling. In this way we can avoid ‘unexpected runtime errors’ and so the error message will become more meaningful for the end-user. (throwDgcError).

Example of using error handling

Without error handling

With error handling

Code snippet 

//throw an error if the asset already exists

if (b_asset_exists)

{

                   def dgcError = new DGCException(“An asset with the   name

                                             Accuracy already exists in this domain. Please     

                                             change the name and click on Next, or Cancel

                                             the workflow”)

        dgcError.setTitleCode (“Error: Asset Creation”);

        throw dgcError;

}

Scripting methodology

General

USE API v2 (namespace of BPMN file should be: https://www.collibra.com/apiv2)

Note: This is a thorough Collibra recommendation with the tendency to become a requirement, since v1 will no longer be supported in the future versions of DGC. In addition, it is recommended to avoid 3rd party libraries since in the near future some items probably may no longer be supported/allowed. If third-party libraries are used, it should be documented in Workflow Design Specification document in detail.

Add (logic) logging in the script, For example:

  • 1 line of logging at the beginning of the script (start <script name>)
  • Logging per important block in the script
  • 1 line of logging at the end of the script (end <script name>)

In case of parallel development or multiple vendors working on different workflows (means a lot of logging), it is recommended to use prefixes to set your code log apart from other workflows or log info.

e.g. prefix = “workflow Name >>> ” + prefix_task = “Task Name >>> ” + logging,  then it is easy to find out where exactly the code has been interrupted.

For performance reasons and use of the log file, cleanup meaningless development logging in case the development phase is over. You can also use a global variable and some logic in your scripts to enable or disable extra loggers you have implemented in the development phase. That could ease a future debug in case of issues.

Note: Never ever log sensitive information. Logs are “public”. Please be aware that everyone having access to the console can see it.

There are different levels of loggerApi you can define to see more details in the log-file depends on the Logger Level:

Add comment, for example

  • On top of every script, tell what the script will do
  • Comment per important block in the script

Indentation level for conditional code (if, switch – case,… – statements).

Example:

If (<condition>)              {              What to do in case condition is true              }      Else {              }   

Example of logging and comment

def prefix = "[WF-best_practices… Update Asset] "loggerApi.info(prefix + "Start")//check if the attribute 'description' already exists in this assetloggerApi.info(prefix + "check if the attribute 'description' already exists in this asset")IF (<Check if attribute NOT exists>)                  {                  //Description attribute does not exists                  loggerApi.info(prefix + "Description attribute does not exists")                  <Add attribute>                              //Create Description attribute                  loggerApi.info(prefix + "Create description attribute")                  <Update attribute>                  //Description attribute created                  loggerApi.info(prefix + "Description attribute created")                  }    else                  {…………………………………………………………………………………………………………………………………………………………………………

results in log file (console)

 Variables

  • Variable names should be self-explanatory. Also, the data type should be clear in case you see the variable. It would be good if the name of the variable explains the scope of the variable as well. (eg. Global variables defined in the start event VS local variables that only will be used during 1 script task)
  • Try to avoid recycling variables in case the context of the variable is not the same

Some examples of bad/valid variables

  • ‘DomCom’: The meaning of this variable is unclear. Maybe it contains the domain of a selected community. In that case, it’s better to call it DomainOfCommunity
  • ‘AssetExists’: This asset is self explainable. This is most probably a boolean variable that is the result of an existence check of the asset.

Workflow Diagram

The Diagram of the workflow should be clear. Avoid a spiderweb. For readability use different lines and avoid crossing flow lines.

Example

 

 

Modular building blocks

Use separate (sub) workflows for items that will be used multiple times.

Release management

Name convention for project-specific workflows: +<functionality_workflow> + ‘_’ + <project name>

With

  • Functionality_workflow: Describe in a few words the functionality of the workflow. For example, this can be: onboard_asset, define_ownership,…
  • Project_name: name of the customer (in case of a long name, use a trigram)

Every released workflow should contain a workflow specification document. This document contains:

  • Description of the functionality of the workflow
  • Overview of input parameters
  • Extra important information

To avoid duplicate workflows and version mismatches, a workflow doesn’t need to contain a version number, but every version will be stored in a source versioning system (e.g. SVN), and the workflow with the maximum number in the DGC environment. This should be done per environment. 

Release cycle

The release cycle should always be respected

  1. Development environment
  2. User acceptance environment
  3. Production environment

Testing

For every workflow, a test plan should be created.

At least every exclusive gateway should be added to the test plan. Something like:

  • What will happen if exclusive gateway = True
  • What will happen if exclusive gateway = False

API v1 / API v2

The 5.7.X will be the latest version where API v1 will be allowed.

Starting from 5.7, a banner will show (in the workflow page) if the workflow still contains old API v1 code and so not ready for the next version of Collibra

Developer workflow forum

More actual information about workflows can be found at: https://university.collibra.com/developer/workflow/

It contains:

  • latest API
  • templates
  • actual information

CLI

The `Collibra CLI` is the perfect workbench for every Collibra developer!

It’s a centralized place where you can:
– Generate a workflow project with a template and all the dependencies set up.
– Upload your workflows to your instance (available through Eclipse too)
– Analyses the usage of API v1 in your workflows and generate a report helping you to convert to API v2

You have to login to comment.