Apache NiFi – Configure DataFlow & NiFi Policies

Apache NiFi is an open source project mainly designed to support automation of data flows between systems.

This blog is part of a complete guide divided in 3 separate posts:

The complete guide will basically show you how to install and configure an Apache NiFi instance with SSL, LDAP Authentication, policy permissions and also configuring a NiFi cluster using either the embedded zookeeper service or an already configured zookeeper quorum in your current environment.


Configure NiFi & DataFlow Policies

If you followed the previous blog, you will now have a NiFi instance set-up with HTTPS, LDAP User Authentication and an admin user to start editing the required policies.

There are two types of policies in Apache NiFi and these are configured separately:

  1. The Policy to access the Web UI of NiFi, viewing templates, configuration history, etc… This can be accessed from the clickable button in the top-right corner near the username. NiFiPolicybutton
  2. The Policy to view/modify processors, clearing flowfile queues, creating dataflows, viewing DataFlow policies, etc… This can be access by clicking on the key icon as show below

NiFiDataFlowPolicies


Apache NiFi Policies

  • Configuring the policies to access NiFi itself. Near your admin username, you will notice a clickable button which will show you various options:

NiFiWebUI2

We will start by adding some LDAP users and creating NiFi groups to make policy configuration easier.

  • Click on the “Users” option and a larger window will open, showing you current NiFi users and Groups.
  • Click on the NiFiAddUser button to add either a user or a group.
  • You will notice two tickable options:
    • Individual
      • This is where you can add LDAP users. Use the same user account format used in LDAP to “link” them together (e.g. name.surname)
    • Group
      • This is where you can add or create new groups. To “link” groups to LDAP you would need to do further configurations inside the authorizers.xml file.
  • We will start by creating new NiFi groups called “NiFi-Admin”, “NiFi-Operator” and “NiFi-ReadOnly”.

.NiFinewgroup

  • Once the groups are created, you can then start adding users to the specified groups.
  • You can add created users to the group by clicking on the NiFiEditUser near the username and selecting the group you want.

NiFiAddUsertogroup

Now that the users and groups have been created, you would need to setup the necessary policies for users to access your NiFi instance.

  • Click on the “Policies” option and a larger window will open, showing you current NiFi users and Groups depending on the policy.
  • The policies are in the form of a drop-down list. You would need to go through them to configure the specific policies depending on users and/or groups.

NiFiPolicies

  • Adding a user and/or group is similar to creating them. Click on the NiFiAddUser
  • Search for the required user and/or group and add as required.

NiFiAddgrouptopolicy

  • Some policies will have another drop-down list to give you the option to configure whether the permission should be that of “view” or “modify“.

NiFiPolicies2


Apache NiFi DataFlow Policies

As stated above, the dataflow policies are separate from the NiFi policies. These policies need to be set for users to start creating their own dataflows.

  • On the left hand-side you will notice an “Operate” box. Click on the key icon to access the dataflow policies.

NiFiDataFlowPolicies

  • You will notice that the policy format is the same as the users policy. The policies are in the form of a drop-down list.NiFiDataFlowPolicies2

A better explanation of the policies:

  • View the component
    • Allows the user to view what processors are being used and how the dataflow is configured. If the permission is not set for the user, the user will just see the following:

NiFiDataFlowPolicies3

  • Modify the component
    • Allows the user to create his own dataflows, modify/configure processor configurations, start, deleting, etc…

NiFiDataFlowPolicies4

  • View the data
    • Allows the user to view/list the data found inside the queues. Important to note is that the “Node Identity” configured in authorizers.xml should be given permission to this policy

NiFiDataFlowPolicies5

  • Modify the data
    • Allows the user to empty the data found inside the queues should they become stuck or no longer needed. Important to note is that the “Node Identity” configured in authorizers.xml should be given permission to this policy

NiFiDataFlowPolicies6

  • View the policies
    • Allows the user to view the dataflow policies currently in place
  • Modify the policies
    • Allows the user to modify the dataflow policies for all other users

 

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: