Apache NiFi is an open source project mainly designed to support automation of data flows between systems.
This blog is part of a complete guide divided in 3 separate posts:
- Part 1: Apache NiFi – Basic installation with HTTPS/SSL & LDAP Configuration
- Part 2: Apache NiFi – Configure Data Flow & NiFi Policies
- Part 3: Apache NiFi – Cluster configuration
The complete guide will basically show you how to install and configure an Apache NiFi instance with SSL, LDAP Authentication, policy permissions and also configuring a NiFi cluster using either the embedded zookeeper service or an already configured zookeeper quorum in your current environment.
Configure NiFi & DataFlow Policies
If you followed the previous blog, you will now have a NiFi instance set-up with HTTPS, LDAP User Authentication and an admin user to start editing the required policies.
There are two types of policies in Apache NiFi and these are configured separately:
- The Policy to access the Web UI of NiFi, viewing templates, configuration history, etc… This can be accessed from the clickable button in the top-right corner near the username.
- The Policy to view/modify processors, clearing flowfile queues, creating dataflows, viewing DataFlow policies, etc… This can be access by clicking on the key icon as show below
Apache NiFi Policies
- Configuring the policies to access NiFi itself. Near your admin username, you will notice a clickable button which will show you various options:
We will start by adding some LDAP users and creating NiFi groups to make policy configuration easier.
- Click on the “Users” option and a larger window will open, showing you current NiFi users and Groups.
- Click on the button to add either a user or a group.
- You will notice two tickable options:
- This is where you can add LDAP users. Use the same user account format used in LDAP to “link” them together (e.g. name.surname)
- This is where you can add or create new groups. To “link” groups to LDAP you would need to do further configurations inside the authorizers.xml file.
- We will start by creating new NiFi groups called “NiFi-Admin”, “NiFi-Operator” and “NiFi-ReadOnly”.
- Once the groups are created, you can then start adding users to the specified groups.
- You can add created users to the group by clicking on the near the username and selecting the group you want.
Now that the users and groups have been created, you would need to setup the necessary policies for users to access your NiFi instance.
- Click on the “Policies” option and a larger window will open, showing you current NiFi users and Groups depending on the policy.
- The policies are in the form of a drop-down list. You would need to go through them to configure the specific policies depending on users and/or groups.
- Adding a user and/or group is similar to creating them. Click on the
- Search for the required user and/or group and add as required.
- Some policies will have another drop-down list to give you the option to configure whether the permission should be that of “view” or “modify“.
Apache NiFi DataFlow Policies
As stated above, the dataflow policies are separate from the NiFi policies. These policies need to be set for users to start creating their own dataflows.
- On the left hand-side you will notice an “Operate” box. Click on the key icon to access the dataflow policies.
- You will notice that the policy format is the same as the users policy. The policies are in the form of a drop-down list.
A better explanation of the policies:
- View the component
- Allows the user to view what processors are being used and how the dataflow is configured. If the permission is not set for the user, the user will just see the following:
- Modify the component
- Allows the user to create his own dataflows, modify/configure processor configurations, start, deleting, etc…
- View the data
- Allows the user to view/list the data found inside the queues. Important to note is that the “Node Identity” configured in authorizers.xml should be given permission to this policy
- Modify the data
- Allows the user to empty the data found inside the queues should they become stuck or no longer needed. Important to note is that the “Node Identity” configured in authorizers.xml should be given permission to this policy
- View the policies
- Allows the user to view the dataflow policies currently in place
- Modify the policies
- Allows the user to modify the dataflow policies for all other users