Business Problems: Overcoming the Confidentiality and Integration challenges with Hadoop Clusters using Centrify

Background

Hadoop implementations present multiple challenges to enterprises at the Operating System layer(*), I like to categorize them in 2 areas:

Confidentiality

• Hadoop clusters are unsecure by default.  What that means is that there's no service-to-service authentication and that privileged users have access to world-readable information and can elevate to privileged Hadoop accounts.
• Multiple clusters are needed because of the development nature of the apps.  Typically at least a DEV/QA and PROD environments are needed, depending on the risk profile of the organization, each environment may be in different isolated environments and require different access control rules.
• Different types of users need access:  From the SysAdmin, to the Hadoop Admin, to the Data Scientist, they all have different access and privileged needs.
• The data classification of the business intelligence may require additional controls.  What if the cluster crunches Personal, Financial, Health, Energy or Card data?   SOx, PCI, HIPAA, NERC or FERC compliance is needed.

Integration

• Kerberos:  Many organizations balk at the proposition of standing-up a separate MIT Kerberos implementation; and even if AD is an option, test environments may be in a one-way trust.
• Different organizations == Different requirements, therefore the devil is in the details:
• Process
• Technology/Infrastructure
• People
• Regulations

(*) There are additional security challenges, like how to protect data at the Hadoop layer, for this, your trusted Hadoop vendor (Cloudera, Hortonworks, MapR, etc) have an ecosystem of applications.  Centrify can provide identity information to those apps.

Technical Briefing

The following videos provide technical demos on how Centrify can overcome these challenges

Putting it all together

• Centrify allows for OS level integration for Linux and UNIX systems that enables:
• Centralized Administrations of multiple Hadoop Clusters
• Regardless of how complex your AD may be 
• No schema extensions or software in domain controllers
• Using UNIX frameworks
• Kerberos just works out of the box
• Leverage AD fully:  Kerberos, Group Policy, PKI
• Centrify enables the implementation of strong access controls to enforce
• Least access
• Least privilege (RBAC- not password Centric)
• Easy attestation and reporting
• Separation of Duties
• Works on Windows to eliminate the problem of the persistent administrator
• For environments with Personal, Financial, Health or Card data
• Session transcription 
• Session replay
• Event consolidation
• Works on Windows
• Hadoop-exclusive features:
• adkeytab for advanced keytab/service account provisioning
• Kerberos infinite ticket renewal parameters and GPOs
• LDAP Proxy to assist apps like Sentry, Hue and Knox
• Partner with Cloudera, MapR and Hortonworks

Centrify + AD + Hadoop = faster, more secure and regulation-aligned big data projects.

Labs: Using Couchbase's upcoming LDAP functionality with AD leveraging Centrify’s LDAP Proxy

Background

Couchbase is including LDAP support in the upcoming March release and we wanted to preview how to leverage your existing Centrify assets to help with this scenario.  The instructions below are for testing purposes only.
This capability will provide the ability to use LDAP users (From AD or other LDAP directories) to be provisioned as Full or Read-only administrators of the web console.

About this configuration

In this configuration  Couchbase is using saslauthd to connect to LDAP.  By leveraging the Centrify LDAP Proxy, this configuration can benefit from the robust capabilities of the Centrify agent (AD sites and services awareness, performance, caching, etc).

What’s required?

• Couchbase Server 3.5.x:  http://latestbuilds.hq.couchbase.com/couchbase-server/sherlock/1415/couchbase-server-enterprise-3.5.0-1415-centos6.x86_64.rpm
• Centrify Agent + LDAP Proxy:  http://www.centrify.com/products/server-suite/active-directory-bridge/
• Cyrus-SASL (saslauthd):  http://linux.die.net/man/8/saslauthd

Instructions (All steps on CentOS 6.x)

Step 1 – Install the Centrify Agent and join AD

1. With a privileged user, install the Centrify agent.
   rpm –Uvh centrifydc-<version>  (e.g. centrifydc-5.2.2-rhel3-x86_64.rpm)
2.  Check if all is OK to join Active Directory
   adcheck <your domain>  (e.g. adcheck corp.contoso.com)
   Make sure you fix any issues (e.g. DNS, DC connectivity, etc)
3. Join active directory (you’ll need an account that can join computers in AD)
   adjoin –z Model –c “ou=Servers,ou=UNIX” –u jerry.seinfeld -V corp.contoso.comJoins the corp.contoso.com in Centrify zone mode (Model) mode, and places the computer on the /UNIX/Servers OU and uses jerrry.seinfeld's  account for the corp.contoso.com domain.

Step 2 – Install, configure, start and test the Centrify LDAP Proxy

1. Install the Centrify LDAP Proxyrpm – Uvh centrify-ldapproxy-<version>  (e.g. centrifydc-ldapproxy-5.2.2-rhel3-x86_64.rpm)
2. Elevate and start the proxy manually or configure a startup script.  For example
   dzdo /usr/share/centrifydc/libexec/slapd  -f /etc/centrifydc/openldap/ldapproxy.slapd.conf -h ldap://cen1.corp.contoso.com
   This makes the LDAP proxy to start reading the indicated startup file and listens to the hostname cen1.corp.contoso.com
3. To test that the LDAP proxy is responding to queries, use ldapsearch:$ /usr/share/centrifydc/bin/ldapsearch -h cen1.corp.contoso.com -x -b "ou=staff,dc=corp,dc=contoso,dc=com" "(cn=George Constanza)"
4. Your search should yield results.

Step 3 – Install and Configure and Test saslauthd for LDAP

1. Install cyrus-sasl (saslauthd)$ dzdo yum install cyrus-sasl
2. Configure SASL for LDAP

• a.       Edit the /etc/sysconfig/saslauthd and make sure the MECH parameter is set to LDAPMECH = ldap
• b.      Configure the LDAP parameters for SASL the config file should be on /etc/saslauthd.conf. You need to add the ldap_server parameter (the system running the Centrify LDAP proxy) as well as your search base and the attribute that corresponds to the user name.  With Centrify you can use the AD name (samAccountName) or the uid contained in the PosixAccount class.  In my test scenario:
  ldap_servers: ldap://cen1.corp.contoso.comldap_search_base: ou=Staff,dc=corp,dc=contoso,dc=com=ldap_filter: (samAccountName=%u)
  
  Note that in a production environment you’ll use multiple proxies for HA and secure LDAP (ldaps://) and most likely your Couchbase configuration will be clustered.  Use the same availability controls for the Authentication that you are using for the cluster.
• c.       Start the saslauthd service (or set it to start automatically with chkconfig)$ dzdo service saslauthd start

1. Test saslauthd for LDAPUse the testsaslauth script to test LDAP authentication:/usr/sbin/testsaslauthd -u george.constanza -p <cleartextpw> -f /var/run/saslauthd/mux
2. If everything is well set up, the output should be:  0: OK "Success."

Steps 1 to 3 - Potential misconfigurations:

• Firewall ports are not open for LDAP
• The Proxy is not started or hasn't started with the appropriate protocol or hostname.
• The configuration of saslauthd is incorrect (look at /etc/sysconfig/saslauthd OR /etc/saslauthd.conf)
• Your LDAP filters are not correct.

Step 4 – Install Couchbase and Perform Initial Setup

Official instructions here: http://docs.couchbase.com/admin/admin/Install/rhel-installing.html

1. Obtain the new bits that support LDAP: wget http://latestbuilds.hq.couchbase.com/couchbase-server/sherlock/1415/couchbase-server-enterprise-3.5.0-1415-centos6.x86_64.rpm
2. Add the following lines to the /etc/rc.localfor i in /sys/kernel/mm/*transparent_hugepage/enabled; doecho never > $i; donefor i in /sys/kernel/mm/*transparent_hugepage/defrag; doecho never > $i; done 
3. Save and reboot the server.
4. Install couchbase$ dzdo rpm --install couchbase-server-version.rpm
5. Provided that your firewall ports are open, you can go to a browser and navigate to the Couchbase admin page:  http://your-hostname:8091
6. Follow the steps to complete the initial configuration (hostname, cluster, sample apps, Administrative user, etc)

Note: In my environment I performed a reboot.  Keep in mind that if you ran the Centrify LDAP proxy manually, you need to launch it again and restart saslauthd before the next step.

Step 4 – Set up Couchbase for LDAP and test settings

1.  Log in to the couchbase administrative interface with your administrator account.
2. Navigate to Settings > LDAP Auth Setup
3. Under Setup, check the Enable Checkbox and press Save.
4. Add a Full or Read-Only administrator
   (I used the AD username, since my LDAP filter uses samAccountName).  E.g. George.Constanza
5. To test the user, use the Validate box.
   
6. Sign out of the Console.  Attempt login with the newly assigned Admin.   If your admin is Full, they’ll get access to all the settings, if they are read-only, they’ll get the settings grayed out.

Summary

Setup was fairly simple using the Centrify LDAP proxy.  The benefits are tangible because the underlying adclient agent serves as a way not to have to maintain multiple LDAP entries and for additional performance gains.  Keep in mind, this was a "quick-and-dirty" setup, in reality you would have to account for LDAPS and HA.

As far as Couchbase goes, I was very impressed with how simple it was to set up and things just worked. They have been adding security features incrementally, and we are hoping that we can see them leverage PAM, Kerberos, PKI and to be able to leverage NSS UNIX users/groups for at least administrative interfaces; this will enable Centrify users to expose AD capabilities and accelerate the security posture of the platform.  We are keeping our eyes and ears locked on these newer technologies.

Quick Video Playlist (11 minutes total)

Centrify + Hadoop Update - February 2015

Back in august 2014, I published a blog entry titled "Using Active Directory and Centrify to Accelerate your Linux-based Big Data Deployments" that helped illustrate how to use Centrify for Servers for the purposes of Big Data with Hadoop.

In the meantime Centrify has been gearing-up for the release of Centrify Suite 2015, part of what's coming is improvements on all popular Hadoop implementations with Cloudera, Hortonworks and MapR.  As a preview, David has released a few of companion whitepapers:

• Centrify Identity and Access Management for MapR
• Centrify Identity and Access Management for Cloudera
• Centrify Identity and Access Management for Hortonworks Data Platform

From the post:

"Centrify Server Suite provides Identity, Access and Privilege Management for the Cloudera | Hortonworks | MapR:

• Simplifying AD integration for Cloudera | Hortonworks | MapR to run in secure mode
• Automating service account credential management
• Simplifying access with AD-based user single sign-on authentication
• Ensuring regulatory compliance with least privilege and auditing
• Developer SDKs for secure client application access to Hadoop

Enjoy!!