Advanced Replication Setup for High availability and Performance

In my personal opinion, Oracle leads the market in Directory Product offerings (LDAP Directories). Starting from Oracle Internet Directory (OID), to the latest Oracle Unified Directory (OUD), Oracle definitely provides variety of LDAP Directory related products for integration.

With increasing demand for mobile computing and cloud computing offering, there is a need to standardize LDAP Deployments for Identification, Authentication and (sometimes) Authorization (IAA) services. With a highly scalable, highly performing, highly available, highly stable and highly secure LDAP Directory, these IAA services will be easier to integrate with applications in the cloud or for the mobile applications.

Introduction

Oracle Unified Directory (OUD) is a latest LDAP Directory offering from Oracle Corp. As mentioned in my previous post, OUD comes with three main components. They are:

• Directory Server
• Proxy Server
• Replication Server

Here, Directory Server provides the main LDAP functionality (I assume you already know what an LDAP Directory Server means). Proxy server is used for to proxy LDAP requests (how?). And Replication Server is used for replicating (copying) data from one OUD to another OUD or even to ODSEE server (we will talk more about replication in this post). You can read about my first post on OUD here. In this current article, I will write about replication server and advanced replication setup for Oracle Unified Directory.

Many people want a step by step guide (kind of cheat sheet) to setup something like OUD or OID for replication. Unfortunately I am not going to give you that here. In my personal opinion, that (cheat sheet) is not a right approach at all and will not be helpful in the long run for gaining concepts or knowledge. First of all, we need to give importance to the basic concepts behind how something works.

First of all, read OUD Documentation

Product Documentation must be read before you plan your deployment. You can find the OUD Documentation here. This link is for OUD Version 11.1.1. Make sure to refer the latest product manual. Documentation provides lot of details about the product and save lot of time with investigation later. For Replication, you need to start with “Architecture Reference” Guide.

When do you want to setup replication?

There should be a reason, right? If there is no reason, then there is no need for you to setup replication at all. Instead, you can have a beer and pass the time happily doing something else.

Ideally, you need replication setup for “High Availability” and “Performance”. Usually, there will be multiple instances of OUD Directory Server processes running in Production. Let’s say we need to have around four OUD Directory Servers (and four more for Business Continuity/Disaster Recovery).

Unfortunately, there is no single process to update all the eight OUD Directory Servers in our example. We need to find a mechanism to synchronize the directory entries across these servers.  For this, we need to use the OUD Replication Server Component.

Securing the Replication Traffic

We don’t want network sniffers taking away critical user information (even inside the internal network, it is possible). We need to encrypt the traffic between the replication servers. Do not consider setting up a Replication Server communication without encrypted traffic.

Since OUD provided identity data, all the network traffic is prone to sniffing attacks. Always use encrypted or secure connections to OUD or to any LDAP Directory.

Deciding a Replication Method to use

Next important thing is to decide what replication method you are going to use. This is mostly site specific and you need to know lot of details before deciding a replication method to use. I am planning to use the following sample architecture for this post. Let’s understand our sample OUD Architecture first.

Here are the quick components of the architecture:

• We have one master OUD Server called PROD-01. All the updates to the directory happens here. Most probably, HR System will update the directory. Also, Updates can happen using a custom developed application plug-in for LDAP Directory or using a Identity and Access Management System (IAM) system such as Oracle Identity Manager or Tivoli Identity Manager.
• PROD-02 will be used with PROD-01 for High Availability and Performance in this Production Deployment.
• In Disaster Recovery deployment, we have PROD-03 and PROD-04 servers. These servers need to synchronize the user data from the master server PROD-01.

One way to setup replication is by provisioning users into all the six OUD Directory Servers by an Identity and Access Management (IAM) System (such as Oracle Identity Manager or Tivoli Identity Manager). However this provisioning can be time consuming to complete because it will be treated as updating six different LDAP Directories. So a better way to achieve this is using a Replication Server.

We will continue setting up the Replication Server for this architecture. Lets meet in another post - Until then.

Read More about Advanced Replication Setup

My exploration with Oracle Unified Directory 11g

When I read the blog post about the introduction of a new product Oracle Unified Directory, first thing that came to my mind was, why another LDAP directory from Oracle? Oracle already has two LDAP Directory offerings, ODSEE (Sun Directory) and OID. However my impression changed after attending this webcast from Oracle (which justified about the new product in general). You can find the Q & A from the webcast here. It looks like “unified” summarizes about the new product.

Personally, I have interests in all the products that deal with LDAP Protocol and LDAP Directories. So, I was interested to know more about the new product. I read about Oracle Unified Directory (OUD) 11g release notes here. You can find the OUD documentation for 11g R1 release (11.1.1.5.0) here.

OUD is one of a brand new directoty product from Oracle as part of the Fusion Middleware Identity Management. OUD comes with Oracle Directory Services Plus (ODS Plus) suite. So, if you already have a license for ODS Plus, then you already have a license to use OUD in your enterprise (Please check with Oracle Licensing before any major deployments to see the existing license covers it).

There are too many TLAs (Three Letter Acronyms) used in this post – I tried to use this minimally but that was not easy. If you hate TLAs, here is a quick recap:

OUD – Oracle Unified Directory

OID – Oracle Internet Directory

ODS (ODSEE) – Oracle Directory Server (previously, Sun Java System Directory Server – also, iPlanet Directory)

DIP – Directory Integration Platform

A brand new LDAP v3 Implementation

LDAP Protocol is the standard way for storing directory entries for an enterprise. With more than 5 years of development (mentioned in the webcast), OUD is a LDAP v3 implementation from Oracle. It was also mentioned that OUD was written in Java.

OUD comes with three main components. They are:

• Directory Server
• Proxy Server
• Replication Server

Directory Server provides the main LDAP functionality. Proxy server can be used for proxy LDAP requests. And Replication Server is used for replication from one OUD to another OUD or even ODSEE server.

Embedded Berkeley DB

This is my favorite part of OUD. There is no separate Oracle Database requirement for OUD (unlike Oracle Internet Directory). Personally I like LDAP Directories embedded with an attached database such as Berkeley DB for storing the data entries. Some may argue that we can use Database features in the LDAP Directory. However I feel that we can use file based storage features for the DB files which can be better.

OUD Replication for ODSEE

Replication is one of the major requirements for LDAP Directories now. To setup high availability environments, we need replication to be setup in almost every LDAP Directory deployment. We need to propagate changes from one OUD directory to another using replication for High-Availability.

One of the important features I like about OUD is its replication gateway for ODSEE (also for OUD). Definitely this is aimed for deployments with ODSEE to migrate to the OUD easily.

Directory Integration Platform (DIP)

Oracle DIP provides functionality to synchronize data from one LDAP Directory to another, such as Active Directory (also from database to the LDAP Directory). Oracle DIP was already available earlier. Now, DIP is supported in OUD as well.

Performance and Security

In existing LDAP Directories, we face performance issues related to writing data entries. In OUD, it was mentioned in the webcast that it provides “5 times write” and “3 times read” performance.

OUD will be used for storing user information for authentication and sometimes for authorization purposes. Also, it can be used to store personnel information. So, security is one of the major concerns during the deployment. I need to explore more on this.

Some of the other features:

1. dsconfig command – dsconfig command line is used to configure most of the system administration functions for OUD.
2. ODSM – ODSM is a graphical utility for managing OUD.

So what’s next? Obviously it is time to download the new product and play around with its features.

Installation and Configuration

Installation is a simple task with Oracle Installers. This is no exception for OUD too. OUD installation is a simple process. We just need to download the software and run the runInstaller utility (I have Oracle Linux in my laptop – All my examples below is on Linux).

./runInstaller –jreloc /usr

[I have Oracle Linux 6.0 in my laptop. I installed OUD in this laptop with jre 1.6.0_20 installed under /usr by default. The Java executable is /usr/bin/java/]

OUD installation is a simple process with 7 step installation process:

1. Welcome
2. My Oracle Support Updates
3. Prerequisite checks
4. Installation Location
5. Installation Summary
6. Installation Progress
7. Installation Complete

[Since the prerequisite checks failed in my laptop (Issue with Linux Version), I have to tweak little bit for the OUD installation, by editing /etc/redhat-release and /etc/oracle-release files]

Once Installation is completed, then we need to configure OUD based on what components we want to deploy.

For each component, OUD comes with a GUI utility for configuration. We can configure OUD for Directory Server, Proxy Server and/or Replication Server. We can run these utilities using command line options for configuration too. The utilities are located under the install folder:

oud-setup

oud-proxy-setup

oud-replication-setup

Once configuration is completed using these utilities, you can use the commands in bin folder to start or stop the components. You can refer the installation guide for more details on this.

Note:

I faced issues during the Server Startup for using the jre in /usr (delivered with Oracle Linux). I will write about it in my next post.

All right… that’s all for now. I will continue to write more on OUD for advanced options for deployment. Also, I will be deploying the ODS connector in OIM to integrate with OUD. We will meet in another post with more details on them. Until then

Read More about Oracle Unified Directory

Let’s talk about Oracle’s Secure Enterprise Search (SES) 11g

Introduction

Since the acquisition of Sun few years ago, Oracle is now has more things to offer (especially in the hardware side). From a company of Software Products, Oracle is now a company with lot of hardware stuff as well. Oracle’s Exadata and Exalogic are some of the top-tier hardware offerings that we all know. They are based on Sun SPARC hardware. There are lot of things to talk about… However, in this post, I wanted to explore something less talked about.

If you are an Oracle Shop running Enterprise Applications from Oracle, then you may want to look for Enterprise Search Application from Oracle to provide search capabilities for your intranet applications (especially for content management systems). I came to know about this recently when I was reading the Release Value Propositions for Peopletools 8.52. Then, I realized that this product is used in many other products from Oracle.

Functionality of SES

Secure Enterprise Search (SES) 11g (11.1.2) is a product from Oracle for Search Operations in enterprise systems.  Also, Oracle’s Secure Enterprise Search (SES) comes with Oracle Database 11g Enterprise Edition – for use with limited license with Oracle database 11g. SES 11g requires Weblogic Application Server for the functionality (so, obviously it uses lot of Java for sure).

Oracle SES can crawl, search and index for several source types. Some of the content types that are built-in for SES are web content, files, emails, database tables and other SES sources. Also, using connectors you can use many of the content management products for search purposes.

Here are some of the Oracle products that uses/will use SES as part of providing search operations:

• Proposed Peopletools 8.52
• Fusion Applications
• Oracle iAS/Portal
• EBS
• Siebel
• Web Center etc

My personal opinion is, installing something is the simple thing to do with any of the Oracle Products that I know of. If you can understand some of the basic concepts behind Oracle Installers, then you are all set with the installation, nothing complicated here – installation is easy. During the SES installation, you need to make sure the port numbers and the data storage locations are correctly setup. Configuring a product for a specific implementation is something more work to do, some conceptual knowledge will be required at this time.

Most of the time, contents are not public for SES to search. So the search engine should provide crawling and indexing functions for private content. So, a kerberos based authentication or LDAP based authentication can be used in SES as an authentication plug-ins.

SES Scheduler is used to run jobs for crawling and related purposes. Also, we can write a custom Scheduled Tasks for SES using Search API.

If you have some basic understanding of the search engine concepts, then I think SES Administration Tool is simple and easy to understand.

SES Connectors

For searching, there are variety of content available from products from different vendors. SES can perform search and index operations in variety of other target systems using SES Connectors. Obviously, in heterogeneous IT environments, the content is not available in one single source or systems. So, there are different connectors available. Oracle SES 11g connectors are delivered free with the SES product for:

• Microsoft Exchange
• NTFS File Systems
• JDBC Connections to Oracle and MS SQL Server
• Microsoft Sharepoint
• Oracle Portal 9/10 etc.

There are other SES Connectors available for different products, especially for content management systems. However it looks like they need a separate license to be purchased. You can check the available SES Connectors here.

SES and Oracle products

I checked few of the products that are using/planning to use SES. There are other Oracle Products too. This is only a short list that I know of:

Peopletools 8.52

In the next release of Peoplesoft’s Peopletools (expected in Q3/Q4 2011), SES framework will be used in the Peoplesoft Systems. Peoplesoft Applications already use Verity Software for the Search Operations. We need to wait until Peopletools 8.52 release to see what things are going to change.

To know more about PeopleSoft Application Search in next release of Peopletools, you can check here.

Fusion Middleware and Applications

Web Center uses SES as Search Provider. Also, Fusion Application uses SES as the default search Provider.

Oracle iAS/Portal

Going forward, SES will replace the Oracle’s earlier UltraSearch as the Search Provider in newer versions of Portal.

Oracle EBS

Latest versions of EBS support SES. You may want to check the system certifications for SES on EBS in My Oracle Support.

So, thats it for now. Lets meet you in another post. Until then

Read More about  Oracle’s Secure Enterprise Search (SES) 11g

(Some) Internals of Oracle Identity Manager Access Policies

Introduction

Many enterprises are considering (or already deployed) an identity management solution either for effective IT automation to reduce costs and/or for compliance purposes. Oracle Identity Manager is part of the Oracle’s identity and Access Management (IAM) solution. It provides functionalities such as, automatic user provisioning, compliance reporting, etc.

In my personal opinion, Oracle Identity Manager (OIM) is a wonderful product from Oracle. Many people don’t understand the basic concepts behind how OIM works. Worst thing is, they complain about the vendor product for their own failures in understanding basic concepts.

If you are planning to work with Oracle Identity Manager, then get ready for learning a lot of new things. OIM requires knowledge and you should be familiar with following:

• LDAP Directory – especially Oracle Internet Directory or Oracle Directory Server (formerly Sun/Iplanet Directory)
• Basic understanding of XML
• Programming in Java
• Concepts of Microsoft Active Directory and Microsoft Exchange (if you are planning to integrate them)
• Most importantly, self-initiative and interest to research yourself for things you can’t find in “google”.

Oracle Identity Manager stores all the user information, metadata information, audit information, and everything related to data in the Database (similar to Oracle Internet Directory – OID). There are two supported database environments for OIM to store data. It can be:

• Oracle Database Server
• Microsoft SQL Server

The second major component of OIM is the connectors. OIM connectors provide functionality for connecting to various systems across an enterprise. Good thing about OIM is, there are many connectors available. Also, Oracle is standardizing some of the connector components to get the same feeling across all the connectors. So, if you can understand few connectors, then it will be easier for you to work with the remaining connectors.

Latest OIM connectors can be found here – You can download it as well.

OIM Connector Certification (supported systems for OIM for user provisioning) can be found here.

OIM Connector documentation can be found here.

Basic OIM Concepts

Before we talk about Access Policies, we need to understand few other OIM Concepts. OIM has various objects that work together to achieve the necessary functionality. In an ideal way, OIM should manage the complete lifecycle of user accounts in an enterprise – using automatic ways with no manual intervention during entire lifecycle of user creation, modification and deletion phases.

When a user is created in OIM, there will be corresponding entries available in USR table. USR table has many fields delivered OOTB (OOTB – Out of the box). However for some of the enterprises, this may not be sufficient. We can define additional fields as UDFs (User Defined Fields).

In OIM, almost everything revolves around the user account (I think that is what expected from an identity provisioning software such as OIM). User account is the central piece of data here.

In OIM, Users will be provisioned or de-provisioned with Resources. Resources are a target system, such as, Oracle Internet Directory or Active Directory.

What are OIM Access Policies?

There are three types of objects required to perform automatic provisioning based on policies. When you use Access Policies for auto-provisioning, then it is called as “Policy Based Provisioning”. The main objects required for policy based provisioning are:

• Rules
• Groups
• Access Policies

We can use Rules for placing users to some specific OIM Groups. Once a user is a member of a group, then, Access policies can be used to perform policy-based provisioning in OIM. That’s why we need to understand the dependencies between Rules, Groups and Access Policies.

Rules get evaluated whenever an update is made to the user attributes (such as a password change, email address change etc). Also, we can use the OIM API updateUser() function to re-evaluate rules.

In Design Console, you can use “Policy History” form to view the details of the access policies and resources related to users.

Starting from OIM 9.1.0.2 and later versions (in Fusion Middleware Identity and Access Management 11.1.1.x too), there is a scheduled task called “Evaluate User Policies” delivered OOTB. This task will be useful if you want to provision users by validating all the rules, then automatically adding/removing groups, finally provisioning/de-provisioning resources by access policies.

Some Internals of working

POL table holds details about the Access Policies in OIM database. There are other tables related to OIM Access Policies as well. Some of the interesting ones are:

• POP – data about parent table in Access Policies
• POC – data about child policies in Access Policies
• POG – mapping between access policies and OIM groups (based on pol_key and ugp_key)
• POF – Field Values in Access Policies

In USR Table, there is a field called “USR_POLICY_UPDATE”. I think the values can be null or 1. This field is used when “Evaluate user policies” task is run for the evaluate criteria. This field will determine whether the access policies will be reevaluated next time.

User Policy Profile tables – UPP and UPD tables are important user related tables that stores details about access policies for a user and relevant details. These tables normally referred when “Policy History” form is being used for a user in OIM Design Console.

There are two other history tables UPH and UHD. They are history tables for the corresponding User Policy Profile Tables UPP and UPD.

OIU table has two columns, OIU_POLICY_BASED and OIU_POLICY_REVOKE. Based on my understanding, these two columns are set based on the resources provisioned access policy and “Revoke if no longer applies” setting.

Process form tables (UD_ tables) will contain POL_KEY column populated with Access policy. This POL_KEY column is applicable for the OIM Child tables as well.

In OIM, updating the underlying tables are not recommended and not supported by Oracle. These tables will be used when you investigate to try to find out scenarios such as, why a user was not revoked automatically or why she was not provisioned to a resource automatically.

A Sample Implementation

I was thinking of a scenario to explain the usage of access policies for automatic provisioning of Resources in OIM. You can consider an enterprise trying to move to OIM. They have some of the rules based on which user account will be created or modified or deleted. I just have these few rules as an example (in real world, there can be many up to 100+ or even 200+ rules).

1. All users in HR Department will be part of the AD Group “HR Department”
2. All users with “IT Operations” should be having a unix account server in “exadata-200”

So, in first case, you can define an OIM Rule, that will place the users with “HR Department” value in an OIM Group “Group_HR_Department”. Then whenever user is part of that OIM Group, then the user can be provisioned to “HR Department” AD Group automatically.

In the second case, we can check for the department with the Rules, place the user in a group – then we can define an access policy to provision user account to “exadata-200” automatically.

Closing note

Access Policies are just one of the features of OIM. There are many other features there in OIM. Implementing OIM is easy if you understand these underlying basic concepts. Also, understanding about the target systems will be useful when investigating issues during the implementation.

As in every project, collecting the requirements is important. In OIM implementations, this is really important, more than that, documenting the requirements is important. Also, sufficient amount of testing is another consideration for OIM implementation projects. I will cover the logistic details of an OIM implementation in another post.

As the saying goes “The more you know, the more you know what you don’t know”. This is true for OIM (for so many other things in IT too). There are still some things I don’t know about OIM Access Policies. I am just working with OIM on what I know now (and still learning).  J

Okay. I hope that is it for this post. We will meet in another post with more interesting details about OIM. Until then

Read More about Oracle Identity Manager 

Part I – LDAP Directory for the Cloud – Which one do you recommend?

I am planning to appear for the CISSP exam sometime this year (could be in the month of May – I believe it really needs more time to prepare). For my Exam, I just completed my reading the Access Control chapter. I am using the Shon Harris AIO guide for my CISSP Exam. Whether I take the exam or not, the more knowledge I gain, then I am good with that. Believe me “Access Control” is not an easy chapter for me (though I worked on that domain for last few years. I have to understand lot of terminologies for the CISSP Exam. I still have 9 more domains to complete before start taking other books (Access Control is just one of them). It looks like it needs a lot more preparation than I thought.

Definition of Cloud Computing

Directory as a Service

• Oracle Directory Server (ODS) – formerly Iplanet or Sun LDAP.

• Oracle Internet Directory (OID)

• Microsoft Active Directory (AD)

• IBM Tivoli Directory Server (ITDS)

• Novell’s eDirectory

• OpenLDAP

What do I think?

Anyways, I don’t want to talk about Access Control here. But it is about the webcast by Mark Wilcox from Oracle couple of weeks ago. Mark webcasted a presentation on “Choosing the right Directory for the Cloud”. You can find the recording here.

Let’s try to understand the general definition of cloud computing first. According to “The NIST Definition of Cloud Computing” Version 15, it is a model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction.

According to the definition, there should be shared pool of configurable computing resources. In this context, we are talking about LDAP Directory as a software service that can be configured to provide access through various resources through. In this webcast, Mark talked more about the OID and ODS (see below).

In this context, let’s try to understand how LDAP Directories can be a service in the cloud.

There are many LDAP Directory offerings from various vendors, such as the ones below:

I want to talk more about LDAP directories for the clouds more on covering famous Directories out there. We serve many customers and everyone has their own preference of a LDAP Directory. So, we can’t ignore the other famous LDAP Directories.

When we talk about LDAP Directory for a cloud, we are talking about an LDAP instance for the Cloud application for authentication purposes (in some cases, we can use it for authorization as well).

If you are working with Oracle Products, such as Oracle EBS etc, and you need to consider a integration with LDAP Directory, then I believe Oracle Internet Directory (OID) has more advantages than the others in the list (Also, Oracle certifies most of the Identity Management products for EBS aligning with OID). Main reason is that Oracle Products are certified with OID as a recommended LDAP Directory – they are easy to integrate with the support from the point of the Vendor. Other reasoning is because the data is stored in the database, so you can take advantage of the Database Security Features.

ODS (formerly Sun Java System Directory Server, before that Iplanet Directory Server) is a great product in itself. I am working with this directory for a long time now. The data is stored in the Operating System Files (it internally uses the database structure). ODS follows LDAP v3 protocol standard.

I don’t want to be Oracle-centric in my approach (both of the above two directory servers I mentioned are from Oracle Corp). Mark Wilcox is from Oracle, So he talked more about these two directories in general. Also,

So, how can we provide an LDAP Directory as a service in the cloud? And more importantly what are the important factors that we need to consider while providing this service?

Also, Let’s talk about other directories in coming posts.

Until then

Vijay Chinnasamy

Read More about  LDAP Directory