Let’s talk about Oracle’s Secure Enterprise Search (SES) 11g

Introduction

Since the acquisition of Sun few years ago, Oracle is now has more things to offer (especially in the hardware side). From a company of Software Products, Oracle is now a company with lot of hardware stuff as well. Oracle’s Exadata and Exalogic are some of the top-tier hardware offerings that we all know. They are based on Sun SPARC hardware. There are lot of things to talk about… However, in this post, I wanted to explore something less talked about.

If you are an Oracle Shop running Enterprise Applications from Oracle, then you may want to look for Enterprise Search Application from Oracle to provide search capabilities for your intranet applications (especially for content management systems). I came to know about this recently when I was reading the Release Value Propositions for Peopletools 8.52. Then, I realized that this product is used in many other products from Oracle.

Functionality of SES

Secure Enterprise Search (SES) 11g (11.1.2) is a product from Oracle for Search Operations in enterprise systems.  Also, Oracle’s Secure Enterprise Search (SES) comes with Oracle Database 11g Enterprise Edition – for use with limited license with Oracle database 11g. SES 11g requires Weblogic Application Server for the functionality (so, obviously it uses lot of Java for sure).

Oracle SES can crawl, search and index for several source types. Some of the content types that are built-in for SES are web content, files, emails, database tables and other SES sources. Also, using connectors you can use many of the content management products for search purposes.

Here are some of the Oracle products that uses/will use SES as part of providing search operations:

• Proposed Peopletools 8.52
• Fusion Applications
• Oracle iAS/Portal
• EBS
• Siebel
• Web Center etc

My personal opinion is, installing something is the simple thing to do with any of the Oracle Products that I know of. If you can understand some of the basic concepts behind Oracle Installers, then you are all set with the installation, nothing complicated here – installation is easy. During the SES installation, you need to make sure the port numbers and the data storage locations are correctly setup. Configuring a product for a specific implementation is something more work to do, some conceptual knowledge will be required at this time.

Most of the time, contents are not public for SES to search. So the search engine should provide crawling and indexing functions for private content. So, a kerberos based authentication or LDAP based authentication can be used in SES as an authentication plug-ins.

SES Scheduler is used to run jobs for crawling and related purposes. Also, we can write a custom Scheduled Tasks for SES using Search API.

If you have some basic understanding of the search engine concepts, then I think SES Administration Tool is simple and easy to understand.

SES Connectors

For searching, there are variety of content available from products from different vendors. SES can perform search and index operations in variety of other target systems using SES Connectors. Obviously, in heterogeneous IT environments, the content is not available in one single source or systems. So, there are different connectors available. Oracle SES 11g connectors are delivered free with the SES product for:

• Microsoft Exchange
• NTFS File Systems
• JDBC Connections to Oracle and MS SQL Server
• Microsoft Sharepoint
• Oracle Portal 9/10 etc.

There are other SES Connectors available for different products, especially for content management systems. However it looks like they need a separate license to be purchased. You can check the available SES Connectors here.

SES and Oracle products

I checked few of the products that are using/planning to use SES. There are other Oracle Products too. This is only a short list that I know of:

Peopletools 8.52

In the next release of Peoplesoft’s Peopletools (expected in Q3/Q4 2011), SES framework will be used in the Peoplesoft Systems. Peoplesoft Applications already use Verity Software for the Search Operations. We need to wait until Peopletools 8.52 release to see what things are going to change.

To know more about PeopleSoft Application Search in next release of Peopletools, you can check here.

Fusion Middleware and Applications

Web Center uses SES as Search Provider. Also, Fusion Application uses SES as the default search Provider.

Oracle iAS/Portal

Going forward, SES will replace the Oracle’s earlier UltraSearch as the Search Provider in newer versions of Portal.

Oracle EBS

Latest versions of EBS support SES. You may want to check the system certifications for SES on EBS in My Oracle Support.

So, thats it for now. Lets meet you in another post. Until then

Read More about  Oracle’s Secure Enterprise Search (SES) 11g

(Some) Internals of Oracle Identity Manager Access Policies

Introduction

Many enterprises are considering (or already deployed) an identity management solution either for effective IT automation to reduce costs and/or for compliance purposes. Oracle Identity Manager is part of the Oracle’s identity and Access Management (IAM) solution. It provides functionalities such as, automatic user provisioning, compliance reporting, etc.

In my personal opinion, Oracle Identity Manager (OIM) is a wonderful product from Oracle. Many people don’t understand the basic concepts behind how OIM works. Worst thing is, they complain about the vendor product for their own failures in understanding basic concepts.

If you are planning to work with Oracle Identity Manager, then get ready for learning a lot of new things. OIM requires knowledge and you should be familiar with following:

• LDAP Directory – especially Oracle Internet Directory or Oracle Directory Server (formerly Sun/Iplanet Directory)
• Basic understanding of XML
• Programming in Java
• Concepts of Microsoft Active Directory and Microsoft Exchange (if you are planning to integrate them)
• Most importantly, self-initiative and interest to research yourself for things you can’t find in “google”.

Oracle Identity Manager stores all the user information, metadata information, audit information, and everything related to data in the Database (similar to Oracle Internet Directory – OID). There are two supported database environments for OIM to store data. It can be:

• Oracle Database Server
• Microsoft SQL Server

The second major component of OIM is the connectors. OIM connectors provide functionality for connecting to various systems across an enterprise. Good thing about OIM is, there are many connectors available. Also, Oracle is standardizing some of the connector components to get the same feeling across all the connectors. So, if you can understand few connectors, then it will be easier for you to work with the remaining connectors.

Latest OIM connectors can be found here – You can download it as well.

OIM Connector Certification (supported systems for OIM for user provisioning) can be found here.

OIM Connector documentation can be found here.

Basic OIM Concepts

Before we talk about Access Policies, we need to understand few other OIM Concepts. OIM has various objects that work together to achieve the necessary functionality. In an ideal way, OIM should manage the complete lifecycle of user accounts in an enterprise – using automatic ways with no manual intervention during entire lifecycle of user creation, modification and deletion phases.

When a user is created in OIM, there will be corresponding entries available in USR table. USR table has many fields delivered OOTB (OOTB – Out of the box). However for some of the enterprises, this may not be sufficient. We can define additional fields as UDFs (User Defined Fields).

In OIM, almost everything revolves around the user account (I think that is what expected from an identity provisioning software such as OIM). User account is the central piece of data here.

In OIM, Users will be provisioned or de-provisioned with Resources. Resources are a target system, such as, Oracle Internet Directory or Active Directory.

What are OIM Access Policies?

There are three types of objects required to perform automatic provisioning based on policies. When you use Access Policies for auto-provisioning, then it is called as “Policy Based Provisioning”. The main objects required for policy based provisioning are:

• Rules
• Groups
• Access Policies

We can use Rules for placing users to some specific OIM Groups. Once a user is a member of a group, then, Access policies can be used to perform policy-based provisioning in OIM. That’s why we need to understand the dependencies between Rules, Groups and Access Policies.

Rules get evaluated whenever an update is made to the user attributes (such as a password change, email address change etc). Also, we can use the OIM API updateUser() function to re-evaluate rules.

In Design Console, you can use “Policy History” form to view the details of the access policies and resources related to users.

Starting from OIM 9.1.0.2 and later versions (in Fusion Middleware Identity and Access Management 11.1.1.x too), there is a scheduled task called “Evaluate User Policies” delivered OOTB. This task will be useful if you want to provision users by validating all the rules, then automatically adding/removing groups, finally provisioning/de-provisioning resources by access policies.

Some Internals of working

POL table holds details about the Access Policies in OIM database. There are other tables related to OIM Access Policies as well. Some of the interesting ones are:

• POP – data about parent table in Access Policies
• POC – data about child policies in Access Policies
• POG – mapping between access policies and OIM groups (based on pol_key and ugp_key)
• POF – Field Values in Access Policies

In USR Table, there is a field called “USR_POLICY_UPDATE”. I think the values can be null or 1. This field is used when “Evaluate user policies” task is run for the evaluate criteria. This field will determine whether the access policies will be reevaluated next time.

User Policy Profile tables – UPP and UPD tables are important user related tables that stores details about access policies for a user and relevant details. These tables normally referred when “Policy History” form is being used for a user in OIM Design Console.

There are two other history tables UPH and UHD. They are history tables for the corresponding User Policy Profile Tables UPP and UPD.

OIU table has two columns, OIU_POLICY_BASED and OIU_POLICY_REVOKE. Based on my understanding, these two columns are set based on the resources provisioned access policy and “Revoke if no longer applies” setting.

Process form tables (UD_ tables) will contain POL_KEY column populated with Access policy. This POL_KEY column is applicable for the OIM Child tables as well.

In OIM, updating the underlying tables are not recommended and not supported by Oracle. These tables will be used when you investigate to try to find out scenarios such as, why a user was not revoked automatically or why she was not provisioned to a resource automatically.

A Sample Implementation

I was thinking of a scenario to explain the usage of access policies for automatic provisioning of Resources in OIM. You can consider an enterprise trying to move to OIM. They have some of the rules based on which user account will be created or modified or deleted. I just have these few rules as an example (in real world, there can be many up to 100+ or even 200+ rules).

1. All users in HR Department will be part of the AD Group “HR Department”
2. All users with “IT Operations” should be having a unix account server in “exadata-200”

So, in first case, you can define an OIM Rule, that will place the users with “HR Department” value in an OIM Group “Group_HR_Department”. Then whenever user is part of that OIM Group, then the user can be provisioned to “HR Department” AD Group automatically.

In the second case, we can check for the department with the Rules, place the user in a group – then we can define an access policy to provision user account to “exadata-200” automatically.

Closing note

Access Policies are just one of the features of OIM. There are many other features there in OIM. Implementing OIM is easy if you understand these underlying basic concepts. Also, understanding about the target systems will be useful when investigating issues during the implementation.

As in every project, collecting the requirements is important. In OIM implementations, this is really important, more than that, documenting the requirements is important. Also, sufficient amount of testing is another consideration for OIM implementation projects. I will cover the logistic details of an OIM implementation in another post.

As the saying goes “The more you know, the more you know what you don’t know”. This is true for OIM (for so many other things in IT too). There are still some things I don’t know about OIM Access Policies. I am just working with OIM on what I know now (and still learning).  J

Okay. I hope that is it for this post. We will meet in another post with more interesting details about OIM. Until then

Read More about Oracle Identity Manager 

Checking on Oracle Fusion Applications

Sun Blogger Vijay Tatkar wrote in his blog about the eight technology innovations mentioned by Larry Ellision during his Oracle Open World Keynote speech during last week. Nearly half of them were Sun Hardware related (such as Exadata, ExaLogicSun ultraSPARC t3 etc). Here is the list:

1. Fusion Apps
2. Unbreakable Linux Kernel
3. Solaris Express 11
4. unltraSPARC t3 chip
5. mySQL 5.5
6. exadata
7. exalogic
8. Java 7 and 8

Since the beginning, I am always interested to know more about Fusion Apps, mainly out of curiosity. Oracle Fusion Applications were formally introduced during the Oracle Open World last week (during Open World 2010). As per Oracle Release, this was one of the major innovation or next big thing for Oracle. As you are aware, Fusion Middleware Products were released already. Now, it is time to talk about the Fusion Applications.
You may be already aware; I started my IT career as a Web Developer in a small web hosting company. I used to write perl CGI code and hosting them on Linux Servers running with Apache Web Server and mySQL Database. I got bored (or I wanted a change, I am not sure!) with that job and then moved into the Unix System Administration. I worked as a Sun Solaris Admin and HP Unix Admin for some time. Then I worked in both Peoplesoft System Administration for nearly 7 years and currently working in an Oracle Identity Management (which is part of Oracle Fusion Middleware products) project for nearly past one year.
So, the question is “now what?” And how can we develop the knowledge for Fusion Apps Administration.
I am not sure when Fusion Apps will be deployed full-fledged instead of the other ERP Applications like Peoplesoft. I don’t think it will be near soon, but may be after few years, Oracle may buy in customers who are going to do a new implementation of some ERP Applications.
You know what, Fusion middleware for Fusion Apps is like PeopleTools for Peoplesoft Applications. PeopleTools Technology is the abstract layer on top of which all the Peoplesoft Applications run on. PeopleTools was originally built on C and C++ and finally evolved into a Java Technology. However I still feel some of them are C++ code. While Fusion Middleware is more Java and J2EE apps, I believe Fusion Apps will be more J2EE apps than Peoplesoft. I need to spend little bit more time on implementing a Fusion Middleware and an application. As of now, I only worked on Identity Management Product Sure and little but of Oracle Portal Technologies.
For an IT Infrastructure Administrator like me (who mainly works on Oracle Server Technologies), I think understanding the Fusion Middleware Stack will be important.
Talk to you later. Until then

Read More about Oracle Fusion Applications

Oracle Internet Directory LDAP Relpica States in Fusion Middleware 11g

Oracle Internet Directory LDAP Relpica States in Fusion Middleware 11g (11.1.1)

In the Oracle Fusion Middleware 11g Documentation (I think I was referring to Version 11.1.1 of the doco), you can find OID Administrator’s Guide. As the name suggests, this is the top most important and valuable guide for Oracle Internet Directory Administrators. I think I have read most of this guide already. However I still refer this guide, since there is a lot of information provided in this guide (and it is a reference guide too).

I want to write about the LDAP Replica states mentioned in the Appendix D (How Replication Works) of this Guide. In Fusion middleware, Oracle provides lot of details about Oracle Internet Directory Replication. Earlier this information was scattered around the Oracle Support Website and was difficult to find. Now, I think Oracle collected most of this information in this guide.

If you are working or supporting or planning to implement an OID Replication High Availability environment, then you should be familiar with this section of the topic in the guide. This replica states information will be useful if you are running LDAP Based Replica (Just to refresh your memory, there are two types of Replication possible, ASR based and LDAP Based – ASR is based on Ddatabase Links, while LDAP based replication uses a LDAP Client process).

orclReplicaState Attribute

orclReplicaState attribute stores the Replication State for the LDAP Based Replication Replica. You can check the current Replica State of the OID using the ldapsearch command. (In a Live System that uses LDAP based replication, it will be set to the numeric value of 1 – which means it is in online state).

You need to run the following LDAPSEARCH and check the orclreplicastate attribute as shown below. Please make sure to replace values for the arguments specific to your site, I just gave an example.

ldapsearch -h localhost -p 389 -D cn=orcladmin -w password -b “orclreplicaid=local_replica_ID, cn=replication configuration” -s sub objectclass=*

You need to check the value of the orclreplicastate in the output. Alternatively, you can get the orclreplicastate attribute value directly as shown below example:

ldapsearch -h localhost -p 389 -D cn=orcladmin -w password -b “orclreplicaid=local_replica_ID, cn=replication configuration” -s sub objectclass=* orclreplicastate

The local_replica_ID is specific to your installation, normally it is machine_database. You can check the value using a ldapsearch query as shown below.

Ldapsearch Argument Description:

Argument	Description
-h	Hostname or IP Address of the LDAP Directory ServerI used localhost since I am running this command on the same server where OID is running.
-p	Port Number for the LDAP Directory, default LDAP port is 389, LDAPS port is 636.If you use the port 636, then you should define the –U argument.
-D	Bind DN – LDAP DN for connecting to LDAP Directory
-w	Password for the Bind DN – It is site specific.
-b	Base DN for the search – here it starts from the top.
-s base	Search Scope is base (other values are sub and one)

orclReplicaState possible values in 11g

There are 9 LDAP Replica States mentioned in this guide (In 10g OID, there are only 7 LDAP Replica states, it looks like Oracle added two more LDAP Replica states in 11.1.1). As I mentioned earlier, in a normal production system which uses LDAP based replication, the orclreplicastate will be set to the value of 1 automatically during the start of the replication server first time.

Let’s list the LDAP replica states:

LDAP Replica State	Description
0	Bootstrap - This is one of the important Value. You can setup a new LDAP based consumer replica using this value. Lets talk about it in next blog
1	Online – For regular replication processing.
2	Offline
3	Bootstrap in progress
4	Bootstrap in progress + cn=orclcontext completed
5	Bootstrap completed with failures
6	Database based
7	Sync Schema only (Not Data)
8	Bootstrap without schema sync (Only Data)

In a LDAP replication with high-availability environment, it is a must that you should understand these values and their significance. Let’s talk about these values and how we can exploit this attribute and their values in the coming blogs. Until then

Read More about Oracle Internet Directory

Moving to Oracle Server Technologies

Believe me; life is not easy when you are working with a Vendor Company, such as Hexaware Technologies that I work for (Hexaware is an Oracle Platinum Partner as well). I have to learn all the new things with the little time you get, sometimes you have to learn lot of things in less than few hours. For a person like me, this is exactly what I want and like to do. Learn new things all the time!!! That is my motto!

One thing I like the most here is, I have the freedom to move to other IT technologies that I have little or no experience with. However that was not easy for a person like me or anybody for that matter. You have to keep learning and understand new things that come up.

As you are already aware (or if you are reading my blogs for first time), I started my IT career as the Web Developer with Apache and Perl CGI development (really old technologies!!). After couple of years, I got bored with Web Development. Then, I moved into Unix System Administration, mainly worked on Solaris and HP-UX and related hardware and software. And again, I got bored with UNIX Administration and moved to Peoplesoft Infrastructure and Administrator positions.

I was a happy person (I am still happy!!) for almost 7 years working with Peoplesoft Infrastructure for many clients. Now, I got an opportunity to work in Oracle Server Technologies here, especially Oracle Database, Oracle Identity Management and Oracle Fusion Middleware technologies.

If you are in the IT industry, you have to know one thing for sure. Keep learning. We have to develop a mentality like kids have. They are always curious to learn new things and all the time. This is an important quality you have to develop if you want to excel in IT Technical career. You have to develop curiousness to learn new things (from internet, from other blogs, from collogues, from peers, from managers and almost everywhere!).

I started working in Oracle Server Technologies (Oracle Database, Oracle Application Server, Oracle Fusion middleware, Enterprise Manager etc ) less than a year ago. However, before starting, I had an fundamental understanding of what they are and why do we need them. You cannot build this in one day. You should be aware of other technologies. One major thing that helped me was, my UNIX skills. I am able to solve almost any problems if that runs on UNIX.

Two things you have to understand in UNIX World. Everything is handled as a file and everything runs in the server is a process. If you are able to make these two simple facts, then I am sure you will be able to fix any servers, anything that runs in UNIX/Linux.
Okay, I think we are going off topic. Other than books and internet, I use two simple ways of learning.

a) Blogging

b) Teaching/Mentoring

Both of these are not easy for me. I have to really develop mastery to some level before start teaching someone. Believe me, it is not easy to teach, especially in IT industry, it is difficult with all the new things popping up almost every second. That is why I wanted to start blogging more often and conduct more mentoring classes in Hexaware.

And now, within last one year, I have got quite an expertise on Oracle Server Technologies. During this time, my experience with UNIX, Web Development and Peoplesoft really helped a lot in understanding the architecture of the Oracle Server Technologies. I am still learning new things everyday (that is why I want to write here, at least I can use them later!).

I want to use this new blog site to start sharing knowledge, write about errors or failures and how we handle them (lessons learned) etc. I will start with a new topic here soon. Until then.

Read More about Oracle Server Technologies