Saturday, November 8, 2008

Note to myself

I sometimes take too long to answer comments. I will try to answer any queries that I know within maximum of 2 days from now on. Will try to maintain an average of 1 day.

Apologies to everyone whom I have given an delayed response.

Is your SOA suite is it secure?

I was trying to see if the servers we have is secure enough. What I wanted to know is that wether we were making some errors or the facts were not documented clearly enough. I did find the documentation regarding most problems in some document or the other, but is it clear for admins was the question.

So to find the answer I started looking for SOA suite available online.

Probably this notion would seem stupid why will anyone expose the SOA suite online, I thought that too. But I did an inurl:BPELConsole search and found atleast 10 websites that have their consoles being resulted in the search. Tried a few search engines including google.

The commonly known users are oc4jadmin, but what about bpeladmin and the default users. Nearly 80% of the sites did not have the passwords changed for the users. Could log in some of them seemed to be production sites others were testing sites.

Also a common error was the OWSM user admin with the password as oracle. Sadly most of these users were available to use.

I don't know how to contact the admins of these sites, anyone reading this blog please change the passwords. Go to the user management in the Application Server Console and remove all users that are not used or atleast change the passwords.

All sites had their oc4jadmin passwords changed but it's simply not good enough. Why were these consoles exposed to the internet I have no idea, I don't think it should be done.

Another probability is Denial Of Service attacks or brute force. I don't think (cannot confirm this though) the username for oc4jadmin can be changed. It makes brute force hack even simpler.

Is security still an aferthought for organizations !!! This scares me.



One of the sites I could log into. I have tried to remove all the information regarding who owns the site.

Saturday, November 1, 2008

Learning and implementing the Oracle Fault Management Framework

Why?
The process has some external dependencies, due to a network failure the partner system is down for a few days. The message sent to the partner can be sent later but recovery mechnism can be difficult and redundant in BPEL.

How was it done before?
Without the framework solutions that were possible were
1. Assume that the partner system link or the system would rarely be down, in such cases the process would have to be handled manually. Data fix, using the test case feature etc. This is called wishful thinking.
2. Retry then create a worklist task whenever there is error, requires programming effort and redundancy in process.
3. Rollback using compensation handler may be an option based on design of your process.

What is it in a nutshell?
Instead of handling faults in BPEL by adding catches handle faults use the framework to handle it for you. Both can also be used together. Retry of failed activity, Replay of failed activity scope, Human intervention and many other ways of handling the faults can be provided.

In detail it can be read fro references stated below.

What is the advantage?
1. Generic framework can be reused without coding effort.
2. Will provide resume, retry, continue and modify functionalities.
3. No BPEL change required

Lessons learnt and opinions
While implement the fault handling using this framework, found rather suprisingly that it was very easy to use. The only trouble I had was that in my patch of 10.1.3.3 the post installation steps had not been executed. Without these steps the framework does not catch the fault.

It was rather suprising that the framework can override the fault handling defined in the process. It was a bit difficult to digest but I could not think of any way else it could be designed.

The best resource I could find was http://www.it-eye.nl/weblog/2007/09/10/oracle-bpel-10133-fault-policy-management/
A very good resource to begin.
Also use Oracle documentation http://www.oracle.com/technology/products/ias/bpel/pdf/10133technotes.pdf is of great help.

Using these resources when I started testing my processes I wanted all my faults remote faults to be retried and then sent for human intervention.

But then I started facing issues when an synchronous process is invoked and the process waiting for human intervention the calling process is timed out. So I had to have a seperate policy for synchronous processes which only contain retries and another for asynchronous processes which contain retry and human intervention.

This reminded me of some Oracle document which had said prefer Async process over sync (I think it said it because of performance reasons).

Then I stumbled over an article http://orasoa.blogspot.com/2008/03/bpel-fault-policies-best-practise.html which validated my understandings.

If this works(in production) I'm pretty sure it will (It's working on my PC but cannot celebrate until it goes to production) I will be very pleased.

The framework has limited extensibility only java tasks can be used to extend the framework. It leaves very less scope for out of the box thinking.

Also, the problem I faced was that the activities tab in the BPEL Console needs to be provided to support personnel, none of the other tabs should be accessible. Could not resolve this issue :(. From the information I could gather was that this can only be done by tweaking the code of BPEL Console. ref: http://chintanblog.blogspot.com/2007/12/i-saw-numerous-people-asking-about-bpel_290.html

Special thanks:
To the Oracle team to come up with this feature and the blogs of consultants I have mentioned and all the people who answer questions on the Oracle forum.

Monday, August 4, 2008

Sash IP Messenger

Forces compelling to learn RCP:-
For quite some time I have been involved in creating helper projects, small projects that help in coding, generate code etc. This seemed to be a very nice and effective way of producing a very nice UI for them.
It’s eclipse; I like the IDE and is my IDE of choice for Java. Using RCP would mean I could integrate the tools written with eclipse.
I have done some Swing coding, it’s not a very easy to write a nice UI. RCP provided a dream of producing feature rich applications written in java fast and clean.
Since the release of eclipse 3 the documentation there have been major modifications in the way eclipse works (or so I have read) and made RCP integration easier. I started with reading a book Eclipse Rich Client Platform: Designing, Coding, and Packaging Java Applications - By Jeff McAffer, Jean-Michel Lemieux. Very nice book, made the journey simpler.
When I had just started coding one of my leads had told me, the best way to learn was to implement a small project. Execute it like a real project, and you would learn what are the where a tool can be best utilized, what is the effort required for implementation and any issues with the tool. Quite a few books are written in similar fashion where they build one application from start to the end. So does the book Eclipse Rich Client Platform. This book creates a chat application using the XMPP protocol and smack library.
I decided to create a UI for IP Messenger, since this is the messaging tool I usually use and it does not have any nice UI. This is a free messenger available and does not need a server.
I have just downloaded Ganymede so I created a new Eclipse plug-in project in Ganymede. I realized later that the version I should have used is Eclipse Europa since Ganymede is still to be used widely. I was able to convert the project into an eclipse 3.3 compatible plugin, the only drawback was that I could not use an image with menu manager. I had to replace

MenuManager helpMenu = new MenuManager("&Help", AbstractUIPlugin.imageDescriptorFromPlugin(Application.PLUGIN_ID, IImageKeys.HELP) ,"help");

I had to remove the image from the menu manager to use it with Eclipse Europa. Now it is compatible with both Eclipse Europa and Eclipse Ganymede.
The prerequisites for learning RCP are basic knowledge of Java, SWT and Eclipse. I am still learning new things about RCP everyday but creating a feature rich UI using RCP was fun and fast. The working RCP application can be found at: http://code.google.com/p/sashipmessenger/downloads/list
Features:-
Customized splash screen, the project uses a very simple splash screen. I have just replaced the splash.bmp file in the project. I decided not to use a progress bar, which can easily be added. More customizations are possible. While googling I also found a nice article that used the splash screen for login.



Saving the workbench state. Include the code in the class that extends WorkbenchAdvisor.

public void initialize(IWorkbenchConfigurer configurer) { super.initialize(configurer); configurer.setSaveAndRestore(true); }
This ensures that the windows appear in the same location as when you closed the views.

Use perspective to decide the initial layout of the application
Minimize to system tray. RCP provides a good level of integration with the system tray.



Also to provide a notification when a message has arrived and application is not visible or inactive, the icon changes to a message and a system beep is sound.


In the menu preferences have been added, the preferences configuration is as shown below. This integrates nicely with the eclipse editor if this is used as an eclipse plugin.



The configurations shown are the display name for the user, group name and the log folder. The messenger shows the chat history with a user. The broadcast address is used to specify address not in the same subnet.
The application has two different views and an editor. The contacts view shows all the users on IP Messenger. The console view at the bottom provides information about different events. The different events like message read event, user removed message added etc. can be seen. The editor is open whenever a message is sent or received from a user.


The messenger saves the chat log. When the editor is opened again the log is greyed out.


The views can be closed and moved around.


The view can be opened from the menu.



Users can be found from the find item on the top of contacts view.


The RCP application can be integrated with the existing eclipse. Just drop the plugin jar into the plugin folder of eclipse dump.

Tuesday, May 20, 2008

WS-BPEL 2.0

Derived from : http://docs.oasis-open.org/wsbpel/2.0/Primer/wsbpel-v2.0-Primer.pdf

Copyright © OASIS Open 2007. All Rights Reserved.

This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to OASIS, except as needed for the purpose of developing OASIS specifications, in which case the procedures for copyrights defined in the OASIS Intellectual Property Rights document must be followed, or as required to translate it into languages other than English.

What’s new in WS-BPEL 2.0
As a result of the OASIS Technical Committee’s issues process, the original BPEL4WS 1.1 specification has received several updates. The following list summarizes the major changes that have been incorporated in the WS-BPEL 2.0 specification.

Data Access
  • Variables can now be declared using XML schema complex types
XPath expressions are simplified by using the ‘$’ notation for variable access, for example, $myMsgVar.part1/po:poLine[@lineNo=3]
  • Access to WSDL messages has been simplified by mapping directly mapping WSDL message parts to XML schema element/type variables
  • Several clarifications have been added to the description of the activity’s semantics
  • The keepSrcElementName option has been added to in order to support XSD substitution groups or choices
  • The ignoreMissingFromData has been added to automatically some of operation, when the from data is missing.
  • An extension operation has been added to the activity
  • A standardized XSLT 1.0 function has been added to XPath expressions
  • The ability to validate XML data has been added, both as an option of the activity and as a new activity
  • Variable initialization as part the of variable declaration has been added
Scope Model

  • New scope snapshot semantics have been defined
  • Fault handling during compensation has been clarified
  • The interaction between scope isolation and control links have been clarified
Enrichment of fault catching model

  • A activity has been added to fault handlers
  • The has been added to scopes
  • The exitOnStandardFault option has been added to processes and scopes
Message Operations

  • The join option has been added to correlation sets in order to allow multiple participants to rendezvous at the same process with a deterministic order
  • Partner link can now be declared local to a scope
  • The initializePartnerRole option has been added to specify whether an endpoint reference must be bound to a partner link during deployment
  • The messageExchange construct has been added to pair up concurrent and activities
Important differences between BPEL 1 and 2 as imported from wsbpel-v2.0-Primer.

Most of the features added seems to be lessons learn't, a good way to improve. It does not drastically change the specifications but seems to be more friendly for the developer. Many problems that I am facing with the current implementation with BPEL 1.1 would be removed using this specification when the product vendor decides to implement it.

Sadly one point was missing support for XPath 2.0 and XSLT 2.0. The FAQs [http://www.oasis-open.org/committees/download.php/23858/WS-BPEL-2.0-FAQ.html] clearly state 'WS-BPEL2.0 is based upon XPath 1.0 and XSLT 1.0'. I thought XSLT 2.0 and XPath 2.0 were huge advancements but not being supported I was very disappointed. I believe the product vendors would still support them along with XPath 1.0 and XSLT 1.0 as standard extensions.

Monday, April 21, 2008

Oracle SOA suite - Retrieving process information

The problem statement
To retrieve the information of a process from dehydration store even when I did not have the instance id using data present in the request of the process. (This is a part of a bigger problem I am trying to resolve)

I have an a unique field in the BPEL input. How can I find the BPEL instance and the related instances (This BPEL is executed which in turn executes several other BPEL processes ). I could not modify the existing BPEL so I was left with only one option that was to use the Oracle BPEL Process Manager Client API. This API has very less documentation, most of the help I got was from some blogs specially http://orasoa.blogspot.com/2007/06/calling-bpelesb-webservice-from.html and the API documentation http://download-uk.oracle.com/docs/cd/B31017_01/integrate.1013/b28986/toc.htm.

Driving to the beach
The Marc Kelderman blog solved one of the biggest problems I faced the 'jar hell'. I had tried several other posts but was not successful in setting up the project correctly, but the jar files as specified by him worked wonderfully.

Finally my classpath contained :-
connector15.jar, ejb.jar, oc4j-internal.jar, optic.jar, orabpel.jar, orabpel-ant.jar, orabpel-boot.jar, orabpel-common.jar, orabpel-exts.jar, orabpel-thirdparty.jar, oracle_http_client.jar, orawsdl.jar, xmlparserv2.jar. I am using SOA Suite version 10.1.3.3.

Now I setup a new java project in eclipse with these jars in my classpath. I then setup a server configuration file

Getting your feet wet

## server_config.properties
java.naming.factory.initial=com.evermind.server.rmi.RMIInitialContextFactory
java.naming.provider.url=opmn:ormi://someserver.com:6004:oc4j_soadqa/orabpel
java.naming.security.principal=myname
dedicated.connection=true
java.naming.security.credentials=mypwd
Load the properties:-

 prop = new Properties();
InputStream resourceAsStream = BPELManagerControl.class
.getClassLoader().getResourceAsStream("server_config.properties");
prop.load(resourceAsStream);
The next piece of code I wrote was to retrieve all the instances of a specific BPEL process,



  Locator locator = new Locator("domainname", prop);
WhereCondition whereProcessId = new WhereCondition("process_id = ?");
whereProcessId.setString(1, "myprocessname");
IInstanceHandle[] instanceHandles = locator
.listInstances(whereProcessId);

Taking the dive

And using the instance handles I could display the states and instance id's of the processes. Now this was just the beginning of learning of how to use the API. I needed to modify it to suite my requirements. The previous piece of code returns all record in no particular order, this would make my task very difficult. By trial and error I realized that the API was using a view admin_list_ci to query and all its fields could be used in the query. Thus I added the following to order by creation_date (My instance would be one of the current instances in the server) desc. The next problem was if there was an error the list would continue processing infinitely. So I decided that my instance would be one of the last 50 instance executed on the server. This was a safe assumption since I would be searching immediately after submission. Thus my code became:-


   Locator locator = new Locator("domainname", prop);
WhereCondition whereProcessId = new WhereCondition("process_id = ?");
whereProcessId.setString(1, "myprocessname");
whereProcessId.append("ORDER BY CI_Creation_Date desc");
IInstanceHandle[] instanceHandles = locator
.listInstances(whereProcessId,0,50);
The next problem to resolve was ho do I find if the given IInstanceHandle handle was the instance I was searching for. I needed to search if my application specific id was present in the request of the instance.



  • The IInstanceHandle object had a getField method which seemed to suite my requirements (get the request variable and get the xml from it), but I realized it could only be used for a process that is not finished, thus had to drop the idea of using this method.
  • The only other way to get the data I could find was using the debug and audit xmls. In the BPELConsole along with the flow of the executed instance it also can display the Audit and Debug xmls. Corresponding methods were getAuditTrail and getDebugTrace that gave the dump of the whole BPEL instance data. I decided to use the getAuditTrail as the debug trace referred to the XML as an id (probably a refernce to some other table) which I could not find. Audit trail seemed to be working thus I decided to use it.
   String auditTrailXML = instanceHandle.getAuditTrail();
String XPATH = "//event[@label=\"receiveInput\"]/details/text()";
String receiveInput = Utils.xPathEvaluator(auditTrailXML, XPATH);
XPathFactory factory = XPathFactory.newInstance();
XPath xPath = factory.newXPath();

NamespaceContext ctx = new NamespaceContext() {
public String getNamespaceURI(String prefix) {
String uri;
if (prefix.equals("ns1"))
uri = "http://www.sash.com/Schema/Declaration";
else
uri = null;
return uri;
}

public Iterator getPrefixes(String val) {
return null;
}

public String getPrefix(String uri) {
return null;
}
};

xPath.setNamespaceContext(ctx);
String XPATH2 = "//ns1:appid/text()";
XPathExpression xPathExpression = xPath.compile(XPATH2);
String appid = xPathExpression.evaluate(new InputSource(
new StringReader(receiveInput)));

And from the appid compare with the appid we had and keep on looping until the instance is found. Thus I was successful in retrieving the instance id.

To find all the related instances find the current instance and get it's handle. Then use the following where condition to retrieve related instances:-

    WhereCondition wpi = new WhereCondition("ROOT_ID=?");
wpi.setString(1, instanceHandle.getRootId());

Conclusion

  1. The whole code was mostly based on trial and error and basic API documentation.
  2. The API can be very helpful but very difficult to use
  3. The view admin_list_ci in the BPEL dehydration store can be used to construct the where clause in the query.
  4. Setting up the project is not very simple. The jars have to be correct.
  5. Do you want to use the API? Depends on your requirement. I turned to this API when I had no other option and yes I was satisfied as it resolved my problem.

Friday, March 28, 2008

XSLT / XPATH 2.0

In my current project I was trying to replace a java transformation service to an XSLT. I faced some speed breakers. Googling I came to references of XSLT 2.0 which solves the problems that I was encountering.We are using the Oracle SOA suite 10.3.x. the editor (JDeveloper) only supports XSLT 1.0, but the interesting part is that in text editor changing version of XSLT from 1.0 to 2.0 the parser supports the newer version also.

I just needed group by function like the database. In an array I had to do a grouping. XSLT 2.0 brings the new for-each-group construct. This missing feature from the older xslt version had been a major drawback.

Another simple requirement I had was a sum of products. Say I have multiple items and I need to calculate the total price.


 <items>
<line_item>
<price>20</price>
<quantity>33</quantity>
</line_item>
<line_item>
<price>10</price>
<quantity>4</quantity>
</line_item>
</items>

To calculate the product (20*33 + 10*4) we can calculate it by using XPath . Evalaution of XPath sum(for $a in (//line_item) return ($a/price * $a/quantity)) gives the result. We can use this in our XSLT to calculate the value. The new version of XSLT/XPath brings in features that were long waited.

Some other features include :

  • Output multiple documents from a single transformation
  • Type awareness
  • The resultant tree created by querying the doc, can be queried like any other element
  • custom functions

I hope these standards are adopted soon by everyone with better tool support.