Tag Archives: soa

Deploying ESB projects using Ant

Posted on by
Reply

This post is to summarize and correct some of the information which can be found on this thread: Dealing with changing service wsdl locations in the ESB. (A big thanks to all the people who participated on this thread!)

Let’s start with some issues a lot of people, including me, have run into.

jaxb.properties not found

If you’re using Ant 1.6.5, you need to put some libraries on the classpath before starting ant. In the OTN forum thread some people state that you need to use Ant 1.7, and although Ant 1.7 has some improvements in how it handles classpaths, you do not need to use it. You just need to make sure that the oracle xmlparserv2 jar, the jaxb jars (i’m using version 2.0.5), and apache commons codec jar is on the classpath. In JDeveloper you can do it as displayed below (click image for full size).

Ant esb classpath

NullPointerException when trying to extract a deployment plan

This is caused by a bug in an older version of ESBMetadataMigration.jar. As is mentioned on the thread, this can be solved by using the jar included with Soasuite 10.1.3.3. (you can find it in soasuite/integration/esb/deployment/documentation.zip).

Error detected while reading the Deployment Plan

If you see this error when deploying an ESB, you can solve this by running Ant in a separate JVM. You can specify this in the project properties as is displayed below.

Ant esb jvm

Dynamically determining ESB component GUIDs

Ant has some default tasks that you can use to determine the GUID of a esb service. Using xmlproperty you can read an xml file and extract the value of a guid. Here’s an example:

<xmlproperty file="MyEsb.esbsvc" prefix="myesb"/>
<property name="myesb.guid" value="${myesb.service(guid)}"/>

Complete example

Since Ant 1.6 you have the ability to define macros. Using some simple macros deploying esbs becomes quite simple. Here’s an example:

<target name="deploy">
    <!--
      == Undeploy esb components, by determining the guid from the esbsvc file
      -->
    <xmlproperty file="MyEsb.esbsvc" prefix="myesb"/>
    <property name="myesb.guid" value="${myesb.service(guid)}"/>
    <undeploy-esb guid="${myesb.guid}"/>
    <!--
      == deploy esb project
      -->
    <deploy-esb/>
</target>
<target name="extract.esb.deployment.plan">
  <!--
    == extract deployment plan from deployed esb component
    -->
  <extract-esb-deployment-plan/>
</target>

ESB deployment macros

I’m using the following macro definitions:

<project default="init" name="build-macros" xmlns:esb="antlib:oracle.tip.esb.client.anttasks">
  <taskdef uri="antlib:oracle.tip.esb.client.anttasks"
           resource="oracle/tip/esb/client/anttasks/antlib.xml"
           loaderref="oracle.esb.ant">
    <classpath>
      <path refid="oracle.esbant.lib"/>
      <path refid="apache.commons.httpclient.lib"/>
      <path refid="apache.commons.logging.lib"/>
      <path refid="apache.commons.codec.lib"/>
      <path refid="oracle.esb.lib"/>
      <path refid="java.jaxb.lib"/>
      <path refid="oracle.xmlparser.lib"/>
    </classpath>
  </taskdef>
  <!--
    ============================================================================
    == deploy esb
    ============================================================================
    -->
  <macrodef name="deploy-esb">
    <attribute name="deploymentplanFile"
               default="${basedir}/${src.dir}/${ant.project.name}-esbplan-${build.env}.xml"/>
    <attribute name="esbHost" default="${integration.server.host}"/>
    <attribute name="esbPort" default="${integration.server.port}"/>
    <attribute name="esbAdminUser"
               default="${integration.server.admin.username}"/>
    <attribute name="esbAdminPassword"
               default="${integration.server.admin.password}"/>
    <sequential>
      <echo>Deploying esb project ${ant.project.name} using plan @{deploymentplanFile}</echo>
      <property name="esbplan" location="@{deploymentplanFile}"/>
      <esb:deployESBSuitcase esbmetadataserverhostname="@{esbHost}"
                             esbmetadataserverport="@{esbPort}"
                             username="@{esbAdminUser}"
                             password="@{esbAdminPassword}"
                             deploymentplanfilename="${esbplan}"
                             sourcedirectory="${basedir}"></esb:deployESBSuitcase>
    </sequential>
  </macrodef>
  <!--
    ============================================================================
    == extract esb deployment plan
    ============================================================================
    -->
  <macrodef name="extract-esb-deployment-plan">
    <attribute name="templateDeploymentplanFile"
               default="${basedir}/${srcgen.dir}/${ant.project.name}-esbplan-template.xml"/>
    <sequential>
      <mkdir dir="${srcgen.dir}"/>
      <delete file="@{templateDeploymentplanFile}"/>
      <echo>Creating deployment plan @{templateDeploymentplanFile}</echo>
      <esb:extractESBDeploymentPlan sourcedir="${basedir}"
                                    deploymentplanfile="@{templateDeploymentplanFile}"/>
    </sequential>
  </macrodef>
  <!--
    ============================================================================
    == undeploy esb
    ============================================================================
    -->
  <macrodef name="undeploy-esb">
    <attribute name="guid"/>
    <attribute name="esbHost" default="${integration.server.host}"/>
    <attribute name="esbPort" default="${integration.server.port}"/>
    <attribute name="esbAdminUser"
               default="${integration.server.admin.username}"/>
    <attribute name="esbAdminPassword"
               default="${integration.server.admin.password}"/>
    <sequential>
      <echo>Undeploying ESB from @{esbHost}</echo>
      <esb:undeployESBEntities esbmetadataserverhostname="@{esbHost}"
                               esbmetadataserverport="@{esbPort}"
                               username="@{esbAdminUser}"
                               password="@{esbAdminPassword}">
        <system guid="@{guid}"/>
      </esb:undeployESBEntities>
    </sequential>
  </macrodef>
</project>

Most macros have sensible default values for the attributes which usually means you don’t have to specify any attributes. The properties used in the defaults are read from a property file.

The deployment macro expects a deployment plan in the src directory. You need to specify a build.env property to tell it which deployment to use.

Using XMLType, XMLDB and XQuery

Posted on by
Reply

This is a small example for a coworker, but maybe someone else can also benefit from it. This question was “can we store multiple xml document types in one column and easily query on xml contents”?

First create a table with an XMLType column, and insert some data:

SQL> create table mydocs
  ( id number primary key, doc xmltype);

Table created.

SQL> insert into mydocs values
  ( 1, XMLType('< ?xml version="1.0"?><order><productid>2</productid><amount>5</amount></order>'));

1 row created.

SQL> insert into mydocs values
  (2,XmlType('< ?xml version="1.0"?><customer><name>Pete Jansen</name><phone>+31-30-1234567</phone></customer>'));

1 row created.

Query documents using an xpath expression in the where clause:

SQL> set long 5000
SQL> column xml format a40
SQL> select id, doc as xml from mydocs where existsNode(doc,'/order/productId/text() = "2"') >= 1
SQL> /

        ID XML
---------- ----------------------------------------
         1 < ?xml version="1.0"?>
           <order>
             <productid>2</productid>
             <amount>5</amount>
           </order>

Query values from xml documents:

SQL> select id, extractValue(doc,'/order/productId') as xml from mydocs;

        ID XML
---------- ------------------------------
         1 2
         2

Using xquery to retrieve xml documents:

SQL> select xmlquery('for $i in ora:view("MYDOCS")/ROW/DOC return $i' returning content) as xml from dual
SQL> /

XML
----------------------------------------
<doc><order>
  <productid>2</productid>
  <amount>5</amount>
</order>
</doc><doc><customer>
  <name>Pete Jansen</name>
  <phone>+31-30-1234567</phone>
</customer>
</doc>

And, to continue with the example from my previous post, query the same data using XQLPlus:

E:oraclejdev1111r2lib>java -classpath xquery.jar;xmlparserv2.jar;xsu12.jar;..jdbclibojdbc14.jar;..jliborai18n-collection.jar
;..rdbmsjlibxdb.jar oracle.xquery.XQLPlus

XQuery Command Line Tool
Version 1.0
Enter XQuery statements followed by /;

XQL> set sqlconn default jdbc:oracle:thin:test/test@ora11:1521:orcl
XQL> for $i in ora:view('MYDOCS')/ROW/DOC return $i
  2  /;

Result
----------------------------------------------------
<doc>
   <order>
      <productid>2</productid>
      <amount>5</amount>
   </order>
</doc>

<doc>
   <customer>
      <name>Pete Jansen</name>
      <phone>+31-30-1234567</phone>
   </customer>
</doc>

Executing XQuery statements with XQLPlus

Posted on by
Reply

Oracle’s xquery implementation contains a sqlplus like tool which you can use to run simple xquery statements. You can find the xquery implementation in %JDEV_HOME%/lib. To run it, use the following statement:

E:oraclejdev1111r2lib>java -classpath xquery.jar;xmlparserv2.jar;xsu12.jar;..jdbclibojdbc14.jar;..jliborai18n-collation.jar
oracle.xquery.XQLPlus

XQuery Command Line Tool
Version 1.0
Enter XQuery statements followed by /;

XQL> help

XQuery Command Line Tool
Enter XQuery statements followed by /;
To execute queries in a file from the XQL prompt, enter @ followed by the <filename>
To execute queries in a file from the shell prompt, enter java XQLPlus </filename><filename>
Set command helps set environment variables:
set sqlconn default <jdbc -connect-string>
set sqlconn <name> <jdbc -connect-string>
set echo   (ON | OFF)
set timing (ON | OFF)
set var <variable name> <value>
set initialctx <filename>

A simple xquery example:

XQL> for $i in <doc><row>text1</row><row>text2</row><row>text3</row></doc>//row return $i
  2  /;

Result
----------------------------------------------------
<row>text1</row>

<row>text2</row>

<row>text3</row>

You can use xquery to query data in a database table. First you need to specify the database connection using set sqlconn. when you’ve set the database you can use ora:view to query data from a table:

XQL> set sqlconn default jdbc:oracle:thin:jat/jat@xesoa1:1521:xe
XQL> for $i in ora:view('BOOKS') return $i
  2  /;

Result
----------------------------------------------------
<row>
   <id>1</id>
   <title>Processing XML with Java: A Guide to SAX, DOM, JDOM, JAXP, and TrAX</title>
   <author_id>1</author_id>
</row>

<row>
   <id>2</id>
   <title>Java &amp; XML, 2nd Edition: Solutions to Real-World Problems</title>
   <author_id>2</author_id>
</row>

<row>
   <id>3</id>
   <title>Java, XML, and the JAXP</title>
   <author_id>3</author_id>
</row>

You can also use xquery to query data from xml documents:

XQL> for $i in doc('file:///E:/projects/xquery-demo/xquerydemo-1/src/nl/iteye/xquerydemo1/books.xml')//book where $i/title/text() =
'Xml primer' return $i
  2  /;

Result
----------------------------------------------------
<book category="WEB">
   <title lang="en">Xml primer</title>
   <author>Erik T. Ray</author>
   <year>2003</year>
   <price>39.95</price>
</book>

By default xquery statements are executed by the client side xquery implementation. You can also execute your statements in a database server:

XQL> set server on
  Server Side Execution
XQL> set sqlconn default jdbc:oracle:thin:hr/hr@ora11:1521:orcl
XQL> for $i in <doc><row>1</row></doc> return $i
  2  /;

Result
----------------------------------------------------
<doc><row>1</row></doc>
XQL>

Of course, you can also use sqlplus to execute xquery statements on an Oracle database server.

Btw, you should be able to specify variable values using “set var”, but i haven’t been able to get that to work. And after looking at the source code, i don’t think that’s implemented, at least for client side xquery statements.

Amazon and SOA

Posted on by
Reply

ZDnet has an interesting piece on the use of SOA at Amazon: SOA done right: the Amazon strategy. It’s a summary of a presentation at the Google Seattle Scalability Conference. Although the article is a bit light on details, it does contain some interesting info.

Amazon is a big user of SOA:

Every Amazon web page calls on at least 150 services.

150 seems like a lot, although every page at Amazon does contain a lot of information. It be interesting to know what these services are used for.

The article then goes on to state that you need to have SLA’s for your services:

resource sharing without contracts kills scaling

Rene Wiersma recently posted a message on this subject: SOA Best Practices # 5: Determine SLA’s listing some items which we think need to be in the SLA of a service. The Amazon article mentions the following:

The contract covers what the service promises against certain demand levels. The key is that both the demand side and the supply side are covered by the contract.

One final quote i would like to share:

Saving state to a database guarantees scaling failure.

I hope the presentation will be on youtube soon, worth watching.

Update:
This article has some details on Amazon’s SOA use: A Conversation with Werner Vogels

SOA best practice #9: Use a Canonical Data Model

Posted on by
Reply

The Canonical Data Model pattern is very useful for soa projects. You can find a thorough description of it in Enterprise Integration Pattern (pages 355-360). The goal of this pattern is:

How can you minimize dependencies when integrating applications that use different data formats? …Design a Canonical Data Model that is independent from any specific application. Require each application to produce and consume messages in this common format

EAI tools use canonical models heavily. In a hub and spoke integration solution all messages are first translated into an application independent message format and then into the required format. This way you have less dependencies than in a peer-to-peer integration approach. It’s easier to add new applications if all you have to do is add a translation from a new application to the common model and back.

How does this work on a soa project? The integration functionality, often implemented using bpel processes, is a integration layer which communicates at it’s edges with all sorts of services. These services may internally use all sorts of formats, but your bpel processes shouldn’t be bothered with this. You need to keep this complexity outside your bpel business processes. This means you need to transform the messages to a common message format. This is best done by transforming the messages to a common data format. Either by using an ESB or by creating some custom code.

What you are basically creating is a layer between your integration processes and the services, a transformation or mediation layer. The integration layer is used to define business processes on top of heterogeneous services, the transformation layer is used to hide the heterogeneousness of the services to the integration layer.

Performance problems running SOA Suite on VMWare

Posted on by
Reply

For development and testing purposes i’m using vmware to run Oracle’s SOA suite on my laptop. I’ve installed it on Centos. When running this on my laptop, the performance isn’t really great, the oc4j processes are using a lot of cpu time (continuously 50%). My initial thought was that this was related to my situation: running it on a laptop and not giving it enough memory (the vmware instance has about 1Gb allocated, the laptop has 2Gb). But recently we’ve seen these performance problems at other locations, where the vmware instance has enough memory (2.5Gb). A search on google shows that more people have problems using Oracle’s SOA suite on vmware: Poor performance of oracle soa-suite in vmware.

Has anybody else experienced this problem? Have you found any solutions?

Oracle open sources Toplink

Posted on by
Reply

The part of Toplink which implements JPA, Toplink Essentials, was already open source, and part of the JEE 5 reference implementation, but today Oracle announced that it will completely open source Toplink (pressrelease ). Toplink not only implements JPA, it also support JAXB, SDO and JCA, which means that it’s a pretty complete persistency solution. Oracle is also increasing it’s contribution to Eclipse, by joining the board of directors of the Eclipse foundation, and by creating an Eclipse project called Eclipse Persistence Platform based on Toplink (FAQ (pdf)).

What does this mean? That Oracle is still focusing on the open source part of the Java world. They are not converting a lot of java programmers to JDeveloper and ADF, with ADF being tied to JDeveloper. Instead, open source java programmers are targeted through Eclipse, by adding all the Oracle technologies that may interest 3gl java programmers this IDE.

It also means there is less of a barrier to use Toplink extensions on projects. I always prefer to stick to the standards whenever possible. But JPA does have some missing pieces, which Toplink, but also Hibernate, fixes by included some non standard functionality. Support for stored procedures is one example. In the past I’ve hesitated to use these extensions, because it limits you how you can distribute your application. For example, we have a development server virtual appliance we use on most projects, which includes version control, bug database, build server, etc. This is all based on open source, because it allows you to take it to all your customers. Now we can include applications based on Toplink in this vmware machine.

ADF Training

Posted on by
25

I just finished giving an Oracle ADF training. This is a 3 day training were people learn to use EJB3, JPA, JSF, ADF Faces and ADF Data Controls and Data Bindings. This is a lot of material for 3 days, next time we’ll probably extend the training to 4 or 5 days.

As always, the students aren’t the only ones who learn a lot during a training. I always find that whenever I’m giving a training I’m learning a lot too. To be able to explain how something works, and how you should use it, means you have to really understand it. Just being able to use it, doesn’t mean you understand it well enough to explain it.

I created the training material myself, which was also quite a challenging task. There’s so much information available about the subjects, what do you put in your slides and what do you skip? In the beginning i just put everything i thought relevant on the slides, but this is not a good way to learn about ADF.

So i turned it around, it’s not the slides nor the information that is important. What is important is that the students learn how to use the tools to create the applications that they need. This means that the core of the training should be the exercises. If a student can complete an exercises it means he’s able to use the technology. So i created a lot of exercises, and only put in my slides whatever i though necessary for the students to be able to complete the exercises (to be honest, i still have some slides in there that aren’t really necessary for the exercises, these will be removed the next time i give the training).

It’s also important that the exercises slowly become more complex, it’s a step by step process. I read an interesting statement some time ago on a forum which said that you can only learn what you almost know. I think this is very true, and it means that a training needs a lot of exercises that slowly expect more knowledge of the subject.

Another interesting fact i learned during the training is that ADF Business Components and JHeadstart are quite popular. Personally, I prefer to use standard solutions whenever a good standard solution is available. For example, Java Persistency API. Before JPA, java didn’t have a good standard and broadly supported solution for ORM, so better alternatives became mainstream. But in my opinion JPA is good enough for most applications, so for me JPA is the default pick, unless there’s a good reason to use something else. Using standards has the advantage that there’s a lot of information available, and it’s easy to find developers, which is quite important in today’s market. For nonstandard niche solutions it’s a lot harder to find support and developers.

But most of the students in the training had a Oracle Forms Developer and Designer background, and they really like ADF Business Components and JHeadstart. Business Components because it allows them to reuse their extensive SQL knowledge, and JHeadstart because it’s the only tool that gives them the generation productivity similar to Oracle Designer.

Oracle JDeveloper 10.1.3.2 and WebCenter available

Posted on by
5

Just read on Steve Muench’s weblog that Oracle has released JDeveloper 10.1.3.2 (Fix List, New features in 10.1.3.2) and WebCenter Suite.

The new features of JDeveloper include improved Portlet development support, you can now use JSF for portlets. It also includes Wizards to create Web 2.0 mashups, it’ll be intersting to find out what they actually mean by this.

Another new item in JDeveloper is Oracle MDS, Oracle’s Metadata Services. I have heard of this before, but i thought it was a feature scheduled for the 11 release.

Also included in JDeveloper 10.1.3.2 is declarative security for ADF. This might be interesting. The last time we tried to implement security on a ADF Faces application we had to custom build part of the security as the default JEE declarative security didn’t integrate well with JSF. It looks like this can now all be done declaratively.