Integrating Legacy Applications with Oracle Applications via Oracle Fusion Middleware

I have been managing an integration requirement to automate the transactions sync between Oracle Applications E Business Suite MFG module and legacy mainframe application.

The solution is built on Oracle Fusion Middleware (OFM) SOA Suite (BPEL and Web Services), to interface between the Oracle Applications E Business Suite (Oracle Apps) and the legacy application.

Manufacturing related transactions - Item Master, Shipment, Purchase Order, Shop Order, Cost, and Internal Sales Order are created out of Oracle Apps into FTP site associated with the Oracle Apps. OFM sweeps them moving to legacy FTP sites. Programs on the mainframe load them into legacy application.

Internal Sales Order, PO Receipt, Safety Stock, Sales Order, Shop Order, and Inventory are synced from legacy applications to Oracle Apps. For this, first the files are moved from legacy FTP to the FTP site associated with Oracle Apps. Next the files are loaded into Oracle Apps using the standard EBiz APIs via Concurrent Requests.

Oracle Fusion Middleware Adapter for Oracle Apps provides the following interface types:
1. XML Gateway
2. e-Commerce Gateway
3. Business Events
4. Concurrent Programs
5. Interface Tables and Views
6. PL/SQL API

More about each of these interface types for another time...

In the current solution approach, combinations of option 4 and 6 concepts are used.
1. A PL/SQL script (Stored Procedure) acts as an abstraction layer between OFM and Oracle Apps.
2. The PL/SQL determines which concurrent request to submit within Oracle Apps based on the transaction specific file name that OFM provides as a input parameter.
3. The PL/SQL returns the status of the concurrent program execution along with the ‘request id’ as a return parameter.

Internally within OFM, the Stored Procedure invocation is encompassed into a Web Service. A WSDL is generated within BPEL Process Manager for the Stored Procedure. So whenever a particular transaction is synced from legacy to Oracle Apps, the Web Service is invoked within OFM passing the file name as a parameter. This Web Service invocation is in synchronous mode. The resulting ‘request id’ is communicated to system users via OFM as an email notification.

With this approach, the concurrent program details for each of the listed transactions and the required user responsibility are abstracted to the OFM. OFM uses just a database user account. Any changes in the concurrent programs, or its parameters, etc., does not warrant a change in the OFM, rather are isolated within the PL/SQL.

Hands on with BIT EME-CE’s Managed File Transfer

Installation was quick and easy. Following the BIT’s Quick Start guide, downloaded the zip file bit-eme-ce-1.0.2.zip. Unzipping the file created two folders, each for EME-CE and Protocol Proxy.

My installation structure:
C:\ bit-eme-ce-1.0.2\ eme-ce-1.0.2
C:\ bit-eme-ce-1.0.2\ protocol-proxy-ce-1.0.2

Pre-requisite for EME is JDK 1.5+. When EME starts, it refers to the environment variable JAVA_HOME. JAVA_HOME should point to a valid installed JDK root folder.

Started the Protocol Proxy service first through the start script C:\bit-eme-ce-1.0.2\protocol-proxy-ce-1.0.2\bin\startup.bat.

Next, started the EME service through the start script C:\bit-eme-ce-1.0.2\eme-ce-1.0.2\eme.bat. This is a wrapper to start the Mule.

Protocol Proxy starts the Tomcat web server and FTP server. EME starts the Mule ESB Server, Derby Database on port 1528, Jetty Web Server on port 8888 and ActiveMQ JMS Server.

EME connects to Protocol Proxy during its start up to load the MFT agents. It throws exceptions if it cannot connect to Protocol Proxy.

EME provides browser based interfaces for administration, configuring trading partners, translations, and viewing messages, etc.

The below lists the URLs hosted on the localhost with default ports:

Admin login to EME:
URL: http://localhost:8888/
Login name: admin
Password: password

EME Admin console has features to:
View the status of the Derby, Jetty and JMS services that were started along with EME
View the messages that were transacted with the Trading Partners
View the event log to monitor the errors
To create and manage Trading Partners, setting the profiles and XSLT based message translations
Create Users for EME and manage the ACL for Users

Admin interface to Protocol Proxy:
URL: http://localhost:80/proxy-admin
Login name: eme
Password: password

Protocol Proxy Admin console has features to create Trading Partner users for mainly for testing purposes, limiting to dropping files off and picking files up from Protocol Proxy, without actually being allowed into the enterprise/into firewall, where EME runs.

Trading Partner interface to Protocol Proxy:
URL: http://localhost:80/proxy-portal
Login Name:
Password:

Trading Partner console allows the Trading Partners to send or receive transactions via files with EME either in Binary or ASCII format.

Trading Partners can transact with EME either via the Trading Partner interface to Protocol Proxy over HTTP or over FTP.

When a new Trading Partner is created in EME, a set of folders are created as part of Managed File Transfer feature. By default, EME polls the Proxy for new messages from Trading Partners for every 60 seconds. EME CE reads files from and writes files to directories in the default setup for Managed File Transfer.

{BIT_EME_CE_HOME}/eme-ce-{version}/MFT//in
{BIT_EME_CE_HOME}/eme-ce-{version}/MFT//out

After the start-up, logged into EME Admin console and created a new Trading Partner by the name TP1. Enabled the flag that links the integration for this trading partner between EME and Protocol Proxy.

Logged into Trading Partner Console as TP1. Uploaded a test file with the file name “from_tp1_to_eme.txt” using the Upload.

EME in its schedule poll from Proxy retrieved the file from TP1 and moved it to MFT folder under C:\ bit-eme-ce-1.0.2\ eme-ce-1.0.2\MFT\TP1\in. The file was also listed in the EME Admin console under Messages tab with a link to download.

Next, logged into Protocol Proxy Admin console, and uploaded a file to TP1.


EME transmitted the file “from_eme_to_tp1.txt” to TP1. The file from EME was visible on the Proxy Portal for TP1 to view and retrieve, as well as on the FTP.

Observations:

1. EME has a lot of potential. With its Managed File Transfer, support for various protocols, message translation capabilities the possibilities are endless.

2. The HTTP(S) wrapper to the trading partners to do the standard FTP operations of file upload and download worked right out of the box. Managed File Transfer allows monitoring and controlling of messages in one place. The Message Viewer with its filtering options has potential to view historical transactions and its status. I have not yet explored further to understand the options for purging old messages along with its tranactional data.

3. Tested uploading files with varying sizes through the Proxy Portal using the TP login, starting with few kilo bytes and up to 2 mega bytes and these got transferred to EME flawlessly. However it failed to move a file with 3 MB with an exception at Mule level while reading the file from the trading partner. This test was run on Intel Duo 2.53Ghz with 4 GB RAM on Windows XP.

4. On the Proxy Portal for a TP User, the link “Outbox” lists the messages received from EME, and the link “Inbox” lists the messages sent to EME. The names seem to have interchanged. Inbox from TP perspective should relate to messages arriving to TP. Outbox for messages sent by TP.

5. EME Admin console and Proxy Portal UI is fairly simple and basic.

6. It would be helpful if the audit messages displayed in EME Admin console can be customizable. The search feature to list messages in the EME Admin console between the date ranges did not return correct values.

7. The Trading Partner or User management is limited. There is no option to delete trading partners or users. The available option is to turn off the ‘enabled’ flag.
I have not yet tested the message translation or Protocol Proxy deployment on HTTPS. There is a support for XSLT based translations. As well as supports HTTPS by means of configuring Tomcat for SSL. However, the current version does not provide secure FTP.


Response from BIT mentioned that they have recently tested the newest version and successfully transmitted files up to 1+ gigabytes. The newer version would also provide interface to customize audit messages to track the activities in the EME Admin console, and other enhancements. The 'Outbox' and 'Inbox' labels in Proxy Protocol is something customizable when implemented in a organization.

The new release version 1.1 is scheduled for the first of week of May 2010. Will have to wait for it to try it out...

EME by BIT

I am evaluating a product of Business Integration Technology (BIT) (http://businessintegrationtechnology.com), a business-to-business integration suite called Enterprise Messaging Engine (EME).

My initial intent is to understand the BIT EME solution and use as a benchmark versus what we are doing with Oracle Fusion Middleware. Look at the Open Source offering to see how it can complement Oracle Fusion Middleware or supplement in certain areas. The managed file transfer (MFT), a solution of EME that is something we don’t have now and will be looking closer at this component.

A brief on BIT EME:

EME is BIT’s open source based product powered by Mule (from MuleSoft) the open source enterprise service bus. There are two variations: EME Enterprise Edition which is fully supported complete solution, and EME Community Edition which is a complementary offering of Enterprise Edition for architects to evaluate. Current EME-CE version available is 1.0.2

The EME includes these additional features:
Protocol Proxy for a secure Internet point-of-presence, fully integrated with EME and Mule.
Managed File Transfer solution based on Mule/EME and Protocol Proxy.
Message Viewer for visibility to all enterprise messaging.
Business Documents Processor (BDP) to extend messaging to business users.

The architecture of BIT EME-CE:

EME is built on Mule open-source ESB providing integration framework. It leverages Apache Derby Database, Jetty Web Server, and ActiveMQ JMS Server.

Apache Derby is a Java based open source light weight database, Jetty Web Server is Apache's Java based open source HTTP and Servlet container, ActiveMQ JMS Server is Apache's Java based open source JMS server.

Protocol Proxy is a lightweight Apache Tomcat web server to provide the standard protocol services (FTP, SFTP, HTTP, HTTPS). It is deployable in the DMZ, securing the connectivity from external trading partners. The Protocol Proxy runs Web and FTP servers. By default web runs on port 80. It can be SSL enabled by configuring SSL for Tomcat. FTP runs on default port 21.


Next: Hands on with BIT EME-CE’s Managed File Transfer Service...

Hello Blog World!

My first blog post!