Figure 3 – Target Processing

 

Each Canonical is received by the Target service and processed per Target requirements, then written to disk.

 

Figure 4 – Target Batching

 

Each Large Transaction Notification is received by the Target package.  The batching service verify that all the reported “part” of the original message have been processed and will then batch the data to the target system.

 

Broker: The webMethods Broker is the hub of the system. Its main purpose is to exchange documents between components that are connected to it. The Broker is provided by the webMethods Integration Platform software. It stores the webMethods Documents that are related to the interface point. All webMethods Documents are stored in the client queues of the Broker and then dispatched to the components that subscribe to these documents. For further information, please see the webMethods Integration Platform documentation.

Integration Server: The webMethods Integration Server hosts services that contain the logic of the interface point. It uses a JDBC Adapter to connect to the databases. Subscription is performed through conditional trigger that invokes the appropriate services.

 

 

1       Document Definitions

1.1         LXKLDH.LargeDocumentHandling.docs:processNodeNotification

The enterprise document : LXKEnterprise.docs:processNodeNotification is used to transmit information regarding the node written to file to the service that will process that node. Each enterprise document contains a single instance of the relevant data set.

1.2         LXKLDH.LargeDocumentHandling.docs:largeDocNotification

The enterprise document : LXKEnterprise.docs:largeDocNotification is used to transmit information regarding the original node that was processed on the Source side to the service on the Target side that will batch and send the Target data. Each enterprise document contains a single instance of the relevant data set.

 

 

1       Steps to Implement

1.1.1    Integration Analysis

The following must be defined in order to utilize the large document/transaction handling:

§  Definition of a transaction within the source canonical.

§  Maximum size of source transaction (can it exceed current thresh hold?).

§  Definition of line item within source transaction.

§  Requirements for target transaction (does the transaction need to be processed as a whole on the target side?).

§  Mapping  for source to target (elements from source that need to be passed to target).

1.1.2    Large Document/Transaction Type

There are three different “flavors” of handling large documents and transactions.  Follow the flow below to determine the appropriate process to implement for your project

 

 

1.1.1.1      Large Document Handling – Source

This will split the source document by the defined transaction and process each transaction independently to the target.

§  Create the source package for your project.

§  Invoke the LargeDocHandling.SetUp:InitNewLargeDocHandlingPackage with the following inputs

o    Package – the name of the source package

o    Interface Type = “source”

o    documentTypeName = namespace of source canonical

o    transactionElement = element that defines a single transaction in the source canonical.

o    transactionElementKey = attribute that holds unique id for transaction

o    processNodeServiceName = the name of the flow service that will be created to process the single source transaction

o    splitNodeServiceName = the name of the flow service that will be created to split the source document into single transactions

o    splitTransactions = “false”

o    lineItemElement, lineCount, and transactionHeaderAttributes = null

§  Update the template service with steps for processing and publishing source transaction

1.1.1.2      Large Transaction Handling – Sourc§

  Create the source package for your project.

§  Invoke the LargeDocHandling.SetUp:InitNewLargeDocHandlingPackage with the following inputs

o    Package – the name of the source package

o    Interface Type = “source”

o    documentTypeName = namespace of source canonical

o    transactionElement = element that defines a single transaction in the source canonical.

o    transactionElementKey = attribute that holds unique id for transaction

o    processNodeServiceName = the name of the flow service that will be created to process the single source transaction

o    splitNodeServiceName = the name of the flow service that will be created to split the source document into single transactions

o    splitTransactions = “true”

o    lineItemElement = element that defines the line item within a transaction

o    lineCount = number of lines to process for each split transaction

o    transactionHeaderAttributes = List of attributes from source transaction that are required for target

§  Update the template service with steps for processing and publishing source transaction

1.1.1.1      Large Transaction Handling – Target

§  Create the target package for your project.

§  Copy and rename the following services to your target package:

o    LargeDocumentHandling.SetUp.templates.target:batchToTarget (Copy to .utils)

o    LargeDocumentHandling.SetUp.templates.target:processLTNotification (Copy to .main)

o    LargeDocumentHandling.SetUp.templates.target:processTargetData (Copy to .main)

o    LargeDocumentHandling.SetUp.templates.target:subscribeLagreProduct (Copy to .triggers)

o     

§  Update the template services with steps for processing the target transaction

 

1       WebMethods IS Components

1.1         Packages and Folders

-          Packages/Folders:

§  LXKLDH

o    LXKLDH.LargeDocumentHandling

o    LXKLDH.LargeDocumentHandling.docs

o    LXKLDH.LargeDocumentHandling.triggers

o    LXKLDH.LargeDocumentHandling.SetUp

o    LXKLDH.LargeDocumentHandling.SetUp.templates

o    LXKLDH.LargeDocumentHandling.SetUp.templates.source

o    LXKLDH.LargeDocumentHandling.SetUp.templates.target

1.1.1    Adapter Connections, Services, and Notifications

There are no adapter connections, database services, or notifications utilized for implementing large document/transaction handling.

 

1.1.2    Documents

1.1.2.1   Enterprise Documents

These webMethods Enterprise Documents represent the Enterprise Data for the appropriate business objects, they are publishable to the Broker where noted

1.1.2.2   LXKLDH.LargeDocumentHandling.docs:largeDocNotification

This document is used to publish the notification that all parts of a single transaction have completed processing and been published.  This document is only used when splitting transaction.  This document is publishable to wM Broker.

1.1.2.3   LXKLDH.LargeDocumentHandling.docs:processNodeNotification

This document is used to publish the notification that a single transaction has been written to disk and is ready to be processed.  This document is publishable to wM Broker.

1.1.3    Template Flow Services

1.1.3.1      Large Documents Handling Services (LXKLDH.LargeDocumentHandling.SetUp.templates.source)

These template services are used for implementing Large Document Handling.

1.1.3.1.1       LXKLDH.LargeDocumentHandling.SetUp.templates.source:receiveBizDoc

This is the template service that will be invoked from TN for the specific integration.  It will call

 

LargeDocHandling:SplitInboundNode.

1.1.1.1.1       LXKLDH.LargeDocumentHandling.SetUp.templates.source:processNode

This is the template service that will be invoked from trigger.  It is the template for processing and publishing the source transaction.

1.1.1.2      Larger Transaction Handling – Source (LXKLDH.LargeDocumentHandling.SetUp.templates.source)

These template services are used for implementing source side, Large Transaction Handling.

1.1.1.2.1       LXKLDH.LargeDocumentHandling.SetUp.templates.source:receiveBizDoc

This is the template service that will be invoked from TN for the specific integration.  It will call LargeDocHandling:SplitInboundNode.

1.1.1.2.2       LXKLDH.LargeDocumentHandling.SetUp.templates.source:processNode_splitTransaction

This is the template service that will be invoked from trigger.  It is the template for splitting, processing and publishing the source transaction and transaction notification.  It calls LargeDocHandling.SetUp.templates.source:mapDataNode and LargeDocHandling.SetUp.templates.source:publishCanonical.

1.1.1.2.3       LXKLDH.LargeDocumentHandling.SetUp.templates.source:mapDataNode

This is the template service that will map the transaction header attributes.  It is invoked from LargeDocHandling.SetUp.templates.source:processNode_splitTransaction

1.1.1.2.4       LXKLDH.LargeDocumentHandling.SetUp.templates.source:publishCanonical

This is the template service that will publish the split transactions.  It is invoked from LargeDocHandling.SetUp.templates.source:processNode_splitTransaction

 

1.1.1.3      Larger Transaction Handling – Target (LXKLDH.LargeDocumentHandling.SetUp.templates.target)

These template services are used for implementing target side Large Transaction batching.

1.1.1.3.1       LXKLDH.LargeDocumentHandling.SetUp.templates.target:processTargetData

This is the template service that will subscribe to the split transactions.  It will convert the source transactions to the target format and write the records to disk.

1.1.1.3.2       LXKLDH.LargeDocumentHandling.SetUp.templates.target:processLTNotification

This is the template service that will subscribe to the Large Transaction notifications.  It will verify that all of the transaction parts have completed processing and call the service

1.1.1.3.3       LXKLDH.LargeDocumentHandling.SetUp.templates.target:batchToTarget

This is the template service that will concatenate all of the target transaction parts and send the complete transaction to the target.

 

 

 

1.1.2    Template Triggers

1.1.2.1   LXKLDH.LargeDocumentHandling.SetUp.templates.target:subscribeLagreProduct

This template trigger receives any LargeDocHandling.docs:largeDocNotification canonicals.  The filter must be configured to match the trigger for the target/transaction.

 

1.1.3    Common Flow Services

1.1.3.1   LXKLDH.LargeDocumentHandling:SplitInboundNode

This service splits the provided BizDoc into transactions as defined by the service input variables.  For each transaction, the transaction is written to disk (NodeToFile) and a notification is published to the Broker.

1.1.3.2   LXKLDH.LargeDocumentHandling.java_services:NodeToFile

This service writes the xmlNode to disk as xml using the path and filename provided as input.

1.1.3.3   LXKLDH.LargeDocumentHandling.triggers:invokeProcessNodeService

This service is invoked by the LargeDocHandling.triggers:subscribeNodeNotification trigger.  It will simply invoke the service provided in the LargeDocHandling.docs:processNodeNotification document.

1.1.3.4   LXKLDH.LargeDocumentHandling.java_services:getLargeDocDirs

This service returns the path for the Large Document/Transaction directories for the given package.

 

1.1.4    Common Triggers

1.1.4.1   LXKLDH.LargeDocumentHandling.triggers:subscribeNodeNotification

This trigger receives any LargeDocHandling.docs:processNodeNotification canonicals and invokes the LargeDocHandling.triggers:invokeProcessNodeService service.

 

 

2       Physical Design: Error Handling

 

All error handling will conform to the Lexmark Common Components Design document which details global error handling for both webMethods and JDEdwards.

1.1         Error Notification

Error	Brief Description	Action Taken

 

2       Physical Design: Environment/Architecture Requirements

 

3       Risks

Risk ID	Risk	Risk Description	Mitigation Plan	Status
1	Risk to memory consumption when receiving BizDoc from Trading Networks.	In the case of a large document, the BizDoc, as received from Trading Networks contains the complete document, this can cause the server to exceed it’s allocated memory	1.       If the Trading Networks large document handing is configured correctly the BizDoc will only contain the reference to the document, the complete document will be stored to disk. 2.       The BizDoc is dropped from the pipeline as soon as the xmlNode is extracted
2	Risk to memory consumption when batching transactions to the target	If target batching is required, when all of the transactions have been reassembled for transmission to the target system, the complete transaction could exceed the current thresh hold causing the server to exceed it’s allocated memory	Target batching should only be used when there is no other option.  The complete transaction is only held in memory during the actual transmission to the target system and dropped as soon as there is a response from the target
3	Available disk space/resources	Transaction volume could cause the physical server to run out of available disk space and/or iNodes.	A script will be created to clear all of the “Archive” directories after 24 hours.  This script will be scheduled as a cron job which will be added to the server start-up process flow.

 

4       Issues

Issue Id	Issue	Issue Description	Notes/Status
1	Transmitting LDH over HTTP	Instability on transmitting large files over HTTP to Siebel	Closed. This issue has been approved by LISC

 

5       Design Alternatives