Considerations For Task Data In webMethods Cluster Environment
A Business process consists of one or many automated or manual activities. An automated activity is a step that is performed directly by the execution engine while a manual activity involves human interaction. Manual tasks also known as human tasks in BPM (Business Process Management) nomenclature. As much as processes are geared towards automated tasks, involvement of a human in a process is inevitable. Examples include approving a large loan, overriding a system generated decision etc.
In webMethods (7.1.2), Tasks are run on the Task Engine (TE) in My webMethods Server (MWS). When the Process Engine encounters a task in a process, the control is passed to Task Engine in MWS. The status of the process will change to started state till a Task is completed. The status for the task step itself is queued and the task is eligible for a user from the task queue.
A task consists of 2 parts: Task Info and Task Data. Task Info contains task meta data such as created, updated dates etc and Task Data contains the business data available for display. In J2EE world, task Data closely resembles Data Transfer Object (DTO) in the presentation layer. Within webMethods, task Data is a document in Integration Server and persisted in T_TASK_DATA table as a blob object. Task Data also provides the facility to save data and retrieve them later.
Tasks are updated via WmTaskClient package, web service API or MWS API. The Task Client package is simply a convenient wrapper for the web services running in TE. Task API provides a number of methods but in this blog the focus is on two methods: updateTask and getTask. The update task does what its name suggests but it is an async call. Why not a sync call? One explanation for this could be that task updates are most likely to be triggered by user actions and hence it would make sense the call to return immediately to minimize any delays with UI.
An update to a task consists of 2 steps: Persists task to task data tables and updates the cache maintained by TE. The impact of cache update in TE is not significant in a non clustered environment to warrant a further discussion. However, in a clustered environment, this can present a problem for remote TE web service APIs, including APIs from the WmTaskClient package. Since the application is running on a clustered environment, the getTask API call can be routed by a load-balancer to any node in a MWS cluster. In this case, it is possible that the call may return incorrect (out of date) data, because the cluster node has not been updated yet. The worst is yet to follow. A user without knowledge of this stale data will happy to carry on till they invoke an action to save task data again. At this point, TE will detect that update request contains older data and hence will return an exception (âTask has not completed update cycle, please refreshâ) to the user. In this instance, the most likely outcome for the user is to re-enter data. In the meanwhile, it wouldnât matter if there is no immediate call to fetch data after an update call. This is because after some delay, depending on network and number of nodes in a cluster etc, all the nodes in a cluster will be updated and hence subsequent calls to get task data will return the âupdatedâ data.
To ensure that up-to-date data is always returned, webMethods suggests the following option (applicable only to cluster environments):
The default value is false. If set to true, any task update operation will attempt to synchronously refresh caches on all other cluster nodes and wait until other cluster nodes report back that their caches have been updated. A disadvantage to setting this option to true is that any task update operation will have to wait until all cluster nodes are refreshed. This does not introduce any additional load on the server, but it does increase task update operation response time
Another suggestion (in addition to the above) is setting the lightweight option to true as shown below:
When using the Task Engine to complete a large volume of tasks, the JMS queue (backed up by database table T_TASK_JMS) may become plugged. Part of the solution is to use lightweight queueing (JVM backed queuing) of task events. To turn this setting on, add the following JVM System property to the server.properties.bat or .sh file. The effect of using lightweight queuing is a much lighter load to the database (avoid database table) and faster task-rules processing at the cost of a slightly increased chance of data loss in the case of a software or hardware crash.
None of the above options seem to offer a complete solution to the problem. Therefore, it is necessary to take this limitation in designing an application with Task Data in a clustered environment.
- Size of task data: A smaller data set for Task Data may help with saving and synchronizing with TE caches.
- Avoid updateTask and getTask calls: A void update and immediate get task API call combination as much as possible. This should prevent TEs with outdated caches giving stale data. However, this is difficult to avoid in some situations especially when some processioning activities on Task data, such as populating some missing data based on user inputs, is done before calling the update task. In this instance, UI must invoke get task API call to get the latest data
- Save data in a session: Useful in an application where data are passed from one panel to another panel. Instead of relying on update and get Task API calls to access data, store them in a session for each panel to access.
- Store Task Data elsewhere: Instead on relying on TaskData API to manage task data, use an external table to store data. With this approach, TaskData will simply hold a primary key to an external table where the full data set is retrieved either by a JDBC call or a web service call
- Avoid mutable data in queue columns: When task queues contain columns which are mutable items (items users can change), it will require an immediate update when these items are changed by the user. However, if queues donât contain mutable data, then a call to task update can take place at the end of a data entry process