External Task Workers decouple the execution of tasks from the process orchestration by the Process Engine by means of a pull mechanism.

When automating processes with Camunda, service tasks are usually executed directly by the process engine using explicit custom Java methods. The External Task Worker pattern offers an alternative to this, in which a separate application independently fetches tasks from a list, processes them and then returns the execution result to the engine. We describe the basic application of this pattern and discuss advantages, disadvantages and usage scenarios.

A German version of this blog post can be found here.

External Task Workers process service tasks

The core of any process model are its tasks or activities. In the following example process, we want to obtain a nice hat. In our scenario, this has been fully automated and is therefore modeled as a Service Task.

If the implementation of a corresponding process application uses Camunda's process engine, this engine controls the process flow and in particular also ensures that the service tasks are completed automatically. The most common approach for this is to provide the engine with a Java class as a co-called delegate in which the execution of the taskis implemented - in this case, the retrieval of a hat, e.g. through corresponding REST calls to the internal hat system. For this purpose, the GetHatDelegate implements the JavaDelegate interface.

Once the process flow arrives at the service task, the engine refers the corresponding task to the delegate class and uses its implementation of the execute method to complete the service task. Then, the engine continues in the process. Thus, task processing is performed directly and synchronously by the process engine and the delegate is tightly coupled to the engine or process application.

An alternative to this is the implementation with an External Task Worker. In this pattern, the process engine does not execute the tasks itself using a delegate, but instead writes them into a list, together with all the information necessary for their execution. An External Task Worker is an application independent of the engine that retrieves the tasks from this list via the engine's REST API, executes them and then completes them, i.e. returns their results to the engine, as shown in the following animation.

This means that task execution is achieved using a pull mechanism, thus asynchronously and less strongly coupled to the engine. At this point, we want to briefly point out that communication with the Process Engine does not necessarily have to be done via REST: There is also the option to implement External Task Workers as part of the Java application containing the process engine and to then interact with it via its Java API. However, the advantages of the External Task Worker pattern are more clearly evident when considering communication via REST, hence we will focus on this below.

Of course, we could implement the necessary basics for interacting with the Process Engine ourselves, such as for example  communication with the REST API, appropriate DTOs, pull behavior. However, this is rather tedious and we prefer to use a suitable dependency, e.g. the Java ExternalTaskClient from Camunda. Be aware that even when using Camunda's own client all interactions with the process engine takes place via its REST API.
If we now use Camunda's ExternalTaskClient for Java, then the actual execution of tasks is implemented in the form of an ExternalTaskHandler, analogous to the JavaDelegate above. The code does not change much, but the ExternalTaskClient has to be configured and initialized beforehand, as can be seen below.

ExternalTaskHandler

Configuration

Note

Since the German version of this article was first published, Camunda has extended their support for external task clients by (amongst other features) also providing a rather usefull Spring Boot Starter. An example for using it can be found in the second part of our miniseries on the external task pattern.

Advantages of the External Task Worker

One key question remains: Why should we go through the trouble of implementing an External Task Worker, explicitly pulling and returning tasks, instead of leaving all of this overhead to the Process Engine as in the delegate pattern described above?

The use of External Workers offers a number of advantages over implementing them with e.g. Java delegates, some of which we will now explore in detail.

• Scalability: Not all service tasks can be completed quickly, and for time-wise expensive activities, executing via a Java delegate can quickly become a bottleneck. The same problem can occur if the activity itself can be completed quickly, but occurs very frequently. When implementing the worker as a separate application, it is not a problem to spin up multiple instances of the same appliaction to handle the workload that arises.
• Distributed systems: Most modern processes are not handled on a single machine. (Micro-)Services are used everywhere and by definition require some form of communication. In our example, such a service could be used to fetch some hat information from the hat system. Such a system is probably located behind a firewall. Now, the Process Engine does not necessarily operate behind the same firewall; perhaps we operate on-site in separate domains, or we have outsourced the operation of the Process Engine to an external cloud. If the Process Engine is processing tasks directly, the firewall would need to be configured to allow requests from and, if necessary, responses to the Engine through. If we can run the External Task Worker behind the firewall, part of this configuration is omitted and we only need to allow communication to the outside, to the Process Engine. Especially in the case of hybrid cloud systems (distributed across multiple clouds, possibly not all with the same provider), this one-way only and outbound communication can be beneficial.
• Free choice of technology: Direct integration of delegates only works with Camunda Process Engine through Java classes. External task Workers only need to be able to communicate with the Engine's REST API and execute the task in a manner of their choosing. Hence, you can essentially implement it in the programming language of your choice.  In particular, this makes it possible to entrust teams that are more familiar with C# or any other language with the implementation of workers for completing activities. Similar to the ExternalTaskClient for Java mentioned above, there are now a number of other clients for various languages that simplify REST interaction with the Process Engine, allowing developers to focus on the essential processing of the task. So far, clients for Java and JavaScript are supported by Camunda, but there are also other currently (still) unsupported variants for e.g. Python, Ruby or .Net (see https://github.com/camunda/awesome-camunda-external-clients).
• Retry behavior and availability: A major advantage of using Camunda is its behavior in an error situation. If the execution of an activity using a Java delegate fails, the Process Engine tries again and follows the configured retry behavior. Especially if the completion of the activity depends on systems that are not continuously available (e.g. because maintenance work or updates are performed over night or over the weekend), this often leads to less than elegant solutions. For example, the execution of an activity is attempted several times at 13-hour intervals to get through the night or weekend. This leads to a series of unnecessary calls to a notoriously unreachable system and a deeply disappointed Process Engine.
      If, on the other hand, we entrust an External Worker with the activity, it controls its own retry behavior. Moreover, we could, for example, only ever start it up when we assume that its required systems are also available. Until then, the activities accumulate in the task list of the Process Engine and are all processed once the worker is online again.
• Decoupling: When implemented with Java delegates, the process application handles not only the orchestration of the process, but also the handling of the activities. Modern software architectures are modular, with modules as loosely coupled as possible to prevent, for example, changes at a single point from requiring re-deployments of all other components. The External Task Worker approach is much more in line with the ideal of loosely coupled services than that with Java delegates.

Not perfect in every Conceivable scenario

Of course, the external task pattern is not the ideal solution to all the world's problems. For example, it works asynchronously by design and is therefore unsuitable for tasks requiring synchronous execution.  Furthermore, it has to be considered that replacing Java delegates with External Workers increases the number of applications that need to be deployed, maintained and monitored. Finally, the worker's REST requests to the process engine necessarily increase the amount of overall network traffic.

To reduce the latter problem, the process engine's REST API provides a neat workaround: Obviously, it is inconvenient if the worker is constantly asking the engine for open tasks and returning empty-handedly. Clearly, it is a good idea to stagger these requests with a suitable backoff strategy and ask less and less frequently as long as no new tasks are added to the task list. However, this approach has the disadvantage that the worker may take quite long to get started on a task when one finally does show up in the list. This latency problem is reduced by long polling: Here, the worker declares that it is not dependent on an immediate answer to its REST request and would rather wait some time for a reply if there are currently no open tasks available. This behaviour can significantly improve the worker's response time without requiring more REST calls. Nevertheless, even with long polling, the response time of a worker will always be slower than that of a Java delegate.

Conclusion

External Task Workers decouple task processing from process orchestration by the Process Engine through a pull mechanism. In many scenarios this offers a number of advantages, but it must always be re-evaluated whether such asynchronous processing is suitable for the task in question. At least the implementation of workers is hardly more complicated than the implementation of corresponding Java delegates when using the ExternalTaskClients. Thus, nothing stands in the way of using the External Workers in all conceivable deployment scenarios. A sample implementation to compare Java delegates and External Workers can be found on GitHub. The application is based on Camunda's Maven Archetype.

More on this

As indicated above, the retry and error behavior of External Task Workers can be explicitly controlled. In a second blog post, we explain why you should definitely do this, how to proceed, and what to keep in mind when doing so: Camunda External Tasks - Error-Handling and Retry-Behavior.
The third blog post in this series explains how to make even better use of Camunda External Task Workers with the help of Quarkus: Camunda External Task Workers and Quarkus.