Orchestration & Choreography Guidelines

Cloud , RPA , Integration, Business Process, API, Event Driven

What if choosing an architecture could be as simple as selecting a food item from the menu?

As more & more technologies are evolved, we get more & more choices making it difficult to choose right tools & techniques.

Purpose of this article is to cover following capabilities & provide guidance around when to use what :

1. Cloud Orchestration (AWS)
2. Integration Orchestration (Webmethods.io)
3. Business Process Orchestration (Camunda)
4. API Orchestration/Choreography (Camunda, AWS Step Function, Istio)
5. UI based Orchestration (RPA with Automation Anywhere, uiPath, Microsoft Power Automate)
6. Event Driven Architecture(EDA) — Choreography (Kafka Confluent)

Digital Process Automation Decision Matrix

Cloud based orchestration (AWS- SNS,EventBridge,AppFlow,SQS…)

Cloud based orchestration is best fit for cloud native applications reducing the overhead of multiple hops, monitoring, tracing, correlationid’s, log consolidation etc . Few key services in AWS which provides those capabilities are:

• SNS (Simple Notification Service) : provides instantaneous, push-based delivery (no polling) with extreme throughput. Possible subscribers are API, Lambda, SQS, etc.
• EventBridge : It's an extension of CloudWatch events which enable creating your own eventbus. It is majorly used for SaaS based integration.Throughput of eventbridge is limited to 400 requests per sec while for SNS it is a couple million requests per sec. EventBridge supports more than 13 AWS services as consumer while there are very few options in SNS. EventBridge provides event content related filtering while SNS supports attribute based filtering.
• AppFlow : Provides bi-directional data flow between AWS eco system & SaaS based systems like Salesforce. Difference between EventBridge & AppFlow is EventBridge is one way flow i.e. from SaaS based systems to AWS & doesn’t provide functionalities like data transformation & connector based plugin architecture to SaaS vendors like Salesforce.
• SQS (Simple Queue Service) : Provides pull based model with 2 options Default Queue & FIFO. Highly scale-able & resilient, best recommended as a bridge between high throughput producer to lower throughput consumer
• Step function : Works as coordinator between different AWS services specifically AWS Lambda. It can work well to establish a pipeline between multiple AWS lambdas & smooth handover between them
• Kinesis : Extremely high throughput streaming service. Used when publisher is sending data continuously which need to be processed in real time

DynamoDB streams change management to Salesforce

SNS based pub-sub across 2 AWS accounts

Integration Orchestration (Eg: Mulesoft, Webmethods.io)

Best fitted when you have applications living on-prem would like to talk to SaaS based platforms like Workday, Salesforce, etc. It provides 3rd party connectors out of the box & provides rich data transformation capabilities.

Webmethods.io integration flow from AWS SNS To Salesforce with Data Enrichment & Tranformation

Business Process Orchestration (Eg: Camunda)

Provides orchestration for a business process which can be easily created using BPMN diagram, works as a central process orchestrator specially which requires human intervention. Provides high visibility between different parts of the process & can orchestrate different activities living in different echo system like AWS, on-prem, ECS/EKS.

Loan Processing BPMN Diagram

UI Based Orchestration- RPA (Eg : Automation Anywhere, UiPath, Microsoft Power Automate)

Widely used for integrating with legacy applications which doesn’t have API or any other way to communicate except UI.

Insurance Declaration & Loan System Integration using Microservices based architecture

UI Web Automation using Microsoft PowerAutomate

API based orchestration (Eg: Kubernetes/Istio)

It’s majorly done using choreography than orchestration as there is no central component or broker to co-ordinate the communication between services, it’s purely event driven

Rendering Information in Salesforce by pulling data from external API’s

Event Driven Architecture (Confluent Kafka)

Widely used for event driven architecture & real time streaming, intersects with technology like AWS EventBridge, SNS, SQS & Kinesis however at a much larger scale & more power & customization. Event driven architecture heavily relies on choreography rather than following the traditional orchestration based architecture.

Conclusion

“Every choice you make has an end result.” Similarly choosing the right orchestration technique enable you to build maintainable, scalable & portable architecture.