Brovanture Senior Consultant
In the past, when data load jobs needed to be scheduled in Oracle Cloud EPM, there were two main options; Batches or EPM Automate scripts. Now however, there is the third option with the new Oracle Cloud EPM Pipelines functionality. This functionality was released as part of the new Data Integration interface in June 2023. Pipelines are set to replace the existing batch functionality, currently available in Data Management, that will soon be decommissioned by Oracle.
While it’s always exciting when Oracle release new functionality, it can be sometimes hard to keep up. This switch however is one that is definitely worth the effort! This blog will tell you all about the differences between the old Batch functionality, existing EPM Automate scripting and the new Oracle Cloud EPM Pipelines, and why it’s worth making the switch to Pipelines as soon as possible.
Let’s start with Batches. If you’ve not used them before, Batches are defined and scheduled within data management and are a way to schedule a range of data management load rules and defining parameters for each load.
Batches – Pros:
- Code-free scheduling
- Defined and scheduled within EPM (data management) rather than on an external server
- A series of data load rules can be scheduled in series or parallel
Batches – Cons:
- Schedules are hidden within Data Management, rather than showing in the Jobs console alongside other scheduled jobs
- Cannot use substitution variables for the date parameters so they must be manually updated periodically
- Limited to scheduling data load rules (including any business rules assigned to those data load rules)
- Oracle plans to decommission the batch functionality and it will not be moving to the data Integration interface
EPM Automate scripts
EPM Automate scripts are coded scripts which are stored and scheduled on an EPM Automate server. They are much more flexible than batches, however they require a basic level of batch scripting to write and maintain.
EPM Automate Scripts – Pros:
- Can code in usage of substitution variables and other functions like current date
- Custom email notifications can be coded into the script
- Able to code more complex and flexible error handling
- A range of jobs can be incorporated into the script, as long as there is an EPM Automate command for it
- No plans for Oracle to decommission this functionality
EPM Automate Scripts – Cons:
- Requires code and a basic understanding of batch scripting
- Scripts are stored and scheduled on an external server, rather than directly in Oracle Cloud EPM
- Requires the maintenance of an EPM Automate server and scripts
Now onto the new Pipeline functionality which, in my humble opinion, is a huge improvement! Pipelines are built and viewed in the Data Integration area (under Data Exchange) and listed alongside all the data load rules in the system. Pipelines are made up of Stages, which in turn are made up of Jobs, such as data load rules, business rules, or any other job type which is available in Oracle Cloud EPM.
The Pipeline interface has been designed to help admins visualise the tasks and group them into relevant stage groupings, which can have their own individual settings. Email notifications are built in and contain the status of every job, with a link directly to the log.
Pipeline – Pros:
- Code-free scheduling
- Defined and scheduled within Oracle Cloud EPM rather than on an external server
- Able to visualise the jobs in relevant stage groupings
- Stages can be run in parallel or sequence
- Built-in email notification functionality, which can be sent always, never, or on failure/success only
- Use of substitution variables to set load period parameters
- Ability to abort or skip tasks on error
- Incorporates all types of job (not just data load rules and business rules), including any REST job types
- Scheduled via the normal Job scheduler, so all scheduled tasks can be viewed in a single place
Pipeline – Cons:
- No automatic migration tool from batches to pipeline so schedules must be re-build manually initially
In summary, Pipelines provide the flexibility of EPM Automate scripts without the need to maintain scripts on an external server, while also providing a visual interface within EPM Data Integration for admins to view jobs within stages. Pipelines are a great alternative to Batches which, while code-free, were quite restricted in the jobs that can be scheduled and batches will ultimately will be decommissioned by Oracle at some point in the future.
So, there you have it, make the switch to Pipelines soon to make use of all the new features available and remove the reliance on either EPM Automate servers or Batch functionality!
Until next time
If you want help with Pipelines we already have considerable experience of using them with many of our clients and can be contacted at the phone number at the top of this page or via our contact page HERE.
I also blog on Blogspot HERE