Why Oracle Cloud EPM Pipelines have all the answers!

Lydia Maksoud
Brovanture Senior Consultant

In the past, when data load jobs needed to be scheduled in Oracle Cloud EPM, there were two main options; Batches or EPM Automate scripts. Now however, there is the third option with the new Oracle Cloud EPM Pipelines functionality. This functionality was released as part of the new Data Integration interface in June 2023. Pipelines are set to replace the existing batch functionality, currently available in Data Management, that will soon be decommissioned by Oracle.

While it’s always exciting when Oracle release new functionality, it can be sometimes hard to keep up. This switch however is one that is definitely worth the effort! This blog will tell you all about the differences between the old Batch functionality, existing EPM Automate scripting and the new Oracle Cloud EPM Pipelines, and why it’s worth making the switch to Pipelines as soon as possible.

Batches

Let’s start with Batches. If you’ve not used them before, Batches are defined and scheduled within data management and are a way to schedule a range of data management load rules and defining parameters for each load.

Batches – Pros:

  • Code-free scheduling
  • Defined and scheduled within EPM (data management) rather than on an external server
  • A series of data load rules can be scheduled in series or parallel

Batches – Cons:

  • Schedules are hidden within Data Management, rather than showing in the Jobs console alongside other scheduled jobs
  • Cannot use substitution variables for the date parameters so they must be manually updated periodically
  • Limited to scheduling data load rules (including any business rules assigned to those data load rules)
  • Oracle plans to decommission the batch functionality and it will not be moving to the data Integration interface

EPM Automate scripts

EPM Automate scripts are coded scripts which are stored and scheduled on an EPM Automate server. They are much more flexible than batches, however they require a basic level of batch scripting to write and maintain.

EPM Automate Scripts – Pros:

  • Can code in usage of substitution variables and other functions like current date
  • Custom email notifications can be coded into the script
  • Able to code more complex and flexible error handling
  • A range of jobs can be incorporated into the script, as long as there is an EPM Automate command for it
  • No plans for Oracle to decommission this functionality

EPM Automate Scripts – Cons:

  • Requires code and a basic understanding of batch scripting
  • Scripts are stored and scheduled on an external server, rather than directly in Oracle Cloud EPM
  • Requires the maintenance of an EPM Automate server and scripts

Pipelines

Now onto the new Pipeline functionality which, in my humble opinion, is a huge improvement! Pipelines are built and viewed in the Data Integration area (under Data Exchange) and listed alongside all the data load rules in the system. Pipelines are made up of Stages, which in turn are made up of Jobs, such as data load rules, business rules, or any other job type which is available in Oracle Cloud EPM.

The Pipeline interface has been designed to help admins visualise the tasks and group them into relevant stage groupings, which can have their own individual settings. Email notifications are built in and contain the status of every job, with a link directly to the log.

Brovanture Oracle Cloud EPM Pipelines

Pipeline – Pros:

  • Code-free scheduling
  • Defined and scheduled within Oracle Cloud EPM rather than on an external server
  • Able to visualise the jobs in relevant stage groupings
  • Stages can be run in parallel or sequence
  • Built-in email notification functionality, which can be sent always, never, or on failure/success only
  • Use of substitution variables to set load period parameters
  • Ability to abort or skip tasks on error
  • Incorporates all types of job (not just data load rules and business rules), including any REST job types
  • Scheduled via the normal Job scheduler, so all scheduled tasks can be viewed in a single place

Pipeline – Cons:

  • No automatic migration tool from batches to pipeline so schedules must be re-build manually initially

In summary, Pipelines provide the flexibility of EPM Automate scripts without the need to maintain scripts on an external server, while also providing a visual interface within EPM Data Integration for admins to view jobs within stages. Pipelines are a great alternative to Batches which, while code-free, were quite restricted in the jobs that can be scheduled and batches will ultimately will be decommissioned by Oracle at some point in the future.

So, there you have it, make the switch to Pipelines soon to make use of all the new features available and remove the reliance on either EPM Automate servers or Batch functionality!

Until next time

Lydia

If you want help with Pipelines we already have considerable experience of using them with many of our clients and can be contacted at the phone number at the top of this page or via our contact page HERE.

I also blog on Blogspot HERE