The present document will introduce you to Flows and how to build your first one.
It is important to have in mind that in getOperate Scripts are at the basis of Flows and Apps. To sum up roughly, workflows are state machines represented as DAG to compose scripts together. Learn more in the Script Quickstart in the previous section. You will not necessarily have to re-build each script as you can reuse them from your workspace or from the Hub.
Those workflows can run for-loops, branches (parralellizable) suspend themselves until a timeout or receiving events such as webhooks or approvals. They can be scheduled very frequently and check for new external items to process (what we call “Trigger” script).
The overhead and coldstart between each step is about 20ms, which is faster than any other orchestration engine, by a large margin.
To create your first workflow, you could also pick one from our Hub and fork it. Here, we’re going to build our own flow from scratch, step by step.
From getOperate, click on + Flow
, and let’s get started!
Follow our detailed section on the Flow Editor for more information.
The first thing you’ll see is the Metadata menu. From there, you can set the permissions of the workflow: User (by default, you), and Folder (referring to read and/or write groups).
Also, you can give succinctly a Name, a Summary and a Description to your flow. Those are supposed to be explicit, we recommend you to give context and make them as self-explanatory as possible.
On another tab, you can configure a Schedule* to trigger your flow. Flows can be triggered by any schedules, their webhooks or their UI but they only have only one primary schedule with which they share the same path. This menu is where you set the primary schedule with CRON. The default schedule is none.
Scheduling allows you to define schedules for Scripts and Flows, automatically running them at set frequencies.
Last tab of the settings menu is the Shared Directory.
By default, flows on getOperate are based on a result basis. A step will take as inputs the results of previous steps. And this works fine for lightweight automation.
For heavier ETLs and any output that is not suitable for JSON, you might want to use the Shared Directory
to share data between steps. Steps share a folder at ./shared
in which they can store heavier data and pass them to the next step.
Get more details on the Persistent Storage dedicated page.
Ensure that your data is safely stored and easily accessible whenever required.
When a worker group is defined at the flow level, any steps inside the flow will run on that worker group, regardless of the steps’ worker group. If no worker group is defined, the flow controls will be executed by the default worker group ‘flow’ and the steps will be executed in their respective worker group.
Worker Groups allow users to run scripts and flows on different machines with varying specifications.
You can always go back to this menu by clicking on Settings
on the top lef, or on the name of the flow on the toolbar.
Flows on getOperate are generic and reusable, they therefore expose inputs. Input and outputs are piped together.
Inputs are either:
A workflow is a JSON serializable value in the OpenFlow format.
On the left you’ll find a graphical view of the flow. From there you can architecture your flow and take action at each step.
There are four kinds of scripts: Action, Trigger, Approval and Error handler. You can sequence them how you want. Action is the default script type.
Each script can be called from Workspace or Hub, you can also decide to write them inline.
Your flow can be deepened with additional features, below are some major ones.
For loops are a special type of steps that allows you to iterate over a list of items, given by an iterator expression.
Iterate a series of tasks.
Branches build branching logic to create and manage complex workflows based on conditions. There are two of them:
Split the execution of the flow based on a condition.
At each step, getOperate allows you to customize the number of retries by going on the Advanced
tabs of the individual script. If defined, upon error this step will be retried with a delay and a maximum number of attempts.
Re-try a step in case of error.
At each step you can add Approval Scripts to manage security and control over your flows.
Request approvals can be sent by email, Slack, anything. Then you can automatically resume workflows with secret webhooks after the approval steps.
Suspend a flow until specific event(s) are received, such as approvals or cancellations.
You can find all the flows’ features in their dedicated section.
There are several ways to trigger a flow with getOperate.
Trigger flows on-demand, by schedule or on external events.
You don’t have to explore all flows editor possibilities at once. At each step, test what you’re building to keep control on your wonder. You can also test up to a certain step by clicking on an action (x) and then on Test up to x
.
Iterate quickly and get control on your flow testing.
When you’re done, deploy your flow, schedule it, create and app from it, or even publish it to Hub.
Follow our detailed section on the Flow Editor for more information.
The present document will introduce you to Flows and how to build your first one.
It is important to have in mind that in getOperate Scripts are at the basis of Flows and Apps. To sum up roughly, workflows are state machines represented as DAG to compose scripts together. Learn more in the Script Quickstart in the previous section. You will not necessarily have to re-build each script as you can reuse them from your workspace or from the Hub.
Those workflows can run for-loops, branches (parralellizable) suspend themselves until a timeout or receiving events such as webhooks or approvals. They can be scheduled very frequently and check for new external items to process (what we call “Trigger” script).
The overhead and coldstart between each step is about 20ms, which is faster than any other orchestration engine, by a large margin.
To create your first workflow, you could also pick one from our Hub and fork it. Here, we’re going to build our own flow from scratch, step by step.
From getOperate, click on + Flow
, and let’s get started!
Follow our detailed section on the Flow Editor for more information.
The first thing you’ll see is the Metadata menu. From there, you can set the permissions of the workflow: User (by default, you), and Folder (referring to read and/or write groups).
Also, you can give succinctly a Name, a Summary and a Description to your flow. Those are supposed to be explicit, we recommend you to give context and make them as self-explanatory as possible.
On another tab, you can configure a Schedule* to trigger your flow. Flows can be triggered by any schedules, their webhooks or their UI but they only have only one primary schedule with which they share the same path. This menu is where you set the primary schedule with CRON. The default schedule is none.
Scheduling allows you to define schedules for Scripts and Flows, automatically running them at set frequencies.
Last tab of the settings menu is the Shared Directory.
By default, flows on getOperate are based on a result basis. A step will take as inputs the results of previous steps. And this works fine for lightweight automation.
For heavier ETLs and any output that is not suitable for JSON, you might want to use the Shared Directory
to share data between steps. Steps share a folder at ./shared
in which they can store heavier data and pass them to the next step.
Get more details on the Persistent Storage dedicated page.
Ensure that your data is safely stored and easily accessible whenever required.
When a worker group is defined at the flow level, any steps inside the flow will run on that worker group, regardless of the steps’ worker group. If no worker group is defined, the flow controls will be executed by the default worker group ‘flow’ and the steps will be executed in their respective worker group.
Worker Groups allow users to run scripts and flows on different machines with varying specifications.
You can always go back to this menu by clicking on Settings
on the top lef, or on the name of the flow on the toolbar.
Flows on getOperate are generic and reusable, they therefore expose inputs. Input and outputs are piped together.
Inputs are either:
A workflow is a JSON serializable value in the OpenFlow format.
On the left you’ll find a graphical view of the flow. From there you can architecture your flow and take action at each step.
There are four kinds of scripts: Action, Trigger, Approval and Error handler. You can sequence them how you want. Action is the default script type.
Each script can be called from Workspace or Hub, you can also decide to write them inline.
Your flow can be deepened with additional features, below are some major ones.
For loops are a special type of steps that allows you to iterate over a list of items, given by an iterator expression.
Iterate a series of tasks.
Branches build branching logic to create and manage complex workflows based on conditions. There are two of them:
Split the execution of the flow based on a condition.
At each step, getOperate allows you to customize the number of retries by going on the Advanced
tabs of the individual script. If defined, upon error this step will be retried with a delay and a maximum number of attempts.
Re-try a step in case of error.
At each step you can add Approval Scripts to manage security and control over your flows.
Request approvals can be sent by email, Slack, anything. Then you can automatically resume workflows with secret webhooks after the approval steps.
Suspend a flow until specific event(s) are received, such as approvals or cancellations.
You can find all the flows’ features in their dedicated section.
There are several ways to trigger a flow with getOperate.
Trigger flows on-demand, by schedule or on external events.
You don’t have to explore all flows editor possibilities at once. At each step, test what you’re building to keep control on your wonder. You can also test up to a certain step by clicking on an action (x) and then on Test up to x
.
Iterate quickly and get control on your flow testing.
When you’re done, deploy your flow, schedule it, create and app from it, or even publish it to Hub.
Follow our detailed section on the Flow Editor for more information.