Production Overview

A Diver Solution or Diver Platform implementation involves regularly scheduled extraction, transformation, and load (ETL) tasks. Business rules are applied and validated; cBases or models are created. These tasks can be quite complex and a challenge to design, build, monitor, and maintain. Production helps back-end developers frame their development efforts. Production is a visual tool that provides an overview of a process, while facilitating construction, testing, scheduling, and real-time monitoring of the scripts. One or many Production scripts might be required, depending on the complexity of the data and wanted outcome.

Production Overview

Production Features

Production allows you to:

  • Visually define the job flow by using the graphical user interface
  • Nest Production scripts to break tasks into manageable pieces
  • Optimally organize the processes that involve a large number of scheduled jobs
  • Systematically handle complex job triggers that are often required
  • Manage the logging and identification of the current state of a large number of jobs
  • Run tests and monitor results in real time to quickly find and fix errors
  • Create, save, and reuse run configurations that include only selected portions of a script
  • Setup multiple schedules for a script using different run configurations

Production Scripts

Production scripts, with file name extension prd, describe the flow of the process—which Integrator scripts to run, when to copy files, which cBases to rebuild, and so on. Production scripts are made up of Process Nodes that describe a particular task, and Control Nodes that affect how the flow proceeds.

A script might contain any number of nodes. Each node in the script handles specific tasks. For example, a script might contain various process nodes such as Integ nodes to process Integrator scripts, Build cBase nodes to process Spectre Build scripts, Copy nodes to copy or move files, and FTP nodes to pull or push data from or to a server using FTP. The script might also contain various control nodes such as Fork nodes that split the flow into multiple branches and Wait nodes that merge branches back. Branches can be used to run simultaneous, non-dependent processes such as pulling data from multiple sources. Branches can then be merged back together once all tasks have completed. Production scripts can call sub-scripts to complete particularly complex tasks.

Data and control pass from one node to another as it travels from the Start control node to the End control node in a script. As data travels from node to node, each node performs its specified functions before passing control to the next node in the script.

Various factors determine where the flow proceeds. The direction of the flow is affected by the type of node used in the script. For example, if a Fork node is used, the flow travels to each branch specified by the Fork node based on conditions. If a Loop node is used, the process flow passes through each node in the loop before continuing along the branch. The number of times it passes through the loop depends on the settings configured for the Loop node.

The direction of the flow is also determined by how a node is configured. A node can be configured to pass the flow to one node if it succeeds and to a different node if it fails.

Scripts can be scheduled to run automatically at specified intervals. For more information, see Scheduling Production Scripts. Email notifications can be sent out when a script fails or succeeds. For more information, see Mailing Lists Tab.