xvc pipeline dag

Synopsis

$ xvc pipeline dag --help Generate a dot or mermaid diagram for the pipeline Usage: xvc pipeline dag [OPTIONS] Options: -p, --pipeline-name <PIPELINE_NAME> Name of the pipeline to generate the diagram --file <FILE> Output file. Writes to stdout if not set --format <FORMAT> Format for graph. Either dot or mermaid [default: dot] -h, --help Print help

You can visualize the pipeline you defined with xvc pipeline set of command with the xvc pipeline dag command. It will generate a dot or mermaid diagram for the pipeline.

Examples

As all other pipeline commands, this requires an Xvc repository.

$ git init --initial-branch=main Initialized empty Git repository in [CWD]/.git/ $ xvc init

All steps of the pipeline are shown as nodes in the graph.

We create a dependency between the two steps by using the --dependencies flag to make them run sequentially.

$ xvc pipeline step new --step-name preprocess --command "echo 'preprocess'" $ xvc pipeline step new --step-name train --command "echo 'train'" $ xvc pipeline step dependency --step-name train --step preprocess

It's not very readable but you can supply the result directly to dot and get a more useful output.

$ xvc pipeline dag digraph pipeline{n0[shape=box;label="preprocess";];n1[shape=box;label="train";];n0[shape=box;label="preprocess";];n0->n1;}

The output after dot -Tsvg is:

pipeline-1

When you add a dependency between two steps, the graph shows it as a node. For example,

$ xvc pipeline step dependency --step-name preprocess --glob 'data/*' $ xvc pipeline dag digraph pipeline{n0[shape=box;label="preprocess";];n1[shape=folder;label="data/*";];n1->n0;n2[shape=box;label="train";];n0[shape=box;label="preprocess";];n0->n2;}

pipeline-2

You can use --mermaid option to get a mermaid.js diagram.

$ xvc pipeline dag --format=mermaid flowchart TD n0["preprocess"] n1["data/*"] --> n0 n2["train"] n0["preprocess"] --> n2

The output can be used in Mermaid Live Editor or any web page that support the format.

train
preprocess
data/*