Pipelines can be tested from the editing page. There are 2 options at the top, Test which will run the pipeline in test mode and Debug which will add the debugging to it.
For an unpublished pipeline, this will Validate, Save and then Run the pipeline.
If the pipeline has been published then the Debug will appear next to the Load button.
If there are many origins, you can select the origin from the Test button by clicking on the down arrow.
If the pupeline has been published then you can do the same from the Load button.
To use Debug mode, you can either click it before clicking Test or you can press Test and select the Debug from the Batch Start pop-up
You can do this when you start or re-run any pipeline.
If you select Debug mode
When the pipeline runs, it will be in Debug Mode and so it will look like this.
Note that for an Unpublished pipeline, like this one, the Commit option is greyed out.
If you click on Preview the pipeline will run in Debug Mode.
Using Debug Mode
The debug mode screen looks like this:
If you press on the down arrow,
It will show the the step you are looking at. This is done in 2 ways, firstly, the row is highlighted and secondly, the Debug step appears in the Selected Debug at the top. This number corresponds to the DebugID field.
The data is the state of the data at that step.
To move through the steps, you can use the up and down arrow keys at the top.
Column Selector on data panel
On the data panel, there is a column selector so enable you to select certain columns. This can be useful if you are doing a Transform. In this case, an AddColumn statement has been added to the pipeline. If you go to the line in debug mode, it will only show the result column.
In order to see the source columns, use the column selector to pick the source columns.
And then it makes the transform easier to understand.
Leaving Debug Mode
THere are 4 buttons in the top left which are mainly about leaving Debug Mode
Batch Preview
goes to the Batch Preview Page
Load Again
Load the batch again using a new file
Replay
Creates a new batch using the same file as this batch
Pipeline Editor
Goes to the pipeline editor for the current pipeline.
Getting back to Debug Mode
The symbol for getting back into debug mode is this one
It appears on the batch preview
And on the batch list
How to Publish an Unpublished Pipeline and Commit the Data
Once a pipeline has been run in Test mode, you have the option to publish it and commit the data
How to Tell if a Batch was run in Test Mode
There are 2 ways to tell if a batch has been run in Test mode.
Firstly, on the Batch Preview, there is a Test Mode icon
It is also shown next to the batch status in the Load list
