Step 9 - Use a ConfigMap with a shell script

We used native Groovy code in the process examples above. Let us now use a shell script:

// Step - 9
process addTupleWithProcessHashScript {
  input:
    tuple val(id), val(input), val(config)
  output:
    tuple val("${id}"), stdout
  script:
    def thisConf = config.addTupleWithProcessHashScript
    def operator = thisConf.operator
    def term = thisConf.term
    """
    echo \$( expr $input $operator ${thisConf.term} )
    """
}
workflow step9 {
  Channel.from( [ 1, 2, 3 ] ) \
    | map{ el ->
      [
        el.toString(),
        el,
        [ "addTupleWithProcessHashScript" :
          [
            "operator" : "-",
            "term" : 10
          ]
        ]
      ] } \
    | addTupleWithProcessHashScript \
    | view{ it }
}

Running this (in the same way as before), we get something along these lines:

> nextflow -q run . -entry step9
WARN: DSL 2 IS AN EXPERIMENTAL FEATURE UNDER DEVELOPMENT -- SYNTAX MAY CHANGE IN FUTURE RELEASE
[2, -8
]
[1, -9
]
[3, -7
]

This is because the stdout qualifier captures the newline at the end of the code block. We could look for ways to circumvent that, but that is not the point here.

What’s important to notice here:

1. We can not just retrieve individual entries in config in the shell, we have to let Groovy do that.
2. That means we either first retrieve individual values and store them in a variable,
3. or we use the ${...} notation for that.
4. If we want to use bash variables, the $ symbol has to be escaped.

Obviously, passing config like this requires a lot of typing (especially as additional parameters are introduced) and is error prone.

A DiFlow module selects the appropriate key from a ConfigMap and uses that as its configuration settings. In a sense, we scope the global ConfigMap and use it as a local variable within a module. A module could also update the global ConfigMap and there may be cases where this is necessary, but care should be taken to update the global state like this.

The scoping done can be seen as a Functional Lens, although it’s a poor man’s implementation at that. Furthermore, in some FRP frameworks, so-called reducers or transducers are used for transforming state in an application. We did not (yet) consider the further extension in that direction.