Modern-day computational workflows often combine tens to a hundred different computational steps. These include off-the-shelf bioinformatics tools, trusted open source libraries but also hacked-together scripts. In the end, those ingredients are combined into a single, reproducible pipeline that can be run on different datasets and in different environments.
We expect our operational data workflows to be usable by lab scientists as well as bioinformaticians. We expect devops engineers to integrate them in automation efforts, data engineers to assess data governance policy alignment, etc. As a consequence we often hear from clients and collaborators that the workflows should be user-friendly or easy to use.
The Graphical Interface Instinct
With a team of engineers trained in providing technical services to pharma and biotech scientists, our first instinct is to make sure a graphical user interface is available.
Isn’t that what everyone wants: A good-looking web interface that allows to enter workflow parameters in a user-friendly way?
But is that really what users need? Or better, what all users need?
The Problem With One-Size-Fits-All Interfaces
Do we actually provide value by:
- Offering users a form with 40 parameters that can be filled in, of which half are not really required and could even break the workflow if not filled in correctly?
- Asking users to fill in paths to files and directories on S3 or some other shared file system?
- Letting users fill in the same reference paths each time again for every workflow run?
- Expecting users to adhere to data organization practices and correctly assign output paths for each processed dataset, risking data proliferation or worse?
User-Friendliness Depends on the User
In a recent conversation with a lab team lead at a client, I learned that the lab scientists did not necessarily dislike the form-based approach and did not particularly mind copy/pasting reference paths between workflow runs. But they liked a structured input file more; a CSV file containing all samples with their relevant parameters. Paths to reference files are captured using labels.
When we showcased a proof of concept of such an approach, it clicked. This was the most efficient and user-friendly way for this team.
Engineering Choices Enable Flexible User Interfaces
The fact that we could provide such a CSV input approach for this team was only because of the way we develop Viash-enabled Nextflow workflows where a workflow can be used as a subworkflow to a larger workflow which allows us to create super-workflows that provide different interfaces to the same underlying core functionality.
While creating the super-workflow(s), we made sure to include data organization practices that aligned with the client’s practices. Instead of pointing to input files on S3, users can provide a run identifier. The workflow will match this up with the input files automatically. Output, likewise, is based on a combination of a project and experiment identifier, effectively standardizing not only the computational workflows themselves but also the input and output of those workflows.
Conclusion
True user-friendliness isn’t about creating the most visually appealing interface—it’s about understanding your users’ actual needs and workflows, then designing interfaces that seamlessly integrate with how they already work.
In designing the super-workflows capturing that understanding, we include data governance policies as well as common standards for reference data and other shared parameters. This in turn leads to less errors and time lost.