Introduction
We recently participated in a hackathon that brought together different communities to work on the SpatialData object, file format and related topics. The event was hosted in Padua, Italy from 14-16 April and involved around 30 developers from institutes and companies around the world who contribute to a variety of projects related to the SpatialData format.
What is SpatialData?
SpatialData is designed to be an open and universal framework for processing spatial omics data. It includes a Zarr-based FAIR file format based on the OME-NGFF specification and objects designed to access this storage in a performant way. SpatialData objects can store multiscale images as well as a range of annotations (points, shapes, labels) but the key capability for spatial omics data is the inclusion of additional linked annotation tables, typically available as AnnData objects.
In the Python scverse ecosystem there is a growing collection of packages that use this format to perform a range of analyses. Interoperability has always been a consideration in the design of the format and implementations exist in R and other languages including a range of interactive viewers.
Who participated?
The hackathon brought together people from three key communities involved in spatial omics and image analysis:
- scverse, the core Python single-cell omics analysis software community and the key developers behind the SpatialData format
- OME (the Open Microscopy Environment), a consortium of universities, research labs, industry and developers producing open-source software and format standards for imaging data
- Bioconductor, the core community for Bioinformatics software in R
These groups have different expertise, use cases and outlooks and getting input from each of them is important for the success of the SpatialData format.
What were the topics?
There were four main topics for the event:
- R interoperability - Developing R packages for implementing the SpatialData object stack in R
- Design modernisation - Updates and refinements of the SpatialData format, object and related specifications
- File formats and transformations - Developing new specifications for representing transformations (scaling, cropping, rotations etc.) and related updates to OME specifications
- Visualisation improvements - Updates to plotting libraries and interactive viewing tools
What did we do?
We focused on the R interoperability track, specifically finalising support for the Zarr file format in the anndataR package. The anndataR package provides an R-native implementation of the AnnData object used by the scverse in Python as well as conversion to common R objects for single-cell analysis (SingleCellExperiment and Seurat). Compatibility with the Python implementation is ensured by a rigorous roundtrip testing framework.. The current implementation focuses on the HDF5-based H5AD file as the on-disk storage format but Zarr support is required for anndataR to be used in the R SpatialData package and remove the dependency on an associated Python environment. Read more about anndataR in our preprint.
An initial implementation of Zarr support had been contributed by the community but this was significant new functionality that required greater changes to the package. Over the three days we were able to finalise the contributions, fully integrate them into the package and its testing framework and refactor common code shared between the H5AD and Zarr backends. By the end of the hackathon all checks were passing including over 3500 tests and the functionality was ready to be reviewed for inclusion in an upcoming package release.
Outlook
While it is still developing, the SpatialData format is an important part of the spatial transcriptomics infrastructure. Having a standard file format is vital to building reproducible processing workflows and enables interoperability between tools and languages.
Our experience at the hackathon was highly productive, giving us the opportunity to work with the community to implement important functionality and providing insights into what is planned more widely.
Interested in workflows for your spatial transcriptomics data? Please get in touch.
Acknowledgements
Thank you to the organisers for inviting us to attend, particularly the main organiser Artür Manukyan, and to all the participants for their contributions to anndataR, the wider ecosystem and all the discussions during the event. We look forward to joining similar events in the future!