In the…

Last month I had the privilege of participating in the scverse Hackathon in Heidelberg hosted by the Stegle and Saez labs. It was an exhilarating event packed with exciting developments, intense coding, inspiring collaborations, and of course, pizza. The…

anndata is a commonly used Python package for keeping track of data and learned…

Developing and maintaining pipelines/workflows can be a genuine challenge. Doing this in a collaborative…

It’s been a while since I first wrote about tackling model error using a simple model. It’s about time to come back to it.

For some years, I have been going back an forth between [Spark-Notebook][spark-notebook] and [Apache Zeppelin][zeppelin] for different use-cases. Already 2 years ago, I made a little…

No, I’m not at the spark summit in Brussels. Time is not on my side, too much things to do.

How to use Scala/Spark in such a way that user or machine errors lead to minimal harm?

In this post, I want to get a better sense of the effects of model error by simulating a very simple model for the spreading of a virus. It’s based on an analysis…

In this…

I’m jumping between Scala/Spark coding, some Javascript in between, Python/PySpark and then some R every now and then. This in itself is already a…

I have been using Spark quite a lot for the last year. At first using the Scala interface, but lately more using the Python one.

I’ve been writing about my writing workflow before. Since some aspects of it are related to reproducible research and especially reproducible data analysis, I have collected some…

How many times have…

You’ve seen those fancy html presentations on the web? Reveal.js is a framework to create such things of beauty. And it goes along well with my Markdown based style of writing, even…

You wouldn’t tell from…

We introduced some basic aspects of SLM in the previous posts. We now turn to a list of issues with traditional Service Level Management.

We start this introduction with some examples of traditional metrics for measuring performance.

In business contexts, generally speaking, we believe we act as rational beings. The rise of Business Intelligence and the wonders of Number Crunching and Big Data make us believe we have entered the era of rational…

Say you are the editor in chief of a magazine and you have to decide among two possible…

The (brand new) Belgian government has decided to raise the stakes in the fight on tax fraud. That’s a good thing, especially in times of tight government budgets.

There’s a new word in the Dutch language: infografiek. It’s clearly derived from the English infographic. If this is a trend, it’s only the start. FlowingData has a ni…

CHANGE…

With the rise of VDI (Virtual Desktop Infrastructure, see here for a comprehensive overview) and the momentum it has, one starts to ask similar questions as with conventional server consolidation: what type of virtualization platform is…

Although I have been arguing that performance monitoring and capacity planning require a decent server montoring environment, it also requires more. This extra part comes…

In a previous post, we talked about averages (types of averages) and peaks and how peaks can tell you something about the spreading (variance, standard deviation) of the data.

Now that we are into the topic of performance (of capacity)…

Capacity planning starts with capacity (of performance) monitoring.

Lately…