CASE STUDY 01 / SOFTWARE ENGINEERING · CLOUD & DEVOPS
Making fragmented
research data comparable.
A microbiomics research lab later acquired by a global crop-science company.
FROM FRAGMENTED FILES TO EXPLORABLE DATA
01 Spreadsheets
02 Transformation
& provenance
03 Analysis
04 Exploration
interfaces
KUBERNETES + GITOPS / Hosting the processing, analysis, and exploration services
Illustration of the reported functional flow—not a recovered client architecture. Field-history example: manual entry → Boolean → renamed field.
ENGAGEMENT FOCUS
Elixir processing pipeline
Transformation & provenance
Data-exploration interfaces
Kubernetes & GitOps
THE OBJECTIVE
Make research data comparable across changing spreadsheet formats—and deliver a working platform to process and explore it.
THE CHALLENGE
When spreadsheet versions stop agreeing.
Microbiomics assays and screenings were recorded in spreadsheets across many computers. Field names, types, and formats had drifted apart. A manually entered field could become a Boolean and later be renamed, obscuring the relationship between records.
Researchers needed more than consistent column names. They needed to preserve meaning across versions. Meanwhile, an attempted Kubernetes implementation had stalled, leaving the software without a working deployment platform.
Preserve meaning. Make it deployable.
Transform with provenance
A custom Elixir pipeline ingested, transformed, standardized, and analyzed spreadsheets while preserving provenance. Field history remained part of the transformation flow.
Explore across versions
Separate exploration interfaces supported different ways of examining research data. Elixir was selected for concurrent processing; no measured language-performance advantage is claimed.
Deploy through GitOps
The team established Kubernetes from scratch and deployed 60 microservices. A linting build, images published on GitLab, and Flux reconciliation supported declarative delivery.
ONE DELIVERY EFFORT
Software and infrastructure, together.
The transformation software, analysis services, and exploration interfaces needed a dependable place to run. Autoscaling supported variable processing demand, while Flux reconciled the declared deployment state.
The delivery team combined application engineering with the Kubernetes rollout and GitOps deployment work. Researchers supplied the laboratory workflow and spreadsheet context; the client’s CTO and IT team had begun the infrastructure effort. The work supported scientific research—it did not design the research protocols or perform the science.
A working research-data platform.
60 microservices
Deployed on a Kubernetes platform managed through GitOps.
Comparable data
Custom transformations tracked field changes and preserved provenance across representations.
Scaling support
Autoscaling processing workloads and exploration interfaces on a working platform.
Source: delivery-team project interview. The service count and functional capabilities are reported outcomes; no processing-speed, throughput, or availability measurements are supplied. The later acquisition is client context, not a project outcome.
MORE CASE STUDIES
Lower cloud costs. More frequent releases. →