Data engineering
Pipelines that turn raw operational data into clean, modelled datasets an analyst can query without asking three questions first.
Raw data is rarely wrong on purpose. It is inconsistent because it was produced by systems that were solving a different problem, and the cost of that lands on whoever tries to report on it.
Layered by design
We build warehouses in layers: raw as it arrived and never edited, a cleaned layer where types and keys are enforced, and a modelled layer shaped for the questions people actually ask. Each layer is reproducible from the one before it, so a bad transformation is fixed by rerunning, not by patching rows.
Where it helps most
Having also built the application backends that generate source data, we know why a column is nullable and where the dirty rows come from. Modelling is easier from the side that produced the mess.
/What this includes
Pipeline development
Ingestion, cleaning and transformation that can be rerun safely, with failures that are visible rather than silent.
Warehouse and lakehouse design
Medallion layering on Microsoft Fabric, PostgreSQL or an object store, chosen by workload rather than by fashion.
Dimensional modelling
Star schemas with defined grain and conformed dimensions, so two reports on the same metric cannot disagree.
Migration and modernisation
Moving off spreadsheets, a legacy warehouse or a pipeline nobody remembers writing, with the old output kept for comparison.
/Technologies
- Languages
- TypeScript
- Python
- SQL
- JavaScript
- Dart
- Backend
- NestJS
- FastAPI
- Prisma
- SQLAlchemy
- Alembic
- Celery
- Frontend
- Next.js
- React
- Astro
- Tailwind CSS
- Flutter
- Data and analytics
- Microsoft Fabric
- PySpark
- Power BI
- PostgreSQL
- InfluxDB
- Redis
- Infrastructure
- Docker
- nginx
- Caddy
- Cloudflare
- GitHub Actions
- Linux
/Questions
Questions we get asked
Do we need a warehouse, or is a database enough?
Often a well-modelled PostgreSQL is enough, and saying so costs us a bigger project. A warehouse earns its keep when reporting queries start competing with the application for the same database.
Which platform do you build on?
Microsoft Fabric where the organisation already runs on Microsoft, PostgreSQL where the volume does not justify a separate platform. The choice follows the workload and the team that will run it.
Have a project in mind?
Tell us what you are building and we will tell you honestly whether we are the right fit.
Contact us