Get in touch
Data files flowing through AWS into faster reports and lower costs
AWS Optimization

Global Insurer:
Same data, half the cost,
three times fresher

A global insurance company's AWS bill for data processing kept climbing. Its reports were also slow to update. Its pipelines were reloading data that hadn't changed and running on more computing power than they needed. We tuned them, and costs nearly halved.

  • 48% lower AWS Glue costs
  • 55% faster pipelines from start to finish
  • 3x business reports with data fresher
  • 100% of failed jobs retried automatically, so no more manual reruns

What was going wrong

AWS Glue is the service that moves and cleans the company's data before it reaches reports. Four things were wasting money there:

  • Too much computing power

    Jobs ran with more capacity than they needed, and the extra sat idle while still billing.

  • Reloading everything, every time

    Each run reprocessed the full dataset, even when most of it hadn't changed.

  • Thousands of tiny files

    Data in S3 storage was split into lots of small pieces, which slowed down everything reading it.

  • No visibility

    No one could see what each pipeline cost, how long it took or why it failed.

What we did

  1. 1

    Right-sized every job

    Jobs now get the capacity they need and scale up or down automatically.

  2. 2

    Upgraded to Glue 4.0

    We also reorganised how the data is split up, so the work runs faster.

  3. 3

    Processed only what changed

    New and updated records are picked up, and everything else is skipped.

  4. 4

    Tidied up storage

    Small files were merged into fewer, larger ones in a format built for analytics. Queries now read only the sections they need.

  5. 5

    Added a cost and speed view per job

    CloudWatch now shows what each pipeline costs, how long it took or when it fails.

Before
  • Over-provisioned compute
  • Full data reloads every time
  • Thousands of small files
  • No cost or performance visibility
After
  • Right-sized, automated scaling
  • Glue 4.0 with better partitioning
  • Incremental data processing
  • Optimised storage (larger files)
  • Monitoring with cost & speed per job

Smart moves, instant savings

This is the kind of project our Save Now Pay Later model is built for. A 48% cut on a recurring bill shows up month after month and is easy to verify. Under the model, the client pays nothing upfront, and our fee comes only from those verified savings. The rest of what they save is theirs to keep.

Save Now Pay Later

  • No upfront cost
  • Fee only from verified savings
  • You keep the rest