This page is also available in Deutsch, Español, Français, Italiano, 日本語, and Português.

Huq Cuts Cloud Costs 50% as Data Volume 4x

DoiT guided Huq's migration from AWS to Google Cloud, rebuilding its pipeline on Dataflow and BigQuery to process nearly a billion geo-location records daily.

Meet Huq

Huq measures real-world factors that affect businesses to help them gain a holistic view of their marketplace. The company produces economic and societal benchmarks using mobility data that serves as a proxy for footfall in retail, transport, town centers, and workplaces. This anonymized resource incorporates high volumes of rich, descriptive data that enables complex analysis of consumer behavior.

The Challenge

Huq's customers required increasingly fast turnaround times and more regular access to high-frequency data. The company needed to adapt its data pipeline to respond to customer demands in near real time while maintaining flexibility for customers with varying technical needs.

The Solution

DoiT helped Huq migrate its core data processing pipeline from AWS to Google Cloud using Apache Beam and Google Dataflow. The solution included workflow orchestration with Cloud Composer, data storage with Cloud Storage and BigQuery, and cost optimization through reOptimize CMP and superQuery IDE.

Results

Cut cloud costs by 50% while data volumes quadrupled from 200 million to 800 million data points daily
Built scalable architecture capable of processing almost one billion geo-location records per day
Achieved ability to scale data processing 100-fold without stack changes

We're a compact technical team, which means we're able to move very fast and rapidly respond to changes. But when you're adopting new technologies, you need to leverage cutting-edge expertise. The key reasons we chose DoiT International were the quality of communication and the team's deep technical know-how.
Isambard Poulson, Co-founder and CTO, Huq

Finding the right solutions based on a tailored approach

Isambard turned to DoiT in May 2020 on recommendation from another client. DoiT impressed Huq with its tailored approach, taking time to understand requirements rather than pushing existing products. The onboarding process was smooth and the available tools were highly compelling for Huq's needs.

Building a scalable data pipeline

DoiT helped Huq complete its migration to Google Cloud, building a data pipeline that rapidly ingests, processes, analyzes, and exports almost one billion geo-location records daily. The solution uses Google Cloud Composer for workflow orchestration, Cloud Storage for data loading, Dataflow for enrichment, and BigQuery for client access.

Staying on top of costs with optimization tools

DoiT provided reOptimize CMP for enhanced visibility into cloud usage and cost optimization. Huq also gained access to superQuery IDE, an AI-powered development environment for BigQuery that enables more efficient querying and flexible customer response.

Dramatic cost savings while scaling

With DoiT's help, Huq cut its cloud bill in half while data needs quadrupled. The company now processes 800 million data points daily compared to 200 million previously, with costs falling by 50%. The new stack can scale data processing 100-fold without changes.

See how DoiT helps cloud teams control spend

Explore how Cloud Intelligence™ helps teams improve visibility, governance, and unit economics across cloud environments.

Visit DoiT Talk to our team

What they say

On integrating DataHub, the DoiT team was very hands-on to the point where DoiT support put together the first ingestion script and pipeline to unlock us.

Kyle Espinola, Head of Platform Engineering

“PerfectScale gives us the confidence to automate Kubernetes optimization while letting our engineers focus on building what matters.”

Josh Zarrabi, Infrastructure Engineer

“Without PerfectScale by DoiT, I don't know how we would do it. We'd have to have twice the number of people on my team to stay on top of it. With PerfectScale, we can focus on the more strategic issues.”

David Adkins, Director of Software and Platform Engineering

Connecting IBM and AWS privately was the piece we needed to get right, and we wanted to move faster on it than we could on our own. DoiT worked through the options with us, found the subnet overlap that was breaking the routing, and made it stable. That unblocked everything else.

DevOps Lead, Software Projects

Attribute™'s platform is truly unique. We now have crystal-clear visibility into our cloud spend at the workload and tenant level, and that insight has already led to actionable savings and powerful insights as we further scale our service.

Paul White, VP of Engineering, Shasta Cloud

We have worked with DoiT for many years, and there has been an increasing number of capabilities and features in DoiT Cloud Intelligence. We've embedded features such as Cloud Analytics and Reports in our own FinOps processes, it's become core to what we do.

Martin Lee, Director of Operations

DoiT gave us the confidence to move from experimentation to production. They helped us understand the right way to build AI for the real world.

Milad Rezazadeh, CTO

Attribute™'s cost grouping technology took our cost visibility and allocation to a whole new level. Now, our teams are fully accountable for their budgets, significantly improving our cloud efficiency and helping us minimize unnecessary costs.

Eli Zilbershtein, Head of DevOps, Hippo

You can't tag a customer in a multi-tenant environment. Attribute™ finally shows us what each customer costs and what's driving those costs.

Omri Cohen, Director of Engineering, Platform

Attribute™'s data is truly unmatched. No other solution on the market could deliver the precise customer cost and usage profiles we needed in such a complex infrastructure. Within weeks, the data from Attribute™ transformed our understanding of cost structures, influencing key strategic decisions in pricing, renegotiations, and market positioning.

Jonathan Langer, COO, Claroty

Attribute™ simplified tracking customer costs in our multi-tenant environments. Customer cost measurement is now clear and standardized, and finance gets the business context they need. Integration was quick and required no changes.

Kfir Lippmann, CFO, Salt Security

Attribute™ translates complex cloud bills into actionable, business-centric insights that empower our engineering teams to take true ownership of their costs.

Balamurugan Mohandossgandhi, Head of IT and Infrastructure, PropertyGuru

This has let us get a better idea of what our cost of goods sold really is. It's not every day you come across something that delivers value as quickly as yours did for us. I was seeing useful insights inside the POC, and we had only deployed it to a couple of real clusters.

Jason Moore, Principal DevOps Engineer, Accrete AI

Your cloud bill shouldn't be a mystery

Let us show you what ships this week.

Book a demo