At , we collect and process 30 billions geo-events a month from more than 8 million devices. At the time of writing we collected and enriched more than 300 billions events all stored in BigQuery table for a total of about 85 TB.

In this blog post, I will…


In one of my previous blog posts, we have seen how, at , we used authorised views to reduce costs, complexity, and delivery time. You can read more about it . In this post, we will see how to implement authorised views in production and managing them as code…


Originally published at

At , we collect and process 30 billions events a month from more than 8 million devices. Every day, we enrich, slice, and then deliver data feeds in various forms to our clients mainly via BigQuery, GCS, or S3.
In the past 2 years, we have…


How to use BigQuery superpowers to rewind time.

Originally published at

Who does not like time travel? We all saw it and were fascinated by it in many sci-fi movies, unfortunately science did not crack real-life time travel yet.

However, we can “data time-travel”. Thanks to the amazing BigQuery…


Originally published at on May 10, 2020.

Do you use BigQuery? Are you interested in knowing how to integrate data from different cloud providers into BigQuery? In this blogpost, we will implement a serverless and fully managed system to make available S3 access logs into BigQuery to easily integrate…


The BigQuery team rolled out support for a while ago and they have never stopped improving performances and GIS (Geographic Information System) . This allows users to run complex geo-spatial analytics directly in BigQuery harnessing all its power, simplicity, and reliability.

Hold on your keyboard (or your screen…


In this blogpost, I will explain what partitioning and clustering features in BigQuery are and how to supercharge your query performance and reduce query costs.

Partitioning

Partitioning a table can make your queries run faster while spending less. Until December 2019, BigQuery supported table partitioning only using date data type. Now…


BigQuery supports the “*” wildcard to reference multiple tables or files. You can leverage this feature to load, extract, and query data across multiple sources, destinations, and tables. Let’s see what you can do with wildcards with some examples.

The first thing is definitely loading the data into BigQuery. If…


Back in the early days of Huq we were ingesting a just few millions records per day into our geo-behavioural insights platform. Today that figure is in the hundreds of millions. …


Author’s and . Originally posted on .

Introduction

is Google Cloud Platform product that helps you manage complex workflows with ease.

It is built on top of and is a fully managed service that leverages other GCP products like Cloud SQL, GCS…

Alessandro Puccetti

Italian by birth but citizen of the world by choice Researched network measurement and security Opensource aficionado Juggle billions of events into the cloud

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store