Yesterday, we announced that Google Cloud Platform big data services are taking a big step forward by allowing everyone to use big data the cloud way. Google BigQuery has many new features and is now available in European zones. These improvements were designed to extend BigQuery's performance and capabilities to give you greater peace-of-mind and control over your data.

European Data Location Control

You now have the option to store your BigQuery data in European locations while continuing to benefit from a fully managed service, now with the option of geographic data control, without low-level cluster maintenance headaches. Feel free to contact the Google Cloud Platform technical support team for details on how to set this up.

Streaming Inserts

One of BigQuery's most popular features is the ability to stream data into the service for real-time analysis. To allow such low-latency analysis on very high-volume streams, we've increased the default insert-rate limit from 10,000 rows per second, per table, to 100,000 rows per second, per table. In addition, the row-size limit has increased from 20 KB to 1 MB, and pricing will move from a per-row model to a per-byte model for better flexibility and scale.

Security Features

BigQuery can now tackle a wider range of enterprise applications with the addition of data expiration controls and row-level permissions. Row-level permissions eliminate the need to create different views for different users, allowing secure shared access to systems such as finance or HR. This ensures that you get the information that’s relevant to you. In addition, data in BigQuery will be encrypted at rest.

Google Cloud Platform Logging Integration

Google Cloud Logging provides a powerful set of tools for managing your operations and understanding the systems powering your business; now, it also lets your Google App Engine and Google Compute Engine applications stream their logs into BigQuery. This allows you to perform real-time analysis on your log data and gain insight into how your system is performing and how your users are behaving. By joining application logs with your marketing and partnership data, you can rapidly evaluate the effectiveness of your outreach, or apply context from user profile info into your application logs to quickly assess what behavior resulted from specific customer interactions, providing easy and immediate value to both system administrators and business analysts.

Frequently requested features

Additionally, we’ve implemented a number of new features you’ve been asking for. You can now:
For a full list of features, take a look at the release notes.

Unprecedented scale

BigQuery continues to provide exceptional scale and performance without requiring you to deploy, augment or update your own clusters. Instead, you can focus on getting meaningful insights from massive amounts of data. For example:
  • BigQuery absorbs real-time streams of customer data totaling more than 100 TB per day, which you can query immediately. All this data is in addition to the hundreds of terabytes loaded daily from other sources. If you have fast-moving, large-scale applications such as IoT, you can now make quick, accurate decisions against in-flight applications.
  • We have customers currently running queries that scan multiple petabytes of data or tens of trillions of rows using a simple SQL query, without ever having to worry about system provisioning, maintenance, fault-tolerance or performance tuning.
With BigQuery’s new features, you can analyze even more data and access it faster than before, in brand new ways. To get started, learn more about BigQuery, read the documentation, and try it out for yourself.

-Posted by Andrew Kowal, Product Manager