A fellow Technical Solutions Engineer recently found their Google Cloud Platform project in an interesting state. They could create Compute Engine VM instances that would boot, but could not remotely connect via SSH into any of them. While this problem is often due to a misconfigured firewall rule, a quick check of the rules showed this was not the case, as an SSH rule existed and its SRC_RANGES value was non-discriminatory:

$ gcloud compute firewall-rules list -r .*ssh.*
default-allow-ssh default  tcp:22

We ruled out a system-level firewall misconfiguration, as new systems from default images would not share that issue. As a sanity check, we used tcptraceroute to ensure traffic was reaching the instance:

$ sudo tcptraceroute -P 22
Selected device en0, address, port 22 for outgoing packets
Tracing the path to on TCP port 80 (http), 30 hops max
1  1.247 ms  0.256 ms  0.250 ms
2  * * *
10  * * *
11 ( [closed]  38.175 ms  38.918 ms  38.072 ms

We would expect the last hop to report open, not closed.  Typically, this value means that the instance has responded but the port wasn't open for communication.  With no firewall interference, we knew it had to be something else.  The next step was to grep through the serial port output to see if sshd had started:

$ gcloud compute instances get-serial-port-output gcp-rge0-blog --zone us-central1-a | grep Starting.*sshd
[....] Starting OpenBSD Secure Shell server: sshd
Jan 14 23:19:19 gcp-rge0-blog sshd[1911]: Server listening on port 22.
[ ok ] Starting OpenBSD Secure Shell server: sshd.

Okay, that looked fine.  With the most obvious points of interference ruled out, the network routes were the next best bet:  

$ gcloud compute routes list
NAME                            NETWORK     DEST_RANGE      NEXT_HOP                 PRIORITY
default-route-31a84e4cfff40b29 default                            1000

Now we’ve found the root cause.  The default route for non-local traffic ( had been inadvertently deleted, which caused all external traffic to be lost on the return path.  Recreating the missing route solved the issue:

$ gcloud compute routes create default-internet --destination-range --next-hop-gateway default-internet-gateway
Created [].

$ gcloud compute routes list
NAME                            NETWORK     DEST_RANGE      NEXT_HOP                 PRIORITY
default-route-31a84e4cfff40b29 default                            1000
default-internet     default       default-internet-gateway 1000

Now, the instances are once again reachable by SSH and any other external method. Case closed!

You can find a lot of help and information in the Google Cloud Platform documentation and more information on troubleshooting Compute Engine specifically here.

- Posted by Josh Moore, Technical Solutions Engineer

The emergence of affordable high IOPS storage, such as Google Compute Engine local SSDs, enables a new generation of technologies to re-invent storage. Helium, an embedded key-value store from Levyx, is one such example -- designed to scale with multi-core CPUs, SSDs, and memory efficient indexing.

At Levyx, we believe in a “scale-in before you scale-out” mantra. Often times technology vendors advertise scale-out as a way to achieve high performance. It is a proven approach, but it is often used to mask single node inefficiencies. Without a well balanced system where CPU, memory, network, and local storage are properly balanced, this is simply what we call “throwing hardware at the problem”. Hardware that, virtual or not, customers pay for.

To demonstrate this, we decided to check Helium’s performance on a single node on Google Cloud Platform with a workload similar to the one previously used to showcase Aerospike and Cassandra (200 byte objects and 100 million operations). With Cassandra, the data store contained 3 billion indices. Helium starts with an empty data store. The setup consists of:

  1. Single n1-highcpu-32 instance -- 32 virtual CPUs and 28.8 GB memory.
  2. Four local SSDs (4 x 375 GB) for the Helium datastore. (Note: local-SSDs is limited in terms of create time flexibility and reliability compared to persistence-disks, but the goal of this blog post is to test with highest performing GCP IO devices).
  3. OS: Debian 7.7 (kernel 3.16-0.bpo.4-amd64, NVMe drivers).
  4. The gists and tests are on github.

Scaling and Performance with CPUs

The test first populates an empty datastore followed by reading the entire datastore sequentially and then randomly. Finally, the test deletes all objects. The 100 million objects are in memory with persistence on SSD, which acts as the local storage every replicated system requires. The total datastore size is kept fixed.
image (8).png
  • Single node performance of over 4 Million inserts/sec (write path) and over 9 Million gets/sec (read path) with persistence that is as durable as the local SSDs.
  • 99% (in memory) latency for updates < 15 usec, and < 5 usec for gets.
  • Almost linear scaling helps with the math of provisioning instances.

Scaling with SSDs and Pure SSD Performance

Compute Engine provides high IOPS, low latency local SSDs. To demonstrate a case where data is read purely from SSDs (and not take advantage of memory), let’s run the same benchmark with 4K object size x 5 million objects, and reduce Helium’s cache to a minimal 2% (400 MB) of total data size (20GB). Only random gets performance is shown below because it is a better stress test than sequential gets.

Take aways:
  • Single node SSDs capable of updates at 1.6 GB/sec (400K IOPS) and random gets at 1.9 GB/sec (480K IOPS).
  • IOPS scaling with SSDs.
  • Numbers comparable to fio, a pure IO benchmark.
  • With four SSDs and 256 threads, median latency < 600 usec, and 95% latency < 2 msec.
  • Deterministic memory usage (< 1GB) by not relying on OS page caches.image (10).png
image (12).png
image (11).png

Cost Analysis

The cost of this Google Compute Engine instance for one hour is $1.22 (n1-highcpu-32) + $0.452 (4 x Local SSD) = $1.67. Based on 200-byte objects, this boils down to:

  • 2.5 Million updates per dollar
  • 4.6 Million gets per dollar

To put this in perspective, New York’s population is ~8.4 million; therefore, you can scan through a Helium datastore containing everyone’s record (assuming each record is less than 200 bytes. Eg: name, address and phone) in one second on a single Google Cloud Platform instance for under $2 per hour.


Helium running on Google Compute Engine commodity VMs enables processing data at near memory speeds using SSDs. The combination of Cloud Platform and Helium makes high throughput, low latency data processing affordable for everyone. Welcome to the era of dollar store priced datastores at enterprise grade reliability!

For details about running Helium on Google Cloud Platform, contact

- Posted by Siddharth Choudhuri, Principal Engineer at Levyx

Today's guest post comes from our friends at Tableau: Jeff Feng, Product Manager & Ellie Fields, Vice President of Product Marketing. Tableau, a Google Cloud Platform partner,  is a leader of interactive data visualization software.

“It’s a beautiful thing when best-of-breed technologies Tableau, Google BigQuery and Twitter come together to operate seamlessly in concert with one another.” - Jeff Feng

Next, a Google Cloud Platform Series
Over the month of June, the Tableau team traveled around the world with the Google Cloud Platform team as a proud sponsor of Next, a Google Cloud Platform event series. The teams made stops in New York, San Francisco, Tokyo, London, and Amsterdam where attendees learned about the latest services and features on the platform, and fellow developers and IT professionals shared how they are using Google Cloud Platform to move from idea to an application and/or decision quickly.
Ellie presented a joint demo on Twitter during the Data & Analytics Talk at Next, New York City (Left).  Jeff discussed the activity of Tweets around #GCPNext in Amsterdam (Right).

Visualizing Streamed Tweets with Tableau, Google BigQuery & Twitter
As a part of our presence at the events, we wanted to develop a live demo that highlighted and showcased our technologies. Google BigQuery has the ability to process petabytes of data within seconds and ingest data rapidly. Tableau’s live connectivity to BigQuery enables users to create stunning dashboards within minutes with our drag-and-drop interface, extending the usefulness of BigQuery to all users. For this demo, we decided to visualize  real-time Tweets from Twitter about the #GCPNext conference series.
Overall architecture for visualizing streamed Tweets in BigQuery using Tableau.

We worked together with our friends at Twitter (@TwitterDev) who developed an open-source connector called Twitter-for-BigQuery that streams Tweets directly into BigQuery.  Additionally, the connector can retrieve the last 30 days of data for the defined Tweet stream.  The APIs for the connector are provided by Gnip, which offers enterprise-grade access and filtering for the full Twitter stream. The connector enables users to define the filters for certain hashtags and usernames, and consequently streams tweets matching these filters in real time directly into BigQuery using the Tabledata.insertAll method. For the purposes of our demo, our Tweet stream included hashtags such as #bigdata, #IoT, and #GCPNext as well as usernames such as @Google.

Once the data lands in BigQuery’s tables, the data may be accessed using super-fast, SQL-like queries using the processing power of Google’s infrastructure. Google provides a console with a command line interface that’s great for analysts and developers who know how to write SQL. Tableau enhances the joint solution by providing a drag-and-drop visual interface to the data so that anybody can use it. Plus our live native connector to Google using the BigQuery REST API means a user can leverage our interface while optimized against Google’s massive infrastructure.  Additionally, Tableau and the Google BigQuery team have co-published a best practices whitepaper to help you maximize the value of our joint solution.

Using Tableau Desktop, we connected to the data and built the dashboard below, enabling users to search for keywords within the filtered Tweet stream. Then we published the live data connection to BigQuery and the dashboard to Tableau Online, our hosted analytics platform. Tableau Online is the perfect compliment to BigQuery because the solution is completely no-Ops and maintenance-free. It also supports a live connection to Google BigQuery.

Not only does the dashboard show the overall number of Tweets in the stream and the percentage occurrence of the keyword by date, but you can also visualize the actual Tweet itself by hovering over the marks in the scatter plot below.
Interactive Tableau Online dashboard visualizing live streamed Tweets in Google BigQuery.

In the video below, Ellie shares how you can interact with the Tableau Online visualization we created as well as build a new visualization using the live data connection to BigQuery directly from Tableau Online.

Demo video featuring Tableau Online visualizing live streamed Tweets in Google BigQuery.

What’s Up Next?
At Tableau, we believe that the future of data is in the cloud. We love how Google is innovating on cloud infrastructure and building the cloud services of tomorrow today. That’s why we recently announced a new named connector to Google Cloud SQL. The connector moves Google Cloud Platform and Tableau Online customers one step closer to being able to both host and analyze data completely in the cloud. This connector also compliments our existing native connectors to Google BigQuery and Google Analytics. In the future, we are committed to building broader and deeper integrations with Google to delight our users.

Try It For Yourself!
The beautiful thing about this demo is that the technologies used in the solution are easy to use. To learn more and try it for yourself, please see the following links below:

- Posted by Jeff Feng, Product Manager, and Ellie Fields, VP of Product Marketing, both at Tableau.

Do you want the power and flexibility of public cloud, but are concerned about losing control over data security? We can help. Security is at the core of Google’s architecture - we’ve spent years developing one of the world’s most advanced and secure infrastructures.  We’re committed to providing you great security, and giving you more control over how you manage security on Google Cloud Platform.

Today, we are adding Customer-Supplied Encryption Keys for Google Compute Engine in beta, which allow you to bring-your-own-keys to encrypt compute resources. Google Compute Engine already protects all customer data with industry-standard AES-256 bit encryption. Customer-Supplied Encryption Keys marries the hardened encryption framework built into Google’s infrastructure with encryption keys that are owned and controlled exclusively by you. You create and hold the keys, you determine when data is active or at rest, and absolutely no one inside or outside Google can access your at rest data without possession of your keys. Google does not retain your keys, and only holds them transiently in order to fulfill your request.

Customer-Supplied Encryption Keys are now available in beta in select countries. Starting today, you can access Customer-Supplied Encryption Keys through our API, our Developers Console, and our command-line interface, gcloud. This new functionality is currently rolling out to the Free Trial and will be available soon.

Customer-Supplied Encryption Keys provides you unprecedented control over encryption in the public cloud:

  • Secure: All of your compute assets are encrypted using the industry-leading AES-256 standard, and Google never retains your keys, meaning Google cannot decrypt your data at rest.
  • Comprehensive: Unlike many solutions, Customer-Supplied Encryption Keys cover all forms of data at rest for Compute Engine, including boot and data persistent disks.
  • Fast: Google Compute Engine is already encrypting all of your data at rest, and Customer-Supplied Encryption Keys gives you greater control, without additional overhead.
  • Included Free: We feel that encryption should be enabled by default for cloud services; we’re not going to charge you more for the option to bring your own keys.

"Google Compute Engine gives us the performance and scale to process high-volume transactions in the financial markets. With Customer-Supplied Encryption Keys, we can independently control data encryption for our clients without incurring additional expenses from integrating third-party encryption providers. This control is critical for us to realize the price/performance benefits of the cloud in a highly regulated industry."  
Neil Palmer, CTO of Sungard Consulting Services

Security is as much about control as it is about data protection. With Customer-Supplied Encryption Keys, we are giving you control over how your data is encrypted with Google Compute Engine. Keep in mind, though, if you lose your encryption keys, we won’t be able to help you recover your keys or your data - with great power comes great responsibility!  

Retain control while taking advantage of the cloud. Try Customer-Supplied Encryption Keys and let us know how it’s going on the Google Compute Engine forum. We love hearing from you.

- Posted by Leonard Law, Product Manager