Google Cloud Platform provides a reliable and scalable compute, storage and network infrastructure for all your big data needs. We have worked extensively with the Open Source community to optimize the Hadoop ecosystem onto Cloud Platform. In 2014, we helped simplify the deployment of Apache Hadoop and Apache Spark on Google Cloud Platform by introducing bdutil, a Command Line toolset to accelerate deployment. To reduce cluster startup time, increase interoperability, and streamline storage of the source data and subsequent results, we have also provided connectors to Google Cloud Storage and Google BigQuery.

Today, we’re happy to announce the availability of Hortonworks Data Platform, HDP 2.2, on Google Cloud Platform. HDP 2.2. has been certified by Hortonworks for use on Google Cloud Platform, along with usage of bdutil deployment toolset and the Google Cloud Storage connector. Google and Hortonworks believe in providing a seamless experience for starting and running your hadoop tasks on the cloud. We want users to be focused on developing and analyzing their data, rather than worrying about bringing up Hadoop clusters.

You can take advantage of integrated and certified HDP plugin with bdutil and start deployment of standard clusters in a matter of minutes, with the following command line:

./bdutil deploy -e platforms/hdp/ambari_env.sh

By default, bdutil will deploy a cluster with 5 nodes, per HDP recommendations, along with the latest version of HDP and recommended HDP components. Once deployed, the cluster is ready to run Pig Scripts, MapReduce jobs, Hive Queries, or additional Hadoop services supported by HDP. You’ll also have access to the Ambari GUI to perform additional configuration and setup activities.
For additional information, please visit our bdutil Hortonworks documentation. You can download the bdutil setup scripts in zip format or tar.gz format.

To find out more about our joint collaboration, go here.

-Posted by Ram Ramanathan, Product Manager