When Streak — CRM in your inbox — launched in March 2012, our userbase grew 30% every week for four consecutive months. Today, Streak supports millions of users with only 1.5 back-end engineers. We chose Google App Engine to power our application because it enabled our team to build features fast and scaled with user growth. Plus, we didn’t have to worry about infrastructure.


Streak’s data growth

Here are six tips we’ve learned building on App Engine, and if you’d like even more detail – including an overview of our app’s architecture and 15 minutes of Q&A – you can check out my webinar.  

1. Keep user-facing GET requests fast
This tip isn’t specific to App Engine, as it really applies to most web applications. User-facing GET requests should be quick. App Engine has a 60 second timeout on all requests; frankly, if the total latency after a user interaction is taking longer than 200ms, users will perceive your app as slow. To keep requests fast, you should do your heavyweight processing – such as calculations or complex queries –  either in the background or at write time. That way, when the user requests data (read time), it’s already precalculated and ready to go.

2.  Take advantage of Managed VMs
So, what are Managed VMs? Managed VMs are a new hosting environment for App Engine, enabling you to take advantage of beefier compute resources and run your own custom runtimes. For example, we host our back-end data processing modules on n1-standard-1 machines (1 CPU and 3.75 GB mem), rather than App Engine frontend instances. This provides better performance and cost savings, due to sustained use discounts. Yes, Managed VMs take a littler longer to boot up than an App Engine frontend instance, but they're perfect for our background processing needs.

3. Denormalize for faster reads
Google Cloud Datastore is a NoSQL database so if you’re coming from the RDBMS world, it requires a different approach to data modeling. You have to be comfortable denormalizing and duplicating data, since SQL joins don’t exist. While data duplication might feel uncomfortable, by doing so, your reads will be very fast.  

4.  Break your application into modules
Modules make it easy for you to break your App Engine app into different components. For example, you could have a module for your user-facing traffic and one for background processing. Each module has its own yaml file, so you can set parameters such as instance size, version number, runtime language used, and more. As mentioned above, our backend modules take advantage of Managed VMs for performance/cost benefits, while our frontend module uses App Engine frontend instances that scale quicker. The documentation discusses best practices on how you should structure your app.   

5. Deploy aggressively and use traffic splitting
At Streak, we do continuous deployments because versioning, deployment and rollout is easy with App Engine. In fact, sometimes we deploy up to 20 times per day to get changes into the hands of customers. We aggressively deploy to many production versions of our app and then selectively turn on new features for our users. As we slowly ramp up the traffic to these new versions via traffic splitting, we catch issues early and often. These are usually really easy to deal with because each of our new code deploys has a small set of functionality, so its easy to find the relevant issues in the code base. We also use Google Cloud Monitoring and our own homegrown system (based on #6 below) to monitor these deploys for changes.

6. Use BigQuery to analyze your log files
Application and request logs can give you valuable insights into performance and help you make product improvements. If you’re just starting out, the log viewer’s list of recent requests will be just fine, but once you’ve reached scale you’ll want to do analysis on aggregate data or a specific user’s requests. We’ve built custom code to export our logs to Google BigQuery, but you can now stream your logs directly from the Developers Console. With these insights, my team can build a better user experience.

Watch the webinar
App Engine has been critical to our success. As our application has scaled, so has App Engine and we’ve been able to focus on building features for our customers, rather than ops. To learn more tips about App Engine – including an overview of our architecture and 15 minutes of Q&A – watch the full webinar.

 -Posted by Aleem Mawani, CEO and co-founder, Streak