We've noticted heavy latencies from large growth of App Engine. Increasing popularity is at blame. Frequently we see error logs spike indicating Request Timeouts. The Data Store API is the cause of these Timeouts. Reads, Writes and Updates are problematic.
When Writes fail due to timeout, abandonment occurs. This means the write fails and the transaction is not recorded. Data is not lost however; it goes into a limbo state. This limbo is defined as "Unapplied Writes". When a write fails, it will be logged into an emergency server and saved for later merging. We are grateful for this redundancy, yet troubled by the slowness.
Understanding goes unmentioned when waiting for performance relief. Yet its already been a month! Doesn't this Google company have unbound resources? Struggling to meet demands of growth is problematic. The rate of popularity for App Engine has "grown 25% every two months" says Google.
We have experienced interruptions in our service. This isn't good for customers. Google says that AppEngine Clients "will not be charged for Datastore CPU costs". That is good, though our end users are the most affected. It may be a good time to check out other cloud hosting platforms such as AWS (Amazon Web Services) and RackSpace Cloud Computing . Those interested in more details: Datastore Performance Growing Pains
Google will meet the demand of its popular growing service. Lots of things must happen. Many new components supplied into GAE ecosystem. Google has resources for sure. Surprising that it is lasting so long for Google to solve this. The next month will tell for sure GAE's fate.
