AppNeta makes extensive use of Amazon Web Services for all of our products, making the AWS re:Invent conference of great interest to us. Several team members attended the conference, where many new services were announced. This post is a quick recap of those announcements, while future posts will drill down in more detail.
Kinesis is a real-time streaming data service. It allows you to define message streams and partition (shard) those streams based on capacity (number of transactions, and MB in/MB out.) Writers (producers) and readers (consumers) that then work with those streams. Autoscale groups can scale up and down based on how much data is in a stream, so processing won’t fall behind. It can be thought of as a cloud / managed version of Apache Kafka.
Stream data is persisted and replicated across three availability zones for reliability. Data remains available for reads for 24 hours, and then it is expired. Since the service is for real time data processing, it is expected that applications will process data well before that expiration time. Initially, a Java client library will be provided, though the web service API is documented allowing other languages and platforms to use this service.
CloudTrail is an auditing service that logs AWS API calls and stores those logs in S3, providing a historical record of changes to your cloud environment. Logs can then be processed by either custom applications or through a third party log management service (such as Loggly and SumoLogic.) Apps can receive notification of log delivery through SNS. This service seems useful for troubleshooting operational issues (“Which developer created all those instances?”, “When did a security group change?”, etc.)
New Instance Types
Several new instance types were announced: C3 instances are for CPU bound applications, offering more powerful CPUs than any other instance type. I2 instances were also announced, which are SSD backed, optimized for random IO (over 300K IOPS!) These followed the G2 instance type, which was unveiled earlier in the month, targeting GPU-bound apps such as 3D rendering, etc.
AppStream is a service primarily targeted for improving the user experience on mobile devices. Graphically intensive applications, such as games, can use AppStream to render video on the cloud, and then deliver it to less powerful devices such as phones and tablets. It uses a newly developed, proprietary protocol (“STX”) that responds to changes in network bandwidth and latency, providing the user with the best experience based on current conditions. The clients initially supported are iOS, Android, Windows, and FireOS.
WorksSpaces is Amazon’s entry into the VDI space. They provide a virtual Windows 7 desktop with either 50 or 100 GB of storage, and 3.75 or 7.5 GB of RAM. Office is available, preinstalled, for an extra fee. The desktop provides persistent document storage, with the user’s folder being transparently synced to S3. Active Directory integration, especially important for enterprises, is also available. Clients are available for OS X, Windows desktops, and Android, iOS, and Kindle tablets, making the virtual desktop available from almost anywhere.
AppNeta will be looking at how we can best make use of these new services in the coming months! Kinesis seems especially exciting, since we do near real time processing of billions of events daily.