10. Miscellanies

Amazon MSK

  • for Apache Kafka (alternative for Amazon Kinesis Data Streams)
  • Fully managed, with data stored in EBS for as long as needed (longer than 1 year)
  • Or Amazon MSK serverless, which no capacity concerns
  • Differences with Amazon Kinesis Data Streams
    • Message size is not limited in 1MB(default)
    • Topics with Partitions in Kafka = Shards in Data Streams
    • MSK can only add Partition, but Data Streams can split and merge Shards
    • PlainText or In-flight TLS encryption; Data Streams can only in-flight TLS

Amazon OpenSearch Service (previous “Amazon ElasticSearch”)

  • Previous, “AWS ElasticSearch”
  • Can search on “any” field, even partial match.
  • Two cluster modes, “managed” and “serverless”.
  • Not natively support SQL.
  • OpenSearch Dashboards for visualisation.

AWS SES

  • Simple E-mail Service for sending marketing e-mails (like a Marketo or ConstantContact)

Scalability

  • an application / system can handle greater loads by adapting.
    • Vertical, to increase instance size
    • Horizontal, also called as “elasticity”, to increase number of instances

High Availability

  • running your application / system in at least 2 data centers (== Availability Zones)

Server Name Indication (SNI)

  • allow multiple SSL certificates onto one web server (to serve multiple websites), only works for ALB & NLB and CloudFront

OpenTelemetry

  • (opensource) provides a single set of APIs, libraries, agents, and collector services
    • collects distributed traces and metrics from applications
    • also collects metadata from AWS resources and services
    • Auto-instrumentation Agents
    • Migrate from X-Ray to AWS Distro for Temeletry if wants to standardize with open-source APIs from Telemetry or send traces to multiple destinations (AWS and partners) simultaneously

API Rate Limits

  • DescribeInstances API for EC2 has a limit of 100 calls per seconds
  • GetObject on S3 has a limit of 5500 GET per second per prefix
  • For Intermittent Errors: implement Exponential Backoff
  • For Consistent Errors: request an API throttling limit increase

Service Quotas (Service Limits)

  • Running On-Demand Standard Instances: 1152 vCPU
  • You can request a service limit increase by opening a ticket
  • You can request a service quota increase by using the Service Quotas API

Exponential Backoff

  • If get ThrottlingException intermittently
  • For on 5xx server errors and throttling, not on the 4xx client errors

Signing AWS API requests

  • using Signature v4 (SigV4) to send credentials (access key & secret key)
    • HTTP Header option (signature in Authorization header)
    • Query String option, ex: S3 pre-signed URLs (signature in X-Amz-Signature)

AWS Fault Injection Simulator

  • a managed service that is commonly used in chaos engineering, and not for application development. It enables you to perform fault injection experiments on your AWS workloads to improve the performance and resiliency of your applications

AWS Batch

  • used to efficiently run hundreds of thousands of batch computing jobs in AWS; mostly used for ML model training, simulation, and analysis at any scale

AWS Simple Workflow Service (SWF) 

  • is for executing tasks. Helps developers build, run, and scale background jobs; but it does not provide serverless orchestration to multiple AWS resources

Amazon Pinpoint

  • allows you to engage with your customers across multiple messaging channels.
  • primarily used to send push notifications, emails, SMS text messages, and voice messages.

AWS CloudShell

  • simply a command-line interface used for managing AWS resources from a terminal

AWS IoT Greengrass

  • enable connected devices to run AWS Lambda functions, execute predictions based on machine learning models, keep device data in sync, and communicate with other devices securely even without an Internet connection.

Amazon DynamoDB point-in-time recovery (PITR)

  • provides automatic continuous backups of your DynamoDB table data. Point-in-time recovery (PITR) backups are fully managed by DynamoDB and provide up to 35 days of recovery points at a per second granularity.

Disaster Recovery (DR)

  • DR approaches
    • Backup and restore = lowest cost, just create backups
    • Pilot Light = small part of core services that is running and syncing data or documents
    • Warm Standby = scaled down version of a fully functional environment that is actively running
    • Multi-site = on-prem and in AWS in an active-active configuration
  • For disaster recovery in a different region, create a AMI from your EC2 instance and copy it into a 2nd region. 

AWS Global Accelerator

  • increases availability and performance improve the performance of your network traffic by utilizing the AWS global infrastructure instead of the public Internet.
  • can be expensive
  • runs over AWS global network 
  • directs traffic to optimal endpoints across multiple regions
  • By default, provides you with 2 static IP addresses that are anycast from the AWS edge network. You can migrate existing IPv4 (/24) IPs rather than creating new.

AWS Trusted Advisor 

  • primarily used to provide real-time guidance to help you provision your resources following AWS best practices but does not handle logging or compliance.

AWS Cost Management

  • gain visibility into their cloud spending, identify cost-saving opportunities, and make informed decisions about resource allocation and optimization
  • Steps of Cost Allocation
    • STEP ONE: create user-defined tags with key-value pairs that reflect attributes such as project names or departments to ensure proper categorization of resources
    • STEP TWO: apply these tags to the relevant resources to enable tracking
    • STEP THREE:  enable the cost allocation tags in the Billing console
    • (AFTER) STEP FOUR: Configure tag-based cost and usage reports (AWS Cost Allocation Reports) for detailed analysis in Cost Explorer
  • AWS Budgets is typically used for setting cost and usage limits and receiving alerts; more about monitoring and controlling costs rather than detailed tracking and reporting

Serverless ComponentMax TimeoutComments
API Gateway50 milliseconds – 29 secondsConfigurable
Lambda Function900 seconds (15 minutes)Also limited to 1,000 concurrent executions. If not handled, can lead to throttling issues.
DynamoDB Streams40,000 write capacity units per table 
S3No timeout by default, can be configured to 5-10 secondsUnlimited objects per bucket


  • AWS App Mesh = for application networking for microservices applications
  • AWS Resource Access Manager = share a Transit Gateway connection (only?) with other AWS accounts
  • AWS Server Migration Service (SMS) is for migrating virtual machines
  • AWS CodeStar quickly develop, build and deploy applications on AWS
  • Amazon Lex builds conversational interfaces into an application using voice and text
  • Amazon EMR allows researchers, scientists, businesses, etc. to process vast amounts of data using hosted Hadoop running on EC2 and S3
  • AWS Import/Export send HDDs with data to AWS and they import the data into S3
  • Amazon Connect: call centre
  • Amazon QuickSight: BI
  • Amazon Neptune: interactive graphs of DBs
  • Amazon AppStream: streaming service
  • Amazon Elastic Transcoder: convert video and audio files into versions that play on phones, tablets and PCs
  • CloudSearch: search engine for your site
  • AWS LightSail: easy alternative to setting up a VPC. Product set includes virtual servers (instances), MySQL DBs, HA storage and load balance 
  • AWS IoT Core: connected devices interact securely with cloud applications