Amazon MSK
- for Apache Kafka (alternative for Amazon Kinesis Data Streams)
- Fully managed, with data stored in EBS for as long as needed (longer than 1 year)
- Or Amazon MSK serverless, which no capacity concerns
- Differences with Amazon Kinesis Data Streams
- Message size is not limited in 1MB(default)
- Topics with Partitions in Kafka = Shards in Data Streams
- MSK can only add Partition, but Data Streams can split and merge Shards
- PlainText or In-flight TLS encryption; Data Streams can only in-flight TLS
Amazon OpenSearch Service (previous “Amazon ElasticSearch”)
- Previous, “AWS ElasticSearch”
- Can search on “any” field, even partial match.
- Two cluster modes, “managed” and “serverless”.
- Not natively support SQL.
- OpenSearch Dashboards for visualisation.
AWS Amplify
- create mobile and web applications (aka. ElasticBeans for mobile and web application)
- Authentication (Cognito) + Data Store (AppSync + DynamoDB)
- End-to-End (E2E) test, using Cypress
AWS AppConfig
- deploy dynamic configuration change without code deployment; validate with JSON Schema or Lambda Function
AWS AppSync
- managed service of GraphQL, combining multiples data sources
- retrieve data in “realtime” or “MQTT” of WebSocket
- for mobile apps: local data access and data sync
- Security: API_KEY, AWS_IAM, OPENID_CONNECT, AMAZON_COGNITO_USER_POOLS
Amazon Macie
- discover and protect sensitive data (Personal Identifiable Information, PII) by machine learning and pattern matching, on S3 bucket
AWS SES
- Simple E-mail Service for sending marketing e-mails (like a Marketo or ConstantContact)
Scalability
- an application / system can handle greater loads by adapting.
- Vertical, to increase instance size
- Horizontal, also called as “elasticity”, to increase number of instances
High Availability
- running your application / system in at least 2 data centers (== Availability Zones)
Server Name Indication (SNI)
- allow multiple SSL certificates onto one web server (to serve multiple websites), only works for ALB & NLB and CloudFront
OpenTelemetry
- (opensource) provides a single set of APIs, libraries, agents, and collector services
- collects distributed traces and metrics from applications
- also collects metadata from AWS resources and services
- Auto-instrumentation Agents
- Migrate from X-Ray to AWS Distro for Temeletry if wants to standardize with open-source APIs from Telemetry or send traces to multiple destinations (AWS and partners) simultaneously
EC2 Instance Metadata (IMDS)
- Info about the EC2 instance
- access url is http://169.254.169.254/latest/meta-data, without using an IAM Role needed
- retrieve the IAM Role name from the metadata
- IMDSv2 is more secure, needs 2 steps
- Get Session Token (limited validity) – using headers & PUT
- Use Session Token in IMDSv2 calls – using headers
API Rate Limits
- DescribeInstances API for EC2 has a limit of 100 calls per seconds
- GetObject on S3 has a limit of 5500 GET per second per prefix
- For Intermittent Errors: implement Exponential Backoff
- For Consistent Errors: request an API throttling limit increase
Service Quotas (Service Limits)
- Running On-Demand Standard Instances: 1152 vCPU
- You can request a service limit increase by opening a ticket
- You can request a service quota increase by using the Service Quotas API
Exponential Backoff
- If get ThrottlingException intermittently
- For on 5xx server errors and throttling, not on the 4xx client errors
Signing AWS API requests
- using Signature v4 (SigV4) to send credentials (access key & secret key)
- HTTP Header option (signature in Authorization header)
- Query String option, ex: S3 pre-signed URLs (signature in X-Amz-Signature)
- AWS App Mesh = for application networking for microservices applications
- AWS Resource Access Manager = share a Transit Gateway connection (only?) with other AWS accounts
- AWS Server Migration Service (SMS) is for migrating virtual machines
- AWS Elastic Beanstalk is a PaaS service for describing and provisioning resources. Can be used to quickly deploy and manage applications in AWS. Developers upload applications and Beanstalk handles the deployment details. Note that it’s not serverless, it relies on EC2 instances.
- AWS Simple Workflow Service (SWF) is for executing tasks. Helps developers build, run, and scale background jobs
AWS CodeStar quickly develop, build and deploy applications on AWS- AWS Config manage the config of AWS resources
- AWS Batch batch processing of computing jobs
- Amazon Lex builds conversational interfaces into an application using voice and text
- Amazon EMR allows researchers, scientists, businesses, etc. to process vast amounts of data using hosted Hadoop running on EC2 and S3
- AWS Import/Export send HDDs with data to AWS and they import the data into S3
- Amazon Connect: call centre
- Amazon QuickSight: BI
- Amazon Neptune: interactive graphs of DBs
- AWS Config: tracks resource inventory, config history and config change notifications for the purpose of security and compliance. Assess, audit and evaluate the configurations of AWS resources.
- Amazon AppStream: streaming service
- Amazon Kinesis: collect and process streaming data
- Amazon Elastic Transcoder: convert video and audio files into versions that play on phones, tablets and PCs
- CloudSearch: search engine for your site
- AWS LightSail: easy alternative to setting up a VPC. Product set includes virtual servers (instances), MySQL DBs, HA storage and load balance
- AWS IoT Core: connected devices interact securely with cloud applications