18. ML – Other AWS Services

=== TEXT ===

Amazon Comperehend

  • Natural Language Processing and Text Analytics
  • Input social media, emails, web pages, documents, transcripts, medical records
    • Comprehend Medical
      • tailored only for extracting medical information from unstructured text using natural language processing
  • Extract key phrases, entities, sentiment, language, syntax, topics, and document classifications
    • perfect for evaluating customer feedback
  • Events detection
  • PII Identification & Redaction
  • Targeted sentiment (for specific entities)
  • Compose
    • Entities: Noun with Category, Confidence
    • Key phrases: Noun
    • Language
    • Sentiment: Neutral, Positive, Negative, Mixed
    • Syntax: Noun, Verb, Adposition, Adjustive, …
  • can provide quick sentiment analysis results, it’s not focused on labeling the data for future use in training custom models (in large data set)

Amazon Translate

  • Uses deep learning for translation
  • Supports custom terminology
  • In CSV or TMX format
  • Appropriate for proper names, brand names, etc.

Amazon Transcribe

  • Speech to text
    • Input in FLAC, MP3, MP4, or WAV, in a specified language
    • Streaming audio supported (HTTP/2 or WebSocket)
  • Speaker Identificiation
    • Specify number of speakers
  • Channel Identification
    • i.e., two callers could be transcribed separately
    • Merging based on timing of “utterances”
  • Automatic Language Identification
  • Custom Vocabularies
    • Vocabulary Lists (just a list of special words – names, acronyms)
    • Vocabulary Tables (can include “SoundsLike”, “IPA”, and “DisplayAs”)
  • Practices
    • Call Analytics
    • Medical
    • Subtitling

Amazon TextTract

  • OCR with forms, fields, tables support
  • detect typed and handwritten text in a variety of documents

Amazon Kendra

  • Enterprise search with natural language
    • a search service that uses natural language processing and advanced ML algorithms to return specific answers to search questions from your data
    • to extract information from documents
  • Combines data from file systems, SharePoint, intranet, sharing services (JDBC, S3) into one searchable repository
  • ML-powered (of course) – uses thumbs up / down feedback
  • Relevance tuning – boost strength of document freshness, view counts, etc.
Use CasesAmazon KendraElasticsearchOpenSearch
Improve search experiencesYesYes Yes
Enhance customer interactions and satisfactionYesYes Yes
Natural Language Processing (NLP)YesYes, but with the help of additional configurations or integrations. Yes, but it requires external NLP tools.
Full-text searchYesYesYes
Enterprise searchYesYes, but it requires extra configuration.Yes, but it needs extra configuration.
Distributed searchYesYesYes
Piped query language searchNoYesYes
Integrate search functionality into your SaaS applicationsYesYesYes
Implement multi-tenancy and get rid of fraud and riskYesYesYes
Semantic searchYesYesYes
Application searchYesYesYes
E-commerce searchNoYesYes
Log and event data monitoringNoYesYes
Aggregate and analyze large datasetsNoYesYes
Security Information and Event Management (SIEM)NoYesYes

Amazon OpenSearch

  • AWS version of ElasticSearch
  • search and analytics suite used for a broad set of use cases like real-time application monitoring, log analytics, and website search
  • can be utilized as a vector database, which is ideal for storing and retrieving high-dimensional vectors needed for RAG applications. This enables the chatbot to efficiently search and retrieve relevant document vectors that contain the necessary operational guidelines.
  • Lexical search: the search engine compares the words in the search query to the words in the documents, matching word for word.
    • Searching through technical documentation or legal texts
    • Finding specific product codes or part numbers in a database
    • Locating exact phrases in a large corpus of text
  • Semantic search: uses an ML model to encode text or other media (such as images and videos) from the source documents as a dense vector in a high-dimensional vector space. It similarly codes the query as a vector and then uses a distance metric to find nearby vectors in the multi-dimensional space to find matches. The algorithm for finding nearby vectors is called k-nearest neighbors (k-NN). Semantic search doesn’t match individual query terms—it finds documents whose vector embedding is near the query’s embedding in the vector space and therefore semantically similar to the query. This allows you to return highly relevant items even if they don’t contain any of the words that were in the query.
    • Powering digital assistants and chatbots
    • Enhancing e-commerce product discovery
    • Improving academic research by finding conceptually related papers
AspectLexical SearchSemantic Search
PrecisionHigh for exact word matchesLower for specific word matches, higher for conceptual matches
FlexibilityIn its more common form (also known as “full-text search”), it supports wildcards, morphology, and fuzzy matchingVery high, understands context and intent
SpeedGenerally faster, less computationRequires more computational power
Context AwarenessLimited to literal and morphological matchesHigh, understands meaning and relationships
Result TransparencyEasy to understand match criteriaMore complex, based on advanced models
Result CountExact result count availableOften limited to top results, full count may be unclear
Ideal Use CasesStructured queries, known terminologyOpen-ended questions, concept exploration

=== SPEECH ===

Amazon Polly

  • Neural Text-To-Speech
  • Lexicons
    • Customize pronunciation of specific words & phrases
    • Example: “World Wide Web Consortium” instead of “W3C”
  • SSML
    • Speech Synthesis Markup Language
    • Gives control over emphasis, pronunciation, breathing, whispering, speech rate, pitch, pauses.
  • Speech Marks
    • metadata can encode when sentence / word starts and ends in the audio stream
    • Useful for lip-synching animation

=== VISUAL ===

Amazon Rekognition

  • Computer vision
  • Object and scene detection
  • Image moderation
  • Facial analysis
  • Celebrity recognition
  • Face comparison
  • Text in image
  • Video analysis
    • Objects / people / celebrities marked on timeline
    • People Pathing
  • helps identify inappropriate, unwanted, or offensive content
  • The Nitty Gritty
    • Images come from S3, or provide image bytes as part of request
    • Facial recognition depends on good lighting, angle, visibility of eyes, resolution
    • Video must come from Kinesis Video Streams
      • H.264 encoded
      • 5-30 FPS
      • Favor resolution over framerate
    • Can use with Lambda to trigger image analysis upon upload
    • Can use Custom Labels

AWS Panorama

  • Computer Vision (SDK) at the edge (ie, can be processed in local)
  • Brings computer vision to your existing IP cameras
    • Make predictions locally
    • Process video feeds at the edge (helpful in limited internet bandwidth)
  • AWS Panorama Appliance is the hardware that runs your applications.
    • The software on the AWS Panorama Appliance connects to camera streams, sends frames of video to your application, and displays video output on an attached display
  • Usecase
    • Supply chain logistics
    • Traffic management
    • Detect manufacturing anomalies

=== OTHER : APPLICATIONS ===

Amazon Forecast

  • “AutoML” chooses best model for your time series data
    • CNN-QR
      • Convolutional Neural Network – Quantile Regression
      • Best for large datasets with hundreds of time series
      • Accepts related historical time series data & metadata
    • DeepAR+
      • Recurrent Neural Network
      • Best for large datasets
      • Accepts related forward-looking time series & metadata
    • Prophet
      • Additive model with non-linear trends and seasonality
    • NPTS
      • Non-Parametric Time Series
      • Good for sparse data. Has variants for seasonal / climatological forecasts
    • ARIMA
      • Autoregressive Integrated Moving Average
      • Commonly used for simple datasets (<100 time series)
    • ETS
      • Exponential Smoothing
      • Commonly used for simple datasets (<100 time series)
  • Works with any time series
    • Price, promotions, economic performance, etc.
    • Can combine with associated data to find relationships
  • Inventory planning, financial planning, resource planning
  • Based on “dataset groups,” “predictors,” and “forecasts.”

Amazon Lex

  • Natural-language chatbot engine
  • A Bot is built around Intents
    • Utterances invoke intents (“I want to order a pizza”)
    • Lambda functions are invoked to fulfill the intent
    • Slots specify extra information needed by the intent
      • Pizza size, toppings, crust type, when to deliver, etc.
  • Can deploy to AWS Mobile SDK, Facebook Messenger, Slack, and Twilio
  • Lex Automated Chatbot Designer
    • provide existing conversation transcripts
    • Lex applies NLP & deep learning, removing overlaps & ambiguity
    • Intents, user requests, phrases, values for slots are extracted
    • Ensures intents are well defined and separated
    • Integrates with Amazon Connect transcripts

Amazon Personalise

  • Fully-managed recommender engine
  • API access
    • Feed in data (purchases, ratings, impressions, cart adds, catalog, user demographics etc.) via S3 or API integration
    • provide an explicit schema in Avro format
    • Javascript or SDK
    • GetRecommendations
      • Recommended products, content, etc.
      • Similar items
    • GetPersonalizedRanking
      • Rank a list of items provided
      • Allows editorial control / curation
  • Features
    • Real-time or batch recommendations
    • Recommendations for new users and new items (the cold start
      problem)
    • Contextual recommendations
      • Device type, time, etc.
    • Similar items
    • Unstructured text input
    • Intelligent user segmentation
      • For marketing campaigns
    • Business rules and filters
    • Promotions
      • Inject promoted content into recommendations
      • Can find most relevant promoted content
    • Trending Now
    • Personalized Rankings
  • Terminology
    • Datasets
      • Users, Items, Interactions
    • Recipes
      • USER_PERSONALIZATION
      • PERSONALIZED_RANKING
      • RELATED_ITEMS
      • USER_SEGMENTATION
    • Solutions
      • Trains the model
      • Optimizes for relevance as well as your additional objectives
        • Video length, price, etc. – must be numeric
      • Hyperparameter Optimization (HPO)
    • Campaigns
      • Deploys your “solution version”
      • Deploys capacity for generating real-time
        recommendations
  • Hyperparameter
    • User-Personalization, Personalized-Ranking
      • hidden_dimension (HPO)
      • bptt (back-propagation through time – RNN)
      • recency_mask (weights recent events)
      • min/max_user_history_length_percentile (filter out robots)
      • exploration_weight 0-1, controls relevance
      • exploration_item_age_cut_off – how far back in time you go
    • Similar-items
      • item_id_hidden_dim (HPO)
      • item_metadata_hidden_dim (HPO with min & max range specified)
  • Maintaining Relevance
    • Use PutEvents operation to feed in real-time user behavior
    • Retrain the model
      • They call this a new solution version
      • Updates every 2 hours by default
      • Should do a full retrain (trainingMode=FULL) weekly

Amazon Connect

  • an Amazon Web Services (AWS) public cloud customer contact center service
  • Contact Lens
    • For customer support call centers
    • Ingests audio data from recorded calls
    • Allows search on calls / chats
    • Sentiment analysis
    • Find “utterances” that correlate with successful calls
    • Categorize calls automatically
    • Measure talk speed and interruptions
    • Theme detection: discovers emerging issues

Amazon Fraud Detector

  • Upload your own historical fraud data
  • Builds custom models from a template you choose
  • Exposes an API for your online application
  • Assess risk from:
    • New accounts
    • Guest checkout
    • “Try before you buy” abuse
    • Online payments

Amazon Lookout

  • Equipment, metrics, vision
  • Detects abnormalities from sensor data automatically to detect equipment issues
  • Monitors metrics from S3, RDS, Redshift, 3rd party SaaS apps
  • Vision uses computer vision to detect defects in silicon wafers, circuit boards, etc.

Amazon Monitron

  • End to end system for monitoring industrial equipment & predictive maintenance

AWS Deep Composer

  • AI-powered (music) keyboard
  • Composes a melody into an entire song
  • For educational purposes

AWS DeepRacer

  • Reinforcement learning powered 1/18-scale race car

== OTHER : AWS SERVICES ===

TorchServe

  • Model serving framework for PyTorch
  • Part of the PyTorch open source project from Facebook (Meta?)

AWS Neuron

  • SDK for ML inference specifically on AWS Inferentia chips
  • EC2 Inf1 instance type
  • Integrated with SageMaker or whatever else you want (deep learning AMI’s, containers, Tensorflow, PyTorch, MXNet)

Amazon Augmented AI (A2I)

  • Human review of ML predictions
  • Builds workflows for reviewing low-confidence predictions
  • Access the Mechanical Turk workforce or vendors
  • Integrated into Amazon Textract and Rekognition
  • Integrates with SageMaker
  • Difference with SageMaker Ground Truth
    • Focus:
      • A2I: Focuses on integrating human review into the decision-making process of models post-prediction.
      • Ground Truth: Focuses on creating high-quality labeled datasets during the training phase.
    • Stage of ML Workflow:
      • A2I: Used during the model deployment and prediction phase.
      • Ground Truth: Used during the data preparation and model training phase.
    • Automation:
      • A2I: Adds human review when machine confidence is low.
      • Ground Truth: Combines automated labeling with human corrections to produce accurate training data.

Amazon Athena

  • an interactive query service that enables users to run SQL-based queries directly on data stored in Amazon S3 without complex ETL processes
  • supports many file formats, including JSON and Parquet, making it ideal for querying structured and semi-structured data
  • Serverless

Amazon QuickSight

  • a business intelligence service typically used to create visualizations and dashboards
  • ML Insights
    • uses machine learning to uncover hidden insights and trends, helping organizations identify key drivers and forecast business metrics
    • ML-powered anomaly detection
      • leverages Amazon’s Random Cut Forest algorithm to detect outliers and identify the top contributors to any significant changes, such as spikes in sales or dips in website traffic
    • ML-powered forecasting
      • forecast metrics by handling complex scenarios like seasonality and outliers
    • Autonarratives
      • enhance dashboards with plain language descriptions, enabling users to quickly interpret data and make faster, informed decisions