[D2T2S04] SageMaker를 활용한 Generative AI Foundation Model Training and Tuning

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
J U N E 2 0 2 4
Customizing Foundation Models on
Amazon SageMaker
Miron Perel
Principal AI/ML GTM Specialist
Kristine Pearce
Principal AI/ML GTM Specialist

Today’s Agenda
1. GenAI Trends
2. Gen AI on Amazon SageMaker
3. SageMaker Training Demo
4. Getting Started
G E N E R A T I V E A I

Financial
Services
Healthcare and
Life Sciences
Automotive Manufacturing
Media &
Entertainment
Telecom Energy
Generative AI is transforming all industries

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 4
Innovations at
Amazon
SCALE
How can I
scale this?
Rufus
An expert
shopping assistant
Customer
reviews
On Amazon.com
Amazon
Pharmacy
faster prescriptions and
more helpful support

More than use Amazon SageMaker for GenAI/ML

Primary reasons cited is due to
control and customization for
a given use case
Prefer
fine-tuning or training
models for specific needs
Enterprises increasing
open source usage or
switching in 2024 and onward
80% 78%
90%
GenAI Trends in the Enterprise
Ways Enterprises are Building, Buying and Optimizing FMs (a16z)
Source: https://a16z.com/generative-ai-enterprise-2024/

Cost differentials can skyrocket with high end LLMs at scale
The Problem of Sustained Inference
Price to generate a million
tokens with a open source fine
tuned LLM
Price to generate a million
tokens with a closed source LLM
$60 $3
* Upwards of
** For example
**
*
Source: https://medium.com/@ja_adimi/comparison-cost-analysis-should-
we-invest-in-open-source-or-closed-source-llms-bfd646ae1f74

Common approaches for customizing FMs
Zero-shot Few-shots Retrieval
Augmented
Generation
(RAG)
Fine-tuning Re-Training
Complexity,
Quality,
Cost,
Time
8

Common approaches for customizing FMs
Zero-shot Retrieval
Augmented
Generation
(RAG)
Fine-tuning P/re-Training
Complexity,
Quality,
Cost,
Time
9
Few-shots

Automatically generate
a customer response
with the knowledge of
your products
Make faster decisions
by automatically
extracting data from
documents
PDF
Extraction
Customer
Support
Automating process
of manually
categorizing
documents and
content
Classification
Use an LLM to
understand how
customers feel about
products and services
Customer
Sentiment
72% of enterprises are fine-tuning Domain/Task adapted LLMs balancing customization with cost and efficiency
Use Case Examples of Domain & Task Adapted LLM

Parameter efficient
re-training of FM
parameters with new
training data
Adjusting pre-trained
FMs on a specific
dataset with labeled
examples
Supervised Fine
Tuning (SFT)
Re-Training
Develop more precise
context-aware FMs
based on un/semi-
supervised learning
Domain Adaptation
Fine Tuning (DAFT)
FM is fine-tuned on the
data, with specific
instructions or
guidelines, and formats
Instruction
Fine-Tuning (IT)
72% of enterprises are fine-tuning Domain/Task adapted LLMs balancing customization with cost and efficiency
Fine Tuning & Re-Training Techniques
Fine-tune any open source LLM for your Domain Adapted LLMs (finance, healthcare, programming) OR
your Task Adapted LLMs (data summarization, PDF extraction, sentiment analysis, audio transcription)

FM Customization approaches using Amazon SageMaker
Foundation
model
Customize Task-
specific FM
W H Y Y O U U S E I T
• Faster time to market
• Maximize accuracy for specific tasks
• Achieve domain adaptation
Customizing FMs for task and
domain specific use cases
W H Y Y O U U S E I T
• Build a proprietary or re-train open-source
• Commercialize or deploy for internal use
• Drive revenue and cut operational costs
Build Train Deploy
Build/ReTrain FMs
from scratch

Select Evaluate Customize Deploy
Amazon
SageMaker
Build, train, and
deploy ML models at
scale, including FMs
Jumpstart Clarify
Ground Truth
Training
Studio
Inference
SageMaker Generative AI for ML Practitioners
Customizing FMs for Task & Domain Specific Use Cases
Customization
and control with
TTM / TCO

Discover
Fine tune,
evaluate,
and deploy
Select from the broadest and latest selection of foundation models
A V A I L A B L E I N A M A Z O N S A G E M A K E R J U M P S T A R T

Unique challenges to manage hardware resources
efficiently for large scale FM training
Strategies for
distributed training
Infrastructure
stability
Clusters provision &
management
15
Collect data

Amazon SageMaker for Large Scale FM Training:
Self-healing clusters
reduce training time
by up to 20%
Resilient
environment
SageMaker distributed
training libraries
improve performance
by up to 20%
Streamline distributed
training
Control over compute
environment and
workload scheduling
Optimized resources
utilization (SMHP)
Focus on ML without
the need to manage
infrastructure
Managed training
environment (SMTJ)
OR
AND AND

Demo video

SageMaker Training Jobs for FM Training

Financial
Services
Healthcare and
Life Sciences
Automotive Manufacturing
Media &
Entertainment
Telecom Energy

GenAI Success Stories
H E L P I N G E N T E R P R I S E S T O U S E F O U N D A T I O N M O D E L S
50%
lower costs for
hosting FMs
7 months
reduced time-to-
value from 12-18
months
80%
reduction in
inference latency
66%
cost savings with
GPU utilization
Enhance Customer Experience
Enhance Customer Experience Boost Employee Productivity
Streamline Business Processes

Start your generative AI journey today
Additional Resources
View
product page
Watch deep
dive demo videos
Read
blog posts
Read the
technical docs
SageMaker
Training
Example

Thank you!

[D2T2S04] SageMaker를 활용한 Generative AI Foundation Model Training and Tuning

Related slideshows

More Related Content

Similar to [D2T2S04] SageMaker를 활용한 Generative AI Foundation Model Training and Tuning

Similar to [D2T2S04] SageMaker를 활용한 Generative AI Foundation Model Training and Tuning (20)

Recently uploaded

Recently uploaded (20)

[D2T2S04] SageMaker를 활용한 Generative AI Foundation Model Training and Tuning