Skip to content
@bentoml

BentoML

The easiest way to run AI Inference in the cloud
README.md

Welcome to BentoML 👋 Twitter Follow Slack

BentoML

What is BentoML? 👩‍🍳

BentoML is an open-source model serving library for building model inference APIs and multi-model serving systems with any open-source or custom AI models. It comes with everything you need for serving optimization, model packaging, and simplifies production deployment via ☁️ BentoCloud.

Get in touch 💬

👉 Join our Slack community!

👀 Follow us on X @bentomlai and LinkedIn

📖 Read our blog

Pinned Loading

  1. BentoML BentoML Public

    The easiest way to serve AI/ML models in production - Build Model Inference Service, LLM APIs, Multi-model Inference Graph/Pipelines, LLM/RAG apps, and more!

    Python 6.8k 767

  2. OpenLLM OpenLLM Public

    Run any open-source LLMs, such as Llama 3.1, Gemma, as OpenAI compatible API endpoint in the cloud.

    Python 9.4k 599

Repositories

Showing 10 of 82 repositories
  • llama_index Public Forked from run-llama/llama_index

    LlamaIndex (GPT Index) is a data framework for your LLM applications

    bentoml/llama_index’s past year of commit activity
    Python 1 MIT 4,864 0 0 Updated Jul 25, 2024
  • yatai-image-builder Public

    🐳 Build OCI images for Bentos in k8s

    bentoml/yatai-image-builder’s past year of commit activity
    Go 14 9 4 6 Updated Jul 25, 2024
  • BentoQueue Public
    bentoml/BentoQueue’s past year of commit activity
    Python 0 0 0 3 Updated Jul 25, 2024
  • BentoML Public

    The easiest way to serve AI/ML models in production - Build Model Inference Service, LLM APIs, Multi-model Inference Graph/Pipelines, LLM/RAG apps, and more!

    bentoml/BentoML’s past year of commit activity
    Python 6,832 Apache-2.0 767 151 12 Updated Jul 25, 2024
  • helm-charts Public
    bentoml/helm-charts’s past year of commit activity
    0 0 0 0 Updated Jul 25, 2024
  • byoc Public
    bentoml/byoc’s past year of commit activity
    Shell 0 0 0 0 Updated Jul 24, 2024
  • BentoVLLM Public

    Self-host LLMs with vLLM and BentoML

    bentoml/BentoVLLM’s past year of commit activity
    Python 40 9 4 0 Updated Jul 24, 2024
  • langchain Public Forked from langchain-ai/langchain

    ⚡ Building applications with LLMs through composability ⚡

    bentoml/langchain’s past year of commit activity
    Jupyter Notebook 4 MIT 14,557 0 0 Updated Jul 24, 2024
  • openllm-models Public
    bentoml/openllm-models’s past year of commit activity
    HTML 5 2 0 1 Updated Jul 24, 2024
  • OpenLLM Public

    Run any open-source LLMs, such as Llama 3.1, Gemma, as OpenAI compatible API endpoint in the cloud.

    bentoml/OpenLLM’s past year of commit activity
    Python 9,420 Apache-2.0 599 22 1 Updated Jul 23, 2024
View all repositories

Sponsoring

  • @pdm-project
GitHub Sponsor