The Triton Inference Server provides an optimized cloud and edge inferencing solution.

An LLM-powered knowledge curation system that researches a topic and generates a full-length report with citations.

EuroSys '24: "Trinity: A Fast Compressed Multi-attribute Data Store"

AWS Inter-Region Latency Monitoring

Sample database for SQL Server, Oracle, MySQL, PostgreSQL, SQLite, DB2

[ICML 2024] LLMCompiler: An LLM Compiler for Parallel Function Calling

A blazing fast inference solution for text embeddings models

SGLang is yet another fast serving framework for large language models and vision language models.

⚡ Build your chatbot within minutes on your favorite device; offer SOTA compression techniques for LLMs; run LLMs efficiently on Intel Platforms⚡

"rsync for cloud storage" - Google Drive, S3, Dropbox, Backblaze B2, One Drive, Swift, Hubic, Wasabi, Google Cloud Storage, Azure Blob, Azure Files, Yandex Files

The open source Firebase alternative. Supabase gives you a dedicated Postgres database to build your web, mobile, and AI applications.

Build ChatGPT over your data, all with natural language

Dify is an open-source LLM app development platform. Dify's intuitive interface combines AI workflow, RAG pipeline, agent capabilities, model management, observability features and more, letting yo…

LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.

S-LoRA: Serving Thousands of Concurrent LoRA Adapters

Structured Text Generation

Large Language Model Text Generation Inference

Create LLM agents with long-term memory and custom tools 📚🦙

A non-validating SQL parser module for Python

[Unmaintained, see README] An ecosystem of Rust libraries for working with large language models

Dataframes powered by a multithreaded, vectorized query engine, written in Rust

💻 A fully functional local AWS cloud stack. Develop and test your cloud & Serverless apps offline

LLM Chain for answering questions from documents with citations

Header-only C++/python library for fast approximate nearest neighbors

A high-throughput and memory-efficient inference and serving engine for LLMs

Fast Non-line-of-sight Imaging with Non-planar Relay Surfaces