Anyscale’s Post

FP8 quantization is now available in vLLM - check it out! Quantized inference is one of the best ways to reduce the costs of LLM deployments.

Text Embedding Inference (TEI) to run embedding models released by Hugging Face team , using this we can run many embeddings models in much faster way for #rag applications https://lnkd.in/gkpJie4E

this is actually very impressive

5 Advantages of Support Vector Machine Algorithms. Tutorial by @finesse_ai https://lttr.ai/AMIDy #svm #machinelearning

Exciting updates from the Ultralytics team! 🚀 We've just rolled out a series of enhancements to our RT-DETR models, aiming to boost export performance and flexibility for all users in https://lnkd.in/dbs4kQbj. 🌟 Key updates include: - Export to TorchScript and ONNX formats now possible! - Added missing class names to all RT-DETR models. 🚀 - More flexible RT-DETR model sources, no file extension checks required. 🌐 - Consistent prediction handling and improved validation across models. 🎯 Here's to making machine learning tasks smoother and more efficient for everyone! 🛠️👥 #Ultralytics #MachineLearning #YOLO #RTDETR #TechUpdates

Organisations and businesses, it’s time we embrace machine learning for our talent acquisition strategy, especially for recruitment and selection and learning and development, gen Z skills are more diverse. It’s critical that we embrace these diverse skills and use to retain our gen Z …gone are the days where organisations use classroom training with trainers, gamification and machine learning are good ways of embracing the change in learning and development. #learninganddevelopment #talentaquisition #inclusion #generationZ #embracediversity

GPU's and modern CPU's implement Single Instruction, Multiple Data (SIMD) pipelines allowing multiple operations to be issued in parallel which is critical in Machine Learning where the data sets are often very large. Check out my new article : "The Need for Speed: vector vs for loop" on Medium. It talks about how NumPy effectively uses modern day libraries and makes better use of available data parallelism.

This is a fantastic (if not obvious) result. Using LLMs together in an ensemble outperforms single models. Paper below: https://lnkd.in/ghSWhyGn

it’s quite important to pick your models wisely: > mixtral*/ phi* overfit gsm8k > claude/ gpt/ gemini dont (closed api) > llama 3 70b instruct scores decently well in large open models category > mistral 7b / llama 2 13b / codellama 13b score decently for gpu poor models interestingly mistral 7b v0.1 scores much better than mixtral 8x7b - wonder what changed in their training/ sft dataset

Low latency and high throughput are the most desired features of any ML pipeline. There are 3 ways to decrease memory utilization and inference time for a machine learning pipeline: The three methods for making ML pipelines are: 1. Bucketing sequences of different sizes to increase inference speed 2. Dynamic batch size for maximum resource utilization 3. Hosting multiple models on the same engine for optimal work sharing of the CPU Discover how to make ML pipelines more efficient with bucketing in this article in the comments.