Zi Li

A platform to learn and gain insights as per interest. Look into the GitHub Repositories to gain knowledge on Machine Learning and Artificial Intelligence.

2

A recent breakthrough titled "Matrix Multiplication-Free LLMs" demonstrates a huge advancement in the area of Large Language Models (LLMs) by reducing computational costs. The authors have eliminated MatMul operations from LLMs, claiming to 10 times reduction in memory usage and a 25.6% increase in training speed, all while maintaining strong performance at billion-parameter scales. Paper link: https://lnkd.in/ggph8qXc #AI #machinelearning #deeplearning #LLMs

15

I recently completed a detailed review of the paper "ALBERT: A Lite BERT for Self-supervised Learning of Language Representations" by Lan et al.! This fascinating study presents an innovative approach to language modelling, emphasizing efficiency and effectiveness through a lighter, more parameter-efficient architecture compared to traditional models like BERT. In my review, I delved into the strengths and weaknesses of the paper, discussing its impact and potential areas for improvement. This work has made a significant contribution to the field of natural language processing and has already influenced many advancements in the area. You can read my full review of the paper here: https://lnkd.in/ehxm5jXt #NLP #MachineLearning #ArtificialIntelligence #AI #Research #TechReview #DeepLearning

16

1 Comment

Routing the control flow within an RAG application based on the user’s query intent can help us create more useful and powerful Retrieval Augmented Generation (RAG) based applications. #llms #rag #chatgpt4 #gemini #aiml #llmops https://lnkd.in/d54eVrHZ

8

More opportunities

13

Join us in Irvine on May 15th at 6:30 PM for our latest OC ACM talk, "Understanding Life via Computational Bioinformatics" presented by Dr. Shawn X. Wang, a distinguished professor of computer science at Cal State Fullerton. For more info or to RSVP click here: https://lnkd.in/gX72Ss_g Dr. Shawn X. Wang will provide an insightful exploration of computational bioinformatics, a field dedicated to extracting, organizing, and interpreting biological data to uncover new insights. Divided into two parts, the presentation will first highlight the transformative impact of computational bioinformatics research on enhancing quality of life and deepening our understanding of biology. In the second segment, Dr. Wang will offer a concise overview of his research endeavors, focusing on innovative algorithms and data analysis techniques applied to protein 3D structures, microarrays, and DNA sequence comparisons. Join us for a cohesive journey into the intersection of computation and biology. #acm #compsci #bioinformatics #techtalk #irvine #calstatefullerton #science #innovation

1

New ODE (ordinary differential equation) solvers in #MATLAB from the SUNDIALS suite, provides the ability to perform parameter sensitivity analysis. Learn more here 🔗

4

Introducing AGRaME for Any-Granularity Ranking with Multi-Vector Embeddings to rank at varying levels of granularity while maintaining encoding at a single (coarser) level. Link to paper: https://lnkd.in/gC2TtAWk We propose a multi-granular contrastive loss for training multi-vector approaches and validate its utility with both sentences and propositions as ranking units. We demonstrate the application of proposition-level ranking to post-hoc citation addition in retrieval-augmented generation, surpassing the performance of prompt-driven citation generation. Work done in collaboration with Apple Knowledge Platform Team Omar Attia Yunyao Li (hiring) Saloni Potdar

64

Trim chips, not precision! :). Check out llama-8b running on 8 sockets at 430 tokens/second and 16 bits! The power of dataflow architecture is on full display here. And we have just scratched the surface! A lot more to come. It was also very exciting to see that this update was covered in Andrew Ng's news letter! If you want to work on applications that can be enabled by these new capabilities or are want to work on LLMs, Multimodality, model merging and other techniques, or the infrastructure that supports these capabilities, we are hiring! #generativeAI #LLM

34

Excited to be back in the Bay Area this week to represent Caper and Instacart at the Embedded Vision Summit! Ever curious how to deploy and optimize AI models running on edge devices? I’ll break down some tips and techniques in my talk, "Real-Time Retail Product Classification on Android Devices inside the Caper AI Cart” If you're into AI, machine learning, or just want to learn about the Caper Cart or how we're improving our user's experience by leveraging AI, please feel free to come say hi!

60

2 Comments

Using generative AI, NVIDIA operations built an AI Planner agent, developed on NVIDIA Inference Microservices (NIM). The agent leverages LLM, NeMo Retriever and CuOpt NIM to reduce re-planning time from hours to just seconds. https://nvda.ws/4bIfS2T

2

Here's the technical breakdown of the AI Penalty Challenge authored by Alok Pattani (check out this video if you haven't seen the demo yet: https://lnkd.in/gsD3xjJ2) Key Takeaways: ☁️ Firestore for triggering slow-mo video capture on Android phones & uploading to Google Cloud via Firebase Storage. 🧠 Cloud AutoML for training custom object detectors (ball & goal tracking) to calculate speed & accuracy. 🗣️ Gemini Pro 1.5 on Vertex AI for personalized coaching feedback based on multiple data points. 🎨 Imagen on Vertex AI for instant, prompt-driven image editing for cool player pics. More technical insights from the project to come... #AIChallenge #MachineLearning #SportsTech #GoogleCloud #footballNOTsoccer

90

6 Comments

Thanks Philipp Schmid written this useful blog!

4

The biggest shortcoming I have noticed with open source LLMs is the lack of logprobs. In the author's words. "2. Functionality: commercial models are more likely to support function calling and JSON mode. However, most model providers have no or limited API for logprobs. Logprobs are useful for classification tasks, confidence scoring, and interpretability." https://lnkd.in/gZSbhyKP

1

It's hard to run inference for llms because of their large memory footprint because of all model parameters and activations required at inference time and parallelization is hard because of the autoregressive nature of generating tokens.    So to optimize inference in general you need to lower the memory footprint (use fewer GPUs), compute(FLOPS) and latency.      There are some approaches to make inference efficient(less memory usage and fast):      1. Apply parallelism techniques to run inference across a large number of GPUs      2. Offloading unused data to the CPU and reading it back when needed (less memory usage but high latency)      3. Smart batching strategies like continuous batching (vllm uses this strategy)      4. Compression techniques like quantization, distillation or pruning    5. Architectural optimizations like using PagedAttention (efficiently manages key and value in memory)

14

Less than 20 lines of code is all it takes to get started with Llama-3 on Intel Xeon Processors. Get started with the tutorial below. #IntelXeon #LLM #Developer

6