Richard Finger

Growth with Custom Neural Machine Translation (NMT) & LSPs Precise Translation, Localization #L10N, and Internationalisation #I18N for your software, knowledge bases, training and documentation is essential for international success. Why NMT? Both advanced NMT models and LLMs (GPTs) leverage the state-of-the-art Transformer Architecture. NMTs specialise in translation tasks, resulting in superior accuracy for industry specific e.g. finance, legal, and technical domains. They have been arround for longer then LLMs. NMTs are the engine behind most of the AI translation tools and workflows, such as: ♻️ Hybrid Translation 1. Machine Translation (MT): NMT engine performs the initial translation. 2. Quality Prediction: Tools like ModelFront can analyse the MT output and predict segments with high quality that might not require human editing. 3. Human Post-Editing: Translators focus on segments identified as needing improvement or specific domain expertise. 4. Terminology Management: Some TMS and CAT tools ensure advanced jargon consistency. 5. Machine Learning #ML: Improve 1. MT, ⚠️ consider the ownership of your Translation Memories and Privacy ☑️ The Challenge: Your Custom NMT ✔️ Your options: ✅️ Partner with Language Service Provider (LSP) with Established NMT capabilities: Google Translate | Amazon Translate Microsoft Translator | IBM Watson Translator Lionbridge | XTM Cloud | DeepL WorldServer, Language Weaver RWS | SDL Crowdin | Smartling | Translated | LanguageWire ✅️ Develop - Experiment - Learn In-House (open source) #Python: #OpenNMT ⚡️Sockeye https://lnkd.in/eumhW6DW Fairseq by AI at Meta ⚠️ C++ core T5 by Google Research AI ✔️ Benefits: Superior Accuracy: User-centric, can be critical Integration: Language Services with your workflows and strategy Domain-Specific Terminology Management and Transcreation 👏 Your Expertise Matters! We'd love to hear from practitioners who have tackled these challenges firsthand. Share your experience with custom NMT in the comments! 👏 Have I missed an interesting NMT open source, affordable developers ressource or a #LSP? #Fintech #Regtech #NeuralMachineTranslation #LLM #NLP #AIProductManagement #AITranslation #ProductGrowth

24

24件のコメント

neptune.ai's integration with PyTorch Lightning makes it super easy to track and compare your experiments on Lightning AI. Try out the integration with zero setup: https://lnkd.in/gdf6wwuv

40

Understanding Reinforcement Learning-Based Fine-Tuning of Diffusion Models: A Tutorial and Review Masatoshi Uehara, Yulai Zhao, Tommaso Biancalani, Sergey Levine Abstract This tutorial provides a comprehensive survey of methods for fine-tuning diffusion models to optimize downstream reward functions. While diffusion models are widely known to provide excellent generative modeling capability, practical applications in domains such as biology require generating samples that maximize some desired metric (e.g., translation efficiency in RNA, docking score in molecules, stability in protein). In these cases, the diffusion model can be optimized not only to generate realistic samples but also to explicitly maximize the measure of interest. Such methods are based on concepts from reinforcement learning (RL). We explain the application of various RL algorithms, including PPO, differentiable optimization, reward-weighted MLE, value-weighted sampling, and path consistency learning, tailored specifically for fine-tuning diffusion models. We aim to explore fundamental aspects such as the strengths and limitations of different RL-based fine-tuning algorithms across various scenarios, the benefits of RL-based fine-tuning compared to non-RL-based approaches, and the formal objectives of RL-based fine-tuning (target distributions). Additionally, we aim to examine their connections with related topics such as classifier guidance, Gflownets, flow-based diffusion models, path integral control theory, and sampling from unnormalized distributions such as MCMC. 👉 https://lnkd.in/gmgfxP_Y #machinelearning

1

🎨 FIFO-Diffusion: Generating Infinite Videos from Text without Training - My Key Takeaways from the Paper ❓What? The authors proposed a new method called FIFO-Diffusion for generating infinitely long videos from text without additional training. The method utilizes pre-trained diffusion models that can only generate short video clips. 🎯 Why? Existing approaches for generating long videos based on diffusion models have several limitations - some require a lot of memory, while others lead to a loss of temporal consistency between frames. FIFO-Diffusion allows for the generation of arbitrarily long videos while maintaining quality and temporal consistency. 🛠 How? The approach is based on the idea of diagonal denoising. A sequence of frames with increasing noise levels is placed in a queue. At each step, the cleanest frame is extracted, and a new noisy frame is added to the end. To reduce the gap between training and inference, latent partitioning is proposed. Lookahead denoising is also used for more accurate noise prediction in frames. 🚀 Results? The method allowed for the generation of videos longer than 10,000 frames while maintaining high image quality and consistency of motion and semantics throughout the entire video. FIFO-Diffusion demonstrated superiority over other methods of infinite video generation in qualitative experiments and user evaluations. Moreover, the method works with various base architectures of diffusion models and does not require training or fine-tuning. Paper: https://lnkd.in/d6yWrMEW

1

How I build 𝗟𝗕𝗢 𝗠𝗼𝗱𝗲𝗹𝘀 (a guide you can use) 👇 ~~~ 📌𝗧𝗼𝗱𝗮𝘆'𝘀 𝗔𝗻𝗻𝗼𝘂𝗻𝗰𝗲𝗺𝗲𝗻𝘁: grab the guide here 👉 https://lnkd.in/efD2wcJF ~~~ 𝗧𝗮𝗯𝗹𝗲 𝗼𝗳 𝗖𝗼𝗻𝘁𝗲𝗻𝘁𝘀: ✓ Overall Model Structure ✓ Detailed Model Structure ✓ Raw Data ✓ Support Schedules ✓ Quality of Earnings (or "QofE") ✓ Three Statement Model ✓ Transaction and Scenarios ✓ Summaries ✓ Control Panel and Error Checking ~~~ 👋 Hey, I'm Chris Reilly, and I teach Financial Modeling based on real Private Equity and FP&A experience 👉 https://bit.ly/46xfrp5

203

4件のコメント

MAX⚡️ Graph API allows you to build your entire AI inference pipeline in Mojo🔥. Watch this video to see how you can get started with the MAX Graph API, from a simple example to implementing PyTorch graphs with custom ops for faster performance. 🚀 https://lnkd.in/gppJKq3A #ai #llm #modular #mojo #max #engineering #tech #startup #future #artificialintelligence #graph #cloud #pytorch #onnx

24

𝗗𝗼 𝗬𝗼𝘂 𝗪𝗮𝗻𝘁 𝘁𝗼 𝗠𝗮𝘀𝘁𝗲𝗿 𝗠𝗮𝗰𝗵𝗶𝗻𝗲 𝗟𝗲𝗮𝗿𝗻𝗶𝗻𝗴? 🚀 >҉ 𝗠𝗮𝗰𝗵𝗶𝗻𝗲 𝗟𝗲𝗮𝗿𝗻𝗶𝗻𝗴 𝗮𝘁 𝗦𝘁𝗮𝗻𝗳𝗼𝗿𝗱: Learn from Andrew Ng's famous Machine Learning course. ↳ Resource: https://lnkd.in/gU7se3VY >҉ 𝗙𝗿��𝗲 𝗖𝗼𝗱𝗲 𝗖𝗮𝗺𝗽'𝘀 𝗠𝗮𝗰𝗵𝗶𝗻𝗲 𝗟𝗲𝗮𝗿𝗻𝗶𝗻𝗴 𝗪𝗶𝘁𝗵 𝗣𝘆𝘁𝗵𝗼𝗻: Get hands-on with Python and Machine Learning fundamentals. ↳ Resource: https://lnkd.in/gcddnRKp >҉ 𝗞𝗵𝗮𝗻 𝗔𝗰𝗮𝗱𝗲𝗺𝘆'𝘀 𝗜𝗻𝘁𝗿𝗼 𝘁𝗼 𝗠𝗮𝗰𝗵𝗶𝗻𝗲 𝗟𝗲𝗮𝗿𝗻𝗶𝗻𝗴: An introductory course that covers basic concepts. ↳ Resource: https://lnkd.in/gPDvs88p >҉ 𝗛𝗮𝗿𝘃𝗮𝗿𝗱'𝘀 𝗜𝗻𝘁𝗿𝗼 𝘁𝗼 𝗔𝗜 𝗮𝗻𝗱 𝗠𝗮𝗰𝗵𝗶𝗻𝗲 𝗟𝗲𝗮𝗿𝗻𝗶𝗻𝗴: A comprehensive YouTube series covering AI and ML topics. ↳ Resource: https://lnkd.in/gKv8iHtz >҉ 𝗚𝗼𝗼𝗴𝗹𝗲'𝘀 𝗠𝗮𝗰𝗵𝗶𝗻𝗲 𝗟𝗲𝗮𝗿𝗻𝗶𝗻𝗴 𝗖𝗿𝗮𝘀𝗵 𝗖𝗼𝘂𝗿𝘀𝗲: Interactive tutorials and exercises from Google. ↳ Resource: https://lnkd.in/dhEssFpc >҉ 𝗧𝗲𝗻𝘀𝗼𝗿𝗙𝗹𝗼𝘄'𝘀 𝗢𝗳𝗳𝗶𝗰𝗶𝗮𝗹 𝗧𝘂𝘁𝗼𝗿𝗶𝗮𝗹𝘀: Deep dive into TensorFlow with practical examples. ↳ Resource: https://lnkd.in/g5vp3eYw >҉ 𝗞𝗮𝗴𝗴𝗹𝗲 𝗟𝗲𝗮𝗿𝗻: Hands-on learning through real-world data science problems. ↳ Resource: https://lnkd.in/dx72fpPz >҉ 𝗠𝗜𝗧 𝗢𝗽𝗲𝗻𝗖𝗼𝘂𝗿𝘀𝗲𝗪𝗮𝗿𝗲: Explore MIT's free and open machine learning course materials. ↳ Resource: https://lnkd.in/dTzWz8FV >҉ 𝗘𝗱𝘆𝘀𝘀𝗲𝘆 𝗖𝗼𝗱𝗶𝗻𝗴 𝗕𝗼𝗼𝘁𝗰𝗮𝗺𝗽: A free, comprehensive guide to get started with ML. ↳ Resource: https://lnkd.in/gi4BNRES >҉ 𝗛𝗮𝗻𝗱𝘀-𝗢𝗻 𝗠𝗮𝗰𝗵𝗶𝗻𝗲 𝗟𝗲𝗮𝗿𝗻𝗶𝗻𝗴 𝘄𝗶𝘁𝗵 𝗦𝗰𝗶𝗸𝗶𝘁-𝗟𝗲𝗮𝗿𝗻 𝗮𝗻𝗱 𝗧𝗲𝗻𝘀𝗼𝗿𝗙𝗹𝗼𝘄: A free online version of this popular book is available. ↳ Resource: https://lnkd.in/gPsxsCSj ---- ҉. Bonus: 400+ Data Science Resources https://lnkd.in/gv9yvfdd Premium Data Science Interview Resources https://lnkd.in/gPrWQ8is 45+ Mathematics Books Every Data Scientist Needs https://lnkd.in/ghBXQfPc #Creditgoestotheowner.

1

A few words on how to optimize GPU applications for both training / serving use-cases and save $$$: Inefficiencies creep in across one of these 3 dimensions: 1) {Python, PyTorch, kernel launch} overhead 2) FLOPs 3) Memory bandwidth (host<->device, device<->device, HBM<->SRAM) [Re 3) I'll focus only on HBM<->SRAM here: for the host<->device you have to be smart with your memory pinning and for the device<->device have good HW interconnects and be smart about what data you communicate] How do we optimize these 3 bottlenecks? ### Overhead First of all what is it? It's your python interpreter, it's your PyTorch framework trying to figure out which GPU kernel to launch (depending on the dimensions of your matrix, its dtype, target GPU type (RTX4090, A100, H100...), etc.), and it's the actual overhead associated with the CPU/GPU-side kernel launch. To reduce the overhead there is a couple of techniques people use: a) Don't use Python - use C/C++ (obviously you're giving away flexibility and ease of prototyping here) b) jit tracing or CUDA graphs (lower level, but similar semantics) - which basically, as the word suggests, trace out all the decisions that your framework made in the first iteration (e.g. what GPU kernel to use) without having to rerun that same logic again. Just replay. ### FLOPs a) Make sure most of your FLOPs come from bf16/fp16 (or lower precision) matmul as that's where the highest FLOPs/sec throughput is, that's what NVIDIA is optimizing. b) Reduce FLOPs by being smarter with your maths (e.g. see FlashAttention 2 paper for some examples) or shard c) Buy better HW with higher FLOPs/sec! ### Mem BW (this is where most optimizations truly happen) I'll focus mostly on inference latency. Basically during every single decode step you have to: 1) Load your model weights 2) Load KV cache 3) Compute the fwd pass Let's see how the model bandwidth formula looks like: (num_params * num_bytes_per_param * tokens/sec) / (mem_bw) Where num_params is KV cache + model weights. How do we minimize this? 1) Reduce number of params! :) 1.1) Model distillation 1.2) Weight pruning (rarely used afaik) 2) Reduce number of bytes in your weights: 2.1) Quantize your weights to 8 bits or less 2.2) Don't use the big model use the draft model (speculative decoding) 3) Reduce your KV cache: 3.1) Use MQA/GQA (that's how Mistral/Llama, etc. save up 4x) 3.2) Quantize the cache activations 3.3) Make sure you're not wasting memory by using static KV cache logic (e.g. use PagedAttention) 4) Increase your mem bandwidth N times by using N GPUs (tensor parallelism). 5) Reduce HBM(DRAM)<->SRAM data movement (kernel fusion) 6) Optimize your attn mechanism: 6.1) Use sparse patterns (allows you to skip blocks in your CUDA kernels to reduce computation) 6.2) Flash Attention 2 7) Quantize activations I think those are the main techniques on top of my head, lmk in the comments if I missed any significant one!

451

27件のコメント

H2O.ai Just Released Its Latest Open-Weight Small Language Model, H2O-Danube3, Under Apache v2.0 Quick read: https://lnkd.in/gWPFYMWz Paper: https://lnkd.in/gqhyEi-4 Model Card: https://lnkd.in/gWSt8ShK Researchers at H2O.ai have introduced the H2O-Danube3 series to address these challenges. This series includes two main models: H2O-Danube3-4B and H2O-Danube3-500M. The H2O-Danube3-4B model is trained on 6 trillion tokens, while the H2O-Danube3-500M model is trained on 4 trillion tokens. Both models are pre-trained on extensive datasets and fine-tuned for various applications. These models aim to democratize language models’ use by making them accessible and efficient enough to run on modern smartphones, enabling a wider audience to leverage advanced NLP capabilities. The H2O-Danube3 models utilize a decoder-only architecture inspired by the Llama model. The training process involves three stages with varying data mixes to improve the quality of the models. In the first stage, the models are trained on 90.6% web data, which is gradually reduced to 81.7% in the second stage and 51.6% in the third stage. This approach helps refine the model by increasing the proportion of higher-quality data, including instruct data, Wikipedia, academic texts, and synthetic texts. The models are optimized for parameter and compute efficiency, allowing them to perform well even on devices with limited computational power. The H2O-Danube3-4B model has approximately 3.96 billion parameters, while the H2O-Danube3-500M model includes 500 million parameters. H2O.ai Pascal Pfeiffer Philipp Singer Yauhen Babakhin Gábor Fodor Nischay Dhankhar Sri Satish Ambati

46

1件のコメント

5 Best Github Repos to help you become a better ML / AI engineer: 1. Applied ML. All About Machine Learning in Production by Eugene Yan (26k stars) - https://lnkd.in/gwrkyJtf 2. Awesome Scalability. Build scalable ML systems by Binh Nguyen (56k stars) - https://lnkd.in/ghshNVKh 3. Creme de la Creme of Free AI courses by Piotr Skalski (4.7k stars) - https://lnkd.in/gy4cPJZ6 4. 60 Implementations of Deep Learning papers by labml.ai (60k stars) - https://lnkd.in/gnNE8Ptm 5. Made with ML. Learn to design, develop, and deploy production-grade ML applications by Goku Mohandas (36k stars) - https://lnkd.in/gjCQqjiC If you find these helpful... 👍 React ♻️ Share 💬 Comment So more people can learn. --- Happy building and experimenting!

3,092

47件のコメント

New Post: Deepdive: Custom Collate in DataLoader, Gemini cookbook, Build LLM from scratch Optimize your large language model (LLM) training with PyTorch by using a custom collate function in DataLoader. This technique dynamically pads text https://buff.ly/3RSNvXS

1

LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders 🚀

3

DiffUCO: A Diffusion Model Framework for Unsupervised Neural Combinatorial Optimization Quick read: https://lnkd.in/gEDWwdc8 Paper: https://lnkd.in/gbQJNvAN Code: https://lnkd.in/gsW599Yw

22

Small Language Models (not LLMs) are underrated: Ever since ChatGPT, we are hooked with the idea of large language models where it is this ultimate source of intelligence. However, after working with LLMs for a bit over a year now, I realized that while LLMs are great, it's small language model that is the hidden gem. Here is why I think they are underrated: - Fast: One of the biggest features is since they are smaller in nature, they are meant to be able to run faster than large language models, so you would get the response quicker. - Offline: As they are smaller in size, they will be able to run on a device without needing any internet connection, which also means you can use it even when you are offline! - Domain specific: Since it is smaller, it is also easier to train small models for specific tasks like on-device translation. One of the recent small language debuts has to be Apple Intelligence, where they have their small 3B (GPT-3 has 175B) parameter model to be the first model that the user will interact with, and gradually route it to their Apple server models and then to ChatGPT. Come and try it out and feel it for yourself at HuggingChat. They have a small model, Phi-3-mini-4k-instruct, that you can try now: https://buff.ly/4bwnwN1. We might see David vs Goliath stories again in this space!

5

Introduction to Daft ( ... vs Polars). Distributed DataFrames for Python (built with Rust) https://lnkd.in/g9vTT9zv #daft #polars #dataengineering #dataframes

81

14件のコメント

𝐭𝐱𝐭𝐚𝐢 𝐁𝐌25 𝐯𝐬 𝐋𝐚𝐧𝐠𝐂𝐡𝐚𝐢𝐧 𝐁𝐌25 LangChain and LlamaIndex both use Rank-BM25. txtai has a custom BM25 implementation.

6