Nathaniel Cook

The right model combined with domain specific fine tuning (Marqtune) equals massive improvements to relevance.

10

2 Comments

Generative AI and LLMs trained specifically for coding have allowed me to create with code again, and I'm so grateful. ❤️ 🤖 You see, my programming skills have been rusty for over a decade since I left an IC software engineering role. 😕 I've been using ChatGPT with Code Analyzer for data analysis, writing shell scripts, and simple Python with fewer iterations. Today, I wanted to tackle a bigger (for me) project and try out Github CoPilot. I prompted CoPilot to write code in the same way I would write code—in small pieces, focusing on adding complexity iteratively and validating each step. And y'all - coding was actually delightful again. So get back out there all you rusty programmers! ps - I'm building a script that will query ArXiv for recently published LLM-related research papers and posting a summary and a link into my GenAI Slack community. It's been a fun project to play with the OpenAI API, get experience with Google Cloud, test CoPilot's quality, and optimize summarization prompts. And once done, it will make it easier to stay on top of research 😍

47

4 Comments

I was chatting about translation with Andrew Ng at an event several weeks ago. It's fun to see him post a workflow with the seed of the AI Review concept that's live in LILT today. === This is along the path to what many in the industry are doing. Two comments: 1. Replace step 2 with a formalized reflection approach such as AutoMQM. 2. BLEU isn't discriminative at the accuracy levels modern MT systems achieve. Use a trained metrics like BLEURT or COMET to see real differences between systems. 20 years ago we called this idea automatic post-editing, where you'd train a second string transducer to rewrite initial MT output. However, those systems were very difficult to train and cascade. Using LLMs in cascades is significantly more effective.

42

Evaluations of LLM applications is a big buzzword right now... but when do you really need new tools? Our latest blog post from Wenzhe Xue cuts through the noise to show with examples how built-in libraries such as pytest are often enough to get started. As complexity grows along the following dimensions, we're here to help! Metric based ➡ Human review Off the shelf eval or LLM as a judge ➡ Custom eval models trained on your data Offline ➡ Online, realtime https://lnkd.in/gGb3Sjah

22

Ok let's talk about Devin. The AI software engineer that originally got a lot of hype. 👇 For every type of job out there, there is a funded startup trying to make a ‘Devin’ for it. But as a lot of us saw, when Devin was really put to the test, the AI spent hours on handling a task that could have been accomplished by simply looking at the read-me and spitting out a line of code. 🫣 So, there are a lot of claims on the market that might not meet expectations, but here’s my take: it won’t be this way forever. Whether or not Devin specifically gets better, who’s to say? But AI will get stronger at planning and executing work. I see a clear future where tasks that are purely transactional are the kind of work that will be given fully to AI. Because when you do, you lower cost and overall friction for that given function—making things easier for everyone. So maybe it’s time to start thinking about what jobs we actually want AI to take from us… #ai #infosec #artificialintelligence #presales

27

8 Comments

2023 was a breakout year for investment in generative AI startups, with equity funding of almost $22B across 426 deals. The 5 largest rounds all went to companies focused on core generative AI infrastructure: * OpenAI, AI poster child and maker of ChatGPT ($10B corporate minority) * Inflection AI, which focuses on human-computer interfaces ($1.3B Series B) * Anthropic, an AI model developer and research outfit ($1.8B across 2 corporate minorities) * Databricks, a data integration and analytics platform ($504M Series I) * Aleph Alpha, Germany-based LLM developer ($500M Series B) When coupled with the pace of change and the compression of time and cost, makes anyone wonder, given the apparent speed of the commoditization of these LLM engines, what will the ROIC on funds deployed actually look like over time?

1

This is a very interesting article on sparse vectors, which can help overcome limitations of vector search related to domain specific keywords. And of course, the code in the example is clear and clean! Sparse vectors have a few values and many zeros. Dense vectors have limited or no zeroes. But really, what we're talking about semantic search on terms where the context is not present or harder to infer. The author presents this as specialized keyword search such as medical terms or other domain specific terminology. "These scores reflect the significance of each term in the context of the document or query, guiding the model to allocate more weight to terms that are likely to be more meaningful for retrieval purposes." Highly recommend reading. Nice work Paul Iusztin. And a nod to Qdrant and their vector search technology that supports sparse vectors. https://lnkd.in/gcBWfTg8

3

Especially, in software development, this is a very underrated skill. Everything starts from product docs and end with docs (feature documentations). Most people just find it convenient to speak a few lines on call, but fail to lay down the requirement/ or the impact areas in detail, leading to poor planning, poor designs, and increased timelines. Any developer who can write clearly, naturally has a clear thinking. "Writing gives poor thinking nowhere to hide. When your invisible thoughts are made visible, you are forced to confront them as they are, not as you wish them to be. You can’t simply take a few minutes here and there, get the gist of the problem, and expect to have clear thinking and unique insights. Good thinking, like good writing, demands patience." - taken from Farnam Street.

50

Transformer Reel and Real... In the realm of deep learning, the Transformer model, inspired by the movie franchise, doesn't quite metamorphose like its cinematic counterpart. However, it does utilize attention mechanisms to understand data and relationships better. Yet, these mechanisms can be slow due to their quadratic complexity and are limited by input data length. To address this challenge, researchers are devising ways to accelerate attention mechanisms without sacrificing effectiveness. Techniques like making attention matrices sparser or using simpler approximations are being explored in models such as Reformer, Routing Transformer, and Linformer. However, striking the right balance between speed and accuracy remains elusive. Enter the Data Dependent Convolution Network, a novel innovation akin to the shape-shifting abilities of movie Transformers. This network adjusts itself based on input data, allowing the model to efficiently handle long sequences without complexity overload. It can process up to 131,000 elements while maintaining speed. TAt the heart of Orchid lies its innovative data-dependent convolution layer, which dynamically adjusts its kernel based on input data through a specialized conditioning neural network. We've designed two straightforward conditioning networks that ensure shift equivariance in the adaptive convolution operation. This dynamic convolution kernel, combined with gating operations, provides Orchid with remarkable expressivity while maintaining efficiency and quasi-linear scalability, especially for long sequences. We've conducted thorough evaluations of Orchid across various domains, including language modeling and image classification, to demonstrate its performance and versatility. Our experiments show that Orchid not only surpasses traditional attention-based architectures like BERT and Vision Transformers with smaller model sizes but also extends the feasible sequence length beyond the limitations of dense attention layers. This accomplishment marks a significant advancement toward developing more efficient and scalable deep learning models for sequence modeling. In practical terms, Orchid's breakthroughs empower researchers and developers to construct more efficient and potent models for analyzing vast datasets. Whether deciphering lengthy text passages or decoding intricate genetic sequences, Orchid presents a promising solution for accelerating deep learning tasks while upholding accuracy. https://lnkd.in/dQHBARrS #ai #aiinnovation #llm #generativeai #deeplearning

1

Common Culprits Behind CrashLoopBackOff There are a few usual suspects that can trigger endless crashes and restarts: 👻Resource constraints: If a pod doesn't have enough memory allocated, it can run out and crash. Carefully configure pod resource requests and limits. 👻Image issues: Using an image without proper permissions or pulling the wrong image can lead to crashes. 👻Configuration errors: Simple mistakes like typos in the pod spec can prevent containers from starting correctly. 👻External service problems: If your pod relies on an external service or database that's unavailable, it's going to have issues. 👻Uncaught exceptions: Bugs or errors in application code that aren't handled properly will cause crashes. 👻Liveness probe misconfiguration: If liveness probes are configured poorly, they can cause unnecessary restarts. https://hubs.la/Q02vTSxF0

14

1 Comment

I can't believe how much I use Chat GPT for random conversations, technical and otherwise. From debugging i2c on a Pi Pico W to calculating statistical distributions, to nutrition, and everything in-between. I'm a perpetual AI skeptic, but every time I dive into one of these new AI tools, I'm blown away by how quickly they are evolving and how useful they are. Yesterday, I saved a new file in VS-Code and when I had to type the name, I thought, "Why wasn't that named for me?" In a few months, I bet it will be. If you're not doing this, do yourself a favor and pin Chat GPT in your browser. Try it for a week and see. The more you use the tools the more you will start to see opportunity and progress.

7

4 Comments

👠 Toto, we’re not in Kansas anymore — quantum is reducing computational cost of weather predictions. Coming to you from tornado valley, we like the sound of that. 🌪️ Today's dispatches from quantum: ⚆ A new method using tensor network algorithms reduces circuit depth and improves efficiency in adiabatic quantum computing ⚆ Chiba University debuts a quantum algorithm to cut costs in weather prediction data assimilation using quantum annealing ⚆ Yuval Boger of QuEra Computing Inc. makes the case for on-prem quantum setups with superior control, security, and reduced latency; recommends a hybrid cloud approach as the tech matures. ⚆ Frederic Jacquet stresses the importance of post-quantum cryptography and NIST’s initiatives in tackling its implementation challenges ⚆ Fault-tolerant quantum computers can simulate NMR spectra, reducing computational complexity and costs ⚆ Scott Buchholz of Deloitte breaks down quantum computing, along with its potential risks and uses ⚆ Dr. Paul Terry, Photonic Inc.'s CEO, discusses the potential of quantum computing and the evolution towards scalable, fault-tolerant systems ⚆ Q-CTRL Fire Opal error suppression enhances the accuracy of a conditional quantum GAN on IBM hardware by 30-40% ⚆ Reinforcement learning optimizes adaptive estimation tasks for quantum sensors, with techniques implemented in the qsensoropt library ...and more. 📰 Stay quantum-savvy. The Daily Qubit offers daily insights into the rapidly evolving world of quantum computing. Join leaders in academia, founders, and industry experts in staying informed. Sign up at www.thedailyqubit.com and never miss a qubit.  #quantumcomputing #quantumphysics #innovation

14

5 Comments

Check out our latest post on Clappy.ai, "Leverage Layout-Aware RAG for More Accurate Retrieval of Knowledge from Documents." Discover how to harness the power of Retrieval-Augmented Generation (RAG) for documents in PDF, Doc, and Markdown formats. Dive into the specifics of implementation and explore our reference repository to get started with advanced RAG for your knowledge base. Don't miss out on this essential read! Continue reading here for all the details: The Power of Hierarchical Indexing with RAG. (https://lnkd.in/gbiVHfcH)

2

Anthropic just released Claude Sonnet 3.5, a new frontier model that is faster *and* smarter! We move quickly at Sourcegraph, so we're excited to announce we've already rolled it out for users to try: * On the web, check out https://s0.dev (and you can compare head to head with other models, using Sourcegraph-provided code context) * Install the Cody VS Code extension (available tomorrow in the nightly build): https://lnkd.in/gq8DWkTe * Install Cody for JetBrains IDEs (available in the next nightly build): https://lnkd.in/gyJqVzFA

34

2 Comments

The OP is wrong about the count, I think they are counting all models in HF as LLMs. Because there are only 700K+ total models in HF. But it sure begs the question why ~100K text generation models (if not LLMs). For perspective sentence-transformer library based models are only ~7K Related reads ⭕️ Elon’s idiot index for LLMs: https://lnkd.in/eWsbM45J ⭕️ How open is your LLM: https://lnkd.in/eY6JE7Ae

15

3 Comments

Got an email "GCP's Apache Kafka service is now live" at noon. After dinner, spent half an hour confirming it works well with Timeplus. I don't think you can make the bootstrap server publicly accessible. Has to setup a bastion host or run your app within the same project/VPC. Setting up the SASL username/pwd was not easy. Follow the guide carefully https://lnkd.in/gcKEVh3g and you may need to disable the iam.disableServiceAccountKeyCreation Anyway, if your workload are in GCP already, having such Kafka service in GCP is nice way to burn your cloud credits. Yet another Kafka vendor, great.

33

2 Comments