Bin Yu, PhD’s Post

Google's new flagship AI model, "Gemini," is set to be a direct competitor to GPT-4 and boasts computing power 5 times that of GPT-4. Trained on Google's TPUv5 chips, it's capable of simultaneous operations with a massive 16,384 chips. The dataset used for training this model is around 65 trillion tokens, and it's multi-modal, accepting text, video, audio, and pictures. Moreover, it can produce both text and images. The training also included content from YouTube and used advanced training techniques similar to "AlphaGo-type" methods. Google plans to release the Gemini model to the public in December 2023.

I think that the main problem for the current AI industry is that the most of AI use cases that I see now are here to tackle cost centers but not to increase earnings. LLMs are good for automating routine and time-consuming jobs but we have not received proof of increased earnings from the market. Some B2B-SaaS companies may bring some value but it won't be trillions that are predicted by many. BTW, I have a paid ChatGPT account and use it daily. I do understand that it can cover a lot of daily tasks but there is no proper API to allow my bot to book a restaurant reservation or a barber time slot. So many use cases for B2C are out of the market in fact.

What do you think is essential about understanding the business nature of LLMs? 1. More parameters (say millions vs. billions) don't mean one model is better. 2. At the end of the day, you need to think about the cost of the request and balance your user economy. 3. Some tasks are cheaper and easy to pass with less advanced LLMs. 4. Replication of A.I. products, on average, takes six months. Keep this in mind. So, play with a gamified or playful environment first and think about how to launch fast. 5. A.I. is an endless fuel for dopamine. You can use it carefully. Credits go to Kristina Kashtanova for sharing news about "Gemini." Hopes once a day, LinkedIn will hear and finally invent some reputational credits (on web3 🤔 😏) so I can share them literally. #llms #generativeai

5 times more computation power. will Gemini overtake Chatgpt in Accuracy and precision.

This AI Paper from Apple Introduces a Weakly-Supervised Pre-Training Method for Vision Models Using Publicly Available Web-Scale Image-Text Data https://lnkd.in/gfJHWXRA In recent times, contrastive learning has become a potent strategy for training models to learn efficient visual representations by aligning image and text embeddings. However, one of the difficulties with contrastive learning is the computation needed for pairwise similarity between image and text pairs, especially when working with large-scale datasets. In recent research, a team of researchers has presented a new method for pre-training vision models with web-scale image-text data in a weakly supervised manner. Called CatLIP (Categorical Loss for Image-text Pre-training), this approach solves the trade-off between efficiency and scalability on web-scale image-text datasets with weak labeling. By extracting labels from text captions, CatLIP views image-text pre-training as a...

Here is a new AI competitor of GPT from Google called "Gemini" ✨. Gemini is set to be a direct competitor to GPT-4 and boasts the computing power 5 times that of GPT-4 💥. Trained on Google's TPUv5 chips, it's capable of simultaneous operations with a massive 16,384 chips. The dataset used for training this model is around 65 trillion tokens, and it's multi-modal, accepting text, video, audio, and pictures. Moreover, it can produce both text 📄 and images 🖼️. The training also included content from YouTube and used advanced training techniques similar to "AlphaGo-type" methods. Google plans to release the Gemini model to the public in December 2023 🗓️.

Once I reached out to a well respected professor asking him about how does one do neural network design. He said that it was a "black art" and very little was available as a guideline to start except for going through papers and attempting to recreate it and hope you get it right Same continues to this day with LLMs, while established players like OpenAI , Anthropic release papers, its mostly to boast on how they beat the competition than having any meaningful information on how they achieved SOTA and what were their design behind it. In midst of this last week, Apple came out with the MM1 paper detailing their first model. Though it might not be path breaking, it was a real delight to go through the paper as they listed out some key lessons while training to do Large Multi Modal models, giving inside view on what it takes to train a multi modal models. Some of their key insights include: 1. Encoder Lesson: Image resolution has the highest impact, followed by model size and training data composition. 2. Vision-Language (VL) Connector Lesson: Number of visual tokens and image resolution matters most, while the type of VL connector has little effect. 3. Data Lesson 1: Interleaved data is instrumental for few-shot and text only performance, while captioning data lifts zero-shot performance. 4. Data Lesson 2: Text-only data helps with few-shot and text-only performance. 5. Data Lesson 3: Careful mixture of image and text data can yield optimal multimodal performance and retain strong text performance. 6. Data Lesson 4: Synthetic (caption) data helps with few-shot learning. Here is the link - https://lnkd.in/gBnW7pxb #lmm #llm #llmops #mle #ml #ds #genai

One way to source data nowadays is to use audio-to-text models to transcribe audio. The transcribed text is then fed into hungry AI models for training. Interestingly, despite these advancements, there’s still a lot of discussion about data storage issues today. The real question is, do we really have enough data for the computing power we currently have?

Using Google's AI Search: the possibility of learning quantum computing on your own: DIY Learning? while it is possible, it can be challenging without a structured degree program. It is suggested that it may take 100 to 200 hours of self-learning to grasp the basics and progress to an intermediate or advanced level. #google #googlesearch #technology #googleai #vizard

The global race to General Artificial Intelligence (#AGI)....💡In a nutshell, the simple rule: more compute equals more powerful AI models 📈 - will be valid for some time. According to Microsoft's CTO Kevin Scott, there is no indication of reaching diminishing marginal returns regarding the power of AI models as compute power increases. ✨This insight is particularly noteworthy given #Microsoft's role in training some of the largest #AI models. That man knows! 😏 Means, the more money you throw at this, the better your chips and higher your energy bill as a company or government, the better your AI! PS: A way to compete here is with agentic workflows, combining cheaper more feasible models to work toghether. Tests have shown that a few GPT3s can outperform a GPT4. A difference of 100s of Millions of dollars that went into training a GPT4 vs a GPT3.