Annie Lee’s Post

Today we are releasing T-Projection, a tool that can automatically generate labeled data for +100 languages!! For most languages on Earth, very few or no manually labeled data is available. This significantly hinders the development of NLP systems for languages other than English 📉 T-Projection can automatically project labeled sequences from English (or any other high-resource language) into more than 100 target languages with near human-level performance. 🤯 How does T-Projection works? For many tasks, such as text classification, you can just translate the text. However, sequence labeling tasks, which involve span level annotations, require an additional step called annotation projection.The majority of previous published work on this line of research explores the application of word-alignments. But word alignments often produce partial, incorrect or missing annotation projections. T-Projection, decomposes the label projection task into two subtasks: (i) Candidate generation: in which a set of projection candidates using a multilingual T5 model is generated and, (ii) Candidate selection: in which the candidates are ranked based on translation probabilities. The result is that we can automatically generate labeled data for over 100 languages, reducing the scarcity of labeled data for most languages other than English. T-Projection is built on top of 🤗Transformers and 🤗Accelerate from Hugging Face so using it is straightforward! Paper 📖 : https://lnkd.in/dnMQZz5j Code 💻 : https://lnkd.in/dJxKDHW4 W/ Rodrigo Agerri German Rigau at HiTZ Zentroa

We're really excited about this work that explores how to make complex data easier to understand for all of us. At Aiforsite, we're using Large Language Models to turn complicated data language into simple English. This study explains how we use this technology to create a range of questions that help us dig deeper into data and make it useful for you. Plus, we're making sure this solution works for everyone, no matter their style of asking questions. So, what's in it for you? You'll be able to access and understand complex data like never before, no tech expertise required. It's like having a translator for a foreign language you didn't even know existed! We're working hard on fine-tuning this process and can't wait to share more with you soon. Stay tuned to see how we're making data user-friendly for everyone! See more of the technical information in the article below from Saad Azhar. #aiadoption #artificialintelligence #aiadvancements

🤯Exciting stuff alert! Saad has been writing on Medium about how Aiforsite is using AI to make complex data super easy for everyone to understand. This is game-changing. It's like having a personal translator for deep data analysis. Can't wait to see how this simplifies things for everyone. For the technical article, follow the link below. Stay tuned! #DataMadeEasy #AI #ArtificialIntelligence #AIAdoption

🔍 Transforming Data Retrieval at Aiforsite: Harnessing the Power of AI At AiForSite, we're on an exciting journey to revolutionize data retrieval. We're leveraging the capabilities of Large Language Models, particularly GPT-4, to innovate an English-to-Cypher translation mechanism, aiming for greater efficiency and accessibility in data extraction. Here's a snapshot of our approach: 1️⃣ We're developing a testing and improvement pipeline for translating natural language to Cypher query language. 2️⃣ We're employing robust tools such as Neo4j, GPT-4 API, and a custom Python pipeline. 3️⃣ We're using GPT-4 to generate queries based on complexity levels. 4️⃣ We're working to neutralize query style bias by generating diverse query versions, reflecting various personality models. With this initiative, we underline the transformative potential of AI and Knowledge Graphs in data retrieval and processing. By leveraging models like GPT-4, we strive to create systems that are more robust, accurate, and accessible. Stay connected for in-depth insights as we further delve into the components of this initiative in our upcoming posts. Join us in revolutionizing data retrieval! Share your thoughts and insights. https://lnkd.in/dzFUMEtB #AiForSite #AI #NLP #Neo4j #GPT4 #DataScience #DataRetrieval #Cypher #LargeLanguageModels #DigitalTransformation #KnowledgeGraphs

🚀📚**MarIA**, a family of **Spanish language models**, represents a significant leap in natural language understanding (NLU) for Spanish. Let's explore these models: 1. **RoBERTa-base and RoBERTa-large**: - These models are arguably the **largest and most proficient** language models in Spanish. - They were pretrained using a massive corpus of **570GB** of clean and deduplicated texts, containing **135 billion words** extracted from the **Spanish Web Archive** crawled by the **National Library of Spain** between **2009 and 2019**[^10^]. - You can use them for various NLP tasks, such as text generation, sentiment analysis, and question answering. 2. **GPT-2-base and GPT-2-large**: - These transformer-based models are also part of MarIA. - Like RoBERTa, they were pretrained using the same massive Spanish corpus. - They offer powerful capabilities for text generation and understanding[^10^]. 3. **Longformer-base-4096-bne-es**: - This model is based on the **Longformer** architecture and is designed for the **Catalan language**. - It allows processing larger contexts (up to **4096 tokens**) as input without additional aggregation strategies. - Pretraining involved both short and long documents in Catalan[^10^]. 4. **Aguila-7B**: - A **7B-parameter LLM** trained on a mixture of **Spanish, Catalan, and English data**, totaling **26 billion tokens**. - It builds upon the state-of-the-art **Falcon-7b** English language model¹. These models empower developers, researchers, and industry professionals to create innovative applications in Spanish. Whether you're building chatbots, analyzing sentiment, or exploring creative text generation, MarIA has you covered! 🌟🇪🇸 For example, here's how you can use RoBERTa-base for masked language modeling: ```python from transformers import AutoModelForMaskedLM, AutoTokenizer, FillMaskPipeline from pprint import pprint tokenizer_hf = AutoTokenizer.from_pretrained('PlanTL-GOB-ES/roberta-base-bne') model = AutoModelForMaskedLM.from_pretrained('PlanTL-GOB-ES/roberta-base-bne') model.eval() pipeline = FillMaskPipeline(model, tokenizer_hf) text = "¡Hola <mask>!" res_hf = pipeline(text) pprint([r['token_str'] for r in res_hf]) ``` Feel free to explore these models and unleash their potential! (1) [2107.07253] MarIA: Spanish Language Models - arXiv.org. https://lnkd.in/du4bc7va. (2) Spanish Language Models - GitHub. https://lnkd.in/duSNN7Nv. (3) Introduction to Large Language Models - Español | Coursera. https://lnkd.in/dmusCedd. (4) Large Language Models (LLMs): Qué es, Usos y Tipos de LLMs. https://lnkd.in/dtsSZiHY. (5) undefined. https://lnkd.in/d4rvFTtg. (6) undefined. https://lnkd.in/dD4a98t9. (7) undefined. https://lnkd.in/dZyrwd9v.

English is not the only language in NLP. Day 6 of #100DaysofAI - Preprocessing non-English and multilingual texts. Most Python packages and models were initially built predominantly for English. However, other languages present their own challenges with processing the texts and hence they need techniques of their own. For example, languages such as Chinese do not use spaces as their separators. Japanese has a writing system with multiple scripts. In German, there are compound words without spaces and umlauts. Arabic is written from right to left unlike the others. Many languages also have diacritics. They are small marks or symbols added to letters to alter their pronunciation and(or) meaning. For example, the word "schon" in German means already and "schön" means nice. These unique characters can either be normalized, removed, or replaced based on the context and difficulty level. For example, the ö in schön can be replaced with oe to make it a new word schoen. This can be done leveraging regular expressions and the multi-lingual support of the famous packages NLTK, Spacy, TextBlob and Gensim. Additionally, there are some other libraries specific to certain languages. 1️⃣ Hindi and Indian Languages - https://lnkd.in/eUnXwsrW 2️⃣ Chinese - https://lnkd.in/eUVV2bdG 3️⃣ Japanese - https://lnkd.in/eB4xPbqn 4️⃣ Korean - https://lnkd.in/eeDqWKP9 Now, let us consider a single dataset that has texts of multiple languages like CommonCrawl. How do you preprocess such a dataset? You can use the package Polyglot https://lnkd.in/eXRgf9cq to identify the language and apply an appropriate technique or package to preprocess the text. Do you work with any non-English languages? What are the challenges you have faced? Let me know in the comments below. On Day 7 of #100DaysofAI, we will discuss tokenization methods used in Transformer based LLMs along with their applications, pros and cons. #ai #tech #generativeai #datascience #newyear #machinelearning #learning #definitions #nlp #texts #naturallanguageprocessing

📢 Exciting news! Introducing ZS-A2T, a zero-shot framework that translates transformer attention in VQA models to natural language. This approach allows us to gain human-understandable insights about the model without any training. By leveraging a pre-trained language model, we can identify tokens that describe the regions in the input image that the VQA model attended to. Our framework achieves state-of-the-art performance in the zero-shot setting on textual explanation datasets for VQA. Check out the code here: #AI #MachineLearning #DeepLearning #ComputerVision #NLP #DataScience #BigData #ZeroShotTranslation #VQA #Transformers #LanguageModeling #StateOfTheArt

In computing, stop words are words which are filtered out before or after processing of natural language data (text). Stop word

RakutenAI-7B: A Suite of Japanese-Oriented Large Language Models that Achieve the Great Performance on the Japanese Language Model Quick read: https://lnkd.in/g2yXGc7n Rakuten Group, Inc. researchers have introduced RakutenAI-7B, a suite of Japanese-oriented LLMs. The suite includes foundation models alongside instruction- and chat-tuned models, released under the Apache 2.0 license. These models are designed to accommodate the Japanese language better, incorporating extended vocabularies and improved tokenization techniques for enhanced performance. RakutenAI-7B‘s methodology encompasses extending the vocabulary of its tokenizer to 48,000 tokens, significantly improving the processing of Japanese text by enhancing the character-per-token rate. This strategic expansion was essential for efficiently managing the complexities of the Japanese script. In parallel, the model benefitted from rigorous data filtering techniques aimed at refining the quality of training datasets. These datasets, purged of personally identifiable information and low-quality inputs, were approximately 175 billion tokens in size, ensuring the model’s outputs are coherent and relevant. This comprehensive approach, utilizing advanced tokenization and meticulous data curation, underscored the model’s preparation for high-caliber performance across various NLP tasks. Paper: https://lnkd.in/gJa78tNd #artificialintelligence #datascience

Just read an interesting paper (published last week) about text embeddings!  The researchers have proposed a novel approach using Large Language Models (LLMs) to tackle a major hurdle in fine-tuning LLMs for specific NLP tasks – the scarcity of labeled datasets in various languages. Their solution? Generate synthetic datasets through LLMs! By prompting these models to create labeled data for a particular task, they then use this generated data to fine-tune LLMs for NLP tasks requiring text embeddings, like Information Retrieval (IR). They've created synthetic datasets in nearly 100 (93 to be exact) languages, covering hundreds of thousands of embedding tasks. And guess what? Their approach shows promising results on the MTEB benchmark. But, there's a catch. The study doesn't fully address the issue of 'hallucination' in these datasets. It's a crucial point – how reliable can these synthetic datasets be if they're potentially contaminated with hallucinated data? #NLP #MachineLearning #DataScience #Innovation #LanguageModels #Research