Congrats Bo-Han Lu on the fantastic presentation of our work at LREC-COLING 2024! A great collaboration with Richard Tzong-Han Tsai. We translated Hokkien-Mandarin/English using LLaMA 2-7B by standardizing the Han writing script and evaluating with GPT-4. Enhancing Hokkien Dual Translation by Exploring and Standardizing of Four Writing Systems https://lnkd.in/gjWPw7fT Bo-Han Lu, Yi-Hsuan Lin, En-Shiun Annie Lee, Richard Tzong-Han Tsai Machine translation focuses mainly on high-resource languages (HRLs), while low-resource languages (LRLs) like Hokkien are relatively under-explored. The study aims to address this gap by developing a dual translation model between Hokkien and both Traditional Mandarin Chinese and English. We employ a pre-trained LLaMA 2-7B model specialized in Traditional Mandarin Chinese to leverage the orthographic similarities between Hokkien Han and Traditional Mandarin Chinese. Our comprehensive experiments involve translation tasks across various writing systems of Hokkien as well as between Hokkien and other HRLs. We find that the use of a limited monolingual corpus still further improves the model's Hokkien capabilities. We then utilize our translation model to standardize all Hokkien writing systems into Hokkien Han, resulting in further performance improvements. Additionally, we introduce an evaluation method incorporating back-translation and GPT-4 to ensure reliable translation quality assessment even for LRLs. The study contributes to narrowing the resource gap for Hokkien and empirically investigates the advantages and limitations of pre-training and fine-tuning based on LLaMA 2.
Annie Lee’s Post
More Relevant Posts
-
📃Scientific paper: Large language models effectively leverage document-level context for literary translation, but critical errors persist Abstract: Large language models (LLMs) are competitive with the state of the art on a wide range of sentence-level translation datasets. However, their ability to translate paragraphs and documents remains unexplored because evaluation in these settings is costly and difficult. We show through a rigorous human evaluation that asking the Gpt-3.5 (text-davinci-003) LLM to translate an entire literary paragraph (e.g., from a novel) at once results in higher-quality translations than standard sentence-by-sentence translation across 18 linguistically-diverse language pairs (e.g., translating into and out of Japanese, Polish, and English). Our evaluation, which took approximately 350 hours of effort for annotation and analysis, is conducted by hiring translators fluent in both the source and target language and asking them to provide both span-level error annotations as well as preference judgments of which system's translations are better. We observe that discourse-level LLM translators commit fewer mistranslations, grammar errors, and stylistic inconsistencies than sentence-level approaches. With that said, critical errors still abound, including occasional content omissions, and a human translator's intervention remains necessary to ensure that the author's voice remains intact. We publicly release our dataset and error annotations to spur future research on evaluation of document-level literary translation. ;Comment: preprint (31 pages) Discover the rest of the scientific article on es/iode ➡️https://etcse.fr/1Q0Hx
To view or add a comment, sign in
-
Translation strategies demystified If foreignisation is aimed at bringing the target text audience closer to the source text audience, there should be techniques a translator uses to preserve the target text culture. These techniques help the translator achieve the goal set at the very beginning. Some of these techniques are: 1. Literal translation Literal translation is the rendering of a text from one language to another “word-for-word” while conveying the sense of the original text to the target language 2. Borrowing Borrowing could be defined as the transfer of source language lexeme combinations into the target language without any semantic adaptation 3. Calque In linguistics, a calque or loan translation is a word or phrase borrowed from another language by literal, word-for-word or root-for-root translation Just like foreignisation has its techniques that work towards preserving some elements of the source text, domestication on the other hand, also has techniques that enable the translator bring the source text closer to the target text audience by “domesticating”. We have: 1. Modulation Modulation consists of using a phrase that is different in the source and target languages to convey the same idea 2. Adaptation This form of translation is common in the translation or rewriting of plays wherein the SL culture is swapped with the TL culture and the text is rewritten. 3. Reformulation This has to do with restructuring source text phrases, clauses, sentences and paragraphs so that they are different in the target text Having a grip on these techniques by translators shouldn't be undermined because in the case where a translator is asked to justify his translation, he'll need to elaborate using these techniques Happy world translation day in advance… #World translation day #September30 #Raisingawareness #Englishtranslator #Pidgintranslator
To view or add a comment, sign in
-
Google's latest research sheds light on an interesting aspect of translation: machines tend to be more "conservative" compared to humans. 🤖 What does that mean? Translation divergence occurs when translated content structurally differs from the original sentences. This can be attributed to linguistic variations or the unique preferences of individual translators. Such divergences are common in human translations, and they are also used to train machine translation systems. The results indicate that machine translation tends to be more "conservative" than human translation. It exhibits less morphosyntactic diversity, adheres to specific patterns more closely, and aligns more directly with the original sentence structure. Relying solely on machine translation can lead to sentences that may not sound as natural in the target language. 🔗 https://buff.ly/48QWMpo #Translation #GoogleResearch #Language #GlyphLanguageServices
Machine Translation Is More ‘Conservative’ Than Human Translation, Google Says
https://slator.com
To view or add a comment, sign in
-
"As maintained by the ITI, human translators “bring nuanced understanding, creative flair and an innate feel for their working languages, honed through years of experience and specialisation.” Adversely, as stated in the SFT’s manifesto, “algorithms do not have the capacity to understand, question and take account of context.” The consensus is clear — while AI can be a useful tool and increase productivity, it cannot replace the multidimensional perspectives and cultural sensitivity of human translators." #interpreter #translation #translationservices #translationagency #traduction #languages #languageservices https://lnkd.in/g4Q4fxH7
Translator and Interpreter Associations Warn Against Overly Relying on AI
https://multilingual.com
To view or add a comment, sign in
-
👁 As a language and culture enthusiast, I find Microsoft's daily screen pictures endlessly fascinating. These images do more than just showcase the beauty of our world; they also highlight the intricacies and joys of translation. ✨ Take today's picture, for instance. It's paired with a Chinese translation: ▶ "经过长途跋涉来到这里的游客,既能欣赏到美丽的景色,又能感受到耐人寻味的历史。" This translation, while technically accurate, exhibits a pronounced direct-from-English style, particularly in the use of linking words such as "经过,” “既能,” and “又能.” These, though common in English, are somewhat unnatural in Chinese. ❕ The structure of the translated sentence awkwardly separates the subject "游客" (tourists/travelers) from the rest of the sentence in Chinese. A direct back-translation into English might read: "The visitors who traveled a long way to come here can both appreciate the beautiful scenery and experience the history worth patiently savoring." A smoother English rendition could be: "After a long journey, visitors are greeted with breathtaking scenery and a rich, immersive history upon arrival." For a more natural, fluid Chinese translation from the current one, consider: ▶ "游客长途跋涉来到这里,欣赏美景之余还能感受耐人寻味的历史。” This phrasing more fittingly aligns with Chinese grammatical structures and sentence balance. The phrase “耐人寻味” particularly stands out. Generally found in expressions like “耐人寻味的历史故事” or “耐人寻味的历史古迹,” it conveys a history that is layered and invites thoughtful exploration. The phrase “耐心寻味的历史,” as used in this Microsoft translation, merits further discussion for its unique application 🧐 . An even more expressive and vivid trans-creation might be: "游客翻山越岭、长途跋涉来到这里,尽情饱览美丽自然风光,感受悠久人文历史。” 👀 This is, of course, my interpretation without seeing the original English text. Yet, it underscores a crucial aspect of language translation: the importance of cultural and linguistic sensitivity in preserving the original message's essence and beauty. What are your thoughts on this translation? Have you encountered similar nuances in other translated content? Feel free to share your insights! 🌐📝 #litranslators #translations #translationservice #chineselanguage #fridayvibes
To view or add a comment, sign in
-
M.A. Holder | Language Development Team Lead | CAT Tool Accessibility Advocate | Localization Section Head | Assistant Lecturer | Localization Business Coach | Mentor | English<>Arabic Translator | Reviewer | LQA|Tester
Dear my LinkedIn connections, I have created another survey for my PhD thesis to allow the respondents to choose their language pair to answer the questions accordingly. My thesis topic is “Automatic Evaluation VS. Human Verification of English-Arabic Machine Translation.” My thesis aims at exploring to what extent the results of the automatic machine translation (MT) evaluation metrics, like MT Quality Estimation (MTQE)/Prediction (MTQP), the reference-free metric (i.e. DeMT™ Estimate API, COMET QE etc.), and Quality Evaluation, the reference-based metric (i.e. BLEU, TER, etc.), are accurate in comparison with the human verification, like Error Typology (i.e. DQF-MQM metric, etc.) and Rating (i.e. a 5-likret scale, etc.), in the English-Arabic language pair. It would be highly appreciated if machine translation engine developers, CAT Tools/TMS/language solution developers, language service providers (that use MT engines and evaluate their raw output quality for enhancements and optimizations), translators/localizers, and researchers who are working on machine translation evaluation and any relevant fields in any language pair; answer this survey in order to get more insights for my dissertation purposes. If there is any question not applicable to your own expertise, please choose "Other" and type "N/A" in your comment. If there is a question from 11-16 not applicable to your own expertise, you can skip it. Please submit your answers to these questions within a week. NB: The final results of this survey will be shared with you. Sharing your personal information is optional and will be kept confidential. Your understanding and cooperation are highly appreciated. https://lnkd.in/dQ5GavUE
Automatic Evaluation VS. Human Verification of Machine Translation
docs.google.com
To view or add a comment, sign in
-
How effective are open-source language models for translation tasks: https://lnkd.in/dk2Y25wY
Open Source LLMs in the Context of Translation | LLM Explorer Blog
llm.extractum.io
To view or add a comment, sign in
-
Inglés - English - 英文 - الإنجليزية - английский The Revenge of the Translators In the era of Chat GPT and company, professional translators (most people when they think of translators in general think of Google Translator, Chat GPT, or DeepL) are left with the doubtfully honorable mission of showing, pointing with the finger, pointing on the board, underlining in red, crossing out, deleting, warning, ringing the bells, sounding the alarm signal, activating the emergency warning that Chat GPT, Google Translator, or DeepL have made a translation error. Professional translators, the people who dedicate ourselves to translation, have become "informants" of Artificial Intelligence. That's all we have left. But watch out: I think we're not the only ones. Please, write in the comments your profession and tell me if you think the same about it (I'm thinking of lawyers, tax advisors, dubbing actors, cameramen, teachers, writers, journalists, politicians... yes, politicians too). I'm looking forward to your comments. Oh, there will be versions of this post in 50 languages, many of which I don't master and besides, I don't feel like checking if the article will be well translated or not. Let's publish. Click.
To view or add a comment, sign in
-
Lead Data Scientist | AI Leader | General Manager at Reliance Jio | LLM & GenAI Pioneer | AI Evangelist
𝐄𝐯𝐚𝐥𝐮𝐚𝐭𝐢𝐧𝐠 𝐋𝐚𝐫𝐠𝐞 𝐋𝐚𝐧𝐠𝐮𝐚𝐠𝐞 𝐌𝐨𝐝𝐞𝐥𝐬 𝐢𝐧 𝐍𝐨𝐧-𝐄𝐧𝐠𝐥𝐢𝐬𝐡 𝐋𝐚𝐧𝐠𝐮𝐚𝐠𝐞𝐬: 𝐈𝐧𝐬𝐢𝐠𝐡𝐭𝐬 𝐟𝐫𝐨𝐦 𝐭𝐡𝐞 𝐌𝐌𝐋𝐔 𝐁𝐞𝐧𝐜𝐡𝐦𝐚𝐫𝐤 📘 𝐖𝐡𝐚𝐭 𝐢𝐬 𝐭𝐡𝐢𝐬 𝐩𝐚𝐩𝐞𝐫 𝐚𝐛𝐨𝐮𝐭? Specifically, it investigates how automatic translations impact the performance of LLMs using the Massive Multitask Language Understanding (MMLU) benchmark translated into Spanish. 🤖 First key aspect The study highlights the dependence of LLM evaluation results on the quality of automatic translations, revealing significant discrepancies in performance due to translation errors. 📊 Second key aspect It identifies and analyzes specific test items that produce different answers when translated, showing how translation mistakes can skew evaluation results. 🧠 Third key aspect The research makes a case for not just translating benchmarks but also adapting them to the target language, ensuring more accurate assessments of LLM capabilities. 🚀 𝐖𝐡𝐲 𝐢𝐬 𝐭𝐡𝐢𝐬 𝐚 𝐛𝐫𝐞𝐚𝐤𝐭𝐡𝐫𝐨𝐮𝐠𝐡? ⏱ First reason This study underscores the importance of accurate translations in evaluating LLMs, a critical factor as these models are increasingly adopted globally. 📈 Second reason By revealing how translation errors affect LLM performance, the research highlights the need for more robust and language-specific evaluation methods. 🌍 Third reason The findings emphasize the necessity of improving non-English benchmarks to ensure fair and accurate evaluations of LLMs in diverse linguistic contexts. 🔬 𝐊𝐞𝐲 𝐅𝐢𝐧𝐝𝐢𝐧𝐠𝐬 🔧 First finding A significant fraction of failing test items in Spanish can be attributed to mistakes in the automatic translation of the benchmark. 🧩 Second finding The study found that discrepancies in LLM performance between English and Spanish often stem from translation errors rather than the model's inherent capabilities. 🛠 Third finding The research advocates for revising and adapting benchmarks in non-English languages to enhance the reliability of LLM evaluations. 🔍 𝐈𝐦𝐩𝐥𝐢𝐜𝐚𝐭𝐢𝐨𝐧𝐬 𝐟𝐨𝐫 𝐭𝐡𝐞 𝐅𝐮𝐭𝐮𝐫𝐞 🌐 First implication Improving the quality of translations in benchmarks can lead to more accurate assessments of LLM performance across different languages. 🚗 Second implication Developing language-specific benchmarks can help in better understanding and improving LLM capabilities in non-English contexts. 📈 Third implication The study highlights the need for ongoing refinement of evaluation methodologies to keep pace with the global adoption of LLMs. 💡 𝐓𝐚𝐤𝐞𝐚𝐰𝐚𝐲𝐬 🎯 First takeaway Accurate translation is crucial for fair evaluation of LLMs in different languages. 🔄 Second takeaway Identifying and correcting translation errors can significantly improve the reliability of LLM performance assessments. s 🌍📊🧠 #AI #MachineLearning #LanguageModels #GlobalAI
To view or add a comment, sign in
CAIO at SpassMed
2moI found that Llama 3 is amazing and performed better than expected. Cheers and look forward to hearing more from you Annie Lee 🙏