Nexusflow

🚀Calling all developers of copilots and AI agents! Introducing 🐦⬛NexusRaven V2, a 13B function calling LLM surpassing GPT-4 in real-world zero-shot tool use. ✨ Highlights of 🐦⬛NexusRaven V2: 💪Superior Performance:  NexusRaven V2 surpasses GPT-4 up to 7% on complex nested and parallel zero-shot function calling scenarios. 🗣️Commercially Permissive: NexusRaven V2 was instruction-tuned from AI at Meta's CodeLlama-13B, without using proprietary LLM generated data. 📚Realistically Benchmarked: NexusRaven V2 is evaluated on hundreds of high quality examples, shown in our Hugging Face leaderboard. 💻 Dive deeper into our tech: Blog: https://t.ly/drUh_ Huggingface: https://t.ly/LRJrq Function Calling Leaderboard: https://t.ly/XCwAh Discord: https://lnkd.in/ggHjVCZN nexusraven package: https://t.ly/e7xzb GitHub: https://t.ly/81zVT Experience 🐦⬛NexusRaven V2’s potential in our travel and restaurant advisor demo!  https://t.ly/k_YzT Try a Colab notebook to interact with NexusRaven V2: https://t.ly/Ge1Pj . The model is also available on Together AI. #NexusRavenV2 #OpenSource #GenAI #LLM #AI #functioncalling #copilot #leaderboard #AIAgents

Exciting News: Athene-70B Claims #8 Position on Chatbot Arena! In numerous business scenarios, user experience and customer satisfaction are very important. Chatbot Arena has established itself as the gold standard for assessing user preferences for LLMs. As a model fine-tuned from Llama-3-70B, Athene-70B highlights Nexusflow's expertise in customizing model behavior for chat applications. Following AI at Meta's release of Llama-3.1 this week, we are thrilled to continue delivering strong model suites built upon open foundation models.

Big kudos to AI at Meta for advancing open models & tool use! Honored that Nexus FC benchmark powered the evaluation of Llama 3.1. Explore the benchmark on GitHub: https://lnkd.in/gUD2Ka9S. Stay tuned for more breakthroughs from us soon! #ToolUse #AIInnovation #MetaAI

🚀 Introducing Athene-70B: Redefining Post-Training for Open Models! We’re thrilled to release Athene-Llama3-70B, an open-weight chat model fine-tuned from Meta's Llama-3-70B. With an impressive Arena-Hard score of 77.8%, Athene-70B is approaching top proprietary models like GPT-4o and Claude-3.5-Sonnet. Experience it now in public testing on Chatbot Arena! Blog: https://lnkd.in/g-uHVwEV  HuggingFace Model: https://lnkd.in/gwuB5QiP  Discord: https://lnkd.in/gJwN2X7w 🌟 #AI #MachineLearning #LLM #Nexusflow #Athene70B #ChatbotArena #Llama3

🚀 Exciting News! Check out our brand new short course on function calling, the foundation for AI agents, created by our CEO Jiantao Jiao and founding engineer Venkat Srinivasan, in collaboration with DeepLearning.AI and the legendary Andrew Ng! 🌟

🔍 What Starling-LM-7B-beta's excellent performance tells us about benchmarks I compared the performance of Nexusflow's model across various benchmarks. In the Chatbot Arena Leaderboard (https://lnkd.in/eBaze79A), this 7B model impressively outperforms many larger models, including GPT-3.5-Turbo, Mixtral, Gemini Pro, and every fine-tuned version of Llama 2 70B. It is also significantly better than Mistral-7B-Instruct-v0.2, which ranks 30th with an Elo score of 1073. In my leaderboard (https://lnkd.in/gjWKF-5u), its average score is below Mistral-7B-Instruct-v0.2's but the evaluation also suggests that Starling is better: - Much higher AGIEval score (+5.71), which is an excellent benchmark. - Higher BigBench (+2.31) and GPT4All (+2.06) scores. BigBench is as good as AGIEval. - Worse TruthfulQA (-10.37), which shows that this benchmark is unreliable. Don't trust it. On the other hand, we also see that it doesn't perform much better than OpenChat 3.5 0106, which is the base model that Starling used. However, OpenChat is also ranked 30th with an Elo score of 1072. This is something that this benchmark suite couldn't capture. Can MT-bench capture it? Not really, Starling gets a score of 8.12 vs. 7.8 for OpenChat. It's also outperformed by GPT-3.5-Turbo and Mixtral. Not too surprising considering that it evaluates models on a set of multi-turn questions (conversational). Then MMLU, surely? Not at all. Starling obtained an MMLU of 65.14 vs. 65.04 for OpenChat, 60.78 for Mistral, and 71.88 for Mixtral. If I had to make a guess, I would say that Starling's PPO fine-tuning significantly improves the usefulness (but not the accuracy) of its answers, which is not correctly captured by these benchmarks. This shows a potential gap in the current evaluation pipeline. Rather than introducing another evaluation set, it'd be more efficient to use an LLM-as-a-judge, focusing specifically on the usefulness of the responses. It doesn't mean that the Chatbot Arena is perfect. For example, it doesn't handle conversations, and simply increasing verbosity is known to improve the Elo scores. This is exactly what Starling does, being generally a lot more verbose than OpenChat. Considering their excellent performance, I'd be very curious to see how 7B merges would rank on the Chatbot Arena Leaderboard (cc Arcee.ai). 👀 🤗 Model: https://lnkd.in/eyC7hfDF

Have we really squeezed out the capacity of a compact chat model? Thrilled to see our latest open model, Starling-7B, ranks 13th among all models in Chatbot Arena!  🚀 As a 7B model, Starling surpasses larger open and proprietary models, including Claude-2, GPT-3.5-Turbo, Gemini Pro, Mixtral 8x7B and Llama2-70B, and is currently the best 7B chat model in Chatbot Arena! #Chatbot #AI #LLM 

The beta version of Starling finally arrived! 🚀 Presenting Starling-LM-7B-beta, our cutting-edge 7B language model fine-tuned with RLHF! 🌟 Also introducing Starling-RM-34B, a Yi-34B-based reward model trained on our Nectar dataset, surpassing our previous 7B RM in all benchmarks. ✨ We've fine-tuned the latest Openchat model with the 34B reward model, achieving MT Bench score of 8.12 while being better at hard prompts compared to Starling-LM-7B-alpha. Testing will soon be available on lmsys. Please stay tuned! 🔗 HuggingFace links: [Starling-LM-7B-beta] https://lnkd.in/gT2EAN3r [Starling-RM-34B] https://lnkd.in/gxwbcdsi Discord Link: https://lnkd.in/g6Q3UiM8 Since the release of Starling-LM-7B-alpha, we've received numerous requests to make the model commercially viable. Therefore, we're licensing all models and datasets under Apache-2.0, with the condition that they are not used to compete with OpenAI. Enjoy!

🚀 Presenting Starling-LM-7B-beta, our new cutting-edge 7B language model fine-tuned with RLHF! 🌟 Also introducing Starling-RM-34B, the workhorse Reward Model behind the Starling-LM-7B-beta, ranking #1 in the latest RewardBenchmark from Nathan Lambert and the Allen Institute for AI (AI2) team. 🔗 HuggingFace links: [Starling-LM-7B-beta] https://lnkd.in/ecM4JXG5 [Starling-RM-34B] https://lnkd.in/erTkfu4N 🔗 Discord Link: https://lnkd.in/eBE73FaF 🔗 RewardBench from @allenai_org: https://lnkd.in/eKHJaFjJ Since the release of Starling-LM-7B-alpha, we've received numerous requests to make the model commercially viable. Therefore, we're licensing all models and datasets under Apache-2.0, with the condition that they are not used to compete with OpenAI. Enjoy!

📢 Powerful information extraction app built by Stefano Fiorucci and deepset Haystack team using #NexusRaven-V2 LLM for function calling! 🔥 We are thrilled to empower high-quality Gen AI apps with our compact LLMs and toolings.