Matei Zaharia’s Post

At Databricks, we've built an awesome model training and tuning stack. We now used it to release DBRX, the best open source LLM on standard benchmarks to date, exceeding GPT-3.5 while running 2x faster than Llama-70B: https://lnkd.in/ghPNSRyw DBRX is a 132B parameter MoE model with 36B active params and fine-grained (4-of-16) sparsity and 32K context. It was trained using the dropless MoE approach pioneered by my student Trevor Gale in MegaBlocks (and based on his code). Technical details: https://lnkd.in/ghPNSRyw You can try DBRX on Databricks model serving and playground today, with an OpenAI-compatible API! And most importantly, if you want to tune, RLHF or train your own model, we have everything we needed to build this from scratch.

At Databricks, we've built an awesome model training and tuning stack. We now used it to release DBRX, the best open source LLM on standard benchmarks to date, exceeding GPT-3.5 while running 2x faster than Llama-70B: https://lnkd.in/ghPNSRyw DBRX is a 132B parameter MoE model with 36B active params and fine-grained (4-of-16) sparsity and 32K context. It was trained using the dropless MoE approach pioneered by my student Trevor Gale in MegaBlocks (and based on his code). Technical details: https://lnkd.in/ghPNSRyw You can try DBRX on Databricks model serving and playground today, with an OpenAI-compatible API! And most importantly, if you want to tune, RLHF or train your own model, we have everything we needed to build this from scratch.

At Databricks, we've built an awesome model training and tuning stack. We now used it to release DBRX, the best open source LLM on standard benchmarks to date, exceeding GPT-3.5 while running 2x faster than Llama-70B: https://lnkd.in/ghPNSRyw DBRX is a 132B parameter MoE model with 36B active params and fine-grained (4-of-16) sparsity and 32K context. It was trained using the dropless MoE approach pioneered by my student Trevor Gale in MegaBlocks (and based on his code). Technical details: https://lnkd.in/ghPNSRyw You can try DBRX on Databricks model serving and playground today, with an OpenAI-compatible API! And most importantly, if you want to tune, RLHF or train your own model, we have everything we needed to build this from scratch.

At Databricks, we've built an awesome model training and tuning stack. We now used it to release DBRX, the best open source LLM on standard benchmarks to date, exceeding GPT-3.5 while running 2x faster than Llama-70B: https://lnkd.in/ghPNSRyw DBRX is a 132B parameter MoE model with 36B active params and fine-grained (4-of-16) sparsity and 32K context. It was trained using the dropless MoE approach pioneered by my student Trevor Gale in MegaBlocks (and based on his code). Technical details: https://lnkd.in/ghPNSRyw You can try DBRX on Databricks model serving and playground today, with an OpenAI-compatible API! And most importantly, if you want to tune, RLHF or train your own model, we have everything we needed to build this from scratch.

At Databricks, we've built an awesome model training and tuning stack. We now used it to release DBRX, the best open source LLM on standard benchmarks to date, exceeding GPT-3.5 while running 2x faster than Llama-70B: https://lnkd.in/ghPNSRyw DBRX is a 132B parameter MoE model with 36B active params and fine-grained (4-of-16) sparsity and 32K context. It was trained using the dropless MoE approach pioneered by my student Trevor Gale in MegaBlocks (and based on his code). Technical details: https://lnkd.in/ghPNSRyw You can try DBRX on Databricks model serving and playground today, with an OpenAI-compatible API! And most importantly, if you want to tune, RLHF or train your own model, we have everything we needed to build this from scratch.

At Databricks, we've built an awesome model training and tuning stack. We now used it to release DBRX, the best open source LLM on standard benchmarks to date, exceeding GPT-3.5 while running 2x faster than Llama-70B: https://lnkd.in/ghPNSRyw DBRX is a 132B parameter MoE model with 36B active params and fine-grained (4-of-16) sparsity and 32K context. It was trained using the dropless MoE approach pioneered by my student Trevor Gale in MegaBlocks (and based on his code). Technical details: https://lnkd.in/ghPNSRyw You can try DBRX on Databricks model serving and playground today, with an OpenAI-compatible API! And most importantly, if you want to tune, RLHF or train your own model, we have everything we needed to build this from scratch.

At Databricks, we've built an awesome model training and tuning stack. We now used it to release DBRX, the best open source LLM on standard benchmarks to date, exceeding GPT-3.5 while running 2x faster than Llama-70B: https://lnkd.in/ghPNSRyw DBRX is a 132B parameter MoE model with 36B active params and fine-grained (4-of-16) sparsity and 32K context. It was trained using the dropless MoE approach pioneered by my student Trevor Gale in MegaBlocks (and based on his code). Technical details: https://lnkd.in/ghPNSRyw You can try DBRX on Databricks model serving and playground today, with an OpenAI-compatible API! And most importantly, if you want to tune, RLHF or train your own model, we have everything we needed to build this from scratch.

At Databricks, we've built an awesome model training and tuning stack. We now used it to release DBRX, the best open source LLM on standard benchmarks to date, exceeding GPT-3.5 while running 2x faster than Llama-70B: https://lnkd.in/ghPNSRyw DBRX is a 132B parameter MoE model with 36B active params and fine-grained (4-of-16) sparsity and 32K context. It was trained using the dropless MoE approach pioneered by my student Trevor Gale in MegaBlocks (and based on his code). Technical details: https://lnkd.in/ghPNSRyw You can try DBRX on Databricks model serving and playground today, with an OpenAI-compatible API! And most importantly, if you want to tune, RLHF or train your own model, we have everything we needed to build this from scratch.

At Databricks, we've built an awesome model training and tuning stack. We now used it to release DBRX, the best open source LLM on standard benchmarks to date, exceeding GPT-3.5 while running 2x faster than Llama-70B: https://lnkd.in/ghPNSRyw DBRX is a 132B parameter MoE model with 36B active params and fine-grained (4-of-16) sparsity and 32K context. It was trained using the dropless MoE approach pioneered by my student Trevor Gale in MegaBlocks (and based on his code). Technical details: https://lnkd.in/ghPNSRyw You can try DBRX on Databricks model serving and playground today, with an OpenAI-compatible API! And most importantly, if you want to tune, RLHF or train your own model, we have everything we needed to build this from scratch.

At Databricks, we've built an awesome model training and tuning stack. We now used it to release DBRX, the best open source LLM on standard benchmarks to date, exceeding GPT-3.5 while running 2x faster than Llama-70B: https://lnkd.in/ghPNSRyw DBRX is a 132B parameter MoE model with 36B active params and fine-grained (4-of-16) sparsity and 32K context. It was trained using the dropless MoE approach pioneered by my student Trevor Gale in MegaBlocks (and based on his code). Technical details: https://lnkd.in/ghPNSRyw You can try DBRX on Databricks model serving and playground today, with an OpenAI-compatible API! And most importantly, if you want to tune, RLHF or train your own model, we have everything we needed to build this from scratch.