Gemma is a new generation of open models from Google aimed at assisting developers and researchers in building AI responsibly. Gemma is a family of lightweight, state-of-the-art open models built from the same research and technology used to create the Gemini models. It includes model weights in two sizes: Gemma 2B and Gemma 7B, each released with pre-trained and instruction-tuned variants, you can try the model on Perplexity Labs now.
Google is releasing a Responsible Generative AI Toolkit to provide guidance and essential tools for creating safer AI applications with Gemma. This comes just a week after Google announced Gemini Pro 1.5. The models are available worldwide and can run on various platforms, including laptops, workstations, and Google Cloud. Gemma is designed with AI Principles at the forefront, and extensive evaluations have been conducted to ensure the safety and reliability of the models. The Responsible Generative AI Toolkit includes safety classification, debugging, and guidance tools.
Gemma also supports a wide variety of tools and systems, including multi-framework tools, cross-device compatibility, and cutting-edge hardware platforms. Furthermore, free credits for research and development are being offered to enable developers and researchers to start working with Gemma. The models are optimized for Google Cloud, and advanced customization is available with fully-managed Vertex AI tools or with self-managed GKE. Researchers can apply for Google Cloud credits of up to $500,000 to accelerate their projects. More information about Gemma and quickstart guides can be accessed on ai.google.dev/gemma.
Pushing open source
Google is no stranger to open source having literally releasing Transformers and many open source tools. Google is reinforcing its commitment to open-source innovation with the release of Gemma, creating a ripple of anticipation across the AI landscape. This isn’t Google’s first foray into open-source AI—its release of the transformative Transformers library laid a cornerstone for what has become a thriving ecosystem of machine learning models and applications. Gemma’s introduction into this space marks a substantial stride forward for open-source AI by not only providing a new suite of powerful tools but also by championing responsible AI practices.
Comparing to Other Models
A Redditor from the r/locallama community recently benchmarked Gemma 7B against other pre-trained 7Bs on the Open LLM Leaderboard, noting its standout performance boost in GSM8K, which highlights Gemma’s enhanced capabilities in mathematical reasoning.
Despite the specialized improvement in mathematical tasks with the 7B Gemma model, it doesn’t quite surpass the overall performance of the Mistral 7B. It seems to have a slight edge over Llama 7B, perhaps due to its architectural similarities to Llama 2, such as RMSNorm and RoPE. Notable differences include the use of a GeLU activation function instead of SiLU, a larger context window of 8k (twice that of Llama 2), wider intermediate layers, and fewer layers overall, at 28 compared to Llama’s 32. After hours of hands-on testing, it’s clear that Gemma 7B falls short of becoming a go-to over the established Mistral 7B models for broader applications.