Google has unveiled Gemma, a suite of open models designed to advance both the safety and performance of artificial intelligence (AI) systems. Derived from the same research and technology that powers the powerful Gemini models, Gemma represents a significant leap forward in various benchmarks for language comprehension, reasoning, and safety.
Advancing AI safety and performance
Gemma, developed by the Google DeepMind Gemma Team, is built on the foundation of the transformer decoder, incorporating several enhancements such as Multi-Query Attention, RoPE Embeddings, GeGLU Activations, and RMSNorm. These enhancements are crucial to its outstanding performance across diverse domains, including dialogue, reasoning, mathematics, and code generation.
Gemma offers flexibility for efficient deployment and development and is available in two variants. The 7 billion parameter model is optimized for GPU and TPU platforms, while the 2 billion parameter model is tailored for CPU and on-device applications. Both models were meticulously trained on extensive datasets, employing similar architectures and training methodologies as the esteemed Gemini model family.
Ensuring responsible AI development
In addition to performance enhancements, the Gemma team introduces the Responsible Generative AI Toolkit, providing guidance and essential tools for crafting safer AI applications. Techniques such as automated filtering of sensitive information from training sets, supervised fine-tuning, and reinforcement learning from human feedback have been employed to enhance the efficacy and safety of Gemma models.
The release of Gemma into the AI development ecosystem is expected to catalyze the creation of a myriad of beneficial applications, particularly in realms like science, education, and the arts. Moreover, Gemma’s responsible deployment holds the promise of bolstering the safety of frontier models, thereby fostering the next wave of LLM innovations.
With Gemma, Google aims to address concerns surrounding the safe deployment of Large Language Models while pushing the boundaries of AI performance. By providing open access to cutting-edge models and promoting responsible AI development practices, Google endeavors to accelerate progress in AI research and application.
Gemma represents a significant milestone in advancing AI safety and performance. With its innovative architecture, tailored models, and emphasis on responsible AI development, Gemma is poised to make substantial contributions across various domains. As the AI landscape continues to evolve, Gemma stands as a testament to Google’s commitment to pushing the boundaries of AI while ensuring its responsible and ethical use.