Gemma Tuner Multimodal: Fine-Tune Text, Images, and Audio on Apple Silicon

Gemma Tuner Multimodal allows users to fine-tune the Gemma model on text, images, and audio directly on Apple Silicon Macs, without the need for NVIDIA GPUs. This toolkit supports various modalities using LoRA (Low-Rank Adaptation) for efficient model tuning. Users can perform text-only fine-tuning using local CSV files, or enhance image and text capabilities for tasks like captioning and Visual Question Answering (VQA). Audio and text fine-tuning is uniquely supported natively on Apple Silicon, providing a streamlined path for developing domain-specific applications such as medical transcription or legal deposition analysis.

The toolkit is designed to handle large datasets by streaming data from Google Cloud Storage (GCS) or BigQuery, eliminating the need to store terabytes of data locally. This makes it ideal for users who need to work with extensive datasets but are constrained by local storage limitations.

Gemma Tuner Multimodal is built on top of Hugging Face's Gemma checkpoints and utilizes PEFT LoRA for model adaptation. It supports both Gemma 3n and Gemma 4 models, with specific configurations for each. The tool is designed to be user-friendly, featuring a wizard that guides users through model selection, dataset configuration, and training processes.

Installation requires Python 3.10+ and macOS 12.3+ with native arm64 support. The package includes a comprehensive CLI for dataset preparation, model fine-tuning, evaluation, and export. Users can also visualize training progress if desired.

Overall, Gemma Tuner Multimodal offers a powerful solution for developers looking to create multimodal AI applications on Apple Silicon, with the flexibility to adapt models to specific domains and languages.