Fine-Tuning Gemma 4 with GPUs on Cloud Run: From 89% to 94% in Animal Classification

Gemma 4 is Google's new generation of open models, built with improved reasoning capabilities and a more efficient architecture. The model ships under the Apache 2.0 license and features an updated architecture that requires changes to loading logic and data preparation compared to previous Gemma versions.

Architectural changes in Gemma 4

Key improvements:

• Native processing of images, video, and audio • Built-in support for function calling and structured JSON • Dynamic token counts for multimodal inputs

Gemma 4 requires the image to be passed before the text, using an <image> placeholder that tells the processor where to insert the image tokens. Instructions are merged with the user prompt into a single chat template. Unlike Gemma 3, the number of image tokens is now dynamic, so tokenizing the text separately can cause a length mismatch. The fix is to search for the response tokens in the full array starting from the end, which pinpoints the masking boundary accurately.

Memory optimization on Cloud Run

The infrastructure runs on an NVIDIA RTX PRO 6000 with 96 GB of VRAM. Gemma 4 31B in bfloat16 precision takes up roughly 62 GB of VRAM. Applying 4-bit QLoRA quantization through the bitsandbytes library cuts that down to 18–20 GB, leaving enough headroom for long-context processing and multimodal data.

One crucial detail: when using QLoRA, standard LoRA can ignore the model wrapper and cause errors. The fix is to set the PEFT parameter target_modules="all-linear" to stabilize the training process.

Results and training parameters

Gemma 4's baseline accuracy on the test set was 89%. After fine-tuning on the full Oxford-IIIT Pet dataset, accuracy rose to 94%. The configuration used: Rank 64, Alpha 64, learning rate 5e-5.

Running this on Google Cloud requires a project with billing enabled, the Cloud Run and Cloud Build APIs activated, and a Hugging Face token to access the model weights. It's recommended to first test the script on CPU with the 2B model and a minimal dataset before scaling up to a full run.

Model weights are uploaded to Google Cloud Storage in the same region as the Cloud Run service to speed things up. The image is built via Cloud Build, specifying the region, project, and repository name. The final Cloud Run job is created with NVIDIA RTX PRO 6000 support, the GCS bucket mounted as a local disk, and arguments to launch the fine-tuning process with paths to the model and output directory.