Looks at one picture, several pictures, a document, or a video. The 3 GB figure is for an iPhone 17 Pro Max using Lookahead Expert Scheduling, which loads experts from storage as they are needed. The full weights still have to be stored. On a desktop with ordinary llama.cpp or Ollama, plan for the whole Q4_K_M file plus the vision projector, about 20.8 GB.
File
19.7 GB
License
Apache 2.0
Used for
Reading a scan
Quality note
OpenBMB's larger vision model for pictures, documents, and video. The 35B-A3B size is a mixture of experts.
Fits on: Gaming PC, 24 GB GPU, MacBook Pro, 32 GB, Mini PC, 64 GB, and 6 more