MiniCPM-V-4.7-1B-int4-ov

• Model creator: OpenBMB
• Original model: openbmb/MiniCPM-V-4.7-1B

EXPERIMENTAL MODEL This model has not been fully validated with OpenVINO and uses development versions of Optimum Intel and OpenVINO GenAI.

Description

This is the MiniCPM-V-4.7-1B model converted to the OpenVINO IR (Intermediate Representation) format with INT4 weight compression by NNCF.

MiniCPM-V 4.7 supports text, image, and video inputs.

Quantization Parameters

For --weight-format int4, the Optimum Intel default requests the following NNCF weight-compression parameters:

• mode: INT4_ASYM
• group_size: 128
• ratio: 1.0

Confirm the parameters against the generated openvino_config.json before publishing the model files. For more information, see the OpenVINO weight compression guide.

Compatibility

The examples below use:

• Optimum Intel main with MiniCPM-V 4.7 export support
• Transformers 5.18.0
• Matching nightly OpenVINO, OpenVINO Tokenizers, and OpenVINO GenAI builds with VLM support

Optimum Intel currently declares transformers<5.11; the 5.18.0 requirement for this model may cause a dependency warning until upstream aligns its package metadata.

Running Model Inference with Optimum Intel

Install the development dependencies:

pip install -U "git+https://github.com/huggingface/optimum-intel.git@main" --extra-index-url https://download.pytorch.org/whl/cpu
pip install -U "transformers==5.18.0" Pillow requests
pip install --pre -U openvino --extra-index-url https://storage.openvinotoolkit.org/simple/wheels/nightly

Run image inference with Optimum Intel:

from io import BytesIO

import requests
from PIL import Image
from transformers import AutoProcessor
from optimum.intel.openvino import OVModelForVisualCausalLM

model_id = "OpenVINO/MiniCPM-V-4.7-1B-int4-ov"
processor = AutoProcessor.from_pretrained(model_id)
model = OVModelForVisualCausalLM.from_pretrained(model_id)

image_url = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/ai2d-demo.jpg"
response = requests.get(image_url, timeout=30)
response.raise_for_status()
image = Image.open(BytesIO(response.content)).convert("RGB")
messages = [{"role": "user", "content": [{"type": "image"}, {"type": "text", "text": "Describe this image."}]}]
text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=[text], images=[image], return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=128)
print(processor.batch_decode(outputs[:, inputs["input_ids"].shape[1] :], skip_special_tokens=True)[0])

Running Model Inference with OpenVINO GenAI

Install the matching development packages:

pip install -U huggingface_hub requests Pillow
pip install --pre -U openvino openvino-tokenizers openvino-genai --extra-index-url https://storage.openvinotoolkit.org/simple/wheels/nightly

Download the exported model from Hugging Face:

from huggingface_hub import snapshot_download

model_path = snapshot_download("OpenVINO/MiniCPM-V-4.7-1B-int4-ov")

Run image inference with OpenVINO GenAI:

from io import BytesIO

import numpy as np
import openvino as ov
import openvino_genai as ov_genai
import requests
from PIL import Image

pipe = ov_genai.VLMPipeline(model_path, "CPU")

url = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/ai2d-demo.jpg"
response = requests.get(url, timeout=30)
response.raise_for_status()
with Image.open(BytesIO(response.content)) as image:
    image_tensor = ov.Tensor(np.asarray(image.convert("RGB"))[None])

history = ov_genai.ChatHistory([{"role": "user", "content": "Describe this image."}])
history.set_extra_context({"enable_thinking": False})
config = ov_genai.GenerationConfig()
config.max_new_tokens = 128

def streamer(chunk):
    print(chunk, end="", flush=True)
    return False

_ = pipe.generate(history, image=image_tensor, generation_config=config, streamer=streamer)

For more text, image, and video examples, see the MiniCPM-V 4.7 OpenVINO notebook.

Limitations

Check the original model card for limitations. This OpenVINO export has not been fully validated.

Legal information

The original MiniCPM-V-4.7-1B model declares the Apache-2.0 license.

Disclaimer

Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel's Global Human Rights Principles. Intel's products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.