How to build a binary convolution neural network with OpenVINO?

[godhj93]

I would like to build a customized binarized neural network.

It seems that OpenVINO supports compiling binary convolution as described at https://docs.openvino.ai/2023.3/openvino_docs_ops_convolution_BinaryConvolution_1.html?utm_source=chatgpt.com

However, I do not understand how to get this function works for Intel CPU.

Is there any tutorial or script for that?

Many thanks.

[rkazants]

Hi @godhj93,

Here is an example I got from https://github.com/openvinotoolkit/openvino/blob/master/src/bindings/python/tests/test_graph/test_create_op.py#L65:

import openvino as ov
import openvino.opset11 as ov_opset
import numpy as np

strides = np.array([1, 1])
pads_begin = np.array([0, 0])
pads_end = np.array([0, 0])
dilations = np.array([1, 1])
mode = "xnor-popcount"
pad_value = 0.0

input = ov_opset.parameter([1, 1, 9, 9], name="Input0", dtype=np.float32)
kernel = ov_opset.parameter([1, 1, 3, 3], name="Input1", dtype=np.float32)

binary_conv = ov_opset.binary_convolution(
    input, kernel, strides, pads_begin, pads_end, dilations, mode, pad_value,
)

ov_model = ov.Model([binary_conv], [input, kernel])

Note that there is no framework model (PyTorch, TensorFlow) which we convert with this operation. It looks obsolete operation that is not used in practice. Please describe your case where you are using it.

Best regards,
Roman

[godhj93]

Dear @rkazants,

Thank you very much for your valuable contributions.

I am currently exploring the development of a customized binary neural network for tasks such as segmentation or depth estimation. I came across the Class ov::pass::ConvToBinaryConv function described in the OpenVINO documentation (https://docs.openvino.ai/2025/api/c_cpp_api/classov_1_1pass_1_1_conv_to_binary_conv.html), and it appears that this may allow conversion of convolution operations—originally built in PyTorch or TensorFlow—into binary convolutions.

I would greatly appreciate any advice or best practices you could share on how to properly utilize this function within a workflow, especially for models targeting these types of tasks.

Thank you in advance for your guidance.

Best regards,
HJ

[rkazants]

Hi @godhj93,

You need quantize your CNN model into 2-bit so that convolution kernel will contain +1/-1 coeffs only and input to convolution is also quantized with level=2. After that, you can pass your model to conversion using ovc tool or ov.convert_model and you can expect that this transformation pass will be applied and your regular convolutional operation is transformed into binary convolution. This transformation is applied automatically if the pattern matches.

Best regards,
Roman

[godhj93]

Hi @rkazants,

Thank you again for your helpful guidance.

I'm currently trying to convert a convolution to a binary convolution following your suggestion. However, I haven't been able to get the transformation to trigger correctly. The part that's most confusing to me is the FakeQuantize operation — I'm not sure whether it should originate from PyTorch, ONNX, or be added by OpenVINO.

Here's the script I'm currently using:

import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.onnx
import openvino as ov
from openvino.tools.mo import convert_model

# 1. OpenVINO-style fake quantization
class OpenVINOStyleFakeQuant(nn.Module):
    def __init__(self, output_low=0.0, output_high=1.0, threshold=0.5):
        super().__init__()
        self.output_low = output_low
        self.output_high = output_high
        self.threshold = threshold

    def forward(self, x):
        return torch.where(x >= self.threshold,
                           torch.tensor(self.output_high, device=x.device),
                           torch.tensor(self.output_low, device=x.device))

# 2. Binary convolution model
class BinaryConvModel(nn.Module):
    def __init__(self, in_channels, out_channels, kernel_size, stride=1, padding=0,
                 output_low=0.0, output_high=1.0):
        super().__init__()
        self.fake_quant = OpenVINOStyleFakeQuant(output_low, output_high)

        weight = torch.randint(0, 2, (out_channels, in_channels, kernel_size, kernel_size)) * 2 - 1
        self.register_buffer("weight", weight.float())

        self.stride = stride
        self.padding = padding

    def forward(self, x):
        x_bin = self.fake_quant(x)
        return F.conv2d(x_bin, self.weight, stride=self.stride, padding=self.padding)

# 3. Convert PyTorch → OpenVINO
model = BinaryConvModel(in_channels=3, out_channels=4, kernel_size=3, padding=1)
model.eval()
converted_model = ov.convert_model(model)

# 4. Print operation list
core = ov.Core()
print("Operation List:")
for op in converted_model.get_ordered_ops():
    print(op.get_type_name())

# output
Operation List:
Parameter
Constant
ConvertLike
Constant
ConvertPromoteTypes
ConvertLike
Constant
ConvertLike
GreaterEqual
Constant
Constant
Select
Constant
Convolution
Result

As you can see, the BinaryConv node is not present.
Instead, it looks like my torch.where logic is converted into GreaterEqual + Select, which I assume doesn't match the required pattern for ConvToBinaryConv to trigger.

Would you be able to clarify:

Where exactly the FakeQuantize operation should originate from?

And how I can structure the PyTorch or ONNX graph so that OpenVINO recognizes the pattern and applies the transformation?

Thanks again in advance — your insights are very helpful.