ByteAtlas

Connecting Python and C++ with gRPC is straightforward when both sides are generated from the same .proto file. The difficult part is keeping that contract, the generated code, and the runtime configuration aligned after the first demo works.

I treat the protocol definition as a versioned boundary, not as a convenient way to serialize a Python dictionary into a C++ struct. Python and C++ never call each other directly. They agree on a service name, method path, message schema, status model, and transport behavior. The generated stub and service base class enforce that agreement in each language.

This guide builds a Python client and synchronous C++ server on Linux. The example is deliberately small, but it includes the pieces that tend to be missing from short tutorials: repeatable code generation, CMake linkage, deadlines, input validation, and error diagnosis.

How the Python-to-C++ call actually works

• A shared .proto file defines messages and RPC methods.
• protoc generates protobuf message classes for both languages.
• The gRPC plugins generate a Python stub and a C++ service interface.
• The Python stub serializes a request into Protocol Buffers and sends it over an HTTP/2 channel.
• The C++ server deserializes the request, calls the implementation, and serializes the response.
• A gRPC status travels separately from the response message.

The wire format is language-neutral. That does not make application semantics language-neutral. A field may be present but interpreted differently, a timeout may exist only on one client, or one deployment may run older generated code. Interoperability comes from managing the contract, not merely choosing gRPC.

Project layout

grpc-python-cpp/
├── CMakeLists.txt
├── proto/
│   └── inference.proto
├── generated/
│   ├── cpp/
│   └── python/
├── cpp/
│   └── server.cc
└── python/
    └── client.py

Generated files live outside the handwritten source directories. Whether you commit them is a team decision, but generation must be reproducible. The fragile setup is one where a developer generated Python stubs months ago, another machine generated C++ sources from a newer proto, and neither artifact records how it was created.

Define one contract for both languages

// proto/inference.proto
syntax = "proto3";

package byteatlas.inference.v1;

service InferenceService {
  rpc Predict(PredictRequest) returns (PredictResponse);
}

message PredictRequest {
  string request_id = 1;
  repeated float features = 2;
}

message PredictResponse {
  string request_id = 1;
  repeated float scores = 2;
  int32 predicted_class = 3;
}

The package becomes part of the fully qualified service name. In this example, the RPC path includes byteatlas.inference.v1.InferenceService. Changing the package or service name without regenerating and redeploying both sides commonly appears as UNIMPLEMENTED, even though a server is listening on the expected port.

Field numbers are the durable part of a protobuf message. Add new fields with new numbers. Do not reuse a removed field number for a different meaning. Renaming a field is usually wire-compatible; changing its number or incompatible type is not. Reserve deleted numbers and names when a schema will evolve across independently deployed services.

Generate Python and C++ code

Create the output directories and install the Python tooling:

mkdir -p generated/python generated/cpp
python -m venv .venv
source .venv/bin/activate
python -m pip install grpcio grpcio-tools protobuf

Generate the Python message classes, type hints, and gRPC bindings:

python -m grpc_tools.protoc \
  -I proto \
  --python_out=generated/python \
  --pyi_out=generated/python \
  --grpc_python_out=generated/python \
  proto/inference.proto

This produces inference_pb2.py, inference_pb2.pyi, and inference_pb2_grpc.py. The _pb2 module contains messages and descriptors. The _pb2_grpc module contains the client stub and server registration helpers.

Generate the C++ protobuf and gRPC sources:

protoc \
  -I proto \
  --cpp_out=generated/cpp \
  --grpc_out=generated/cpp \
  --plugin=protoc-gen-grpc="$(command -v grpc_cpp_plugin)" \
  proto/inference.proto

The C++ side receives inference.pb.h, inference.pb.cc, inference.grpc.pb.h, and inference.grpc.pb.cc. Keep the protoc, protobuf runtime, and gRPC plugin toolchain compatible. A generated file is source code, but it still carries assumptions about the runtime API it will compile against.

Implement the C++ server

// cpp/server.cc
#include <algorithm>
#include <iostream>
#include <memory>
#include <numeric>
#include <string>

#include <grpcpp/grpcpp.h>
#include "inference.grpc.pb.h"

namespace api = byteatlas::inference::v1;

class InferenceService final
    : public api::InferenceService::Service {
 public:
  grpc::Status Predict(
      grpc::ServerContext* context,
      const api::PredictRequest* request,
      api::PredictResponse* response) override {
    if (request->features_size() != 4) {
      return grpc::Status(
          grpc::StatusCode::INVALID_ARGUMENT,
          "features must contain exactly four values");
    }

    if (context->IsCancelled()) {
      return grpc::Status(
          grpc::StatusCode::CANCELLED,
          "request was cancelled");
    }

    const float sum = std::accumulate(
        request->features().begin(),
        request->features().end(),
        0.0f);

    response->set_request_id(request->request_id());
    response->add_scores(-sum);
    response->add_scores(sum);
    response->set_predicted_class(sum >= 0.0f ? 1 : 0);

    return grpc::Status::OK;
  }
};

int main() {
  const std::string address = "0.0.0.0:50051";
  InferenceService service;

  grpc::ServerBuilder builder;
  builder.AddListeningPort(
      address,
      grpc::InsecureServerCredentials());
  builder.RegisterService(&service);

  std::unique_ptr<grpc::Server> server = builder.BuildAndStart();
  if (!server) {
    std::cerr << "Failed to start gRPC server\n";
    return 1;
  }

  std::cout << "Listening on " << address << "\n";
  server->Wait();
}

The generated Service class defines the C++ method signature. Returning INVALID_ARGUMENT is better than placing an error string inside a successful response. It lets clients distinguish validation failures from transport failures and decide what is safe to retry.

The server binds to 0.0.0.0 so it can accept connections through container or remote interfaces. The Python client should not dial 0.0.0.0; it should use a routable host such as localhost, a container service name, or a DNS name.

Build the C++ server with CMake

# CMakeLists.txt
cmake_minimum_required(VERSION 3.20)
project(grpc_python_cpp LANGUAGES CXX)

set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)

find_package(Protobuf CONFIG REQUIRED)
find_package(gRPC CONFIG REQUIRED)

add_executable(inference_server
    cpp/server.cc
    generated/cpp/inference.pb.cc
    generated/cpp/inference.grpc.pb.cc
)

target_include_directories(inference_server PRIVATE
    generated/cpp
)

target_link_libraries(inference_server PRIVATE
    gRPC::grpc++
    protobuf::libprotobuf
)

cmake -S . -B build -G Ninja
cmake --build build
./build/inference_server

If CMake cannot find gRPCConfig.cmake or ProtobufConfig.cmake, the issue is package discovery, not the RPC code. Install gRPC and protobuf through one consistent package or source-build prefix, then point CMAKE_PREFIX_PATH at that prefix. Mixing a system protobuf library with generated code and plugins from an unrelated installation is a common way to create linker or runtime incompatibilities.

Implement the Python client

# python/client.py
import grpc

import inference_pb2
import inference_pb2_grpc


def main() -> None:
    channel = grpc.insecure_channel("localhost:50051")

    try:
        grpc.channel_ready_future(channel).result(timeout=3.0)
    except grpc.FutureTimeoutError:
        raise SystemExit("gRPC server did not become ready")

    stub = inference_pb2_grpc.InferenceServiceStub(channel)
    request = inference_pb2.PredictRequest(
        request_id="request-001",
        features=[0.4, -0.2, 1.1, 0.3],
    )

    try:
        response = stub.Predict(request, timeout=2.0)
    except grpc.RpcError as error:
        print("RPC failed")
        print("code:", error.code())
        print("details:", error.details())
        raise SystemExit(1) from error
    finally:
        channel.close()

    print("request_id:", response.request_id)
    print("scores:", list(response.scores))
    print("predicted_class:", response.predicted_class)


if __name__ == "__main__":
    main()

PYTHONPATH=generated/python \
  .venv/bin/python python/client.py

The readiness wait separates “the channel never connected” from “the RPC returned an application error.” The per-call timeout becomes a gRPC deadline. Official gRPC guidance recommends setting realistic deadlines because clients otherwise may wait indefinitely.

Do not treat every RpcError as retryable. INVALID_ARGUMENT means the request should change. UNAVAILABLE may be transient. DEADLINE_EXCEEDED is ambiguous for state-changing operations because the server may have completed the work before the client stopped waiting.

What to verify before calling it integrated

• Generate both languages from the same proto revision.
• Start the C++ server and confirm the expected address is listening.
• Call a valid request and verify identifiers and numeric outputs.
• Call with the wrong feature count and verify INVALID_ARGUMENT.
• Stop the server and verify the client reports connectivity failure.
• Use an intentionally short timeout and verify deadline handling.
• Run through the same hostnames, container network, and TLS mode used in deployment.

A localhost success only proves the application logic and local transport agree. It does not prove container DNS, ingress protocol support, certificates, proxies, load balancers, or production deadlines are correct.

Common errors and practical fixes

UNAVAILABLE: failed to connect to all addresses

Check whether the server is running and listening on the interface the client can reach. Inside containers, localhost points to the current container, not another service. Use the Compose or Kubernetes service name. Also check whether one side expects TLS while the other uses insecure credentials.

ss -ltnp | grep 50051
getent hosts inference-server

UNIMPLEMENTED: method not found

This usually means the client and server disagree on the fully qualified service or method name, or the service was not registered with the server builder. Regenerate both sides from the same proto and confirm builder.RegisterService(&service) executes before BuildAndStart().

DEADLINE_EXCEEDED

The deadline may be too short, but increasing it blindly can conceal queueing or a blocked backend. Measure where time is spent, include network overhead, and check cancellation inside expensive server work. A client abandoning the call does not automatically make arbitrary application code stop immediately.

grpc_cpp_plugin not found

protoc can generate protobuf messages without being able to generate gRPC C++ services. Confirm the plugin is installed and resolve its exact path:

command -v protoc
command -v grpc_cpp_plugin
protoc --version

If they come from different prefixes, fix the toolchain rather than copying the plugin binary into a random directory.

Python cannot import inference_pb2

The generated directory is not on Python’s import path, or generation used a different proto include root. Use a stable package layout, add __init__.py files if packaging the generated code, and run with the intended PYTHONPATH. Inspect the import at the top of inference_pb2_grpc.py instead of guessing.

C++ undefined references to grpc or protobuf

Compiling the generated .cc files is not enough. Link the server against gRPC::grpc++ and protobuf::libprotobuf using imported CMake targets. If the error mentions generated message methods, verify inference.pb.cc is part of the executable. If it mentions service or stub symbols, verify inference.grpc.pb.cc is included.

RESOURCE_EXHAUSTED: message larger than max

Large tensors or batches can exceed configured send or receive limits. Raising the limit on only the client or only the server does not solve the contract. First ask whether one large unary message is the right API. Streaming, chunking, shared object storage, or a compact tensor representation may be more appropriate.

The call succeeds but values are wrong

This is usually an application-contract problem rather than transport corruption. Check feature ordering, units, normalization, enum values, default values, and whether an older producer omitted a newly added field. Include a request ID and log the decoded shape and schema version at service boundaries without logging sensitive payloads.

Design choices before production

The synchronous C++ API is a reasonable starting point, but each active RPC consumes server resources differently from an asynchronous or callback design. Choose the server API after understanding expected concurrency and blocking behavior. Do not switch to an asynchronous implementation merely because it sounds more scalable; it adds state-management complexity that needs a real workload to justify it.

Use TLS and authenticated credentials outside trusted local development. Add health checking and reflection only when their operational exposure is understood. Define retry policy around idempotency, not around a generic list of status codes. For model inference, consider whether requests should be unary, streamed, or batched by a server-side scheduler.

Conclusion

A Python-to-C++ gRPC integration has three layers to keep aligned: the protobuf contract, generated language bindings, and runtime behavior. Most failures that look like networking problems are mismatches between those layers.

Start with one versioned proto, generate both sides reproducibly, return meaningful status codes, and set deadlines from the first client implementation. Once that path is reliable, TLS, streaming, retries, and performance tuning can be added without guessing whether the basic contract is already broken.

For further detail, use the official gRPC introduction, Python quick start, C++ quick start, and deadlines guide. If the C++ build fails before the server starts, the CMake undefined reference guide covers the linker side of the problem.