Pybind11 Stubgen Guide

1. Where to Set output_dir for pybind11-stubgen?

The output_dir is where the generated .pyi file will be saved. It does not affect Python’s ability to import the module—the .pyi file is just for type hints and documentation (used by IDEs like VSCode/PyCharm).

Example

1pybind11-stubgen my_module -o ./stubs

• This generates ./stubs/my_module.pyi.
• Python will still import my_module from wherever the .so file is located (e.g., build/lib.linux-x86_64-3.8/).

2. How to Specify .so Output Directory in CMake?

In your CMakeLists.txt, you can control where the .so file is built using:

• LIBRARY_OUTPUT_DIRECTORY (for single-config generators like Unix Makefiles).
• RUNTIME_OUTPUT_DIRECTORY (for shared libraries on Windows).
• CMAKE_LIBRARY_OUTPUT_DIRECTORY (global setting for all libraries).

Example CMake Snippet:

1set(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/bin)  # .so goes to ./build/bin/
2add_library(my_module MODULE my_module.cpp)
3set_target_properties(my_module PROPERTIES PREFIX "")  # Avoid 'lib' prefix on Unix

• After building, the .so file will be in ./build/bin/my_module.so.

3. Location Relationship Between .so and .pyi Files

The .so and .pyi files do not need to be in the same directory for Python to use them. Here’s how they interact:

Recommended Structure:

project_root/
├── build/           # CMake build dir
│ └── bin/
│  └── my_module.so  # Compiled module
├── stubs/           # Manually created
│ └── my_module.pyi  # Generated stubs
└── src/
  └── my_module.cpp  # C++ source

Making Python Find the .so File:

Option 1: Install the .so to Python’s site-packages:

1install(TARGETS my_module DESTINATION ${PYTHON_SITE_PACKAGES})

Option 2: Set PYTHONPATH:

1export PYTHONPATH=/path/to/build/bin:$PYTHONPATH

Option 3: Move .so to your Python script’s directory.

4. Automating Stub Generation with CMake

You can add a custom command to generate stubs after building the .so file:

1find_program(PYBIND11_STUBGEN pybind11-stubgen)
2if(PYBIND11_STUBGEN)
3  add_custom_command(
4    TARGET my_module POST_BUILD
5    COMMAND ${PYBIND11_STUBGEN} -o ${CMAKE_SOURCE_DIR}/stubs my_module
6    COMMENT "Generating .pyi stubs for my_module"
7  )
8endif()

• This runs pybind11-stubgen automatically after building my_module.so.

5. Full Workflow Example

1. Build the .so file:

   1mkdir build && cd build
   2cmake .. -DPYTHON_EXECUTABLE=$(which python)
   3make
   

   • Output: build/bin/my_module.so

2. Generate stubs:

   1pybind11-stubgen my_module -o ../stubs
   

   • Output: project_root/stubs/my_module.pyi

3. Use in Python:

   1import my_module  # Finds my_module.so via PYTHONPATH
   

6. Fianl Solution

1python -m pybind11-stubgen your_module

Key Takeaways

• The .pyi file is for developer tools (not runtime), so its location is flexible.
• The .so file must be in sys.path for import my_module to work.
• Use CMAKE_LIBRARY_OUTPUT_DIRECTORY to control where CMake puts the .so file.
• Stub files are often placed in a stubs/ directory (PEP 561) or alongside the .so.