pybind11 - 如果访问了结构的任何成员，则 boost::optional 结构的 std::vector 成员将被清空

Posted 2023-02-23

【中文标题】pybind11 - 如果访问了结构的任何成员，则 boost::optional 结构的 std::vector 成员将被清空

【英文标题】：pybind11 - std::vector member of boost::optional struct is emptied if any members of the struct are accessed

【发布时间】：2020-07-07 11:18:18

【问题描述】：

我在 C++ 中有一个包含 3 个结构的分层数据结构，我在我的最小工作示例 (mwe) House、Room 和 Objects 中对其进行了重命名。 Room 是 boost::optional 类型，并且有一个 std::vector<Object> 成员，其中包含此房间中的所有对象。对象只是一些数字的容器。

我知道这对于此类信息来说过于复杂，但在原始代码中是必需的，并且不能轻易更改。由于我们不使用 c++17，因此我尝试将其更改为 std::experimental::optional，但这破坏了我们 c++ 代码中的某些部分，我不知道它是否真的能解决问题。

在 C++ 中，我完全没有问题，boost::optional 和所有成员变量都可以正常工作。但是在绑定所有内容后，我遇到了一个奇怪的问题，即只要我访问 Room 的任何成员变量，std::vector 对象就会被清空。这可以是长度、宽度、面积或对象本身，如 Python 示例中所示。如果对象第一次被访问，它们实际上是正常返回的，但是当尝试第二次访问时，它们就像被移动了一样消失了。如果您执行x = myHouse.kitchen.objects.copy()，同样的行为也适用。该列表在x 中，甚至可以多次访问，但厨房中的信息立即丢失。奇怪的是，这也只适用于对象，所有其他双重成员都可以无限期地访问。

为了方便编译所有东西给mwe和调试，我把所有东西都塞进了一个cpp文件中，这在原始代码中显然不是这样。对于 mwe，c++ 代码是通过 cppimport 编译和包含的，但手动编译也不会改变任何内容。

这是mwe：

mwe.cpp：

#include <boost/optional.hpp>
#include <iostream>
#include <map>
#include <string>
#include <vector>
#include <pybind11/complex.h>
#include <pybind11/pybind11.h>
#include <pybind11/stl.h>

/* implementing structs */
struct Object 
  Object() = default;
  Object(double price, double height) : price(price), height(height) 
  ;

  double price = 0.0;
  double height = 0.0;
;

struct Room 
  Room() = default;

  double length;
  double width;
  std::vector<Object> objects; // this is the buggy vector

  double area() const 
    return length * width;
  ;
;

struct House 
  House() = default;

  boost::optional<Room> bedroom;
  boost::optional<Room> kitchen;
  boost::optional<Room> livingroom;

  std::map<std::string, std::vector<Object>> getObjects() 
    std::map<std::string, std::vector<Object>> out;
    if (bedroom) 
      out.insert(std::make_pair("bedroom", bedroom->objects));
    
    if (kitchen) 
      out.insert(std::make_pair("kitchen", kitchen->objects));
    
    if (livingroom) 
      out.insert(std::make_pair("livingroom", livingroom->objects));
    
    return out;
  ;
;

/* everything works fine in C++ -> get data this way to have complete object map */
House initSomethingInCpp() 
  auto myHouse = House();
  myHouse.bedroom = Room();
  myHouse.kitchen = Room();

  myHouse.bedroom->length = 10.0;
  myHouse.bedroom->width = 2.0;
  myHouse.kitchen->length = 5.0;
  myHouse.kitchen->width = 3.0;

  std::vector<Object> bedroomObjects;
  std::vector<Object> kitchenObjects;

  Object closet = Object(100.0, 2.5);
  Object bed = Object(200.0, 1.0);
  Object oven = Object(500.0, 1.5);
  Object table = Object(50.0, 1.5);

  bedroomObjects.push_back(closet);
  bedroomObjects.push_back(bed);
  kitchenObjects.push_back(oven);
  kitchenObjects.push_back(table);

  myHouse.bedroom->objects = bedroomObjects;
  myHouse.kitchen->objects = kitchenObjects;

  return myHouse;
;

namespace pybind11 
/* taking care of boost type */
namespace detail 
/* boost::optional */
template<typename T>
struct type_caster<boost::optional<T>> : optional_caster<boost::optional<T>> ;
 // namespace detail
 // namespace pybind11


/* binding structs */
void init_house(pybind11::module& main) 
  pybind11::class_<House> house(main, "House");
  house.def(pybind11::init<>());

  house.def_readwrite("bedroom", &House::bedroom);
  house.def_readwrite("kitchen", &House::kitchen);
  house.def_readwrite("livingroom", &House::livingroom);
  house.def("get_objects", &House::getObjects);
;

void init_room(pybind11::module& main) 
  pybind11::class_<Room> room(main, "Room");
  room.def(pybind11::init<>());
  room.def_readwrite("length", &Room::length);
  room.def_readwrite("width", &Room::width);
  room.def_readwrite("objects", &Room::objects);
  room.def_property_readonly("area", &Room::area);
;

void init_objects(pybind11::module& main) 
  pybind11::class_<Object> object(main, "Object");
  object.def(pybind11::init<>());
  object.def(pybind11::init<double, double>());

  object.def_readonly("price", &Object::price);
  object.def_readonly("heigth", &Object::height);
;

/* define module and bind init_in_cpp function */
PYBIND11_MODULE(mwe, m) 
 init_house(m);
 init_room(m);
 init_objects(m);
 m.def("init_something_in_cpp", &initSomethingInCpp);
;

执行.py：

import cppimport
#cppimport.set_quiet(False)
#cppimport.force_rebuild()
mod = cppimport.imp('mwe')

# get data
myHouse = mod.init_something_in_cpp()

print("\n")
print("all data is here")
objs = myHouse.get_objects()
print(objs)
print(myHouse.kitchen.area) # by accessing kitchen members, the objects list is emptied
print("\n")

print("kitchen objects are now missing")
objs = myHouse.get_objects()
print(objs)
print("but area still works:")
print(myHouse.kitchen.area) # everything but objects still works
print("\n")

print("also works directly with same variable")
print("bedroom objects are accessed:")
print(myHouse.bedroom.objects)
print("bedroom objects are accessed again:")
print(myHouse.bedroom.objects)

执行给出以下输出：

all data is here
'bedroom': [mwe.Object object at 0x7fbc9c2a43f0, mwe.Object object at 0x7fbc9c2a4670], 'kitchen': [mwe.Object object at 0x7fbc9c2a4c30, mwe.Object object at 0x7fbc9c2a4df0]
15.0

kitchen objects are now missing
'bedroom': [mwe.Object object at 0x7fbc9c2a4e70, mwe.Object object at 0x7fbc9c2a4eb0], 'kitchen': []
but area still works:
15.0

also works directly with same variable
bedroom objects are accessed:
[mwe.Object object at 0x7fbc9c2a4c30, mwe.Object object at 0x7fbc9c2a4df0]
bedroom objects are accessed again:
[]

【参考方案1】：

原来这实际上是 pybind11 release 2.5 https://github.com/pybind/pybind11/issues/1919中的一个错误@

在当前的 master 分支和未来的版本中已修复。