Boost.Python 从 C++ 创建对现有 Python 对象的新引用

Posted 2023-02-21

【中文标题】Boost.Python 从 C++ 创建对现有 Python 对象的新引用

【英文标题】：Boost.Python create new reference to existing Python object from C++

【发布时间】：2014-11-21 11:10:35

【问题描述】：

我正在使用 Boost.Python 包装一个 C++ 类 X。在创建这个类的一个对象的那一刻，我想在本地命名空间中插入一个对该对象的附加引用（这样我就可以用一个固定的名称来引用他新创建的对象，比如说lastX）。我试过 在X::X() 构造函数中使用

X::X() 
   
   boost::python::object locals(boost::python::borrowed(PyEval_GetLocals()));
   boost::python::object me(this);
   locals["lastX"]=me;
   

但这不起作用（lastX 被创建但它什么都没有；从 Python 打印它会导致段错误）。我可能应该使用我自己的 init 函数，但我也不知道如何获取对那里新创建的 Python 对象的引用。

有什么想法吗？谢谢。

【问题讨论】：

需要这种类型的功能可能表明代码有异味，值得花时间寻找替代解决方案，例如修补初始化程序以将弱引用存储到定义明确的注册器中，而不是修改调用者的当地人。

【参考方案1】：

为此，必须修改调用堆栈上的另一帧。请注意，这取决于 Python 实现。例如，在 Python 2.7 中，inspect 模块和sys.settrace() 可用于修改特定框架上的locals()。

我强烈建议使用 Python 解决方案，就像在 this 答案中所做的那样，并猴子修补所需的类的 __init__ 函数。例如，下面将修补 Spam 类，以将引用新构造的 Spam 实例的名为 last_spam 的变量插入调用者的框架：

def _patch_init(cls, name):
    cls_init = getattr(cls, '__init__', None)
    def patch(self):
        caller_frame = inspect.currentframe(1)
        new_locals = caller_frame.f_locals
        new_locals[name] = self
        _force_locals(caller_frame, new_locals)
        if cls_init:
            cls_init(self)
    setattr(cls, '__init__', patch)

_patch_init(Spam, 'last_spam')

spam = Spam()
assert(spam is last_spam)

---

尽管如此，Boost.Python 也可以做到这一点。为此，必须：

修改框架的locals()。 在对象构造期间访问self 实例。

请注意，这些可能是相当高级的主题。

修改框架的locals()

这与this 答案中使用的方法相同，它依赖于 Python 实现，但用 C++ 编写。在大多数执行路径中，帧的locals() 不能将新变量写入其中。但是，启用系统跟踪后，调试器和其他工具经常使用的帧跟踪功能可以修改帧的locals()。

/// @brief Trace signature type.  Boost.Python requires MPL signature for
///        custom functors.
typedef boost::mpl::vector<
  boost::python::object, // return
  boost::python::object, // frame
  boost::python::object, // event
  boost::python::object // argt
> trace_signature_type;

/// @brief A noop function for Python.  Returns None.
boost::python::object noop(
  boost::python::tuple /* args */,
  boost::python::dict /* kw */
)

  return boost::python::object();


/// @brief Inject new_locals into the provided frame.
void inject_locals_into_frame(
  boost::python::object frame,
  boost::python::dict new_locals
)

  namespace python = boost::python;
  // Force tracing by setting the global tracing function to any non-None
  // function.
  // # if not sys.gettrace():
  if (!PyThreadState_Get()->c_tracefunc)
  
    // Use the sys.settrace function to prevent needing to re-implement the
    // trace trampoline.
    //   # import sys
    python::object sys(python::handle<>(PyImport_ImportModule("sys")));
    //   # sys.settrace(lambda *args, **keys: None)
    sys.attr("__dict__")["settrace"](python::raw_function(&detail::noop));
  

  // Create trace function.
  //   # def trace(frame, event, arg):
  python::object trace = python::make_function([new_locals](
        python::object frame,
        python::object /* event */,
        python::object /* arg */
      )
      
        // Update the frame's locals.
        //   # frame.f_locals.update(new_locals)
        frame.attr("f_locals").attr("update")(new_locals);

        // Set the frame to use default trace, preventing this
        // trace functor from resetting the locals on each trace call.
        //   # del frame.f_trace
        frame.attr("f_trace").del();

        //   # return None
        return boost::python::object();
      ,
    python::default_call_policies(),
    trace_signature_type());

  // Set the frame to use the custom trace.
  //   # frame.f_trace = trace
  frame.attr("f_trace") = trace;

通过上面的代码，可以使用inject_into_frame_locals() 来更新给定框架的局部变量。例如，以下代码会将引用42 的变量x 添加到当前帧中：

// Create dictionary that will be injected into the current frame.
namespace python = boost::python;
python::dict new_locals;
new_locals["x"] = 42;

// Get a handle to the current frame.
python::object frame(python::borrowed(
  reinterpret_cast<PyObject*>(PyEval_GetFrame())));

// Set locals to be injected into the frame.
inject_into_frame_locals(frame, new_locals);

---

在对象构造期间访问self 实例。

给定一个 C++ 对象，不能使用 Boost.Python API 来定位该对象所在的 Python 对象。因此，当 Boost.Python 正在构造对象时，必须访问 self。

有几个自定义点：

在构造过程中修改暴露的类以接受PyObject*。可以让 Boost.Python 提供 PyObject* 实例，专门用于 has_back_reference：

struct foo

  // Constructor.
  foo(PyObject* self)
  
    namespace python = boost::python;
    python::handle<> handle(python::borrowed(self));
    trace::inject_into_current_frame("last_foo", python::object(handle));
  

  // Boost.Python copy constructor.
  foo(PyObject* self, const foo& /* rhs */)
  
    namespace python = boost::python;
    python::handle<> handle(python::borrowed(self));
    trace::inject_into_current_frame("last_foo", python::object(handle));
  
;

namespace boost 
namespace python 
  // Have Boost.Python pass PyObject self to foo during construction.
  template <>
  struct has_back_reference<foo>
    : boost::mpl::true_
  ;
 // namespace python
 // namespace boost

BOOST_PYTHON_MODULE(example)

  namespace python = boost::python;
  python::class_<foo>("Foo", python::init<>());

将T 公开为由派生自T 的HeldType 持有。当HeldType公开派生自T时，会在构造时提供PyObject*：

struct bar ;

struct bar_holder
  : public bar

  bar_holder(PyObject* self)
  
   namespace python = boost::python;
    python::handle<> handle(python::borrowed(self));
    trace::inject_into_current_frame("last_bar", python::object(handle));
  

  bar_holder(PyObject* self, const bar& /* rhs */)
  
    namespace python = boost::python;
    python::handle<> handle(python::borrowed(self));
    trace::inject_into_current_frame("last_bar", python::object(handle));
 
;

BOOST_PYTHON_MODULE(example)

  namespace python = boost::python;
  python::class_<bar, bar_holder>("Bar", python::init<>());

禁止 Boost.Python 使用 boost::python::no_init 生成默认初始化程序，然后使用自定义策略将自定义工厂函数注册为 __init__ 方法。自定义对象构造的一个关键函数是boost::python::make_constructor 函数。当提供指向 C++ 函数或指向成员函数的指针时，它将返回一个 Python 可调用对象，当调用该对象时，将调用该函数并创建 Python 对象。但是，Boost.Python 试图对 C++ 函数隐藏特定于 Python 的细节，因此没有明确提供 self。不过，可以使用自定义CallPolicy，并在precall 或postcall 函数中访问Python 参数。为了访问self 参数，必须了解make_constructor 策略的实现细节，它将参数访问偏移1。例如，当尝试访问驻留在索引处的self 参数时0，必须请求-1 索引。

// Mockup models.
class spam ;

// Factor functions for the models will be necessary, as custom constructor
// functions will be needed to provide a customization hook for our models.
spam* make_spam()  return new spam(); 

template <typename BasePolicy = boost::python::default_call_policies>
struct custom_policy
  : BasePolicy
     
  template <typename ArgumentPackage>
  PyObject* postcall(const ArgumentPackage& args, PyObject* result)
  
    namespace python = boost::python;

    // Chain to base policy.
    result = BasePolicy::postcall(args, result);

    // self is the first argument.  It is an implementation detail that
    // the make_constructor policy will offset access by 1.  Thus, to 
    // access the actual object at index 0 (self), one must use -1.
    python::object self(python::borrowed(
      get(boost::mpl::int_<-1>(), args)));

    return result;
  
;

BOOST_PYTHON_MODULE(example)

  namespace python = boost::python;
  python::class_<spam>("Spam", python::no_init)
    .def("__init__", python::make_constructor(
      &make_spam, 
      custom_policy<>()))
   ;

再一次，默认初始化器可以被抑制，make_constructor 返回的对象可以被修饰。当__init__ 方法被调用时，装饰器对象将被调用，它将提供初始化器的所有参数，包括self。装饰器需要委托给从make_constructor 返回的对象。使用 boost::python::make_function() 将 C++ 仿函数转换为可调用的 Python 对象。请注意make_function 文档并未声明它支持自定义函子。但是，它在内部用于函子支持。

// Mockup models.
class egg ;

// Factor functions for the models will be necessary, as custom constructor
// functions will be needed to provide a customization hook for our models.
egg* make_egg()  return new egg(); 

template <typename Fn>
class custom_constructor

public:

  typedef boost::python::object result_type;

public:

  custom_constructor(Fn fn)
    : constructor_(boost::python::make_constructor(fn))
  

  /// @brief Initialize python object.
  template <typename ...Args>
  result_type operator()(boost::python::object self, Args... args)
  
    return constructor_(self, args...);
  

private:
  boost::python::object constructor_;
;

template <typename Fn>
boost::python::object make_custom_constructor(Fn fn)

  // Use MPL to decompose the factory function signature into the
  // desired Python object signature.
  typedef /* ... */ signature_type;

  // Create a callable python object from custom_constructor.
  return boost::python::make_function(
    custom_constructor<Fn>(fn),
    boost::python::default_call_policies(),
    signature_type());


BOOST_PYTHON_MODULE(example)

  namespace python = boost::python;
  python::class_<egg>("Egg", python::no_init)
    .def("__init__", make_custom_constructor(&make_egg))
    ;

---

这是一个完整的例子demonstrating上述所有方法：

#include <boost/function_types/components.hpp>
#include <boost/mpl/insert_range.hpp>
#include <boost/python.hpp>
#include <boost/python/raw_function.hpp>

namespace trace 
namespace detail 

/// @brief Trace signature type.  Boost.Python requires MPL signature for
///        custom functors.
typedef boost::mpl::vector<
  boost::python::object, // return
  boost::python::object, // frame
  boost::python::object, // event
  boost::python::object  // arg
> trace_signature_type;

/// @brief A noop function for Python.  Returns None.
boost::python::object noop(
  boost::python::tuple /* args */,
  boost::python::dict /* kw */
)

  return boost::python::object();


 // namespace detail

/// @brief Inject new_locals into the provided frame.
void inject_into_frame_locals(
  boost::python::object frame,
  boost::python::dict new_locals
)

  namespace python = boost::python;
  // Force tracing by setting the global tracing function to any non-None
  // function.
  // # if not sys.gettrace():
  if (!PyThreadState_Get()->c_tracefunc)
  
    // Use the sys.settrace function to prevent needing to re-implement the
    // trace trampoline.
    //   # import sys
    python::object sys(python::handle<>(PyImport_ImportModule("sys")));
    //   # sys.settrace(lambda *args, **keys: None)
    sys.attr("__dict__")["settrace"](python::raw_function(&detail::noop));
  

  // Create trace function.
  //   # def trace(frame, event, arg):
  python::object trace = python::make_function([new_locals](
        python::object frame,
        python::object /* event */,
        python::object /* arg */
      )
      
        // Update the frame's locals.
        //   # frame.f_locals.update(new_locals)
        frame.attr("f_locals").attr("update")(new_locals);

        // Set the frame to use default trace, preventing this
        // trace functor from resetting the locals on each trace call.
        //   # del frame.f_trace
        frame.attr("f_trace").del();

        //   # return None
        return boost::python::object();
      ,
    python::default_call_policies(),
    detail::trace_signature_type());

  // Set the frame to use the custom trace.
  //   # frame.f_trace = trace
  frame.attr("f_trace") = trace;


/// @brief Helper function used to setup tracing to inject the key-value pair
///        into the current frame.
void inject_into_current_frame(
  std::string key,
  boost::python::object value)

  // If there is no key, just return early.
  if (key.empty()) return;

  // Create dictionary that will be injected into the current frame.
  namespace python = boost::python;
  python::dict new_locals;
  new_locals[key] = value;

  // Get a handle to the current frame.
  python::object frame(python::borrowed(
    reinterpret_cast<PyObject*>(PyEval_GetFrame())));

  // Set locals to be injected into the frame.
  inject_into_frame_locals(frame, new_locals);


 // namespace trace

/// APPROACH 1: has_back_reference

struct foo

  // Constructor.
  foo(PyObject* self)
  
    namespace python = boost::python;
    python::handle<> handle(python::borrowed(self));
    trace::inject_into_current_frame("last_foo", python::object(handle));
  

  // Boost.Python copy constructor.
  foo(PyObject* self, const foo& /* rhs */)
  
    namespace python = boost::python;
    python::handle<> handle(python::borrowed(self));
    trace::inject_into_current_frame("last_foo", python::object(handle));
  
;

namespace boost 
namespace python 
  // Have Boost.Python pass PyObject self to foo during construction.
  template <>
  struct has_back_reference<foo>
    : boost::mpl::true_
  ;
 // namespace python
 // namespace boost

/// APPROACH 2: custom holder

struct bar ;

struct bar_holder
  : public bar

  bar_holder(PyObject* self)
  
    namespace python = boost::python;
    python::handle<> handle(python::borrowed(self));
    trace::inject_into_current_frame("last_bar", python::object(handle));
  

  bar_holder(PyObject* self, const bar& /* rhs */)
  
    namespace python = boost::python;
    python::handle<> handle(python::borrowed(self));
    trace::inject_into_current_frame("last_bar", python::object(handle));
  
;

/// APPROACH 3: custom call policy

struct spam ;

/// @brief CallPolicy that injects a reference to the returned object
///        into the caller's frame.  Expected to only be used as a
//         policy for make_constructor.
template <typename BasePolicy = boost::python::default_call_policies>
struct inject_reference_into_callers_frame
  : BasePolicy

  inject_reference_into_callers_frame(const char* name)
    : name_(name)
  

  template <typename ArgumentPackage>
  PyObject* postcall(const ArgumentPackage& args, PyObject* result)
  
    // Chain to base policy.
    result = BasePolicy::postcall(args, result);

    // self is the first argument.  It is an implementation detail that
    // the make_constructor policy will offset access by 1.  Thus, to 
    // access the actual object at index 0 (self), one must use -1.
    namespace python = boost::python;
    python::object self(python::borrowed(
      get(boost::mpl::int_<-1>(), args)));

    // Inject into the current frame. 
    trace::inject_into_current_frame(name_, self);

    return result;
  

private:
  std::string name_;
;

// Factor functions for the models will be necessary, as custom constructor
// functions will be needed to provide a customization hook for our models.
spam* make_spam()  return new spam(); 

/// APPROACH 4: decorated constructor
//
struct egg ;

namespace detail 

/// @brief A constructor functor that injects the constructed object
///        into the caller's frame.
template <typename Fn>
class inject_constructor

public:

  typedef boost::python::object result_type;

public:

  /// @brief Constructor.
  inject_constructor(
    const char* name,
    Fn fn
  )
    : name_(name),
      constructor_(boost::python::make_constructor(fn))
  

  /// @brief Initialize the python objet.
  template <typename ...Args>
  result_type operator()(boost::python::object self, Args... args)
  
    // Initialize the python object.
    boost::python::object result = constructor_(self, args...);

    // Inject a reference to self into the current frame.
    trace::inject_into_current_frame(name_, self);

    return result;
  

private:
  std::string name_;
  boost::python::object constructor_;
;

 // namespace detail

/// @brief Makes a wrapper constructor (constructor that works with
///        classes inheriting from boost::python::wrapper).
template <typename Fn>
boost::python::object make_inject_constructor(
  const char* name,
  Fn fn)

  // Decompose the factory function signature, removing the return type.
  typedef typename boost::mpl::pop_front<
    typename boost::function_types::components<Fn>::type
  >::type components_type;

  // Python constructors take the instance/self argument as the first
  // argument, and returns None.  Thus, inject python::objects into the
  // signature type for both the return and 'self' argument.
  typedef typename boost::mpl::insert_range<
    components_type, 
    typename boost::mpl::begin<components_type>::type,
    boost::mpl::vector<boost::python::object, boost::python::object>
  >::type signature_type;

  // Create a callable python object from inject_constructor.
  return boost::python::make_function(
    detail::inject_constructor<Fn>(name, fn),
    boost::python::default_call_policies(),
    signature_type());


egg* make_egg()  return new egg(); 

BOOST_PYTHON_MODULE(example)

  namespace python = boost::python;

  // APPROACH 1: has_back_reference
  python::class_<foo>("Foo", python::init<>());

  // APPROACH 2: custom holder
  python::class_<bar, bar_holder>("Bar", python::init<>());

  // APPROACH 3: custom call policy
  python::class_<spam>("Spam", python::no_init)
    .def("__init__", python::make_constructor(
      &make_spam,
      inject_reference_into_callers_frame<>("last_spam")))
    ;

  // APPROACH 4: decorated constructor
  python::class_<egg>("Egg", python::no_init)
    .def("__init__", make_inject_constructor("last_egg", &make_egg))
    ;

互动使用：

>>> import example
>>> foo = example.Foo()
>>> assert(foo is last_foo)
>>> bar = example.Bar()
>>> assert(bar is last_bar)
>>> spam = example.Spam()
>>> assert(spam is last_spam)
>>> egg = example.Egg()
>>> assert(egg is last_egg)

【讨论】：

这是一个非常有用的答案，它很好地解释了 boost::python 内部，足以帮助我解决多个其他问题。向作者致敬！！