c++ boost进程间交换（复制）非共享和共享字符串向量

Posted 2023-02-21

【中文标题】c++ boost进程间交换（复制）非共享和共享字符串向量

【英文标题】：c++ boost interprocess exchange (copy) non-shared and shared string vector

【发布时间】：2017-06-04 23:45:16

【问题描述】：

我想实现this 问题的答案中实现的目标，但对于字符串向量，即字符串对象。我希望能够编写可用于堆内存或共享内存的通用代码。我已经模拟了代码并编写了一个类来实现一个字符串向量，该向量适用于堆内存或共享内存。我让它大部分工作。但是，当我尝试从堆分配的字符串向量初始化/创建字符串的共享内存向量时，我遇到了编译错误。在上面提到的问题中，共享内存向量是从非共享内存向量初始化的（引用“这是因为...... MAGIC！”）。不确定是否是因为该问题正在处理易于复制和移动的整数向量。我已经为共享内存字符串类提供了复制构造函数，但它仍然不起作用。我在下面提供了我的代码（注释掉了给我编译错误的代码）。任何帮助表示赞赏。谢谢。

#include <boost/interprocess/managed_shared_memory.hpp>
#include <boost/interprocess/allocators/allocator.hpp>
#include <boost/interprocess/containers/vector.hpp>
#include <boost/interprocess/containers/string.hpp>
#include <boost/container/scoped_allocator.hpp>
#include <iostream>
#include <string>
#include <sstream>

namespace bip = boost::interprocess;

namespace generic


template <typename T, typename Alloc/* = std::allocator<T>*/ >
using vector = bip::vector<T, typename Alloc::template rebind<T>::other >;

template <typename Alloc/* = std::allocator<T>*/ >
using GenericString = bip::basic_string<char, std::char_traits<char>
                           , typename Alloc::template rebind<char>::other>;

template <typename Alloc>
class String

public:
  typedef Alloc allocator_type; // ties in with uses_allocator/scoped_allocator

  String(Alloc alloc = Alloc())
  : mString(alloc)
  
  

  String(const std::string str, Alloc alloc = Alloc())
  : mString(str.c_str(), alloc)
  
  

  String(const char* str, Alloc alloc = Alloc())
  : mString(str, alloc)
  
  

  template <typename OtherAlloc>
  String(String<OtherAlloc> const& other, Alloc alloc = Alloc())
  : mString(other.mString.begin(), other.mString.end(), alloc)
  
  

  std::string ToString()  return std::string(mString.begin(), mString.end()); 

  friend std::ostream& operator<<(std::ostream& os, const String& str)
  
    os << str.mString;
    return os;
  

private:
  template<typename OtherAlloc> friend struct String;
  GenericString<Alloc> mString;
;

template <typename Alloc>
class StringVector

public:
  typedef Alloc allocator_type; // ties in with uses_allocator/scoped_allocator
  typedef vector<String<Alloc>, Alloc> GenericStringVector;
  using Iterator = typename GenericStringVector::iterator;
  using ConstIterator = typename GenericStringVector::const_iterator;

  StringVector(Alloc alloc = Alloc())
  : mStrings(alloc)
  
  

  template <typename OtherAlloc>
  StringVector(StringVector<OtherAlloc> const& other, Alloc alloc = Alloc())
  : mStrings(other.mStrings.begin(), other.mStrings.end(), alloc)
  
  

  void Add(const String<Alloc> str)  mStrings.emplace_back(str); 

  void Remove(Iterator iter)  mStrings.erase(iter); 

  void Clear()  mStrings.clear(); 

  void Print() const
  
    std::cout << "[";
    for (ConstIterator it = mStrings.begin(); it != mStrings.end(); it++)
      std::cout << (it == mStrings.begin() ? "" : ", ") << *it;
    std::cout << "]\n";
  

private:
  template<typename OtherAlloc> friend struct StringVector;
  GenericStringVector mStrings;
;



namespace heap

  using VAlloc = std::allocator<void>;
  using String = generic::String<VAlloc>;
  using StringVector = generic::StringVector<VAlloc>;


namespace shared

  using VAlloc =boost::container::scoped_allocator_adaptor<
          bip::allocator<void, bip::managed_shared_memory::segment_manager> >;

  using String = generic::String<VAlloc>;
  using StringVector = generic::StringVector<VAlloc>;


int main(void)

  struct shm_remove 
      shm_remove()   bip::shared_memory_object::remove("MySharedMemory"); 
      ~shm_remove()  bip::shared_memory_object::remove("MySharedMemory"); 
   remover;

  std::cout << "Heap based storage: \n";
  heap::StringVector svec;

  svec.Add(heap::String("test1"));
  svec.Add(heap::String("test2"));
  svec.Add(heap::String("test3"));
  svec.Add(heap::String("test4"));
  svec.Print();

  std::cout << "Shared memory storage: \n";
  bip::managed_shared_memory seg(bip::create_only, "MySharedMemory", 65536);
  shared::VAlloc shalloc(seg.get_segment_manager());

  shared::StringVector sh_svec(shalloc);
  sh_svec.Add(shared::String("test1", shalloc));
  sh_svec.Add(shared::String("test2", shalloc));
  sh_svec.Add(shared::String("test3", shalloc));
  sh_svec.Add(shared::String("test4", shalloc));
  sh_svec.Print();


  shared::StringVector sh_svec2(sh_svec, shalloc);
  sh_svec2.Print();

  //shared::StringVector sh_svec3(svec, shalloc);   // gives compile error :(
  //sh_svec3.Print();

包括编译错误：

In file included from src/test_string_vector.cc:5:0:
/usr/include/boost/container/scoped_allocator.hpp: In instantiation of ‘boost::container::container_detail::scoped_allocator_adaptor_base<OuterAlloc>::scoped_allocator_adaptor_base() [with OuterAlloc = boost::interprocess::allocator<void, boost::interprocess::segment_manager<char, boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family>, boost::interprocess::iset_index> >]’:
/usr/include/boost/container/scoped_allocator.hpp:963:7:   required from ‘boost::container::scoped_allocator_adaptor<OuterAlloc, InnerAllocs>::scoped_allocator_adaptor() [with OuterAlloc = boost::interprocess::allocator<void, boost::interprocess::segment_manager<char, boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family>, boost::interprocess::iset_index> >; InnerAllocs = ]’
/usr/include/boost/container/vector.hpp:1174:15:   required from ‘void boost::container::vector<T, Allocator>::assign(FwdIt, FwdIt, typename boost::container::container_detail::enable_if_c<((! boost::container::container_detail::is_convertible<InIt, typename boost::container::allocator_traits<Allocator>::size_type>::value) && ((! boost::container::container_detail::is_input_iterator<InIt>::value) && (! boost::move_detail::is_same<typename boost::container::container_detail::version<Allocator>::type, boost::container::container_detail::integral_constant<unsigned int, 0u> >::value)))>::type*) [with FwdIt = boost::container::container_detail::vec_iterator<generic::String<std::allocator<void> >*, true>; T = generic::String<boost::container::scoped_allocator_adaptor<boost::interprocess::allocator<void, boost::interprocess::segment_manager<char, boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family>, boost::interprocess::iset_index> > > >; Allocator = boost::container::scoped_allocator_adaptor<boost::interprocess::allocator<generic::String<boost::container::scoped_allocator_adaptor<boost::interprocess::allocator<void, boost::interprocess::segment_manager<char, boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family>, boost::interprocess::iset_index> > > >, boost::interprocess::segment_manager<char, boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family>, boost::interprocess::iset_index> > >; typename boost::container::container_detail::enable_if_c<((! boost::container::container_detail::is_convertible<InIt, typename boost::container::allocator_traits<Allocator>::size_type>::value) && ((! boost::container::container_detail::is_input_iterator<InIt>::value) && (! boost::move_detail::is_same<typename boost::container::container_detail::version<Allocator>::type, boost::container::container_detail::integral_constant<unsigned int, 0u> >::value)))>::type = void]’
/usr/include/boost/container/vector.hpp:864:7:   required from ‘boost::container::vector<T, Allocator>::vector(InIt, InIt, const allocator_type&) [with InIt = boost::container::container_detail::vec_iterator<generic::String<std::allocator<void> >*, true>; T = generic::String<boost::container::scoped_allocator_adaptor<boost::interprocess::allocator<void, boost::interprocess::segment_manager<char, boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family>, boost::interprocess::iset_index> > > >; Allocator = boost::container::scoped_allocator_adaptor<boost::interprocess::allocator<generic::String<boost::container::scoped_allocator_adaptor<boost::interprocess::allocator<void, boost::interprocess::segment_manager<char, boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family>, boost::interprocess::iset_index> > > >, boost::interprocess::segment_manager<char, boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family>, boost::interprocess::iset_index> > >; boost::container::vector<T, Allocator>::allocator_type = boost::container::scoped_allocator_adaptor<boost::interprocess::allocator<generic::String<boost::container::scoped_allocator_adaptor<boost::interprocess::allocator<void, boost::interprocess::segment_manager<char, boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family>, boost::interprocess::iset_index> > > >, boost::interprocess::segment_manager<char, boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family>, boost::interprocess::iset_index> > >]’
src/test_string_vector.cc:78:65:   required from ‘generic::StringVector<Alloc>::StringVector(const generic::StringVector<OtherAlloc>&, Alloc) [with OtherAlloc = std::allocator<void>; Alloc = boost::container::scoped_allocator_adaptor<boost::interprocess::allocator<void, boost::interprocess::segment_manager<char, boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family>, boost::interprocess::iset_index> > >]’
src/test_string_vector.cc:151:46:   required from here
/usr/include/boost/container/scoped_allocator.hpp:762:7: error: no matching function for call to ‘boost::interprocess::allocator<void, boost::interprocess::segment_manager<char, boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family>, boost::interprocess::iset_index> >::allocator()’
       
       ^
In file included from /usr/include/boost/interprocess/segment_manager.hpp:38:0,
                 from /usr/include/boost/interprocess/detail/managed_memory_impl.hpp:30,
                 from /usr/include/boost/interprocess/managed_shared_memory.hpp:25,
                 from src/test_string_vector.cc:1:
/usr/include/boost/interprocess/allocators/allocator.hpp:142:4: note: candidate: template<class T2> boost::interprocess::allocator<T, SegmentManager>::allocator(const boost::interprocess::allocator<T2, SegmentManager>&)
    allocator(const allocator<T2, SegmentManager> &other)
    ^
/usr/include/boost/interprocess/allocators/allocator.hpp:142:4: note:   template argument deduction/substitution failed:
In file included from src/test_string_vector.cc:5:0:
/usr/include/boost/container/scoped_allocator.hpp:762:7: note:   candidate expects 1 argument, 0 provided
       
       ^
In file included from /usr/include/boost/interprocess/segment_manager.hpp:38:0,
                 from /usr/include/boost/interprocess/detail/managed_memory_impl.hpp:30,
                 from /usr/include/boost/interprocess/managed_shared_memory.hpp:25,
                 from src/test_string_vector.cc:1:
/usr/include/boost/interprocess/allocators/allocator.hpp:136:4: note: candidate: boost::interprocess::allocator<T, SegmentManager>::allocator(const boost::interprocess::allocator<T, SegmentManager>&) [with T = void; SegmentManager = boost::interprocess::segment_manager<char, boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family>, boost::interprocess::iset_index>]
    allocator(const allocator &other)
    ^
/usr/include/boost/interprocess/allocators/allocator.hpp:136:4: note:   candidate expects 1 argument, 0 provided
/usr/include/boost/interprocess/allocators/allocator.hpp:131:4: note: candidate: boost::interprocess::allocator<T, SegmentManager>::allocator(boost::interprocess::allocator<T, SegmentManager>::segment_manager*) [with T = void; SegmentManager = boost::interprocess::segment_manager<char, boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family>, boost::interprocess::iset_index>; boost::interprocess::allocator<T, SegmentManager>::segment_manager = boost::interprocess::segment_manager<char, boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family>, boost::interprocess::iset_index>]
    allocator(segment_manager *segment_mngr)
    ^
/usr/include/boost/interprocess/allocators/allocator.hpp:131:4: note:   candidate expects 1 argument, 0 provided
Makefile:16: recipe for target 'obj/test_string_vector.o' failed
make: *** [obj/test_string_vector.o] Error 1

【参考方案1】：

我想我现在明白你的意图了。 你想要这段代码 where OtherAlloc = std::allocator , Alloc = bi:allocator

template <typename OtherAlloc>
  StringVector(StringVector<OtherAlloc> const& other, Alloc alloc = Alloc())
  : mStrings(other.mStrings.begin(), other.mStrings.end(), alloc)
  
  

调用您提供的复制构造函数以从堆转换为共享类型

 template <typename OtherAlloc>
  String(String<OtherAlloc> const& other, Alloc alloc = Alloc())
  : mString(other.mString.begin(), other.mString.end(), alloc)
  
  

但是 boost::interprocess::vector::assign(first,last) 没有传递你的 allocator 实例，看起来它使用 Alloc alloc = Alloc() 其中 Alloc = boost::interprocess::allocator 并且它没有有默认构造函数。它与我所有的帖子相关:-) 明天我会尝试编写我的代码，非常具有挑战性的难题。

我今天修改了你的代码，请看下面，看看你是否喜欢它。

#include <boost/interprocess/managed_shared_memory.hpp>
#include <boost/interprocess/allocators/allocator.hpp>
#include <boost/interprocess/containers/vector.hpp>
#include <boost/interprocess/containers/string.hpp>
#include <boost/container/scoped_allocator.hpp>
#include <iostream>
#include <string>
#include <sstream>
#include <algorithm>

namespace bip = boost::interprocess;

namespace generic


template <typename T, typename Alloc/* = std::allocator<T>*/ >
using vector = bip::vector<T, typename Alloc::template rebind<T>::other >;

template <typename Alloc/* = std::allocator<T>*/ >
using GenericString = bip::basic_string<char, std::char_traits<char>
                           , typename Alloc::template rebind<char>::other>;

template <typename Alloc>
class String

public:
  typedef Alloc allocator_type; // ties in with uses_allocator/scoped_allocator

  template<typename ...T>
  String(const std::string &str, T && ...t)
  : alloc_(std::forward<T>(t)...) , mString(str.c_str(), alloc_)
  

  template<typename ...T>
  String(const char* str, T && ...t)
  : alloc_(std::forward<T>(t)...) , mString(str, alloc_)
  

  template <typename OtherAlloc, typename ...T>
  String(String<OtherAlloc> const& other, T && ...t)
  : alloc_(std::forward<T>(t)...) , mString(other.mString.begin(), other.mString.end(), alloc_)
  

  std::string ToString()  return std::string(mString.begin(), mString.end()); 

  friend std::ostream& operator<<(std::ostream& os, const String& str)
  
    os << str.mString;
    return os;
  

private:
  template<typename OtherAlloc> friend struct String;
  Alloc alloc_;
  GenericString<Alloc> mString;
;

template <typename Alloc>
class StringVector

public:
  typedef Alloc allocator_type; // ties in with uses_allocator/scoped_allocator
  typedef vector<String<Alloc>, Alloc> GenericStringVector;
  using Iterator = typename GenericStringVector::iterator;
  using ConstIterator = typename GenericStringVector::const_iterator;

  StringVector()
  : alloc_(), mStrings(alloc_)
  

  template<typename T>
  StringVector(T && t)
  : alloc_(std::forward<T>(t)), mStrings(alloc_)
  

  template <typename OtherAlloc, typename ...T>
  StringVector(StringVector<OtherAlloc> const& other, T && ...t)
  : alloc_(std::forward<T>(t)...), mStrings(alloc_)
  
      std::transform ( other.mStrings.begin(), other.mStrings.end(), std::back_inserter(mStrings), [&t...](auto &s) 
          return typename GenericStringVector::value_type(s, std::forward<T>(t)...);
      );
  

  void Add(const String<Alloc> str)  mStrings.emplace_back(str); 

  void Remove(Iterator iter)  mStrings.erase(iter); 

  void Clear()  mStrings.clear(); 

  void Print() const
  
    std::cout << "[";
    for (ConstIterator it = mStrings.begin(); it != mStrings.end(); it++)
      std::cout << (it == mStrings.begin() ? "" : ", ") << *it;
    std::cout << "]\n";
  

private:
  template<typename OtherAlloc> friend struct StringVector;
  Alloc alloc_;
  GenericStringVector mStrings;
;



namespace heap

  using VAlloc = std::allocator<void>;
  using String = generic::String<VAlloc>;
  using StringVector = generic::StringVector<VAlloc>;


namespace shared

  using VAlloc =boost::container::scoped_allocator_adaptor<
          bip::allocator<char, bip::managed_shared_memory::segment_manager> >;

  using String = generic::String<VAlloc>;
  using StringVector = generic::StringVector<VAlloc>;


int main(void)

  struct shm_remove 
      shm_remove()   bip::shared_memory_object::remove("MySharedMemory"); 
      ~shm_remove()  bip::shared_memory_object::remove("MySharedMemory"); 
   remover;

  std::cout << "Heap based storage: \n";
  heap::StringVector svec;

  svec.Add(heap::String("test1"));
  svec.Add(heap::String("test2"));
  svec.Add(heap::String("test3"));
  svec.Add(heap::String("test4"));
  svec.Print();

  std::cout << "Shared memory storage: \n";
  bip::managed_shared_memory seg(bip::create_only, "MySharedMemory", 65536);
  auto smp = seg.get_segment_manager();

  shared::StringVector sh_svec(smp);
  sh_svec.Add(shared::String("test1", smp));
  sh_svec.Add(shared::String("test2", smp));
  sh_svec.Add(shared::String("test3", smp));
  sh_svec.Add(shared::String("test4", smp));
  sh_svec.Print();


  shared::StringVector sh_svec2(sh_svec, smp);
  sh_svec2.Print();

  shared::StringVector sh_svec3(svec, smp);   // does not give compile error 
  sh_svec3.Print();

【讨论】：

是的。我需要我的代码需要在堆内存和共享内存系统中工作。因此，我试图提出允许我这样做的基本数据结构。感谢您抽出宝贵时间。

别担心，我已经稍微重构了你的通用代码，它编译它运行。

这太棒了。谢谢你。最后一个忙。您能否解释一下您在 StringVector 复制构造函数中所做的事情。你所做的对我来说似乎是黑魔法。 :)

所以，因为 std::allocator() 构造函数不需要参数，但是 boost::interprocess::allocator(segment_manager *) 需要至少一个指向 segment_manager 的参数指针来实现我使用的两个相同的构造函数StringVector 复制构造函数的可变参数模板。

T && ...t 在我们的例子中被转换为 0...N 参数 0 或 1 。我不传递分配器，但我将参数传递给分配器。对于 Heap pass nothing ，对于 Shared pass segment_manager *。尝试了更长的评论它没有接受它:-(

【参考方案2】：

您没有提供编译错误，但是仅通过简要查看代码和 boost::interprocess 的一些工作知识，我认为您的问题在于 std::allocator 用于 boost::interprocess ，但我不是 100%当然，所以请提供编译错误。 同时，如果你喜欢 您可以查看我们的 Shared、Mmap、Heap 开源堆栈解决方案 memory_types.hpp 我认为你可以使用boost::interprocess::allocator 和boost::interprocess::managed_heap_memory::segment_manager 或类似的东西。 还请指定您是否使用 C++03 或 C++11 编译器，因为我认为在 C++03 中它是 typedef T* pointer_type 这不适用于 boost 共享内存，因为它需要 offset_ptr ，基本上是从内存段的开头偏移而不是绝对地址。

我修改了您使用 shared::StringVector 存储 SHM 结构的代码 所以现在它编译了：

heap::StringVector sh_svec3(svec);   // does not gives compile error :-)
sh_svec3.Print();

【讨论】：

添加了编译错误。不想让这个问题太长...... :(

它为我编译和链接，我使用 gcc 5.4 和 boost 1.61

我假设您使用的是 C++11，否则 template using 将无法编译，所以唯一的问题是您的 boost 版本

你是否取消了最后两行的注释... //shared::StringVector sh_svec3(svec, shalloc); // 给出编译错误 :( //sh_svec3.Print(); 我评论了它们以测试代码的其他部分。抱歉。FWIW，我用 boost 1.58 编译它。

知道了。 boost::interprocess::allocator 没有您的通用代码String(Alloc alloc = Alloc()) : mString(alloc) 中的默认构造函数，所以解决它的唯一方法是正如我在回答中所说，查看我们项目中的示例并使用 boost::interprocess:: managed_heap_memory::segment_manager