如何通过 Boost.MPI 发送 2d Boost.MultiArray 的子数组？

Posted 2023-03-27

【中文标题】如何通过 Boost.MPI 发送 2d Boost.MultiArray 的子数组？

【英文标题】：How to send subarray of 2d Boost.MultiArray via Boost.MPI?

【发布时间】：2020-09-18 08:16:48

【问题描述】：

我需要使用 Boost.MPI 发送对 Boost.MultiArray 的 2d 子数组的引用。 现在我有以下代码：

matrix_type::array_view<2>::type
                    current_process_batch = matrix[boost::indices[range(bias, finish_line)][range(0, width)]];
world.send(rank, BEGIN_TAG, current_process_batch);

但我在尝试执行它时也遇到以下错误：

/usr/local/include/boost/serialization/access.hpp:116:11: error: no member named 'serialize' in 'boost::detail::multi_array::multi_array_view<double, 2>'
        t.serialize(ar, file_version);

谁能帮我把它发送到另一个进程？

【参考方案1】：

你需要实现序列化。

这是一个通用的multi_array<T. Dims>序列化器：

namespace boost::serialization 
    template <typename Ar, typename T, size_t Dims>
        void save(Ar& ar, boost::multi_array<T, Dims> const& ma, unsigned /*version*/) 
            std::array<int, Dims> shape;
            std::copy_n(ma.shape(), Dims, shape.begin());
            ar & make_nvp("shape", shape)
               & make_nvp("data", make_array(ma.data(), ma.num_elements()));
        

    template <typename Ar, typename T, size_t Dims>
        void load(Ar& ar, boost::multi_array<T, Dims>& ma, unsigned /*version*/) 
            std::array<int, Dims> shape;
            ar & make_nvp("shape", shape);

            ma.resize(shape);
            ar & make_nvp("data", make_array(const_cast<T*>(ma.data()), ma.num_elements()));
        

    template <typename Ar, typename T, size_t Dims>
        void serialize(Ar& ar, boost::multi_array<T, Dims>& ma, unsigned version) 
            split_free(ar, ma, version);
        

看Live On Coliru

auto make_multi_array() 
    boost::multi_array<int, 2> ma(boost::extents[7][4]);
    std::iota(ma.data(), ma.data() + ma.num_elements(), 10);
    return ma;


int main() 
    auto const original = make_multi_array();

    
        std::ofstream os("array.txt");
        boost::archive::text_oarchive oa(os);

        oa << original;
    

    std::cout << std::ifstream("array.txt").rdbuf() << "\n";

    
        std::ifstream is("array.txt");
        boost::archive::text_iarchive ia(is);

        boost::multi_array<int, 2> restored;
        ia >> restored;

        assert(restored == original);
    

打印

22 serialization::archive 17 0 0 0 0 2 7 4 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37

更新：观点

嗯。已经接受了？注意到我实际上并没有很好地阅读你的问题。你想序列化一个视图。你不能，真的。您不能“反序列化视图”。

你能做的最多就是反序列化成视图。

还要注意

如果大小差异不大，对整个数组进行序列化可能会（很多）更有效

视图类型不是库的公共接口，即它可能会在未来的 Boost 版本中中断。如有疑问，请自行序列化数组

这里还有一些机器：

添加了对detail::multi_array::multi_array_view<...> 的支持，因为它可以反序列化INTO仅匹配形状的现有视图

添加了对 detail::multi_array::const_multi_array_view<...> 的支持，仅用于序列化。出于显而易见的原因，所述视图只能反序列化为非常量视图。

使用子视图扩展演示，在序列化后修补初始数组，因此我们可以验证将补丁读取到恢复数组的等效视图中与原始数组具有相同的效果（在检查补丁创建了预期的不匹配20 != 999)。

Live On Coliru

#include <boost/archive/text_oarchive.hpp>
#include <boost/archive/text_iarchive.hpp>
#include <boost/serialization/array.hpp>
#include <boost/serialization/nvp.hpp>
#include <boost/serialization/array_wrapper.hpp>
#include <boost/multi_array.hpp>
#include <iostream>
#include <fstream>

namespace boost::serialization 
    template <typename Ar, typename T, size_t Dims>
        void save(Ar& ar, boost::detail::multi_array::const_multi_array_view<T, Dims> const& ma, unsigned /*version*/) 
            std::array<int, Dims> shape;
            std::copy_n(ma.shape(), Dims, shape.begin());
            ar & make_nvp("shape", shape);

            for (auto i = 0; i < shape[0]; ++i) 
                for (auto j = 0; j < shape[1]; ++j) 
                    ar & make_nvp("item", ma[i][j]);
                
            
        

    template <typename Ar, typename T, size_t Dims>
        void load(Ar&, boost::detail::multi_array::const_multi_array_view<T, Dims>&, unsigned) 
            static_assert(not Ar::is_loading::value, "const_multi_array_view immutable");
        

    template <typename Ar, typename T, size_t Dims>
        void serialize(Ar& ar, boost::detail::multi_array::const_multi_array_view<T, Dims>& ma, unsigned version) 
            split_free(ar, ma, version);
        

    template <typename Ar, typename T, size_t Dims>
        void save(Ar& ar, boost::detail::multi_array::multi_array_view<T, Dims> const& ma, unsigned /*version*/) 
            std::array<int, Dims> shape;
            std::copy_n(ma.shape(), Dims, shape.begin());
            ar & make_nvp("shape", shape);

            for (auto i = 0; i < shape[0]; ++i) 
                for (auto j = 0; j < shape[1]; ++j) 
                    ar & make_nvp("item", ma[i][j]);
                
            
        

    template <typename Ar, typename T, size_t Dims>
        void load(Ar& ar, boost::detail::multi_array::multi_array_view<T, Dims>& ma, unsigned /*version*/) 
            std::array<int, Dims> shape;
            ar & make_nvp("shape", shape);

            if (!std::equal(begin(shape), end(shape), ma.shape()))
                throw std::logic_error("multi_array_view shape mismatch");

            for (auto i = 0; i < shape[0]; ++i) 
                for (auto j = 0; j < shape[1]; ++j) 
                    ar & make_nvp("item", ma[i][j]);
                
            
        

    template <typename Ar, typename T, size_t Dims>
        void serialize(Ar& ar, boost::detail::multi_array::multi_array_view<T, Dims>& ma, unsigned version) 
            split_free(ar, ma, version);
        

    template <typename Ar, typename T, size_t Dims>
        void save(Ar& ar, boost::multi_array<T, Dims> const& ma, unsigned /*version*/) 
            std::array<int, Dims> shape;
            std::copy_n(ma.shape(), Dims, shape.begin());
            ar & make_nvp("shape", shape)
               & make_nvp("data", make_array(ma.data(), ma.num_elements()));
        

    template <typename Ar, typename T, size_t Dims>
        void load(Ar& ar, boost::multi_array<T, Dims>& ma, unsigned /*version*/) 
            std::array<int, Dims> shape;
            ar & make_nvp("shape", shape);

            ma.resize(shape);
            ar & make_nvp("data", make_array(const_cast<T*>(ma.data()), ma.num_elements()));
        

    template <typename Ar, typename T, size_t Dims>
        void serialize(Ar& ar, boost::multi_array<T, Dims>& ma, unsigned version) 
            split_free(ar, ma, version);
        


auto make_multi_array() 
    boost::multi_array<int, 2> ma(boost::extents[7][4]);
    std::iota(ma.data(), ma.data() + ma.num_elements(), 10);
    return ma;


int main() 
    using range = boost::multi_array_types::index_range;
    auto original = make_multi_array();
    auto const SLICE = boost::indices[range(1,3)][range(1,3)];

    
        std::ofstream os("array.txt");
        boost::archive::text_oarchive oa(os);

        // only serialize initial array:
        oa << original;

        // modify through a sub-view
        auto sub = original[SLICE];
        original[2][2] = 999;

        // serialize the patch
        oa << sub;

        auto const& const_original = original;
        auto const_sub = const_original[SLICE];
        oa << const_sub; // compiles
    

    std::cout << std::ifstream("array.txt").rdbuf() << "\n";

    
        std::ifstream is("array.txt");
        boost::archive::text_iarchive ia(is);

        // only deserialize initial array:
        boost::multi_array<int, 2> restored;
        ia >> restored;

        assert(restored != original); // entry overwritten with 999

        assert(restored.num_elements() == original.num_elements());
        auto [o_2_2, r_2_2] = std::mismatch(
                original.data(), original.data()+original.num_elements(),
                restored.data());

        assert(*o_2_2 == 999);
        assert(*r_2_2 == 20);

        // now patch in the sub array at the same sub-view:
        auto sub = restored[SLICE];
        ia >> sub;

        // now everything is equal
        assert(restored == original); // now matches!
        assert(*r_2_2 == 999);

        auto const& const_restored = restored;
        auto const_sub = const_restored[SLICE];
        //ia >> const_sub; // WON'T COMPILE
    

【讨论】：

嗯。已经接受了？注意到我实际上并没有很好地阅读你的问题。你想序列化一个视图。你不能，真的。您不能“反序列化视图”。我的答案中的 UPDATE 部分添加了所有内容（带有一些解释）

（添加了关于视图界面效率和未来兼容性的注释）