在 PostgreSQL C 扩展中包含头文件
Posted
技术标签:
【中文标题】在 PostgreSQL C 扩展中包含头文件【英文标题】:Including header files in a PostgreSQL C extension 【发布时间】:2018-08-18 07:19:10 【问题描述】:我在 seal_diff_cpp.cpp
文件中有我的 C 扩展的代码:
extern "C" // C Headers must be inside exter "C" block.
#include <postgres.h>
#include <utils/rel.h>
#include <fmgr.h>
#include <utils/array.h>
#include <utils/builtins.h>
#include <catalog/pg_type.h>
#include <stdlib.h>
#include <stdint.h>
PG_MODULE_MAGIC;
// CPP Header must be outside extern "C" block.
#include <string>
#include <vector>
#include <iostream>
#include <fstream>
#include <sstream>
#include <iterator> // For the ostream_iterator
// External projects c++ libraries compiled and linked on running 'make'.
#include <seal/seal.h>
#include <cppcodec/base64_rfc4648.hpp>
// CPP function can be declared and used inside or outside extern "C" block.
std::stringstream dec(std::string st)
// Decode the base64 string into a stringstream
auto decodeBase64 = cppcodec::base64_rfc4648::decode(st);
std::stringstream decodeBase64SS;
std::move(decodeBase64.begin(), decodeBase64.end(), std::ostream_iterator<unsigned char>(decodeBase64SS));
return decodeBase64SS;
std::string enc(std::string st)
// Create a vector to hold the raw data
std::vector<uint8_t> encodeStream;
// Push all the characters from the raw data string into the vector
for (auto &ch : st)
encodeStream.push_back((unsigned char&&)(ch));
// Encode the vector as base64 string
std::string encodeBase64 = cppcodec::base64_rfc4648::encode(encodeStream);
encodeStream.clear();
return encodeBase64;
std::string seal_diff_operation(std::string decodedLocalEncParamTmp, std::string decodedLocalTmp, std::string decodedLocalTmp)
std::stringstream decodedLocalEncParam;
decodedLocalEncParam.str(decodedLocalEncParamTmp);
std::stringstream decodedLocalT1;
decodedLocalT1.str(decodedLocalTmp);
std::stringstream decodedLocalT2;
decodedLocalT2.str(decodedLocalTmp);
// Execute seal library operations
// Load the ecryption parameters
seal::EncryptionParameters IntegerEncryptorParms;
IntegerEncryptorParms.load(decodedLocalEncParam);
// Set Context and evaluator objects
seal::SEALContext context(IntegerEncryptorParms);
seal::Evaluator evaluator(context);
// Set the Encoder parameters
seal::IntegerEncoder encoder(context.plain_modulus());
// Create Ciphertexts and load Chipertext information into them
seal::Ciphertext number1Encoded;
seal::Ciphertext number2Encoded;
seal::Ciphertext diffEncodedResult;
number1Encoded.load(decodedLocalT1);
number2Encoded.load(decodedLocalT2);
// Do the diff operation on the Ciphertexts and prepare the result for output
evaluator.sub(number1Encoded, number2Encoded, diffEncodedResult);
std::stringstream encResult;
diffEncodedResult.save(encResult);
std::string output = enc(encResult.str());
return output;
extern "C" // PostgreSQL functions be inside extern "C" block.
Datum seal_diff_cpp(PG_FUNCTION_ARGS)
// Get the Parameters
text *t1 = PG_GETARG_TEXT_PP(0);
text *t2 = PG_GETARG_TEXT_PP(1);
text *encParam = PG_GETARG_TEXT_PP(2);
std::string localT1;
std::string localT2;
std::string localEncParam;
localT1 = text_to_cstring(t1);
localT2 = text_to_cstring(t2);
localEncParam = text_to_cstring(encParam);
// Decode the parameters
std::stringstream decodedLocalT1 = dec(localT1);
std::stringstream decodedLocalT2 = dec(localT2);
std::stringstream decodedLocalEncParam = dec(localEncParam);
// Encode the parameters
std::string encodedLocalT1 = enc(decodedLocalT1.str());
//std::string encodedLocalT1 = enc(encResult.str());
std::string encodedLocalT2 = enc(decodedLocalT2.str());
//std::string outputParam = seal_diff_operation(decodedLocalEncParam.str(), decodedLocalT1.str(), decodedLocalT2.str());
// Returns Text
PG_RETURN_TEXT_P(cstring_to_text_with_len(localT1.c_str(), localT1.size()));
;
PG_FUNCTION_INFO_V1(seal_diff_cpp);
我使用这个又快又脏的Makefile
来创建我的.so
文件:
MODULES = seal_diff_cpp
PG_CONFIG = /usr/pgsql-10/bin/pg_config
PGXS = $(shell $(PG_CONFIG) --pgxs)
INCLUDEDIR = $(shell $(PG_CONFIG) --includedir-server)
INCLUDE_SEAL = /usr/local/include/seal
INCLUDE_SEAL_LIB = /usr/local/lib
INCLUDE_CPPCODEC = /usr/local/include/cppcodec
CXX = g++
CXXFLAGS = -std=c++17 -fPIC -Wall -Werror -g -O0 -pthread \
-I$(INCLUDEDIR) -L$(INCLUDE_SEAL_LIB) -l libseal.a -I$(INCLUDE_SEAL) -I$(INCLUDE_CPPCODEC)
include $(PGXS)
seal_diff_cpp.so: seal_diff_cpp.o
$(CXX) -shared -o seal_diff_cpp.so seal_diff_cpp.o
seal_diff_cpp.o: seal_diff_cpp.cpp
$(CXX) $(CXXFLAGS) -o seal_diff_cpp.o -c seal_diff_cpp.cpp
当我尝试将我的 C 扩展名复制到 /usr/pgsql-10/lib/ 文件夹后添加
-- Drop the function first, needed when changing return type
DROP FUNCTION IF EXISTS public.seal_diff_cpp(CHARACTER VARYING, CHARACTER VARYING, CHARACTER VARYING);
-- Create the new function
CREATE OR REPLACE FUNCTION
seal_diff_cpp(CHARACTER VARYING, CHARACTER VARYING, CHARACTER VARYING) RETURNS CHARACTER VARYING AS 'seal_diff_cpp'
LANGUAGE C STRICT;
我得到错误:
错误:无法加载库“/usr/pgsql-10/lib/seal_diff_cpp.so”: /usr/pgsql-10/lib/seal_diff_cpp.so:未定义符号: _ZN4seal9Evaluator3subERNS_10CiphertextERKS1_SQL状态:XX000
这个_ZN4seal9Evaluator3subERNS_10CiphertextERKS1_
是因为使用了seal::<some parameter>
而创建的,#include <seal/seal.h>
在那里所以不会发生这样的错误。 SEAL 库是安装到/usr/local/include/seal
的外部库,它的libseal.a
安装到/usr/local/lib/
。
seal::
对象的第一个声明发生错误,我没有任何问题/错误使用我的 Makefile
将其编译到所需的 seal_diff_cpp.so
文件中。当我注释所有 SEAL 代码时,扩展程序可以与 /usr/local/include/cppcodec
下的第二个外部库 (#include <cppcodec/base64_rfc4648.hpp>
) 完美运行。
我在.cpp
文件或Makefile
中的印章库有什么问题吗?我是否需要将外部SEAL
和cppcodec
库安装到/usr/pgsql-10/include/
或其他pgsql-10
路径中?感觉#include <seal/seal.h>
不行了。
更新
按照 Alan Britles 的建议更改我的 Makefile
并运行 make
后,我得到了这个输出,显示链接没有错误:
g++ -Wl,--no-undefined -std=c++17 -fPIC -Wall -Werror -g -O0 -pthread -I/usr/pgsql-10/include/server -I/usr/local/include/seal -I/usr/local/include/cppcodec -o seal_diff_cpp.o -c seal_diff_cpp.cpp g++ -L/usr/pgsql- 10/lib -L/usr/lib64 -Wl,--按需 -Wl,-rpath,'/usr/pgsql-10/lib',--enable-new-dtags -shared -o seal_diff_cpp.so seal_diff_cpp.o
使用nm -gC seal_diff_cpp.so | grep "seal"
:我得到这个输出:
000000000000e641 T pg_finfo_seal_diff_cpp
000000000000e26c T seal_diff_cpp
000000000000de73 T seal_diff_operation(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)
U seal::Ciphertext::load(std::istream&)
U seal::Ciphertext::operator=(seal::Ciphertext const&)
000000000000f3b0 W seal::Ciphertext::Ciphertext()
000000000000f3b0 W seal::Ciphertext::Ciphertext()
000000000000f42c W seal::Ciphertext::~Ciphertext()
000000000000f42c W seal::Ciphertext::~Ciphertext()
U seal::SEALContext::SEALContext(seal::EncryptionParameters const&)
000000000000f1e6 W seal::SEALContext::~SEALContext()
000000000000f1e6 W seal::SEALContext::~SEALContext()
U seal::IntegerEncoder::IntegerEncoder(seal::SmallModulus const&, unsigned long)
U seal::IntegerEncoder::~IntegerEncoder()
000000000000eb80 W seal::MemoryPoolHandle::MemoryPoolHandle()
000000000000eb80 W seal::MemoryPoolHandle::MemoryPoolHandle()
000000000000eabe W seal::MemoryPoolHandle::~MemoryPoolHandle()
000000000000eabe W seal::MemoryPoolHandle::~MemoryPoolHandle()
U seal::EncryptionParameters::load(std::istream&)
U seal::EncryptionParameters::EncryptionParameters()
000000000000ebd2 W seal::EncryptionParameters::~EncryptionParameters()
000000000000ebd2 W seal::EncryptionParameters::~EncryptionParameters()
000000000000effc W seal::util::BaseConverter::~BaseConverter()
000000000000effc W seal::util::BaseConverter::~BaseConverter()
000000000000f166 W seal::util::SmallNTTTables::~SmallNTTTables()
000000000000f166 W seal::util::SmallNTTTables::~SmallNTTTables()
000000000000eba0 W seal::util::Modulus::~Modulus()
000000000000eba0 W seal::util::Modulus::~Modulus()
000000000000e9bc W seal::util::Pointer::release()
000000000000e986 W seal::util::Pointer::Pointer()
000000000000e986 W seal::util::Pointer::Pointer()
000000000000ea66 W seal::util::Pointer::~Pointer()
000000000000ea66 W seal::util::Pointer::~Pointer()
U seal::BigPoly::~BigPoly()
U seal::BigUInt::~BigUInt()
000000000000ec1c W seal::Evaluator::sub(seal::Ciphertext const&, seal::Ciphertext const&, seal::Ciphertext&)
U seal::Evaluator::sub(seal::Ciphertext&, seal::Ciphertext const&)
U seal::Evaluator::Evaluator(seal::SEALContext const&)
000000000000f276 W seal::Evaluator::~Evaluator()
000000000000f276 W seal::Evaluator::~Evaluator()
0000000000011c16 W __gnu_cxx::new_allocator<seal::SmallModulus>::deallocate(seal::SmallModulus*, unsigned long)
0000000000010afe W __gnu_cxx::new_allocator<seal::SmallModulus>::~new_allocator()
0000000000010afe W __gnu_cxx::new_allocator<seal::SmallModulus>::~new_allocator()
0000000000013e54 W __gnu_cxx::new_allocator<seal::util::SmallNTTTables>::deallocate(seal::util::SmallNTTTables*, unsigned long)
000000000001383a W __gnu_cxx::new_allocator<seal::util::SmallNTTTables>::~new_allocator()
000000000001383a W __gnu_cxx::new_allocator<seal::util::SmallNTTTables>::~new_allocator()
U seal::Ciphertext::save(std::ostream&) const
000000000000ebfe W seal::SEALContext::plain_modulus() const
000000000000ebc0 W seal::EncryptionParameters::plain_modulus() const
000000000000ff92 W std::allocator<seal::SmallModulus>::~allocator()
000000000000ff92 W std::allocator<seal::SmallModulus>::~allocator()
00000000000114fe W std::allocator<seal::util::SmallNTTTables>::~allocator()
00000000000114fe W std::allocator<seal::util::SmallNTTTables>::~allocator()
000000000000eb20 W std::shared_ptr<seal::util::MemoryPool>::shared_ptr(decltype(nullptr))
000000000000eb04 W std::shared_ptr<seal::util::MemoryPool>::shared_ptr()
000000000000eaa2 W std::shared_ptr<seal::util::MemoryPool>::~shared_ptr()
000000000000eaa2 W std::shared_ptr<seal::util::MemoryPool>::~shared_ptr()
0000000000013881 W void std::_Destroy_aux<false>::__destroy<seal::util::SmallNTTTables*>(seal::util::SmallNTTTables*, seal::util::SmallNTTTables*)
0000000000011c06 W void std::_Destroy_aux<true>::__destroy<seal::SmallModulus*>(seal::SmallModulus*, seal::SmallModulus*)
000000000000eb40 W std::__shared_ptr<seal::util::MemoryPool, (__gnu_cxx::_Lock_policy)2>::__shared_ptr()
000000000000ea82 W std::__shared_ptr<seal::util::MemoryPool, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr()
000000000000ea82 W std::__shared_ptr<seal::util::MemoryPool, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr()
000000000000ffae W std::_Vector_base<seal::SmallModulus, std::allocator<seal::SmallModulus> >::_Vector_impl::~_Vector_impl()
000000000000ffae W std::_Vector_base<seal::SmallModulus, std::allocator<seal::SmallModulus> >::_Vector_impl::~_Vector_impl()
0000000000010060 W std::_Vector_base<seal::SmallModulus, std::allocator<seal::SmallModulus> >::_M_deallocate(seal::SmallModulus*, unsigned long)
0000000000010028 W std::_Vector_base<seal::SmallModulus, std::allocator<seal::SmallModulus> >::_M_get_Tp_allocator()
000000000000ffca W std::_Vector_base<seal::SmallModulus, std::allocator<seal::SmallModulus> >::~_Vector_base()
000000000000ffca W std::_Vector_base<seal::SmallModulus, std::allocator<seal::SmallModulus> >::~_Vector_base()
000000000001073a W std::_Vector_base<seal::util::SmallNTTTables, std::allocator<seal::util::SmallNTTTables> >::_Vector_impl::~_Vector_impl()
000000000001073a W std::_Vector_base<seal::util::SmallNTTTables, std::allocator<seal::util::SmallNTTTables> >::_Vector_impl::~_Vector_impl()
000000000001151a W std::_Vector_base<seal::util::SmallNTTTables, std::allocator<seal::util::SmallNTTTables> >::_M_deallocate(seal::util::SmallNTTTables*, unsigned long)
00000000000107b4 W std::_Vector_base<seal::util::SmallNTTTables, std::allocator<seal::util::SmallNTTTables> >::_M_get_Tp_allocator()
0000000000010756 W std::_Vector_base<seal::util::SmallNTTTables, std::allocator<seal::util::SmallNTTTables> >::~_Vector_base()
0000000000010756 W std::_Vector_base<seal::util::SmallNTTTables, std::allocator<seal::util::SmallNTTTables> >::~_Vector_base()
0000000000010b2f W std::allocator_traits<std::allocator<seal::SmallModulus> >::deallocate(std::allocator<seal::SmallModulus>&, seal::SmallModulus*, unsigned long)
0000000000013845 W std::allocator_traits<std::allocator<seal::util::SmallNTTTables> >::deallocate(std::allocator<seal::util::SmallNTTTables>&, seal::util::SmallNTTTables*, unsigned long)
000000000000f574 W std::vector<seal::SmallModulus, std::allocator<seal::SmallModulus> >::~vector()
000000000000f574 W std::vector<seal::SmallModulus, std::allocator<seal::SmallModulus> >::~vector()
000000000000fbbe W std::vector<seal::util::SmallNTTTables, std::allocator<seal::util::SmallNTTTables> >::~vector()
000000000000fbbe W std::vector<seal::util::SmallNTTTables, std::allocator<seal::util::SmallNTTTables> >::~vector()
0000000000013873 W seal::util::SmallNTTTables* std::__addressof<seal::util::SmallNTTTables>(seal::util::SmallNTTTables&)
0000000000013e76 W void std::_Destroy<seal::util::SmallNTTTables>(seal::util::SmallNTTTables*)
0000000000010b09 W void std::_Destroy<seal::SmallModulus*>(seal::SmallModulus*, seal::SmallModulus*)
0000000000010036 W void std::_Destroy<seal::SmallModulus*, seal::SmallModulus>(seal::SmallModulus*, seal::SmallModulus*, std::allocator<seal::SmallModulus>&)
000000000001154f W void std::_Destroy<seal::util::SmallNTTTables*>(seal::util::SmallNTTTables*, seal::util::SmallNTTTables*)
00000000000107c2 W void std::_Destroy<seal::util::SmallNTTTables*, seal::util::SmallNTTTables>(seal::util::SmallNTTTables*, seal::util::SmallNTTTables*, std::allocator<seal::util::SmallNTTTables>&)
所以我猜 SEAL 库已正确链接,但我确实有一些重要的符号带有未定义 (U
),因此我仍然从 PostgreSQL 收到相同的错误。但是为什么这些符号被标记为未定义?
【问题讨论】:
尝试在你的makefile中使用这个CXXFLAGS = -std=c++17 -fPIC -Wall -Werror -g -O0 -pthread \ -I$(INCLUDEDIR) -L$(INCLUDE_SEAL_LIB) -lseal -I$(INCLUDE_SEAL) -I$(INCLUDE_CPPCODEC)
,并使用objdump检查符号是否在libseal.a和so
文件中
尝试将--no-undefined
添加到您的链接命令中,这将使 GCC 在链接时报告未定义的符号,而不是让它们在运行时潜入您
完成后,将库链接器标志从编译命令移动到链接命令
@leiyc 我已经更新了我的帖子,看起来 SEAL 库链接正确。
@leiyc 我确实有一些带有U
标志的符号。如果我没记错的话,这意味着它们是未定义的。
【参考方案1】:
GCC 有一个在链接时报告未定义符号的选项,默认情况下共享库是关闭的。通过--no-undefined
开关打开它。当存在未定义的符号而不是在运行时报告它们时,这将使链接器失败。
您的问题是您将库链接命令传递给编译命令(它们被忽略)而不是链接命令。尝试按如下方式修改您的 make 文件:
CXXFLAGS = -std=c++17 -fPIC -Wall -Werror -g -O0 -pthread \
-I$(INCLUDEDIR) -I$(INCLUDE_SEAL) -I$(INCLUDE_CPPCODEC)
LDFLAGS = -L$(INCLUDE_SEAL_LIB) -l libseal.a -pthread
seal_diff_cpp.so: seal_diff_cpp.o
$(CXX) $(LDFLAGS) -shared -o seal_diff_cpp.so seal_diff_cpp.o
seal_diff_cpp.o: seal_diff_cpp.cpp
$(CXX) $(CXXFLAGS) -o seal_diff_cpp.o -c seal_diff_cpp.cpp
注意:不将-pthread
传递给链接器也可能会导致一些微妙的问题。
【讨论】:
我已添加将最后一行更改为$(CXX) -Wl,--no-undefined $(CXXFLAGS) -o seal_diff_cpp.o -c seal_diff_cpp.cpp
,使用make
创建此输出g++ -Wl,--no-undefined -std=c++17 -fPIC -Wall -Werror -g -O0 -pthread -I/usr/pgsql-10/include/server -I/usr/local/include/seal -I/usr/local/include/cppcodec -o seal_diff_cpp.o -c seal_diff_cpp.cpp g++ -L/usr/pgsql-10/lib -L/usr/lib64 -Wl,--as-needed -Wl,-rpath,'/usr/pgsql-10/lib',--enable-new-dtags -shared -o seal_diff_cpp.so seal_diff_cpp.o
@TalG 您需要将此标志添加到链接器命令而不是编译器命令
@AlanBritles,谢谢,现在我得到了错误:)。但我仍然不知道我为什么要得到它们。我已经链接并包含了所有内容。
您是否按照我的建议将-l libseal.a
移至您的链接器命令?
@AlanBritles,我在我的Makefile
中使用你的版本@以上是关于在 PostgreSQL C 扩展中包含头文件的主要内容,如果未能解决你的问题,请参考以下文章
如何在 Qt Creator 的 CMakeLists.txt 中包含头文件?