Skip to content

Commit

Permalink
fix: dlopen error, install udf dir and doc (#3326)
Browse files Browse the repository at this point in the history
  • Loading branch information
vagetablechicken authored Jun 27, 2023
1 parent 82d1929 commit 53724ed
Show file tree
Hide file tree
Showing 5 changed files with 24 additions and 8 deletions.
1 change: 1 addition & 0 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -328,3 +328,4 @@ install(
DESTINATION tools
PERMISSIONS OWNER_EXECUTE OWNER_WRITE OWNER_READ GROUP_EXECUTE GROUP_READ WORLD_EXECUTE WORLD_READ
)
install(DIRECTORY DESTINATION udf)
13 changes: 12 additions & 1 deletion docs/zh/deploy/install_deploy.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,18 @@

### 操作系统

发行的预编译包支持:CentOS 7.x, Ubuntu 20.04, SUSE 12 SP3, macOS 12。对于其他操作系统发行版本,预编译包未做充分测试,无法保证其完全兼容。你可以尝试 [从源码编译](compile.md),来支持其他的操作系统。
发行的预编译包支持:CentOS 7.x, Ubuntu 20.04, SUSE 12 SP3, macOS 12。因此Linux要求glibc version >= 2.17。对于其他操作系统发行版本,预编译包未做充分测试,无法保证其完全兼容。你可以尝试 [从源码编译](compile.md),来支持其他的操作系统。

```{note}
Linux可通过以下命令来检查系统的支持情况。
````shell
cat /etc/os-release # most linux
cat /etc/redhat-release # redhat only
ldd --version
strings /lib64/libc.so.6 | grep ^GLIBC_
````
通常ldd版本>=2.17,libc.so.6中也会有`GLIBC_2.17`,也就是该系统支持glibc 2.17的程序/动态库运行。如果系统的glibc版本低于2.17,则需要尝试从源码编译。
```

### 第三方组件依赖

Expand Down
6 changes: 5 additions & 1 deletion docs/zh/openmldb_sql/ddl/CREATE_FUNCTION.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,4 +53,8 @@ CREATE AGGREGATE FUNCTION special_sum(x BIGINT) RETURNS BIGINT OPTIONS (FILE = '
创建一个输入参数是bigint并且支持null, 返回类型是bigint并且支持返回null聚合函数,动态库文件为libtest_udf.so
```sql
CREATE AGGREGATE FUNCTION count_null(x BIGINT) RETURNS BIGINT OPTIONS (FILE = 'libtest_udf.so', ARG_NULLABLE=true, RETURN_NULLABLE=true);
```
```

```{warning}
如果集群版本<=0.8.0,TabletServer部署目录中默认没有`udf`目录,启动TabletServer后再创建目录并拷贝udf动态库,将无法生效(环境变量问题)。需保证TabletServer启动前存在udf目录,如果已启动,需要将所有TabletServer重启使环境变量生效。
```
2 changes: 1 addition & 1 deletion docs/zh/openmldb_sql/udf_develop_guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -153,7 +153,7 @@ g++ -shared -o libtest_udf.so examples/test_udf.cc -I /work/OpenMLDB/include -st
```

### 2.3 拷贝动态库
编译过的动态库需要被拷贝到 TaskManager 和 tablets中。如果 TaskManager 和 tablets中不存在`udf`目录,请先创建
编译过的动态库需要被拷贝到 TaskManager 和 tablets中。如果 TaskManager 和 tablets中不存在`udf`目录,请先创建并重启这些进程(保证环境变量生效)
- tablet的UDF目录是 `path_to_tablet/udf`
- TaskManager的UDF目录是 `path_to_taskmanager/taskmanager/bin/udf`

Expand Down
10 changes: 5 additions & 5 deletions hybridse/src/udf/dynamic_lib_manager.cc
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,8 @@ DynamicLibManager::~DynamicLibManager() {
handle_map_.clear();
}

base::Status DynamicLibManager::ExtractFunction(const std::string& name, bool is_aggregate,
const std::string& file, std::vector<void*>* funs) {
base::Status DynamicLibManager::ExtractFunction(const std::string& name, bool is_aggregate, const std::string& file,
std::vector<void*>* funs) {
CHECK_TRUE(funs != nullptr, common::kExternalUDFError, "funs is nullptr")
std::shared_ptr<DynamicLibHandle> so_handle;
{
Expand All @@ -45,7 +45,8 @@ base::Status DynamicLibManager::ExtractFunction(const std::string& name, bool is
}
if (!so_handle) {
void* handle = dlopen(file.c_str(), RTLD_LAZY);
CHECK_TRUE(handle != nullptr, common::kExternalUDFError, "can not open the dynamic library: " + file)
CHECK_TRUE(handle != nullptr, common::kExternalUDFError,
"can not open the dynamic library: " + file + ", error: " + dlerror())
so_handle = std::make_shared<DynamicLibHandle>(handle);
std::lock_guard<std::mutex> lock(mu_);
handle_map_.emplace(file, so_handle);
Expand Down Expand Up @@ -96,8 +97,7 @@ base::Status DynamicLibManager::RemoveHandler(const std::string& file) {
}
}
if (so_handle) {
CHECK_TRUE(dlclose(so_handle->handle) == 0, common::kExternalUDFError,
"dlclose run error. file is " + file)
CHECK_TRUE(dlclose(so_handle->handle) == 0, common::kExternalUDFError, "dlclose run error. file is " + file)
}
return {};
}
Expand Down

0 comments on commit 53724ed

Please sign in to comment.