测试环境：Redhat 7 源码：gpdb-master（2020.08.20） gpdb-6X_STABLE 源码编译安装请参考：[Greenplum 6.X 从源码快速搭建demo环境](Greenplum 6.X 从源码快速搭建demo环境) # 一、下载源码 [https://github.com/greenplum-db/gpdb](https://github.com/greenplum-db/gpdb) # 二、依赖 假设：源码路径：/opt/gpdb/，bin安装到：/opt/gp_bin/。 ## 依赖库 ```bash sudo yum install apr apr-util bash bzip2 curl krb5 libcurl libevent libxml2 libyaml zlib openldap openssh openssl openssl-libs perl readline rsync R sed tar zip ``` ## devel依赖库 ```bash libzstd、libzstd-devel、apr-devel、libcurl-devel、bzip2-devel、xerces-c-devel ``` # 三、升级gcc gpdb master目前要求gcc支持C++14，但是Redhat 7自带的gcc版本是4.8，尚不支持”C++14“，需要至少升级到gcc 4.9.0。 如果不升级gcc，编译阶段会报以下错误： ```bash checking whether g++ supports C++14 features with -std=c++14... no checking whether g++ supports C++14 features with +std=c++14... no checking whether g++ supports C++14 features with -h std=c++14... no checking whether g++ supports C++14 features with -std=c++1y... no checking whether g++ supports C++14 features with +std=c++1y... no checking whether g++ supports C++14 features with -h std=c++1y... no configure: error: *** A compiler with support for C++14 language features is required. ``` 这里直接升级到gcc 9.3，参考：[CentOS 7 升级gcc至9.3](../../../Coding/编译/CentOS 7 升级gcc至9.3) # 四、安装pygresql5 gpdb-master 已经将PyGreSQL 从4.0.0 升级至 5.1.2，需要通过以下方式升级pygresql： ```bash pip install pygresql ``` 如果不升级，在后续的”make create-demo-cluster”阶段会报以下错误： ```bash Error: unable to import module: No module named pg ``` 参考：[Update PyGreSQL from 4.0.0 to 5.1.2](https://github.com/greenplum-db/gpdb/commit/f5758021be347aea069c3ef930bae6354a2dd8db) # 五、配置demo 该demo会在本地服务器，部署GP集群，配置包括： - 3个segment - 3个mirror - 数据目录在：/opt/gpdb/gpAux/gpdemo/datadirs/ 如果需要定制demo集群，则需要修改：gpAux/gpdemo/Makefile。包括：端口号、segment个数、是否有mirror等。 ```makefile PORT_BASE ?= 6000 NUM_PRIMARY_MIRROR_PAIRS ?= 3 BLDWRAP_POSTGRES_CONF_ADDONS := fsync=off $(BLDWRAP_POSTGRES_CONF_ADDONS) ifeq ($(WITH_MIRRORS), ) WITH_MIRRORS = true endif ifeq ($(WITH_MIRRORS), true) WITH_STANDBY = true endif ``` ## 修改demo默认端口号 默认的端口号6000与Xserver端口冲突，如果不修改，在有GUI的Linux环境中，几乎总会端口冲突。 ```bash vim gpAux/gpdemo/Makefile # PORT_BASE 修改为不会冲突的端口号 PORT_BASE ?= 6000 ``` ## 编译 ```bash cd /opt/gpdb/ # --prefix 指定安装路径 ./configure --with-perl --with-python --with-libxml --with-gssapi --prefix=/opt/gp_bin # 以上默认是release版本，如果需要debug，则： CFLAGS="-O0 -g3 -ggdb3" ./configure --with-perl --with-python --with-libxml --with-gssapi --enable-debug --enable-debug-extensions --prefix=/opt/gp_bin # 编译安装 make -j8 make install ``` 注：编译debug版本时，如果不指定CFLAGS，仅使用enable-debug参数，编译出来的查询是加了优化参数的，不便于debug。 ## make clean 如果需要重新编译，重新编译前： ```bash make distclean ``` 参考：[「实战系列」Greenplum 编译、安装、调试](https://blog.csdn.net/gp_community/article/details/106237585) ## 为本地用户配置互信 ```bash [gpadmin@test-4 ~]$ ssh-keygen [gpadmin@test-4 ~]$ ssh-copy-id gpadmin@localhost ``` # 六、部署demo ```bash # GP环境变量 source /opt/gp_bin/greenplum_path.sh # 部署demo集群并启动服务 cd /opt/gpdb/ make create-demo-cluster # 集群demo环境变量 source gpAux/gpdemo/gpdemo-env.sh ``` ## 问题1：No psutil make create-demo-cluster阶段，如果报告以下错误： ```txt ImportError: No module named psutil ``` 则，需要安装python模块： ```bash [gpadmin@bogon ~]$ curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py [gpadmin@bogon ~]$ python get-pip.py [gpadmin@bogon ~]$ pip install --upgrade psutil==5.7.0 ``` 注意：因为GP自带了Python，当“source greenplum\_path.sh”后，实际上使用的是GP的Python的，需要在编译环境中安装psutil。 ```bash [gpadmin@bogon gpdb]$ which python /opt/gp/greenplum/greenplum-db-6.9.0/ext/python/bin/python ``` ## 问题2：ping host FAILED 安装失败，报告以下warning： ```txt 20200601:11:06:44:029396 gpstart:bogon:gpadmin-[WARNING]:-Ping to host: 'bogon' FAILED 20200601:11:06:44:029396 gpstart:bogon:gpadmin-[WARNING]:-Skipping startup of segdb on bogon directory /opt/software/gpdb-6X_STABLE/gpAux/gpdemo/datadirs/dbfast1/demoDataDir0 Ping Failed <<<<<< ``` 怀疑GP脚本使用了ping6而不是ping，可以通过以下方法鉴别： ```bash [gpadmin@bogon gpdb]$ cd /opt/gpdb/gpAux/gpdemo/ [gpadmin@bogon gpdb]$ rm -rf datadirs/* [gpadmin@bogon gpdb]$ gpinitsystem -D -a -c clusterConfigFile "" …… 20200601:11:11:35:033652 gpstart:bogon:gpadmin-[DEBUG]:-Running Command: /bin/ping6 -c 1 bogon 20200601:11:11:35:033652 gpstart:bogon:gpadmin-[DEBUG]:-[worker0] finished cmd: ping cmdStr='/bin/ping6 -c 1 bogon' had result: cmd had rc=2 completed=True halted=False ``` 可以修改以下脚本，然后重新make install： ```python [gpadmin@bogon gpdb]$ vim gpMgmt/bin/gppylib/commands/unix.py 296 def run(self, validateAfter=False): 297 if curr_platform == LINUX or curr_platform == DARWIN or curr_platform == OPENBSD: 298 # Get the family of the address we need to ping. If it's AF_INET6 299 # we must use ping6 to ping it. 300 301 try: 302 # addrinfo = socket.getaddrinfo(self.hostToPing, None) 303 addrinfo = socket.getaddrinfo(self.hostToPing, None, socket.AF_INET) # 修改后 304 if addrinfo and addrinfo[0] and addrinfo[0][0] == socket.AF_INET6: ``` 该问题可以参考：[Missing steps for first installation #171](https://github.com/greenplum-db/gpdb/issues/171) ## 问题3：Failed to load GLFW3 shared library ```txt 20210101:15:07:33:2262125 gpinitsystem:test-4:yz-[INFO]:-Restarting the Greenplum instance in production mode Error: unable to import module: Failed to load GLFW3 shared library. 20210101:15:07:34:2262125 gpinitsystem:test-4:yz-[FATAL]:-Error from Greenplum instance shutdown, check log files Script Exiting! ``` 原因是安装python3环境安装了pg模块，需要卸载pg模块，使用pygresql5代替之： ```bash sudo pip3 uninstall pg ``` 该问题可以参考：[Import error when trying to load import "pg" module in python3](https://bugzilla.redhat.com/show_bug.cgi?id=1503042)