以下过程在CentOS 7环境上验证通过。

一、下载安装包

https://github.com/greenplum-db/gpdb/releases/tag/6.9.0

二、官方文档

三、环境准备

假设在以下节点安装集群(万兆网),安装脚本在10.10.0.7节点上。

snippet.txt
10.10.0.4   # slaver
10.10.0.6   # slaver
10.10.0.7   # master, slaver

以下操作除特别说明外,均在10.10.0.7节点上进行。

配置ssh

所有节点ssh配置(/etc/ssh/sshd_config)修改以下项:

snippet.bash
UseDNS no
GSSAPIAuthentication no
MaxStartups 1000
MaxSessions 1000

注:修改后,需要重启sshd服务!

配置sysctl.conf

必须安装以上gp文档修改/etc/sysctl.conf,否则后续安装可能会出各种奇怪问题。

配置limits.conf

/etc/security/limits.conf调整以下配置:

snippet.txt
* soft nofile 65536
* hard nofile 65536
* soft nproc 131072
* hard nproc 131072

安装依赖库

snippet.bash
sudo yum install apr apr-util bash bzip2 curl krb5 libcurl libevent libxml2 libyaml zlib openldap openssh openssl openssl-libs perl readline rsync R sed tar zip

四、集群安装

创建用户

snippet.bash
[root@test-4 ~]# useradd gpadmin

sudo权限

所有节点gpadmin用户配置sudo权限:

snippet.bash
[root@test-4 ~]# vim /etc/sudoers
# 增加:
gpadmin ALL=(ALL)       NOPASSWD: ALL

配置hostname

所有节点配置hostname:

snippet.bash
[root@test-4 ~]# vim /etc/hosts
# 10 Gigabit Network
10.10.0.4 sc10g_4 gp10g_4
10.10.0.6 sc10g_6 gp10g_6
10.10.0.7 sc10g_7 gp10g_7

创建hostfile

snippet.bash
[gpadmin@test-4 ~]$ vim /home/gpadmin/hostfile
gp10g_4
gp10g_6
gp10g_7
 
# other节点指:除当前.7节点之外的host
[gpadmin@test-4 ~]$ vim other_hosts
gp10g_4
gp10g_6

准备路径

在安装rpm之前,务必准备好以下目录,安装程序不会自动创建这些目录。

snippet.bash
[root@test-4 ~]# mkdir -p /opt/gp/greenplum      # install path
[root@test-4 ~]# mkdir -p /opt/gp/gpmaster
[root@test-4 ~]# mkdir -p /opt/gp/gpdata
[root@test-4 ~]# chown -R gpadmin:gpadmin /opt/gp

查看安装文件

snippet.bash
[gpadmin@test-4 software]$ pwd
/opt/gp/software
[gpadmin@test-4 software]$ ll
total 80548
-rwxrwxrwx 1 gpadmin gpadmin 68501280 Jul 15 09:07 greenplum-db-6.9.0-rhel7-x86_64.rpm
-rwxrwxrwx 1 gpadmin gpadmin  6987179 Jul 15 09:07 Pivotal_Greenplum_6_9_Documentation.pdf

安装rpm

snippet.bash
[gpadmin@test-4 software]$ sudo rpm --install ./greenplum-db-6.9.0-rhel7-x86_64.rpm --prefix=/opt/gp/greenplum
[gpadmin@test-4 software]$ sudo chown -R gpadmin:gpadmin /opt/gp/greenplum
[gpadmin@test-4 software]$ source /opt/gp/greenplum/greenplum-db/greenplum_path.sh

配置互信

所有节点执行:

snippet.bash
[gpadmin@test-4 ~]$ ssh-keygen
[gpadmin@test-4 ~]$ ssh-copy-id gpadmin@gp10g_4
[gpadmin@test-4 ~]$ ssh-copy-id gpadmin@gp10g_6
[gpadmin@test-4 ~]$ ssh-copy-id gpadmin@gp10g_7

或者,在.7节点上执行:

snippet.bash
[gpadmin@test-4 ~]$ gpssh-exkeys -f ~/hostfile

验证互信:

snippet.bash
[gpadmin@test-4 ~]$ gpssh -f ~/hostfile
=> pwd
[gp10g_4] /home/gpadmin
[gp10g_7] /home/gpadmin
[gp10g_6] /home/gpadmin
=> exit

准备安装到其它节点

snippet.bash
# 打包
[gpadmin@test-4 ~]$ cd /opt/gp/greenplum/
[gpadmin@test-4 greenplum]$ tar -cf gp.tar greenplum-db-6.9.0/
 
# other节点创建目录
[gpadmin@test-4 greenplum]$ gpssh -f ~/other_hosts
=> mkdir -p /opt/gp/greenplum
[gp10g_4]
[gp10g_6]
=> mkdir -p /opt/gp/gpdata
[gp10g_4]
[gp10g_6]
 
# scp到other节点
[gpadmin@test-4 greenplum]$ gpscp -f ~/other_hosts gp.tar =:/opt/gp/greenplum/
 
# 解包
[gpadmin@test-4 greenplum]$ gpssh -f ~/other_hosts
=> cd /opt/gp/greenplum/
[gp10g_4]
[gp10g_6]
=> tar -xf gp.tar
[gp10g_4]
[gp10g_6]
=> ln -s ./greenplum-db-6.9.0 greenplum-db
[gp10g_4]
[gp10g_6]

配置环境变量

所有节点配置:

snippet.bash
[gpadmin@test-4 ~]$ vim ~/.bash_profile
# 添加
source /opt/gp/greenplum/greenplum-db/greenplum_path.sh
export MASTER_DATA_DIRECTORY=/opt/gp/gpmaster/gpseg-1
export GPPORT=7456

检查节点连通状况

snippet.bash
[gpadmin@test-4 ~]$ gpcheckperf -f ~/hostfile -r N -d ~/tmp
/opt/gp/greenplum/greenplum-db-6.9.0/bin/gpcheckperf -f ~/hostfile -r N -d /home/gpadmin/tmp
 
-------------------
--  NETPERF TEST
-------------------
……
====================
==  RESULT 2020-07-15T15:02:22.721745
====================
 
Netperf bisection bandwidth test
gp10g_4 -> gp10g_6 = 1084.200000
gp10g_7 -> gp10g_4 = 1109.960000
gp10g_6 -> gp10g_4 = 1118.340000
gp10g_4 -> gp10g_7 = 1113.190000
 
Summary:
sum = 4425.69 MB/sec
min = 1084.20 MB/sec
max = 1118.34 MB/sec
avg = 1106.42 MB/sec
median = 1113.19 MB/sec

gp配置文件

snippet.bash
[gpadmin@test-4 ~]$ cp /opt/gp/greenplum/greenplum-db/docs/cli_help/gpconfigs/gpinitsystem_config ~/
[gpadmin@test-4 ~]$ vim ~/gpinitsystem_config
# 修改以下配置:
declare -a DATA_DIRECTORY=(/opt/gp/gpdata /opt/gp/gpdata /opt/gp/gpdata /opt/gp/gpdata
/opt/gp/gpdata /opt/gp/gpdata /opt/gp/gpdata /opt/gp/gpdata
/opt/gp/gpdata /opt/gp/gpdata /opt/gp/gpdata /opt/gp/gpdata
/opt/gp/gpdata /opt/gp/gpdata /opt/gp/gpdata /opt/gp/gpdata)
 
MASTER_HOSTNAME=gp10g_7
MASTER_DIRECTORY=/opt/gp/gpmaster
MASTER_PORT=7456
PORT_BASE=6345
MACHINE_LIST_FILE=/home/gpadmin/hostfile

注:

gpinitsystem

snippet.bash
[gpadmin@test-4 ~]$ gpinitsystem -c ~/gpinitsystem_config

大部分的错误发生在此步,错误类型也是千差万别,如果无法找到问题,可以复查第二,三章参数配置是否正确。

此步失败后,在重试之前,请在所有节点使用gpadmin用户执行以下操作:

参数-m:

当单个节点的segment数较多时(相对于CPU核数而言),gpinitsystem有时会莫名其妙地失败,此时建议降低-m参数值试试,例如:

snippet.bash
[gpadmin@test-4 ~]$ gpinitsystem -m 25 -c ~/gpinitsystem_config

参数-D:

如果安装失败,但是从log上有看不出什么问题,建议再重试时,使用-D参数,这样可以获得更加详细的log信息,例如:

snippet.bash
[gpadmin@test-4 ~]$ gpinitsystem -D -m 25 -c ~/gpinitsystem_config

五、GP集群使用基础

GreenPlum集群使用基础