Hadoop 分布式集群实战 2—— Hadoop 高可用架构

1 HDFS‑HA 核心组件2 台 NameNodeActive NameNode对外处理客户端读写唯一写元数据Standby NameNode热备不接收客户端请求实时同步元数据故障时升级为 ActiveJournalNode 集群JN奇数3 台起步共享存储保存edits编辑日志。Active 把修改写入 JN 集群Standby 持续读取 JN 的 edits在内存回放保持元数据同步腾讯云。Quorum 机制写入超过半数 JN 成功才算写成功容忍半数以下节点故障。ZooKeeper 集群奇数负责选主、故障检测保存分布式锁防止脑裂同一时刻只允许一个 Active。ZKFCZKFailoverController每个 NameNode 节点上都要跑一个 ZKFC 进程三个职责①健康检测监控本机 NameNode 是否存活②和 ZK 交互抢锁竞选 Active③故障隔离 (fence)旧 Active 故障卡死SSH 远程杀死旧 NN 进程防止脑裂。DataNode同时向两台 NameNode 汇报块信息不管谁是 Active 都上报。IP主机名角色192.168.40.141server1NameNode / DFSZKFailoverController / ResourceManager192.168.40.142server2JournalNode / QuorumPeerMain / DataNode / NodeManager192.168.40.143server3JournalNode / QuorumPeerMain / DataNode / NodeManager192.168.40.144server4JournalNode / QuorumPeerMain / DataNode / NodeManager192.168.40.145server5NameNode / DFSZKFailoverController / ResourceManager2 部署2.1 虚拟机准备新开虚拟机server5[rootserver5 ~]# useradd hadoop# 还是要保证 uid 和 gid 与其他节点相同[rootserver5 ~]# yum install -y nfs-utils[rootserver5 ~]# mount 192.168.40.141:/home/hadoop/ /home/hadoop/2.2 ZooKeeper 集群部署上传安装包并解压[hadoopserver1 hadoop]$ sbin/stop-all.sh[hadoopserver1 hadoop]$cd[hadoopserver1 ~]$tarzxf apache-zookeeper-3.8.6-bin.tar.gz集群配置[hadoopserver2 ~]$cdapache-zookeeper-3.8.6-bin/conf/[hadoopserver2 conf]$cpzoo_sample.cfg zoo.cfg[hadoopserver2 conf]$vimzoo.cfg各节点配置文件相同此处的server.1并不是指 server1下一步需要在/tmp/zookeeper目录中创建 myid 文件写入一个唯一的数字取值范围在1-255。编号1/2/3对应每台节点 myid 文件里面的数字数字和机器一一对应三台机器分别生成 myid编号 1、2、3和zoo.cfg里面server.1 server.2 server.3一一对应并且要删除/tmp下的文件目的是清除旧残留数据防止和 HA 模式自动生成的元数据、日志冲突[hadoopserver2 conf]$rm-rf/tmp/*[hadoopserver2 conf]$mkdir/tmp/zookeeper[hadoopserver2 conf]$echo1/tmp/zookeeper/myid[hadoopserver3 ~]$rm-rf/tmp/*[hadoopserver3 ~]$mkdir/tmp/zookeeper[hadoopserver3 ~]$echo2/tmp/zookeeper/myid[hadoopserver4 ~]$rm-rf/tmp/*[hadoopserver4 ~]$mkdir/tmp/zookeeper[hadoopserver4 ~]$echo3/tmp/zookeeper/myid在各节点启动服务[hadoopserver2 conf]$cd..[hadoopserver2 apache-zookeeper-3.8.6-bin]$ bin/zkServer.sh start[hadoopserver2 apache-zookeeper-3.8.6-bin]$ bin/zkServer.sh statusserver2 的角色是 follower[hadoopserver3 ~]$cdapache-zookeeper-3.8.6-bin/[hadoopserver3 apache-zookeeper-3.8.6-bin]$ bin/zkServer.sh start[hadoopserver3 apache-zookeeper-3.8.6-bin]$ bin/zkServer.sh statusserver3 的角色是 leader[hadoopserver4 ~]$cdapache-zookeeper-3.8.6-bin/[hadoopserver4 apache-zookeeper-3.8.6-bin]$ bin/zkServer.sh start[hadoopserver4 apache-zookeeper-3.8.6-bin]$ bin/zkServer.sh statusserver4 的角色是 follower查看进程2.3 Hadoop 配置[hadoopserver1 ~]$cdhadoop/etc/hadoop/[hadoopserver1 hadoop]$vimcore-site.xml[hadoopserver1 hadoop]$vimhdfs-site.xml在三个DN上依次启动 journalnode[hadoopserver2 apache-zookeeper-3.8.6-bin]$cd[hadoopserver2 ~]$cdhadoop[hadoopserver2 hadoop]$ bin/hdfs--daemonstart journalnode[hadoopserver3 apache-zookeeper-3.8.6-bin]$cd[hadoopserver3 ~]$cdhadoop[hadoopserver3 hadoop]$ bin/hdfs--daemonstart journalnode[hadoopserver4 apache-zookeeper-3.8.6-bin]$cd[hadoopserver4 ~]$cdhadoop[hadoopserver4 hadoop]$ bin/hdfs--daemonstart journalnode格式化 HDFS 集群只在 server1 上执行[hadoopserver1 hadoop]$cd[hadoopserver1 ~]$cdhadoop[hadoopserver1 hadoop]$ bin/hdfs namenode-formatscp-r/tmp/hadoop-hadoop server5:/tmp格式化 ZooKeeper只在 server1 上执行[hadoopserver1 hadoop]$ bin/hdfs zkfc-formatZK启动 HDFS 集群只在 server1 上执行[hadoopserver1 hadoop]$ sbin/start-dfs.sh查看各节点状态查看前端网页server1 为 activeserver5 为standby测试中止 server1 的NameNode 进程[hadoopserver1 hadoop]$kill-940713server1 的前端无法访问server5 变 active恢复 server1[hadoopserver1 hadoop]$ bin/hdfs--daemonstart namenodeserver1 变 standby2.4 yarn 高可用配置[hadoopserver1 hadoop]$pwd/home/hadoop/hadoop[hadoopserver1 hadoop]$vimetc/hadoop/yarn-site.xml启动 yarn 服务[hadoopserver1 hadoop]$ sbin/start-yarn.shserver1 为 standbyserver5 为 active测试中止 server5 的 ResourceManager 进程[hadoopserver5 hadoop]$kill-928361server1 变 active恢复 server5 服务[hadoopserver5 hadoop]$ bin/yarn--daemonstart resourcemanagerserver5 变 standby