hibench7.0编译安装
引言
HiBench是一个大数据基准套件，可以帮助您评测不同大数据平台的性能、吞吐量和系统资源利用率。本文仅介绍如何对hadoop进行测试，其他大数据平台使用请参考官网https://github.com/intel-hadoop/HiBench
软件依赖
HiBench需要java环境，以及Maven管理。
安装java运行环境
新建目录/home/java 然后上传相应的jdk二进制文件到此目录并解压。（本例为aarch64为平台，x86平台一样的操作步骤，亲试）
root@kylin1:/home/java# ll
总用量 71608
drwxr-xr-x 3 root root 4096 4月 8 11:59 ./
drwxr-xr-x 4 root root 4096 4月 8 11:28 ../
drwxr-xr-x 7 root root 4096 4月 8 11:44 jdk1.8.0_201/
-rw-r–r– 1 root root 73312819 4月 8 11:43 jdk-8u201-linux-arm64-vfp-hflt.tar.gz
添加环境变量：/etc/profile 文件末尾增加如下行
export JAVA_HOME=/home/java/jdk1.8.0_201
export CLASSPATH=.:$JAVA_HOME/jre/lib/rt.jar:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export PATH=$PATH:$JAVA_HOME/bin
使其生效
source /etc/profile
查看
root@kylin1:/home/java# java -version
java version “1.8.0_201”
Java(TM) SE Runtime Environment (build 1.8.0_201-b09)
Java HotSpot(TM) 64-Bit Server VM (build 25.201-b09, mixed mode)
安装Maven
下载Maven包
wget http://apache.fayea.com/maven/maven-3/3.6.0/binaries/apache-maven-3.6.0-bin.zip
解压缩
unzip apache-maven-3.6.0-bin.zip -d /usr/local/
添加环境变量
cat /etc/profile,在文件末尾增加如下
export M3_HOME=/usr/local/apache-maven-3.6.0
export PATH=$M3_HOME/bin:$PATH
source /etc/profile
测试Maven环境
mvn -v
看到相应版本信息输出即表明配置正确：
root@kylin3:/usr/local/HiBench-HiBench-7.0# mvn -v
Apache Maven 3.6.0 (97c98ec64a1fdfee7767ce5ffb20918da4f719f3; 2018-10-25T02:41:47+08:00)
Maven home: /usr/local/apache-maven-3.6.0
Java version: 1.8.0_201, vendor: Oracle Corporation, runtime: /home/java/jdk1.8.0_201/jre
Default locale: zh_CN, platform encoding: UTF-8
OS name: “linux”, version: “4.4.58-20180615.kylin.server.yun+-generic”, arch: “aarch64”, family: “unix”
下载HiBench
git clone https://github.com/intel-hadoop/HiBench.git比较慢，建议直接网页下载zip包，然后解压到/usr/local目录下。
root@kylin3:/usr/local/HiBench-HiBench-7.0# pwd
/usr/local/HiBench-HiBench-7.0
安装Hibench
切到HiBench下，执行对应的安装操作，可以选择自己想要安装的模块。以安装hadoop框架为例：
root@kylin3:/usr/local/HiBench-HiBench-7.0# mvn -Phadoopbench -Dspark=2.1 -Dscala=2.11 clean package
更多安装方法参考https://github.com/Intel-bigdata/HiBench/blob/master/docs/build-hibench.md
因为网络原因，下次模块过程中经常出现暂停现象。当前解决办法暂停下载，ctrl+c 然后继续执行上面的命令。
遇到的问题：
[WARNING] Could not get content
org.apache.maven.wagon.TransferFailedException: Failed to transfer file http://archive.apache.org/dist/hive/hive-0.14.0/apache-hive-0.14.0-bin.tar.gz with status code 503
下载hive出错，解决：
root@kylin3:/usr/local/HiBench-HiBench-7.0# pwd
/usr/local/HiBench-HiBench-7.0
root@kylin3:/usr/local/HiBench-HiBench-7.0# vim hadoopbench/sql/pom.xml
修改该文件中，将http改为https即可
org.apache.maven.wagon.TransferFailedException: Failed to transfer file http://archive.apache.org/dist/nutch/apache-nutch-1.2-bin.tar.gz with status code 503
root@kylin3:/usr/local/HiBench-HiBench-7.0/hadoopbench/nutchindexing# pwd
/usr/local/HiBench-HiBench-7.0/hadoopbench/nutchindexing
vim pod.xml
修改如下文件由http改成https
最终结果
[INFO] Reactor Summary:
[INFO]
[INFO] hibench 7.0 …………………………………. SUCCESS [ 0.173 s]
[INFO] hibench-common 7.0 …………………………… SUCCESS [ 10.755 s]
[INFO] HiBench data generation tools 7.0 ……………… SUCCESS [ 12.696 s]
[INFO] hadoopbench 7.0 ……………………………… SUCCESS [ 0.004 s]
[INFO] hadoopbench-sql 7.0 ………………………….. SUCCESS [ 2.355 s]
[INFO] mahout 7.0 ………………………………….. SUCCESS [ 6.676 s]
[INFO] PEGASUS: A Peta-Scale Graph Mining System 2.0-SNAPSHOT SUCCESS [ 0.978 s]
[INFO] nutchindexing 7.0 ……………………………. SUCCESS [ 41.968 s]
[INFO] ————————————————————————
[INFO] BUILD SUCCESS
[INFO] ————————————————————————
[INFO] Total time: 01:15 min
[INFO] Finished at: 2019-04-09T11:58:27+08:00
[INFO] ————————————————————————