标签：hotspot namenode word 适配 web etl data exception hba
由于需要首次手动安装sbt，需要联网，故将虚拟机的网络适配器模式设置为"桥接模式"，这样就可以和互联网相连接。
但是后面执行"spark-shell --master yarn --deploy-mode client" 命令时，无法启动，一直停留在中间状态不动，
如下：
[root@master test_code]# spark-shell --master yarn --deploy-mode client
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
17/05/07 18:07:37 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platfor
m... using builtin-java classes where applicable
后来突然想起来，虚拟机的网络适配器模式没有更改过来，重新设置为"仅主机模式" 后（这是由于，安装Hadoop集群
时，几个VMware虚拟机都是使用的"仅主机模式"），spark-shell 正常启动，如下：
[root@master master]# spark-shell --master yarn --deploy-mode client
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
17/05/07 18:30:12 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
17/05/07 18:30:28 WARN yarn.Client: Neither spark.yarn.jars nor spark.yarn.archive is set, falling back to uploading libraries under SPARK_HOME.
17/05/07 18:31:15 WARN metastore.ObjectStore: Failed to get database global_temp, returning NoSuchObjectException
Spark context Web UI available at http://192.168.1.200:4040
Spark context available as ‘sc‘ (master = yarn, app id = application_1494142860645_0001).
Spark session available as ‘spark‘.
Welcome to
____ __
/ __/__ ___ _____/ /__
_\ \/ _ \/ _ `/ __/ ‘_/
/___/ .__/\_,_/_/ /_/\_\ version 2.1.0
/_/
Using Scala version 2.11.8 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_121)
Type in expressions to have them evaluated.
Type :help for more information.
scala>
BTW：
通过查看active NameNode 状态，可以看到，集群状态不正常，无法与其他节点通信，造成block丢失。如下WebUI 上所示：
There are 35 missing blocks. The following files may be corrupted:
Please check the logs or run fsck in order to identify the missing blocks. See the Hadoop FAQ for common causes and potential solutions.
标签：hotspot namenode word 适配 web etl data exception hba
原文地址：http://www.cnblogs.com/followyourdream/p/6821751.html
