标签：limited wrap dfs min height ant 一个 环境 oop
输入文件下载： https://github.com/tomwhite/hadoop-book/blob/master/input/ncdc/micro-tab/sample.txt
Logging initialized using configuration in file:/usr/local/hive/conf/hive-log4j.properties hive> Create table records(year String,temperature INT,quality INT) > ROW FORMAT DELIMITED > FIELDS TERMINATED BY ‘\t‘ > ; OK
注意：数据是没有固定格式的，因为目前input是用分隔符“\t”分割的。所以上一步中使用了（FIELDS TERMINATED BY ‘\t‘）来
hive> LOAD DATA LOCAL INPATH ‘sample.txt‘ > OVERWRITE INTO TABLE records; Loading data to table default.records Table default.records stats: [numFiles=1, numRows=0, totalSize=51, rawDataSize=0] OK Time taken: 6.03 seconds
整个过程和MapReduce一致，一共耗费30秒。
hive> SELECT year,MAX(temperature) > FROM records > WHERE temperature !=999 AND quality IN (0,1,4,5,9) > GROUP BY year; Query ID = root_20171107090403_c61a6f9a-05d4-4d0f-a97b-d37fb83ef65d Total jobs = 1 Launching Job 1 out of 1 Number of reduce tasks not specified. Estimated from input data size: 1 In order to change the average load for a reducer (in bytes): set hive.exec.reducers.bytes.per.reducer=<number> In order to limit the maximum number of reducers: set hive.exec.reducers.max=<number> In order to set a constant number of reducers: set mapreduce.job.reduces=<number> Starting Job = job_1510015112691_0001, Tracking URL = http://server71:8088/proxy/application_1510015112691_0001/ Kill Command = /usr/local/hadoop/bin/hadoop job -kill job_1510015112691_0001 Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1 2017-11-07 09:05:58,529 Stage-1 map = 0%, reduce = 0% 2017-11-07 09:06:59,061 Stage-1 map = 0%, reduce = 0% 2017-11-07 09:07:11,068 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 16.88 sec 2017-11-07 09:07:53,824 Stage-1 map = 100%, reduce = 67%, Cumulative CPU 20.75 sec 2017-11-07 09:08:03,489 Stage-1 map = 100%, reduce = 100%, Cumulative CPU 28.83 sec MapReduce Total cumulative CPU time: 28 seconds 830 msec Ended Job = job_1510015112691_0001 MapReduce Jobs Launched: Stage-Stage-1: Map: 1 Reduce: 1 Cumulative CPU: 28.83 sec HDFS Read: 8355 HDFS Write: 17 SUCCESS Total MapReduce CPU Time Spent: 28 seconds 830 msec OK 1949 111 1950 22 Time taken: 243.092 seconds, Fetched: 2 row(s)
标签：limited wrap dfs min height ant 一个 环境 oop
原文地址：http://www.cnblogs.com/kouryoushine/p/7798434.html
