您现在的位置 >> Hadoop教程 >> Hadoop实战 >> hadoop专题  
 

Apache Hadoop 2.2.0

【作者:Hadoop实战专家】【关键词:】 【点击:85750次】【2013-08-1】
2013 Apache Software Foundation - Privacy Policy  

相关热门搜索:

大数据标签:hadoop hdfs yarn mapreduce bigdata

Apache > Hadoop

Wiki | SVN  | Last Published: 2013-10-07  | Version: 2.2.0

General

* Overview
* Single Node Setup
* Cluster Setup
* Hadoop Commands Reference
* File System Shell
* Hadoop Compatibility

Common

* CLI Mini Cluster
* Native Libraries
* Superusers
* Service Level Authorization
* HTTP Authentication

HDFS

* HDFS User Guide
* High Availability With QJM
* High Availability With NFS
* Federation
* HDFS Snapshots
* HDFS Architecture
* Edits Viewer
* Image Viewer
* Permissions and HDFS
* Quotas and HDFS
* HFTP
* C API libhdfs
* WebHDFS REST API
* HttpFS Gateway
* Short Circuit Local Reads

MapReduce

* Compatibilty between Hadoop 1.x and Hadoop 2.x
* Encrypted Shuffle
* Pluggable Shuffle/Sort

YARN

* YARN Architecture
* Writing YARN Applications
* Capacity Scheduler
* Fair Scheduler
* Web Application Proxy
* YARN Commands

YARN REST APIs

* Introduction
* Resource Manager
* Node Manager
* MR Application Master
* History Server

Auth

* Overview
* Examples
* Configuration
* Building

Reference

* Release Notes
* API docs
* Common CHANGES.txt
* HDFS CHANGES.txt
* MapReduce CHANGES.txt

Configuration

* core-default.xml
* hdfs-default.xml
* mapred-default.xml
* yarn-default.xml
* Deprecated Properties

Apache Hadoop 2.2.0

Apache Hadoop 2.2.0 consists of significant improvements over the previous stable release (hadoop-1.x).

Here is a short overview of the improvments to both HDFS and MapReduce.

* HDFS Federation

In order to scale the name service horizontally, federation uses multiple independent Namenodes/Namespaces. The Namenodes are federated, that is, the Namenodes are independent and don't require coordination with each other. The datanodes are used as common storage for blocks by all the Namenodes. Each datanode registers with all the Namenodes in the cluster. Datanodes send periodic heartbeats and block reports and handles commands from the Namenodes.

More details are available in the HDFS Federation document.

* MapReduce NextGen aka YARN aka MRv2

The new architecture introduced in hadoop-0.23, divides the two major functions of the JobTracker: resource management and job life-cycle management into separate components.

The new ResourceManager manages the global assignment of compute resources to applications and the per-application ApplicationMaster manages the application???s scheduling and coordination.

An application is either a single job in the sense of classic MapReduce jobs or a DAG of such jobs.

The ResourceManager and per-machine NodeManager daemon, which manages the user processes on that machine, form the computation fabric.

The per-application ApplicationMaster is, in effect, a framework specific library and is tasked with negotiating resources from the ResourceManager and working with the NodeManager(s) to execute and monitor the tasks.

More details are available in the YARN document.

Getting Started

The Hadoop documentation includes the information you need to get started using Hadoop. Begin with the Single Node Setup which shows you how to set up a single-node Hadoop installation. Then move on to the Cluster Setup to learn how to set up a multi-node Hadoop installation.

? 2013 Apache Software Foundation - Privacy Policy

大数据系列hadoop相关文章:

上一篇:Apache Hadoop 2.2.0 下一篇:Apache Hadoop 2.2.0
最新评论
冬日烈焰2014-09-10 06:34:27
大家有多少人是有系统上线的?
小小2014-09-09 01:43:42
刚才又买了本书
志国2014-09-09 01:50:03
写的MR程序直接在linux下运行正常
小杨2014-09-09 09:41:50
hadoop week6 | import java.io.IOException; import java.text.DateFormat; import java.text.SimpleDateFormat; import java.util.Date; import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.conf.Configured; import org http://t.cn/RvVd8kU
执着2014-09-08 09:25:40
『Hadoop技术中心』http://t.cn/8sKbr1N
群子2014-09-08 03:51:47
是的,发现用户规律
^_^2014-09-08 03:33:38
空指针异常 看你代码
小梦2014-09-07 06:52:44
传智播客-hadoop2培训 http://t.cn/RvP0VDV
拼劲2014-09-06 05:16:17
『看Netflix是如何良性融合AWS和Apache Hadoop的!』http://t.cn/8kgN72z
伯爵2014-09-05 05:46:29
为Hadoop数据架构添加SQL能力,这是许多厂商在做的一件事,其背后的原因很简单。尽管Hadoop分布式文件系统(HDFS)为大数据带来并行廉价服务器集群的处理能力,但如果企业能够使用SQL来对查询进行交互的话,那么它就可以达到更好的效果。http://t.cn/RvooHTU
 
  • Hadoop生态系统资料推荐