Skip to content

Commit be343d2

Browse files
committed
Fix yarn/README.md and update docs/running-on-yarn.md
1 parent 67cd752 commit be343d2

File tree

2 files changed

+1
-3
lines changed

2 files changed

+1
-3
lines changed

docs/running-on-yarn.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -116,7 +116,7 @@ For example:
116116

117117
# Building Spark for Hadoop/YARN 2.2.x
118118

119-
Hadoop 2.2.x users must build Spark and publish it locally. The SBT build process handles Hadoop 2.2.x as a special case. This version of Hadoop has new YARN API changes and depends on a Protobuf version (2.5) that is not compatible with the Akka version (2.0.5) that Spark uses. Therefore, if the Hadoop version (e.g. set through ```SPARK_HADOOP_VERSION```) starts with 2.2.0 or higher then the build process will depend on Akka artifacts distributed by the Spark project compatible with Protobuf 2.5. Furthermore, the build process then uses the directory ```new-yarn``` (instead of ```yarn```), which supports the new YARN API. The build process should seamlessly work out of the box.
119+
Hadoop 2.2.x users must build Spark and publish it locally. The SBT build process handles Hadoop 2.2.x as a special case. This version of Hadoop has new YARN API changes and depends on a Protobuf version (2.5) that is not compatible with the Akka version (2.0.5) that Spark uses. Therefore, if the Hadoop version (e.g. set through ```SPARK_HADOOP_VERSION```) starts with 2.2.0 or higher then the build process will depend on Akka artifacts distributed by the Spark project compatible with Protobuf 2.5.
120120

121121
See [Building Spark with Maven](building-with-maven.html) for instructions on how to build Spark using the Maven process.
122122

yarn/README.md

-2
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,5 @@
11
# YARN DIRECTORY LAYOUT
22

3-
Hadoop Yarn related code are organized
4-
53
Hadoop Yarn related codes are organized in separate modules with layered dependency.
64

75
* common : Common codes that will be called into by other modules.

0 commit comments

Comments
 (0)