Skip to content

Commit 1b85bcd

Browse files
committed
[SPARK-20627][PYSPARK] Drop the hadoop distirbution name from the Python version
## What changes were proposed in this pull request? Drop the hadoop distirbution name from the Python version (PEP440 - https://www.python.org/dev/peps/pep-0440/). We've been using the local version string to disambiguate between different hadoop versions packaged with PySpark, but PEP0440 states that local versions should not be used when publishing up-stream. Since we no longer make PySpark pip packages for different hadoop versions, we can simply drop the hadoop information. If at a later point we need to start publishing different hadoop versions we can look at make different packages or similar. ## How was this patch tested? Ran `make-distribution` locally Author: Holden Karau <[email protected]> Closes apache#17885 from holdenk/SPARK-20627-remove-pip-local-version-string.
1 parent 25ee816 commit 1b85bcd

File tree

1 file changed

+3
-3
lines changed

1 file changed

+3
-3
lines changed

dev/create-release/release-build.sh

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -163,9 +163,9 @@ if [[ "$1" == "package" ]]; then
163163
export ZINC_PORT=$ZINC_PORT
164164
echo "Creating distribution: $NAME ($FLAGS)"
165165

166-
# Write out the NAME and VERSION to PySpark version info we rewrite the - into a . and SNAPSHOT
167-
# to dev0 to be closer to PEP440. We use the NAME as a "local version".
168-
PYSPARK_VERSION=`echo "$SPARK_VERSION+$NAME" | sed -r "s/-/./" | sed -r "s/SNAPSHOT/dev0/"`
166+
# Write out the VERSION to PySpark version info we rewrite the - into a . and SNAPSHOT
167+
# to dev0 to be closer to PEP440.
168+
PYSPARK_VERSION=`echo "$SPARK_VERSION" | sed -r "s/-/./" | sed -r "s/SNAPSHOT/dev0/"`
169169
echo "__version__='$PYSPARK_VERSION'" > python/pyspark/version.py
170170

171171
# Get maven home set by MVN

0 commit comments

Comments
 (0)