Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[SPARK-46907][CORE] Show driver log location in Spark History Server
### What changes were proposed in this pull request? This PR aims to show `Driver Log Location` in Spark History Server UI if `spark.driver.log.dfsDir` is configured. ### Why are the changes needed? **BEFORE (or `spark.driver.log.dfsDir` is absent)** ![Screenshot 2024-01-29 at 10 11 06 AM](https://github.com/apache/spark/assets/9700541/6d709b4b-d002-422b-a1df-bb5e1b50b539) **AFTER** ![Screenshot 2024-01-29 at 10 10 25 AM](https://github.com/apache/spark/assets/9700541/83b35a7d-5fc9-443a-a6e5-7b6bd98dbdc6) ### Does this PR introduce _any_ user-facing change? No, this is a new additional UI information only for the users who uses `spark.driver.log.dfsDir` configurations. ### How was this patch tested? Manual. ``` $ mkdir /tmp/history $ mkdir /tmp/driver-logs $ SPARK_HISTORY_OPTS="-Dspark.history.fs.logDirectory=/tmp/history -Dspark.driver.log.dfsDir=/tmp/driver-logs" sbin/start-history-server.sh ``` ### Was this patch authored or co-authored using generative AI tooling? No. Closes apache#44936 from dongjoon-hyun/SPARK-46907. Authored-by: Dongjoon Hyun <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]>
- Loading branch information