[SPARK-33874][K8S][FOLLOWUP] Handle long lived sidecars - clean up lo…

…gging ### What changes were proposed in this pull request? Switch log level from warn to debug when the spark container is not present in the pod's container statuses. ### Why are the changes needed? There are many non-critical situations where the Spark container may not be present, and the warning log level is too high. ### Does this PR introduce _any_ user-facing change? Log message change. ### How was this patch tested? N/A Closes apache#31047 from holdenk/SPARK-33874-follow-up. Authored-by: Holden Karau <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]>
panbingkun · Jan 5, 2021 · 171db85 · 171db85
1 parent cc1d9d2
commit 171db85
Showing 1 changed file with 3 additions and 2 deletions.
diff --git a/...tes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsSnapshot.scala b/...tes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsSnapshot.scala
@@ -93,9 +93,10 @@ object ExecutorPodsSnapshot extends Logging {
                   case _ =>
                     PodRunning(pod)
                 }
-              // If we can't find the Spark container status, fall back to the pod status
+              // If we can't find the Spark container status, fall back to the pod status. This is
+              // expected to occur during pod startup and other situations.
               case _ =>
-                logWarning(s"Unable to find container ${sparkContainerName} in pod ${pod} " +
+                logDebug(s"Unable to find container ${sparkContainerName} in pod ${pod} " +
                   "defaulting to entire pod status (running).")
                 PodRunning(pod)
             }