Skip to content

Commit

Permalink
[SPARK-22666][ML][FOLLOW-UP] Improve testcase to tolerate different s…
Browse files Browse the repository at this point in the history
…chema representation

## What changes were proposed in this pull request?

Improve testcase "image datasource test: read non image" to tolerate different schema representation.
Because file:/path and file:///path are both valid URI-ifications so in some environment the testcase will fail.

## How was this patch tested?

Manual.

Closes apache#22449 from WeichenXu123/image_url.

Authored-by: WeichenXu <[email protected]>
Signed-off-by: Xiangrui Meng <[email protected]>
  • Loading branch information
WeichenXu123 authored and mengxr committed Sep 19, 2018
1 parent cb1b55c commit 6f681d4
Showing 1 changed file with 9 additions and 2 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@

package org.apache.spark.ml.source.image

import java.net.URI
import java.nio.file.Paths

import org.apache.spark.SparkFunSuite
Expand Down Expand Up @@ -58,8 +59,14 @@ class ImageFileFormatSuite extends SparkFunSuite with MLlibTestSparkContext {
.load(filePath)
assert(df2.count() === 1)
val result = df2.head()
assert(result === invalidImageRow(
Paths.get(filePath).toAbsolutePath().normalize().toUri().toString))

val resultOrigin = result.getStruct(0).getString(0)
// covert `origin` to `java.net.URI` object and then compare.
// because `file:/path` and `file:///path` are both valid URI-ifications
assert(new URI(resultOrigin) === Paths.get(filePath).toAbsolutePath().normalize().toUri())

// Compare other columns in the row to be the same with the `invalidImageRow`
assert(result === invalidImageRow(resultOrigin))
}

test("image datasource partition test") {
Expand Down

0 comments on commit 6f681d4

Please sign in to comment.