Skip to content

test: fix flakey snapshot normalization logic #7203

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Apr 14, 2025
Merged

test: fix flakey snapshot normalization logic #7203

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
cb7c643
test: fix flakey snapshot normalization logic
serhalp Apr 14, 2025
10c10ba
test: make normalized snapshot port more explicit
serhalp Apr 14, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 10 additions & 10 deletions tests/integration/__snapshots__/framework-detection.test.js.snap
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,11 @@ exports[`frameworks/framework-detection > should default to process.cwd() and st
◈ Running static server from \\"should-default-to-process-cwd-and-static-server\\"
◈ Setting up local development server

◈ Static server listening to 88888
◈ Static server listening to <SNAPSHOT_PORT_NORMALIZED>

┌──────────────────────────────────────────────────┐
│ │
│ ◈ Server now ready on http://localhost:88888 │
│ ◈ Server now ready on http://localhost:<SNAPSHOT_PORT_NORMALIZED> │
│ │
└──────────────────────────────────────────────────┘"
`;
Expand All @@ -38,11 +38,11 @@ exports[`frameworks/framework-detection > should filter frameworks with no dev c
◈ Running static server from \\"should-filter-frameworks-with-no-dev-command\\"
◈ Setting up local development server

◈ Static server listening to 88888
◈ Static server listening to <SNAPSHOT_PORT_NORMALIZED>

┌──────────────────────────────────────────────────┐
│ │
│ ◈ Server now ready on http://localhost:88888 │
│ ◈ Server now ready on http://localhost:<SNAPSHOT_PORT_NORMALIZED> │
│ │
└──────────────────────────────────────────────────┘"
`;
Expand Down Expand Up @@ -150,11 +150,11 @@ exports[`frameworks/framework-detection > should use static server when --dir fl
◈ Running static server from \\"should-use-static-server-when-dir-flag-is-passed/public\\"
◈ Setting up local development server

◈ Static server listening to 88888
◈ Static server listening to <SNAPSHOT_PORT_NORMALIZED>

┌──────────────────────────────────────────────────┐
│ │
│ ◈ Server now ready on http://localhost:88888 │
│ ◈ Server now ready on http://localhost:<SNAPSHOT_PORT_NORMALIZED> │
│ │
└──────────────────────────────────────────────────┘"
`;
Expand All @@ -168,11 +168,11 @@ exports[`frameworks/framework-detection > should use static server when framewor
◈ Running static server from \\"should-use-static-server-when-framework-is-set-to-static\\"
◈ Setting up local development server

◈ Static server listening to 88888
◈ Static server listening to <SNAPSHOT_PORT_NORMALIZED>

┌──────────────────────────────────────────────────┐
│ │
│ ◈ Server now ready on http://localhost:88888 │
│ ◈ Server now ready on http://localhost:<SNAPSHOT_PORT_NORMALIZED> │
│ │
└──────────────────────────────────────────────────┘"
`;
Expand All @@ -185,11 +185,11 @@ exports[`frameworks/framework-detection > should warn if using static server and
◈ Running static server from \\"should-warn-if-using-static-server-and-target-port-is-configured/public\\"
◈ Setting up local development server

◈ Static server listening to 88888
◈ Static server listening to <SNAPSHOT_PORT_NORMALIZED>

┌──────────────────────────────────────────────────┐
│ │
│ ◈ Server now ready on http://localhost:88888 │
│ ◈ Server now ready on http://localhost:<SNAPSHOT_PORT_NORMALIZED> │
│ │
└──────────────────────────────────────────────────┘"
`;
42 changes: 24 additions & 18 deletions tests/integration/framework-detection.test.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,12 @@ import { normalize } from './utils/snapshots.js'

const content = 'Hello World!'

// Normalize random ports
const normalizeSnapshot = (output, opts) =>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👏

Love to see this vs. one big world-normalizing function.

normalize(output, opts)
.replaceAll(/localhost:\d+/g, 'localhost:<SNAPSHOT_PORT_NORMALIZED>')
.replaceAll(/listening to \d+/g, 'listening to <SNAPSHOT_PORT_NORMALIZED>')

describe.concurrent('frameworks/framework-detection', () => {
test('should default to process.cwd() and static server', async (t) => {
await withSiteBuilder(t, async (builder) => {
Expand All @@ -25,7 +31,7 @@ describe.concurrent('frameworks/framework-detection', () => {
const responseContent = await response.text()

t.expect(responseContent).toEqual(content)
t.expect(normalize(output, { duration: true, filePath: true })).toMatchSnapshot()
t.expect(normalizeSnapshot(output, { duration: true, filePath: true })).toMatchSnapshot()
})
})
})
Expand All @@ -44,7 +50,7 @@ describe.concurrent('frameworks/framework-detection', () => {
const responseContent = await response.text()

t.expect(responseContent).toEqual(content)
t.expect(normalize(output, { duration: true, filePath: true })).toMatchSnapshot()
t.expect(normalizeSnapshot(output, { duration: true, filePath: true })).toMatchSnapshot()
})
})
})
Expand All @@ -64,7 +70,7 @@ describe.concurrent('frameworks/framework-detection', () => {
const responseContent = await response.text()

t.expect(responseContent).toEqual(content)
t.expect(normalize(output, { duration: true, filePath: true })).toMatchSnapshot()
t.expect(normalizeSnapshot(output, { duration: true, filePath: true })).toMatchSnapshot()
})
})
})
Expand All @@ -85,7 +91,7 @@ describe.concurrent('frameworks/framework-detection', () => {
const responseContent = await response.text()

t.expect(responseContent).toEqual(content)
t.expect(normalize(output, { duration: true, filePath: true })).toMatchSnapshot()
t.expect(normalizeSnapshot(output, { duration: true, filePath: true })).toMatchSnapshot()
},
)
})
Expand All @@ -102,7 +108,7 @@ describe.concurrent('frameworks/framework-detection', () => {
true,
).catch((error_) => error_)

t.expect(normalize(error.stdout, { duration: true, filePath: true })).toMatchSnapshot()
t.expect(normalizeSnapshot(error.stdout, { duration: true, filePath: true })).toMatchSnapshot()
})
})

Expand All @@ -112,7 +118,7 @@ describe.concurrent('frameworks/framework-detection', () => {

// a failure is expected since this is not a true create-react-app project
const error = await withDevServer({ cwd: builder.directory }, () => {}, true).catch((error_) => error_)
t.expect(normalize(error.stdout, { duration: true, filePath: true })).toMatchSnapshot()
t.expect(normalizeSnapshot(error.stdout, { duration: true, filePath: true })).toMatchSnapshot()
})
})

Expand All @@ -121,7 +127,7 @@ describe.concurrent('frameworks/framework-detection', () => {
await builder.withNetlifyToml({ config: { dev: { framework: 'to-infinity-and-beyond-js' } } }).build()

const error = await withDevServer({ cwd: builder.directory }, () => {}, true).catch((error_) => error_)
t.expect(normalize(error.stdout, { duration: true, filePath: true })).toMatchSnapshot()
t.expect(normalizeSnapshot(error.stdout, { duration: true, filePath: true })).toMatchSnapshot()
})
})

Expand All @@ -135,7 +141,7 @@ describe.concurrent('frameworks/framework-detection', () => {

// a failure is expected since this is not a true create-react-app project
const error = await withDevServer({ cwd: builder.directory }, () => {}, true).catch((error_) => error_)
t.expect(normalize(error.stdout, { duration: true, filePath: true })).toMatchSnapshot()
t.expect(normalizeSnapshot(error.stdout, { duration: true, filePath: true })).toMatchSnapshot()
})
})

Expand All @@ -148,7 +154,7 @@ describe.concurrent('frameworks/framework-detection', () => {
() => {},
true,
).catch((error_) => error_)
t.expect(normalize(error.stdout, { duration: true, filePath: true })).toMatchSnapshot()
t.expect(normalizeSnapshot(error.stdout, { duration: true, filePath: true })).toMatchSnapshot()
})
})

Expand All @@ -161,7 +167,7 @@ describe.concurrent('frameworks/framework-detection', () => {
() => {},
true,
).catch((error_) => error_)
t.expect(normalize(error.stdout, { duration: true, filePath: true })).toMatchSnapshot()
t.expect(normalizeSnapshot(error.stdout, { duration: true, filePath: true })).toMatchSnapshot()
})
})

Expand All @@ -174,7 +180,7 @@ describe.concurrent('frameworks/framework-detection', () => {
() => {},
true,
).catch((error_) => error_)
t.expect(normalize(error.stdout, { duration: true, filePath: true })).toMatchSnapshot()
t.expect(normalizeSnapshot(error.stdout, { duration: true, filePath: true })).toMatchSnapshot()
})
})

Expand All @@ -198,7 +204,7 @@ describe.concurrent('frameworks/framework-detection', () => {
true,
).catch((error_) => error_)

t.expect(normalize(error.stdout, { duration: true, filePath: true })).toMatchSnapshot()
t.expect(normalizeSnapshot(error.stdout, { duration: true, filePath: true })).toMatchSnapshot()
})
})

Expand Down Expand Up @@ -232,7 +238,7 @@ describe.concurrent('frameworks/framework-detection', () => {
await childProcess
}
const error = await asyncErrorBlock().catch((error_) => error_)
t.expect(normalize(error.stdout, { duration: true, filePath: true })).toMatchSnapshot()
t.expect(normalizeSnapshot(error.stdout, { duration: true, filePath: true })).toMatchSnapshot()
})
})

Expand All @@ -259,7 +265,7 @@ describe.concurrent('frameworks/framework-detection', () => {
}
const error = await asyncErrorBlock().catch((error_) => error_)
t.expect(
normalize(error.stdout, { duration: true, filePath: true }).includes(
normalizeSnapshot(error.stdout, { duration: true, filePath: true }).includes(
'Detected commands for: Gatsby, Create React App. Update your settings to specify which to use. Refer to https://ntl.fyi/dev-monorepo for more information.',
),
)
Expand All @@ -278,7 +284,7 @@ describe.concurrent('frameworks/framework-detection', () => {
true,
).catch((error_) => error_)

t.expect(normalize(error.stdout, { duration: true, filePath: true })).toMatchSnapshot()
t.expect(normalizeSnapshot(error.stdout, { duration: true, filePath: true })).toMatchSnapshot()
})
})

Expand All @@ -299,7 +305,7 @@ describe.concurrent('frameworks/framework-detection', () => {
const responseContent = await response.text()

t.expect(responseContent).toEqual(content)
t.expect(normalize(output, { duration: true, filePath: true })).toMatchSnapshot()
t.expect(normalizeSnapshot(output, { duration: true, filePath: true })).toMatchSnapshot()
})
})
})
Expand All @@ -317,7 +323,7 @@ describe.concurrent('frameworks/framework-detection', () => {

// a failure is expected since this is not a true Gatsby project
const error = await withDevServer({ cwd: builder.directory }, () => {}, true).catch((error_) => error_)
t.expect(normalize(error.stdout, { duration: true, filePath: true })).toMatchSnapshot()
t.expect(normalizeSnapshot(error.stdout, { duration: true, filePath: true })).toMatchSnapshot()
})
})

Expand Down Expand Up @@ -395,7 +401,7 @@ describe.concurrent('frameworks/framework-detection', () => {
const responseJson = await response.json()
t.expect(responseJson).toStrictEqual({ CONTEXT_CHECK: 'PRODUCTION' })

const normalizedText = normalize(output, { duration: true, filePath: true })
const normalizedText = normalizeSnapshot(output, { duration: true, filePath: true })
t.expect(
normalizedText.includes(
`Changes will not be hot-reloaded, so if you need to rebuild your site you must exit and run 'netlify serve' again`,
Expand Down
2 changes: 0 additions & 2 deletions tests/integration/utils/snapshots.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,6 @@ const baseNormalizers = [
// Information about the package and the OS
{ pattern: /netlify-cli\/.+node-.+/g, value: 'netlify-cli/test-version test-os test-node-version' },
{ pattern: /@netlify\/build (\d+\.\d+\.\d+)/g, value: '@netlify/build 0.0.0' },
// normalize random ports
{ pattern: /\d{5}/g, value: '88888' },
// windows specific
{ pattern: /\\/gu, value: '/' },
{ pattern: /\r\n/gu, value: '\n' },
Expand Down
Loading