From 69bf15d02df8c0e6c921987ba74301063f6026d6 Mon Sep 17 00:00:00 2001 From: Genevieve Warren <24882762+gewarren@users.noreply.github.com> Date: Fri, 30 Aug 2024 11:12:25 -0700 Subject: [PATCH 1/5] initial work --- docs/core/project-sdk/msbuild-props.md | 24 +++++++++++++++++++- docs/core/project-sdk/overview.md | 15 ++++++------ docs/core/testing/unit-testing-mstest-sdk.md | 23 +++++++++---------- 3 files changed, 42 insertions(+), 20 deletions(-) diff --git a/docs/core/project-sdk/msbuild-props.md b/docs/core/project-sdk/msbuild-props.md index 499e82c0be0a4..3ec3be5c3ea89 100644 --- a/docs/core/project-sdk/msbuild-props.md +++ b/docs/core/project-sdk/msbuild-props.md @@ -1400,7 +1400,7 @@ The following properties are used for launching an app with the [`dotnet run`](. ### RunArguments -The `RunArguments` property defines the arguments that are passed to the app when it is run. +The `RunArguments` property defines the arguments that are passed to the app when it's run. ```xml @@ -1421,6 +1421,28 @@ The `RunWorkingDirectory` property defines the working directory for the applica ``` +## Test project–related properties + +The following MSBuild properties are documented in this section: + +... + +### IsTestProject +### IsTestingPlatformApplication +### EnableMicrosoftTestingExtensionsCodeCoverage +### EnableMicrosoftTestingExtensionsCrashDump +### EnableAspireTesting +### EnablePlaywright +### EnableMSTestRunner +### EnableNUnitRunner +### GenerateTestingPlatformEntryPoint +### TestingPlatformCaptureOutput +### TestingPlatformCommandLineArguments +### TestingPlatformDotnetTestSupport +### TestingPlatformShowTestsFailure +### TestingExtensionsProfile +### UseVSTest + ## Hosting-related properties The following MSBuild properties are documented in this section: diff --git a/docs/core/project-sdk/overview.md b/docs/core/project-sdk/overview.md index 9aedc5573dd20..1e253233427c6 100644 --- a/docs/core/project-sdk/overview.md +++ b/docs/core/project-sdk/overview.md @@ -8,19 +8,20 @@ no-loc: ["EmbeddedResource", "Compile", "None", "Blazor"] --- # .NET project SDKs -Modern .NET projects are associated with a software development kit (SDK). Each *project SDK* is a set of MSBuild [targets](/visualstudio/msbuild/msbuild-targets) and associated [tasks](/visualstudio/msbuild/msbuild-tasks) that are responsible for compiling, packing, and publishing code. A project that references a project SDK is sometimes referred to as an *SDK-style project*. +Modern .NET projects are associated with a project software development kit (SDK). Each *project SDK* is a set of MSBuild [targets](/visualstudio/msbuild/msbuild-targets) and associated [tasks](/visualstudio/msbuild/msbuild-tasks) that are responsible for compiling, packing, and publishing code. A project that references a project SDK is sometimes referred to as an *SDK-style project*. ## Available SDKs The following SDKs are available: -| ID | Description | Repo | -|-------------------------|------------------------------------------------------|---------------------------------| -| `Microsoft.NET.Sdk` | The .NET SDK | | -| `Microsoft.NET.Sdk.Web` | The .NET [Web SDK](/aspnet/core/razor-pages/web-sdk) | | +| ID | Description | Repo | +|----------------------------|---------------------------------------------------------|----------------------------------------| +| `Microsoft.NET.Sdk` | The .NET SDK | | +| `Microsoft.NET.Sdk.Web` | The .NET [Web SDK](/aspnet/core/razor-pages/web-sdk) | | +| `Microsoft.NET.Sdk.Razor` | The .NET [Razor SDK](/aspnet/core/razor-pages/sdk) | | | `Microsoft.NET.Sdk.BlazorWebAssembly` | The .NET [Blazor WebAssembly SDK](/aspnet/core/blazor#blazor-webassembly) | | -| `Microsoft.NET.Sdk.Razor` | The .NET [Razor SDK](/aspnet/core/razor-pages/sdk) | | -| `Microsoft.NET.Sdk.Worker` | The .NET [Worker Service](../extensions/workers.md) SDK | | +| `Microsoft.NET.Sdk.Worker` | The .NET [Worker Service](../extensions/workers.md) SDK | | +| `MSTest.Sdk` | The MSTest SDK | | The .NET SDK is the base SDK for .NET. The other SDKs reference the .NET SDK, and projects that are associated with the other SDKs have all the .NET SDK properties available to them. The Web SDK, for example, depends on both the .NET SDK and the Razor SDK. diff --git a/docs/core/testing/unit-testing-mstest-sdk.md b/docs/core/testing/unit-testing-mstest-sdk.md index c474e1d407ad1..b43f54853a317 100644 --- a/docs/core/testing/unit-testing-mstest-sdk.md +++ b/docs/core/testing/unit-testing-mstest-sdk.md @@ -8,10 +8,9 @@ ms.date: 02/13/2024 # MSTest SDK overview -[MSTest.Sdk](https://www.nuget.org/packages/MSTest.Sdk) is a [MSBuild project SDK](/visualstudio/msbuild/how-to-use-project-sdk) for building MSTest apps. -It's possible to build a MSTest app without this SDK, however, the MSTest SDK is: +Introduced in .NET 9, [MSTest.Sdk](https://www.nuget.org/packages/MSTest.Sdk) is a [MSBuild project SDK](/visualstudio/msbuild/how-to-use-project-sdk) for building MSTest apps. It's possible to build a MSTest app without this SDK, however, the MSTest SDK is: -* Tailored towards providing a first-class experience for testing with MSTest +* Tailored towards providing a first-class experience for testing with MSTest. * The recommended target for most users. * Easy to configure for other users. @@ -32,9 +31,9 @@ You can enable `MSTest.Sdk` in a project by simply updating the `Sdk` attribute ``` > [!NOTE] -> `/3.3.1` is given as example as it's the first version providing the SDK but it can be replaced with any newer version. +> `/3.3.1` is given as example as it's the first version of the SDK, but it can be replaced with any newer version. -To simplify handling of versions we recommend setting the SDK version at solution level using the _global.json_. For example, your project file would look like: +To simplify handling of versions, we recommend setting the SDK version at solution level using the _global.json_ file. For example, your project file would look like: ```xml @@ -48,7 +47,7 @@ To simplify handling of versions we recommend setting the SDK version at solutio ``` -Then, you would have a _global.json_ file that specifies the `MSTest.Sdk` version as follows: +Then, specify the `MSTest.Sdk` version in the _global.json_ file as follows: ```json { @@ -65,7 +64,7 @@ When you `build` the project, all the needed components are restored and install You don't need anything else to build and run your tests and you can use the same tooling (for example, `dotnet test` or Visual Studio) used by a ["classic" MSTest project](./unit-testing-with-mstest.md). > [!IMPORTANT] -> By switching to the `MSTest.Sdk` you've fully opted-in to using the [MSTest runner](./unit-testing-mstest-runner-intro.md), including with [dotnet test](./unit-testing-platform-integration-dotnet-test.md#dotnet-test---microsofttestingplatform-mode) which requires modifying your CI, local CLI calls and also impacts the available entries of the _.runsettings_. +> By switching to the `MSTest.Sdk`, you opt in to using the [MSTest runner](./unit-testing-mstest-runner-intro.md), including with [dotnet test](./unit-testing-platform-integration-dotnet-test.md#dotnet-test---microsofttestingplatform-mode) which requires modifying your CI, local CLI calls and also impacts the available entries of the _.runsettings_. > You can use `MSTest.Sdk` and still keep the old integrations and tools by instead switching the [runner](#select-the-runner). ## Select the runner @@ -150,12 +149,12 @@ Outside of the selection of the runner and runner specific extensions, `MSTest.S ### .NET Aspire -.NET Aspire is an opinionated, cloud ready stack for building observable, production ready, distributed applications. .NET Aspire is delivered through a collection of NuGet packages that handle specific cloud-native concerns. For more information, see the [.NET Aspire docs](/dotnet/aspire/get-started/aspire-overview). +.NET Aspire is an opinionated, cloud-ready stack for building observable, production ready, distributed applications. .NET Aspire is delivered through a collection of NuGet packages that handle specific cloud-native concerns. For more information, see the [.NET Aspire docs](/dotnet/aspire/get-started/aspire-overview). > [!NOTE] > This feature is available from MSTest.Sdk 3.4.0 -By setting the property `EnableAspireTesting` to `true` you can bring all dependencies and default usings you would need for testing with `Aspire` and `MSTest`. +By setting the property `EnableAspireTesting` to `true`, you can bring all dependencies and default `using` directives you would need for testing with `Aspire` and `MSTest`. ```xml @@ -177,7 +176,7 @@ Playwright enables reliable end-to-end testing for modern web apps. For more inf > [!NOTE] > This feature is available from MSTest.Sdk 3.4.0 -By setting the property `EnablePlaywright` to `true` you can bring all dependencies and default usings you would need for testing with `Playwright` and `MSTest`. +By setting the property `EnablePlaywright` to `true` you can bring all dependencies and default `using` directives you would need for testing with `Playwright` and `MSTest`. ```xml @@ -217,7 +216,7 @@ Add the version to your `global.json`: You can then start simplifying your project. -Removing default properties: +Remove default properties: ```diff - true @@ -226,7 +225,7 @@ Removing default properties: - true ``` -Removing default package references: +Remove default package references: ```diff - Date: Fri, 30 Aug 2024 12:57:50 -0700 Subject: [PATCH 2/5] halfway there --- docs/core/project-sdk/msbuild-props.md | 80 ++++++++++++++++++- docs/core/project-sdk/overview.md | 2 +- docs/core/testing/unit-testing-mstest-sdk.md | 34 ++++---- .../core/testing/unit-testing-platform-faq.md | 8 +- .../testing/unit-testing-platform-intro.md | 6 +- 5 files changed, 103 insertions(+), 27 deletions(-) diff --git a/docs/core/project-sdk/msbuild-props.md b/docs/core/project-sdk/msbuild-props.md index 3ec3be5c3ea89..77b52d5562d42 100644 --- a/docs/core/project-sdk/msbuild-props.md +++ b/docs/core/project-sdk/msbuild-props.md @@ -1425,24 +1425,98 @@ The `RunWorkingDirectory` property defines the working directory for the applica The following MSBuild properties are documented in this section: -... +- [IsTestProject](#istestproject) +- [IsTestingPlatformApplication](#istestingplatformapplication) +- [Enable\[NugetPackageNameWithoutDots\]](#enablenugetpackagenamewithoutdots) +- [EnableAspireTesting](#enableaspiretesting) +- [EnablePlaywright](#enableplaywright) +- [EnableMSTestRunner](#enablemstestrunner) +- [EnableNUnitRunner](#enablenunitrunner) +- [GenerateTestingPlatformEntryPoint](#generatetestingplatformentrypoint) +- [TestingPlatformCaptureOutput](#testingplatformcaptureoutput) +- [TestingPlatformCommandLineArguments](#testingplatformcommandlinearguments) +- [TestingPlatformDotnetTestSupport](#testingplatformdotnettestsupport) +- [TestingPlatformShowTestsFailure](#testingplatformshowtestsfailure) +- [TestingExtensionsProfile](#testingextensionsprofile) +- [UseVSTest](#usevstest) ### IsTestProject + +The `IsTestProject` property signifies that a project is a test project. When this property is set to `true`, validation to check if the project references a self-contained executable is disabled. That's because test projects have an `OutputType` of `Exe` but usually call APIs in a referenced executable rather than trying to run. In addition, if a project references a project where `IsTestProject` is set to `true`, the test project isn't validated as an executable reference. + +> [!NOTE] +> If your project specifies the [MSTest SDK](../testing/unit-testing-mstest-sdk.md), you don't need to set this property. + ### IsTestingPlatformApplication -### EnableMicrosoftTestingExtensionsCodeCoverage -### EnableMicrosoftTestingExtensionsCrashDump + +Setting the `IsTestingPlatformApplication` property to `false` disables the transitive dependency to the [Microsoft.Testing.Platform.MSBuild](https://www.nuget.org/packages/Microsoft.Testing.Platform.MSBuild) package. A *transitive dependency* is when a project that references another project that references a given package behaves as if *it* references the package. You'd typically set this property to `false` in a non-test project that references a test project. + +For more information, see [error CS8892](../testing/unit-testing-platform-faq.md#error-cs8892-method-testingplatformentrypointmainstring-will-not-be-used-as-an-entry-point-because-a-synchronous-entry-point-programmainstring-was-found). + +### Enable\[NugetPackageNameWithoutDots\] + +Use a property with the pattern `Enable[NugetPackageNameWithoutDots]` to enable or disable Microsoft.Testing.Platform extensions. + +For example, to enable the crash dump extension (NuGet package [Microsoft.Testing.Extensions.CrashDump](https://www.nuget.org/packages/Microsoft.Testing.Extensions.CrashDump)), set the `EnableMicrosoftTestingExtensionsCrashDump` to `true`. + +For more information, see [Enable or disable extensions](../testing/unit-testing-mstest-sdk.md#enable-or-disable-extensions). + ### EnableAspireTesting + +When you use the [MSTest project SDK](../testing/unit-testing-mstest-sdk.md), you can use the `EnableAspireTesting` property to bring in all the dependencies and default `using` directives you need for testing with `Aspire` and `MSTest`. + +For more information, see [Test with .NET Aspire](../testing/unit-testing-mstest-sdk.md#net-aspire). + ### EnablePlaywright + +When you use the [MSTest project SDK](../testing/unit-testing-mstest-sdk.md), you can use the `EnablePlaywright` property to bring in all the dependencies and default `using` directives you need for testing with `Playwright` and `MSTest`. + +For more information, see [Playwright](../testing/unit-testing-mstest-sdk.md#playwright). + ### EnableMSTestRunner + +The `EnableMSTestRunner` property ... + ### EnableNUnitRunner + +The `EnableNUnitRunner` property ... + ### GenerateTestingPlatformEntryPoint + +The `GenerateTestingPlatformEntryPoint` property ... + ### TestingPlatformCaptureOutput + +The `TestingPlatformCaptureOutput` property ... + ### TestingPlatformCommandLineArguments + +The `TestingPlatformCommandLineArguments` property ... + ### TestingPlatformDotnetTestSupport + +The `TestingPlatformDotnetTestSupport` property ... + ### TestingPlatformShowTestsFailure + +The `TestingPlatformShowTestsFailure` property ... + ### TestingExtensionsProfile + +When you use the [MSTest project SDK](../testing/unit-testing-mstest-sdk.md), the `TestingExtensionsProfile` property lets you select a profile to use. The following table shows the allowable values. + +| Value | Description | +|----------------|-----------------------------------------------------------------------------------------------| +| `Default` | Enables the recommended extensions for this version of MSTest.SDK. | +| `None` | No extensions are enabled. | +| `AllMicrosoft` | Enable all extensions shipped by Microsoft (including extensions with a restrictive license). | + +For more information, see [MSTest runner profile](../testing/unit-testing-mstest-sdk.md#mstest-runner-profile). + ### UseVSTest +Set the `UseVSTest` property to `true` to switch from the MSTest runner to the [VSTest](/visualstudio/test/vstest-console-options) runner when using the [MSTest project SDK](../testing/unit-testing-mstest-sdk.md). + ## Hosting-related properties The following MSBuild properties are documented in this section: diff --git a/docs/core/project-sdk/overview.md b/docs/core/project-sdk/overview.md index 1e253233427c6..bc0c43cac9744 100644 --- a/docs/core/project-sdk/overview.md +++ b/docs/core/project-sdk/overview.md @@ -21,7 +21,7 @@ The following SDKs are available: | `Microsoft.NET.Sdk.Razor` | The .NET [Razor SDK](/aspnet/core/razor-pages/sdk) | | | `Microsoft.NET.Sdk.BlazorWebAssembly` | The .NET [Blazor WebAssembly SDK](/aspnet/core/blazor#blazor-webassembly) | | | `Microsoft.NET.Sdk.Worker` | The .NET [Worker Service](../extensions/workers.md) SDK | | -| `MSTest.Sdk` | The MSTest SDK | | +| `MSTest.Sdk` | The [MSTest SDK](../testing/unit-testing-mstest-sdk) | | The .NET SDK is the base SDK for .NET. The other SDKs reference the .NET SDK, and projects that are associated with the other SDKs have all the .NET SDK properties available to them. The Web SDK, for example, depends on both the .NET SDK and the Razor SDK. diff --git a/docs/core/testing/unit-testing-mstest-sdk.md b/docs/core/testing/unit-testing-mstest-sdk.md index b43f54853a317..893f6885946dc 100644 --- a/docs/core/testing/unit-testing-mstest-sdk.md +++ b/docs/core/testing/unit-testing-mstest-sdk.md @@ -64,18 +64,20 @@ When you `build` the project, all the needed components are restored and install You don't need anything else to build and run your tests and you can use the same tooling (for example, `dotnet test` or Visual Studio) used by a ["classic" MSTest project](./unit-testing-with-mstest.md). > [!IMPORTANT] -> By switching to the `MSTest.Sdk`, you opt in to using the [MSTest runner](./unit-testing-mstest-runner-intro.md), including with [dotnet test](./unit-testing-platform-integration-dotnet-test.md#dotnet-test---microsofttestingplatform-mode) which requires modifying your CI, local CLI calls and also impacts the available entries of the _.runsettings_. -> You can use `MSTest.Sdk` and still keep the old integrations and tools by instead switching the [runner](#select-the-runner). +> By switching to the `MSTest.Sdk`, you opt in to using the [MSTest runner](./unit-testing-mstest-runner-intro.md), including with [dotnet test](./unit-testing-platform-integration-dotnet-test.md#dotnet-test---microsofttestingplatform-mode). That requires modifying your CI and local CLI calls, and also impacts the available entries of the _.runsettings_. You can use `MSTest.Sdk` and still keep the old integrations and tools by instead switching the [runner](#select-the-runner). ## Select the runner -By default, MSTest SDK relies on [MSTest runner](./unit-testing-mstest-runner-intro.md), but you can easily switch to [VSTest](/visualstudio/test/vstest-console-options) by adding the property `true`. +By default, MSTest SDK relies on [MSTest runner](./unit-testing-mstest-runner-intro.md), but you can switch to [VSTest](/visualstudio/test/vstest-console-options) by adding the property `true`. ## Extend MSTest runner -You can customize `MSTest runner` experience through a set of [NuGet package extensions](./unit-testing-platform-extensions.md). To simplify and improve this experience, MSTest SDK introduces two features. +You can customize `MSTest runner` experience through a set of [NuGet package extensions](./unit-testing-platform-extensions.md). To simplify and improve this experience, MSTest SDK introduces two features: -### MSTest Runner profile +- [MSTest runner profile](#mstest-runner-profile) +- [Enable or disable extensions](#enable-or-disable-extensions) + +### MSTest runner profile The concept of *profiles* allows you to select the default set of configurations and extensions that will be applied to your test project. @@ -119,16 +121,16 @@ For example, to enable the crash dump extension (NuGet package [Microsoft.Testin ``` -This page lists all [extensions](./unit-testing-platform-extensions.md) available. +For a list of all available extensions, see [Microsoft.Testing.Platform extensions](./unit-testing-platform-extensions.md). > [!WARNING] -> Please review the licensing terms for each extension as they may vary. +> It's important to review the licensing terms for each extension as they might vary. Enabled and disabled extensions are combined with the extensions provided by your selected extension profile. -This can be used to enable an additional extension on top of the implicit `Default` profile (as seen in the previous CrashDumpExtension example). +This property pattern can be used to enable an additional extension on top of the implicit `Default` profile (as seen in the previous CrashDumpExtension example). -Or to disable an extension that is coming from the selected profile. In this case disabling `MS Code Coverage` extension by setting `false`: +You can also disable an extension that's coming from the selected profile. For example, disable the `MS Code Coverage` extension by setting `false`: ```xml @@ -145,16 +147,16 @@ Or to disable an extension that is coming from the selected profile. In this cas ## Features -Outside of the selection of the runner and runner specific extensions, `MSTest.Sdk` also provides additional features to simplify and enhance your testing experience. +Outside of the selection of the runner and runner-specific extensions, `MSTest.Sdk` also provides additional features to simplify and enhance your testing experience. -### .NET Aspire +### Test with .NET Aspire .NET Aspire is an opinionated, cloud-ready stack for building observable, production ready, distributed applications. .NET Aspire is delivered through a collection of NuGet packages that handle specific cloud-native concerns. For more information, see the [.NET Aspire docs](/dotnet/aspire/get-started/aspire-overview). > [!NOTE] > This feature is available from MSTest.Sdk 3.4.0 -By setting the property `EnableAspireTesting` to `true`, you can bring all dependencies and default `using` directives you would need for testing with `Aspire` and `MSTest`. +By setting the property `EnableAspireTesting` to `true`, you can bring all dependencies and default `using` directives you need for testing with `Aspire` and `MSTest`. ```xml @@ -176,7 +178,7 @@ Playwright enables reliable end-to-end testing for modern web apps. For more inf > [!NOTE] > This feature is available from MSTest.Sdk 3.4.0 -By setting the property `EnablePlaywright` to `true` you can bring all dependencies and default `using` directives you would need for testing with `Playwright` and `MSTest`. +By setting the property `EnablePlaywright` to `true` you can bring in all the dependencies and default `using` directives you need for testing with `Playwright` and `MSTest`. ```xml @@ -191,11 +193,11 @@ By setting the property `EnablePlaywright` to `true` you can bring all dependenc ``` -## Migrating to MSTest SDK +## Migrate to MSTest SDK Consider the following steps that are required to migrate to the MSTest SDK. -### Update your project(s) +### Update your project When migrating an existing MSTest test project to MSTest SDK, start by replacing the `Sdk="Microsoft.NET.Sdk"` entry at the top of your test project with `Sdk="MSTest.Sdk/3.3.1"` @@ -239,7 +241,7 @@ Finally, based on the extensions profile you're using, you can also remove some ### Update your CI -Once you've updated your projects, if you're using `MSTest runner` (default) and if you rely on `dotnet test` to run your tests, you have to update your CI configuration. For more information, see [dotnet test integration](./unit-testing-platform-integration-dotnet-test.md#dotnet-test---microsofttestingplatform-mode) to guide your understanding of all the required changes. +Once you've updated your projects, if you're using `MSTest runner` (default) and if you rely on `dotnet test` to run your tests, you must update your CI configuration. For more information and to guide your understanding of all the required changes, see [dotnet test integration](./unit-testing-platform-integration-dotnet-test.md#dotnet-test---microsofttestingplatform-mode). Here's an example update when using the `DotNetCoreCLI` task in Azure DevOps: diff --git a/docs/core/testing/unit-testing-platform-faq.md b/docs/core/testing/unit-testing-platform-faq.md index 7ca8ad233a777..29a3868af3860 100644 --- a/docs/core/testing/unit-testing-platform-faq.md +++ b/docs/core/testing/unit-testing-platform-faq.md @@ -12,10 +12,10 @@ This article contains answers to commonly asked questions about `Microsoft.Testi ## error CS8892: Method 'TestingPlatformEntryPoint.Main(string[])' will not be used as an entry point because a synchronous entry point 'Program.Main(string[])' was found -Manually defining an entry point (`Main`) in test project, or referencing a test project into an application that already has an entry point, will result in conflict with the entry point generated by `Microsoft.Testing.Platform`. To avoid this issue take one of these steps: +Manually defining an entry point (`Main`) in a test project or referencing a test project from an application that already has an entry point results in a conflict with the entry point generated by `Microsoft.Testing.Platform`. To avoid this issue, take one of these steps: -- Remove your manually defined entry point, typically `Main` method in _Program.cs_, and let testing platform to generate one for you. +- Remove your manually defined entry point, typically `Main` method in _Program.cs_, and let the testing platform generate one for you. -- Disable the generation of the entry point by setting `false` MSBuild property. +- Disable the generation of the entry point by setting the `false` MSBuild property. -- Completely disable the transitive dependency to `Microsoft.Testing.Platform.MSBuild` by setting `false` MSBuild property. This is needed when you are referencing test project from a non-test project, for example: You have a console application referencing a test application. +- Completely disable the transitive dependency to `Microsoft.Testing.Platform.MSBuild` by setting the `false` MSBuild property in the project that references a test project. This is needed when you reference a test project from a non-test project, for example, a console app that references a test application. diff --git a/docs/core/testing/unit-testing-platform-intro.md b/docs/core/testing/unit-testing-platform-intro.md index 722a484aa09ef..6d10b837594d2 100644 --- a/docs/core/testing/unit-testing-platform-intro.md +++ b/docs/core/testing/unit-testing-platform-intro.md @@ -213,10 +213,10 @@ The directory where the test results are going to be placed. If the specified di The NuGet package [Microsoft.Testing.Platform.MSBuild](https://www.nuget.org/packages/Microsoft.Testing.Platform.MSBuild) provides various integrations for `Microsoft.Testing.Platform` with MSBuild: -- Support for `dotnet test`, see [dotnet test integration](./unit-testing-platform-integration-dotnet-test.md) for more information. +- Support for `dotnet test`. For more information, see [dotnet test integration](./unit-testing-platform-integration-dotnet-test.md). - Support for `ProjectCapability` required by `Visual Studio` and `Visual Studio Code` Test Explorers. -- Automatic generation of the entry point (`Main` method) -- Automatic generation of the configuration file +- Automatic generation of the entry point (`Main` method). +- Automatic generation of the configuration file. > [!NOTE] > This integration works in a transitive way (a project that references another project referencing this package will behave as if it references the package) and can be disabled through the `IsTestingPlatformApplication` MSBuild property. From 97ff32809469704f4878131e07491e18b6234694 Mon Sep 17 00:00:00 2001 From: Genevieve Warren <24882762+gewarren@users.noreply.github.com> Date: Fri, 30 Aug 2024 16:57:33 -0700 Subject: [PATCH 3/5] finish --- docs/core/project-sdk/msbuild-props.md | 30 ++++++++++++++----- .../unit-testing-mstest-runner-intro.md | 4 +-- docs/core/testing/unit-testing-mstest-sdk.md | 4 +++ ...esting-platform-integration-dotnet-test.md | 23 +++++++++----- 4 files changed, 44 insertions(+), 17 deletions(-) diff --git a/docs/core/project-sdk/msbuild-props.md b/docs/core/project-sdk/msbuild-props.md index 77b52d5562d42..633cec2637931 100644 --- a/docs/core/project-sdk/msbuild-props.md +++ b/docs/core/project-sdk/msbuild-props.md @@ -1445,7 +1445,7 @@ The following MSBuild properties are documented in this section: The `IsTestProject` property signifies that a project is a test project. When this property is set to `true`, validation to check if the project references a self-contained executable is disabled. That's because test projects have an `OutputType` of `Exe` but usually call APIs in a referenced executable rather than trying to run. In addition, if a project references a project where `IsTestProject` is set to `true`, the test project isn't validated as an executable reference. > [!NOTE] -> If your project specifies the [MSTest SDK](../testing/unit-testing-mstest-sdk.md), you don't need to set this property. +> If your project specifies the [MSTest SDK](../testing/unit-testing-mstest-sdk.md), you don't need to set this property. It's set automatically. ### IsTestingPlatformApplication @@ -1475,31 +1475,45 @@ For more information, see [Playwright](../testing/unit-testing-mstest-sdk.md#pla ### EnableMSTestRunner -The `EnableMSTestRunner` property ... +The `EnableMSTestRunner` property enables or disables the use of the [MSTest runner](../testing/unit-testing-mstest-runner-intro.md). The MSTest runner is a lightweight and portable alternative to VSTest. + +> [!NOTE] +> If your project specifies the [MSTest SDK](../testing/unit-testing-mstest-sdk.md), you don't need to set this property. It's set automatically. ### EnableNUnitRunner -The `EnableNUnitRunner` property ... +The `EnableNUnitRunner` property enables or disables the use of the [NUnit runner](../testing/unit-testing-nunit-runner-intro.md). The NUnit runner is a lightweight and portable alternative to VSTest. ### GenerateTestingPlatformEntryPoint -The `GenerateTestingPlatformEntryPoint` property ... +Setting the `GenerateTestingPlatformEntryPoint` property to `false` disables the automatic generation of the program entry point in a test project. You might want to set this property to `false` when you manually define an entry point, or when you reference a test project from an executable that also has an entry point. + +For more information, see [error CS8892](../testing/unit-testing-platform-faq.md#error-cs8892-method-testingplatformentrypointmainstring-will-not-be-used-as-an-entry-point-because-a-synchronous-entry-point-programmainstring-was-found). ### TestingPlatformCaptureOutput -The `TestingPlatformCaptureOutput` property ... +The `TestingPlatformCaptureOutput` property controls whether all console output that a test executable writes is captured and hidden from the user when you use `dotnet test` to run `Microsoft.Testing.Platform` tests. By default, the console output is hidden. This output includes the banner, version information, and formatted test information. Set this property to `false` to show this information together with MSBuild output. + +For more information, see [Show complete platform output](../testing/unit-testing-platform-integration-dotnet-test.md#show-complete-platform-output). ### TestingPlatformCommandLineArguments -The `TestingPlatformCommandLineArguments` property ... +The `TestingPlatformCaptureOutput` property lets you specify command-line arguments to the test app when you use `dotnet test` to run `Microsoft.Testing.Platform` tests. The following project file snippet shows an example. + +```xml + + ... + --minimum-expected-tests 10 + +``` ### TestingPlatformDotnetTestSupport -The `TestingPlatformDotnetTestSupport` property ... +The `TestingPlatformDotnetTestSupport` property lets you specify whether VSTest is used when you use `dotnet test` to run tests. If you set this property to `true`, VSTest is disabled and all `Microsoft.Testing.Platform` tests are run directly. ### TestingPlatformShowTestsFailure -The `TestingPlatformShowTestsFailure` property ... +The `TestingPlatformShowTestsFailure` property lets you control whether a single failure or all errors in a failed test are reported when you use `dotnet test` to run tests. By default, test failures are summarized into a _.log_ file, and a single failure per test project is reported to MSBuild. To show errors per failed test, set this property to `true` in your project file. ### TestingExtensionsProfile diff --git a/docs/core/testing/unit-testing-mstest-runner-intro.md b/docs/core/testing/unit-testing-mstest-runner-intro.md index 057431916d62f..e626e2e589566 100644 --- a/docs/core/testing/unit-testing-mstest-runner-intro.md +++ b/docs/core/testing/unit-testing-mstest-runner-intro.md @@ -12,9 +12,9 @@ The MSTest runner is a lightweight and portable alternative to [VSTest](https:// The MSTest runner is open source, and builds on a [`Microsoft.Testing.Platform`](./unit-testing-platform-intro.md) library. You can find `Microsoft.Testing.Platform` code in [microsoft/testfx](https://github.com/microsoft/testfx/tree/main/src/Platform/Microsoft.Testing.Platform) GitHub repository. The MSTest runner comes bundled with `MSTest in 3.2.0-preview.23623.1` or newer. -## Enable MSTest runner in a MSTest project +## Enable MSTest runner in an MSTest project -It's recommended to use [MSTest SDK](./unit-testing-mstest-sdk.md) as it greatly simplifies your project configuration, the updating of the project and ensures a proper alignment of the versions of the platform (MSTest runner) and its extensions. +It's recommended to use [MSTest SDK](./unit-testing-mstest-sdk.md) as it greatly simplifies your project configuration and updating the project, and it ensures a proper alignment of the versions of the platform (MSTest runner) and its extensions. When you use `MSTest SDK`, by default you're opted in to using MSTest runner. diff --git a/docs/core/testing/unit-testing-mstest-sdk.md b/docs/core/testing/unit-testing-mstest-sdk.md index 893f6885946dc..323be2507cd61 100644 --- a/docs/core/testing/unit-testing-mstest-sdk.md +++ b/docs/core/testing/unit-testing-mstest-sdk.md @@ -253,3 +253,7 @@ Here's an example update when using the `DotNetCoreCLI` task in Azure DevOps: - arguments: '--configuration Release' + arguments: '--configuration Release -p:TestingPlatformCommandLineArguments="--report-trx --results-directory $(Agent.TempDirectory) --coverage"' ``` + +## See also + +- [Test project–related properties](../project-sdk/msbuild-props.md#test-projectndashrelated-properties) diff --git a/docs/core/testing/unit-testing-platform-integration-dotnet-test.md b/docs/core/testing/unit-testing-platform-integration-dotnet-test.md index bd625488c81b0..73aabfd18ef7d 100644 --- a/docs/core/testing/unit-testing-platform-integration-dotnet-test.md +++ b/docs/core/testing/unit-testing-platform-integration-dotnet-test.md @@ -30,7 +30,7 @@ This layer runs test through VSTest and integrates with it on VSTest Test Framew ### `dotnet test` - Microsoft.Testing.Platform mode -By default, VSTest is used to run `Microsoft.Testing.Platform` tests. You can enable a full `Microsoft.Testing.Platform` by specifying the `true` setting in your project. This setting disables VSTest and thanks to the transitive dependency to the [Microsoft.Testing.Platform.MSBuild](https://nuget.org/packages/Microsoft.Testing.Platform.MSBuild) NuGet package it will directly run all `Microsoft.Testing.Platform` empowered tests project in your solution. It works seamlessly if you pass a direct `Microsoft.Testing.Platform` test project. +By default, VSTest is used to run `Microsoft.Testing.Platform` tests. You can enable a full `Microsoft.Testing.Platform` by specifying the `true` setting in your project file. This setting disables VSTest and, thanks to the transitive dependency to the [Microsoft.Testing.Platform.MSBuild](https://nuget.org/packages/Microsoft.Testing.Platform.MSBuild) NuGet package, directly runs all `Microsoft.Testing.Platform` empowered test projects in your solution. It works seamlessly if you pass a direct `Microsoft.Testing.Platform` test project. ```xml @@ -56,23 +56,32 @@ By default, VSTest is used to run `Microsoft.Testing.Platform` tests. You can en ``` -In this mode, you can supply extra parameters that will be utilized to call the testing application as follows: +In this mode, you can supply extra parameters that are used to call the testing application in one of the following ways: -1. Beginning with `Microsoft.Testing.Platform` version 1.4 (included with MSTest version 3.6), options can be added after the double dash `--`: +- Beginning with `Microsoft.Testing.Platform` version 1.4 (included with MSTest version 3.6), you can options after the double dash `--` on the command line: ```dotnetcli dotnet test -- --minimum-expected-tests 10 ``` -1. By using the `TestingPlatformCommandLineArguments` MSBuild property: +- By using the `TestingPlatformCommandLineArguments` MSBuild property on the command line: ```dotnetcli dotnet test -p:TestingPlatformCommandLineArguments=" --minimum-expected-tests 10 " ``` + Or in the project file: + + ```xml + + ... + --minimum-expected-tests 10 + + ``` + ## Additional MSBuild options -The MSBuild integration provides options that can be specified in user project or through global properties on the command line, such as `-p:TestingPlatformShowTestsFailure=true`. +The MSBuild integration provides options that can be specified in the project file or through global properties on the command line, such as `-p:TestingPlatformShowTestsFailure=true`. These are the available options: @@ -81,9 +90,9 @@ These are the available options: ### Show failure per test -By default test failures are summarized into a _.log_ file, and a single failure per test project is reported to MSBuild. +By default, test failures are summarized into a _.log_ file, and a single failure per test project is reported to MSBuild. -To show errors per failed test, specify `-p:TestingPlatformShowTestsFailure=true` on the command line, or add `true` property to your project file. +To show errors per failed test, specify `-p:TestingPlatformShowTestsFailure=true` on the command line, or add the `true` property to your project file. On command line: From 9c95e43117f3b8adcde79e133f35262a40895e37 Mon Sep 17 00:00:00 2001 From: Genevieve Warren <24882762+gewarren@users.noreply.github.com> Date: Fri, 30 Aug 2024 17:07:56 -0700 Subject: [PATCH 4/5] fix warnings --- docs/core/project-sdk/msbuild-props.md | 4 ++-- docs/core/project-sdk/overview.md | 2 +- docs/core/testing/unit-testing-mstest-sdk.md | 4 ++-- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/core/project-sdk/msbuild-props.md b/docs/core/project-sdk/msbuild-props.md index 633cec2637931..2f7470cacccdb 100644 --- a/docs/core/project-sdk/msbuild-props.md +++ b/docs/core/project-sdk/msbuild-props.md @@ -1465,13 +1465,13 @@ For more information, see [Enable or disable extensions](../testing/unit-testing When you use the [MSTest project SDK](../testing/unit-testing-mstest-sdk.md), you can use the `EnableAspireTesting` property to bring in all the dependencies and default `using` directives you need for testing with `Aspire` and `MSTest`. -For more information, see [Test with .NET Aspire](../testing/unit-testing-mstest-sdk.md#net-aspire). +For more information, see [Test with .NET Aspire](../testing/unit-testing-mstest-sdk.md#test-with-net-aspire). ### EnablePlaywright When you use the [MSTest project SDK](../testing/unit-testing-mstest-sdk.md), you can use the `EnablePlaywright` property to bring in all the dependencies and default `using` directives you need for testing with `Playwright` and `MSTest`. -For more information, see [Playwright](../testing/unit-testing-mstest-sdk.md#playwright). +For more information, see [Playwright](../testing/unit-testing-mstest-sdk.md#test-with-playwright). ### EnableMSTestRunner diff --git a/docs/core/project-sdk/overview.md b/docs/core/project-sdk/overview.md index bc0c43cac9744..8b64410c67601 100644 --- a/docs/core/project-sdk/overview.md +++ b/docs/core/project-sdk/overview.md @@ -21,7 +21,7 @@ The following SDKs are available: | `Microsoft.NET.Sdk.Razor` | The .NET [Razor SDK](/aspnet/core/razor-pages/sdk) | | | `Microsoft.NET.Sdk.BlazorWebAssembly` | The .NET [Blazor WebAssembly SDK](/aspnet/core/blazor#blazor-webassembly) | | | `Microsoft.NET.Sdk.Worker` | The .NET [Worker Service](../extensions/workers.md) SDK | | -| `MSTest.Sdk` | The [MSTest SDK](../testing/unit-testing-mstest-sdk) | | +| `MSTest.Sdk` | The [MSTest SDK](../testing/unit-testing-mstest-sdk.md) | | The .NET SDK is the base SDK for .NET. The other SDKs reference the .NET SDK, and projects that are associated with the other SDKs have all the .NET SDK properties available to them. The Web SDK, for example, depends on both the .NET SDK and the Razor SDK. diff --git a/docs/core/testing/unit-testing-mstest-sdk.md b/docs/core/testing/unit-testing-mstest-sdk.md index 323be2507cd61..9532be72279d4 100644 --- a/docs/core/testing/unit-testing-mstest-sdk.md +++ b/docs/core/testing/unit-testing-mstest-sdk.md @@ -171,7 +171,7 @@ By setting the property `EnableAspireTesting` to `true`, you can bring all depen ``` -### Playwright +### Test with Playwright Playwright enables reliable end-to-end testing for modern web apps. For more information, see the official [Playwright docs](https://playwright.dev/dotnet/docs/intro). @@ -256,4 +256,4 @@ Here's an example update when using the `DotNetCoreCLI` task in Azure DevOps: ## See also -- [Test project–related properties](../project-sdk/msbuild-props.md#test-projectndashrelated-properties) +- [Test project–related properties](../project-sdk/msbuild-props.md#test-projectrelated-properties) From b5d1a9552fe8bb0176cacd0217f824e46a70a56c Mon Sep 17 00:00:00 2001 From: Genevieve Warren <24882762+gewarren@users.noreply.github.com> Date: Mon, 9 Sep 2024 12:35:49 -0700 Subject: [PATCH 5/5] respond to feedback --- docs/core/project-sdk/msbuild-props.md | 28 +++++++++++++------ ...esting-platform-integration-dotnet-test.md | 2 +- 2 files changed, 21 insertions(+), 9 deletions(-) diff --git a/docs/core/project-sdk/msbuild-props.md b/docs/core/project-sdk/msbuild-props.md index 2f7470cacccdb..0099e25f81437 100644 --- a/docs/core/project-sdk/msbuild-props.md +++ b/docs/core/project-sdk/msbuild-props.md @@ -1444,14 +1444,22 @@ The following MSBuild properties are documented in this section: The `IsTestProject` property signifies that a project is a test project. When this property is set to `true`, validation to check if the project references a self-contained executable is disabled. That's because test projects have an `OutputType` of `Exe` but usually call APIs in a referenced executable rather than trying to run. In addition, if a project references a project where `IsTestProject` is set to `true`, the test project isn't validated as an executable reference. +This property is mainly needed for the `dotnet test` scenario and has no impact when using *vstest.console.exe*. + > [!NOTE] -> If your project specifies the [MSTest SDK](../testing/unit-testing-mstest-sdk.md), you don't need to set this property. It's set automatically. +> If your project specifies the [MSTest SDK](../testing/unit-testing-mstest-sdk.md), you don't need to set this property. It's set automatically. Similarly, this property is set automatically for projects that reference the Microsoft.NET.Test.Sdk NuGet package linked to VSTest. ### IsTestingPlatformApplication -Setting the `IsTestingPlatformApplication` property to `false` disables the transitive dependency to the [Microsoft.Testing.Platform.MSBuild](https://www.nuget.org/packages/Microsoft.Testing.Platform.MSBuild) package. A *transitive dependency* is when a project that references another project that references a given package behaves as if *it* references the package. You'd typically set this property to `false` in a non-test project that references a test project. +When your project references the [Microsoft.Testing.Platform.MSBuild](https://www.nuget.org/packages/Microsoft.Testing.Platform.MSBuild) package, setting `IsTestingPlatformApplication` to `true` (which is also the default value if not specified) does the following: -For more information, see [error CS8892](../testing/unit-testing-platform-faq.md#error-cs8892-method-testingplatformentrypointmainstring-will-not-be-used-as-an-entry-point-because-a-synchronous-entry-point-programmainstring-was-found). +- Generates the entry point to the test project. +- Generates the configuration file. +- Detects the extensions. + +Setting the property to `false` disables the transitive dependency to the package. A *transitive dependency* is when a project that references another project that references a given package behaves as if *it* references the package. You'd typically set this property to `false` in a non-test project that references a test project. For more information, see [error CS8892](../testing/unit-testing-platform-faq.md#error-cs8892-method-testingplatformentrypointmainstring-will-not-be-used-as-an-entry-point-because-a-synchronous-entry-point-programmainstring-was-found). + +If your test project references MSTest, NUnit, or xUnit, this property is set to the same value as [EnableMSTestRunner](#enablemstestrunner), [EnableNUnitRunner](#enablenunitrunner), or `UseMicrosoftTestingPlatformRunner` (for xUnit). ### Enable\[NugetPackageNameWithoutDots\] @@ -1463,33 +1471,35 @@ For more information, see [Enable or disable extensions](../testing/unit-testing ### EnableAspireTesting -When you use the [MSTest project SDK](../testing/unit-testing-mstest-sdk.md), you can use the `EnableAspireTesting` property to bring in all the dependencies and default `using` directives you need for testing with `Aspire` and `MSTest`. +When you use the [MSTest project SDK](../testing/unit-testing-mstest-sdk.md), you can use the `EnableAspireTesting` property to bring in all the dependencies and default `using` directives you need for testing with `Aspire` and `MSTest`. This property is available in MSTest 3.4 and later versions. For more information, see [Test with .NET Aspire](../testing/unit-testing-mstest-sdk.md#test-with-net-aspire). ### EnablePlaywright -When you use the [MSTest project SDK](../testing/unit-testing-mstest-sdk.md), you can use the `EnablePlaywright` property to bring in all the dependencies and default `using` directives you need for testing with `Playwright` and `MSTest`. +When you use the [MSTest project SDK](../testing/unit-testing-mstest-sdk.md), you can use the `EnablePlaywright` property to bring in all the dependencies and default `using` directives you need for testing with `Playwright` and `MSTest`.This property is available in MSTest 3.4 and later versions. For more information, see [Playwright](../testing/unit-testing-mstest-sdk.md#test-with-playwright). ### EnableMSTestRunner -The `EnableMSTestRunner` property enables or disables the use of the [MSTest runner](../testing/unit-testing-mstest-runner-intro.md). The MSTest runner is a lightweight and portable alternative to VSTest. +The `EnableMSTestRunner` property enables or disables the use of the [MSTest runner](../testing/unit-testing-mstest-runner-intro.md). The MSTest runner is a lightweight and portable alternative to VSTest. This property is available in MSTest 3.2 and later versions. > [!NOTE] > If your project specifies the [MSTest SDK](../testing/unit-testing-mstest-sdk.md), you don't need to set this property. It's set automatically. ### EnableNUnitRunner -The `EnableNUnitRunner` property enables or disables the use of the [NUnit runner](../testing/unit-testing-nunit-runner-intro.md). The NUnit runner is a lightweight and portable alternative to VSTest. +The `EnableNUnitRunner` property enables or disables the use of the [NUnit runner](../testing/unit-testing-nunit-runner-intro.md). The NUnit runner is a lightweight and portable alternative to VSTest. This property is available in [NUnit3TestAdapter](https://www.nuget.org/packages/NUnit3TestAdapter) in version 5.0 and later. ### GenerateTestingPlatformEntryPoint -Setting the `GenerateTestingPlatformEntryPoint` property to `false` disables the automatic generation of the program entry point in a test project. You might want to set this property to `false` when you manually define an entry point, or when you reference a test project from an executable that also has an entry point. +Setting the `GenerateTestingPlatformEntryPoint` property to `false` disables the automatic generation of the program entry point in an MSTest, NUnit, or xUnit test project. You might want to set this property to `false` when you manually define an entry point, or when you reference a test project from an executable that also has an entry point. For more information, see [error CS8892](../testing/unit-testing-platform-faq.md#error-cs8892-method-testingplatformentrypointmainstring-will-not-be-used-as-an-entry-point-because-a-synchronous-entry-point-programmainstring-was-found). +To control the generation of the entry point in a VSTest project, use the `GenerateProgramFile` property. + ### TestingPlatformCaptureOutput The `TestingPlatformCaptureOutput` property controls whether all console output that a test executable writes is captured and hidden from the user when you use `dotnet test` to run `Microsoft.Testing.Platform` tests. By default, the console output is hidden. This output includes the banner, version information, and formatted test information. Set this property to `false` to show this information together with MSBuild output. @@ -1511,6 +1521,8 @@ The `TestingPlatformCaptureOutput` property lets you specify command-line argume The `TestingPlatformDotnetTestSupport` property lets you specify whether VSTest is used when you use `dotnet test` to run tests. If you set this property to `true`, VSTest is disabled and all `Microsoft.Testing.Platform` tests are run directly. +If you have a solution that contains VSTest test projects as well as MSTest, NUnit, or XUnit projects, you should make one call per mode (that is, `dotnet test` won't run tests from both VSTest and the newer platforms in one call). + ### TestingPlatformShowTestsFailure The `TestingPlatformShowTestsFailure` property lets you control whether a single failure or all errors in a failed test are reported when you use `dotnet test` to run tests. By default, test failures are summarized into a _.log_ file, and a single failure per test project is reported to MSBuild. To show errors per failed test, set this property to `true` in your project file. diff --git a/docs/core/testing/unit-testing-platform-integration-dotnet-test.md b/docs/core/testing/unit-testing-platform-integration-dotnet-test.md index 73aabfd18ef7d..575bedc7c09fb 100644 --- a/docs/core/testing/unit-testing-platform-integration-dotnet-test.md +++ b/docs/core/testing/unit-testing-platform-integration-dotnet-test.md @@ -67,7 +67,7 @@ In this mode, you can supply extra parameters that are used to call the testing - By using the `TestingPlatformCommandLineArguments` MSBuild property on the command line: ```dotnetcli - dotnet test -p:TestingPlatformCommandLineArguments=" --minimum-expected-tests 10 " + dotnet test -p:TestingPlatformCommandLineArguments="--minimum-expected-tests 10" ``` Or in the project file: