Add opt-in language features to the Qsharp.json manifest (microsoft#1089

) Closes microsoft#543 ## Overview This PR adds two things: 1. The ability to opt-in to language features via the project manifest, `qsharp.init()`, or command-line flags 2. A simple feature, `v2-preview-syntax`, ostensibly created to allow users to opt-in to breaking syntax that's coming in the future, but really created so I could wire up and show how a real language feature works. Only one syntax change was made in this feature: removal of scoped qubit blocks. This is because a scoped qubit block is identical to a scoped block with a qubit in it. `use qubit = Qubit () {};` is equivalent to `{ use qubit = Qubit(); }`. We therefore prefer the latter. ## Implementation Notes This PR includes the addition of `ParserContext`. It became clear to me that language features need to be globally available to _at least_ the parser, and probably other areas of the compiler later. Instead of plumbing this all around manually as an argument, I promoted `Scanner` into a `ParserContext`. This PR also refactors manifest loading in `projectSystem.ts` -- we had some duplicate code in `getManifestThrowsOnParseFailure` and `getManifest`. I DRY'd that up. ## FAQ ### Why opt-in features? In our discussions of evolving the language, it became clear that we prioritize both improving the language, including in breaking ways, and maintaining compatibility. As a result, introducing the notion of being able to opt-in to a specific language feature became a blocker for most further language design work. ### How will packages with different feature sets work together? In general, we should aim to write features that don't break compatibility with other packages. The language server and all compiler scenarios assess language features on a per-package basis, so as long as the individual compilation units are sane and well-formed, their feature sets should be interoperable. ## Tests These tests are considered _passing_ if turning on the language feature `v2-preview-syntax` causes the compiler to reject scoped qubit block syntax. ✅ Histogram: https://github.com/microsoft/qsharp/assets/12157751/631dded2-d016-4933-83fe-bdff67852f50 ✅ Language Service https://github.com/microsoft/qsharp/assets/12157751/964574f6-c6bd-4225-acd4-6eb2afaec41c ✅ Jupyter Notebook https://github.com/microsoft/qsharp/assets/12157751/fc5cc8b8-1c47-4dd9-b478-a233f16fc9e3 ✅ `qsc` https://github.com/microsoft/qsharp/assets/12157751/48e746fc-f7fe-4e9a-a658-d428b640d062 ✅ `qsi` https://github.com/microsoft/qsharp/assets/12157751/8154de70-80c5-444c-ac4e-f33d775a2fa9 ✅ Debugger https://github.com/microsoft/qsharp/assets/12157751/5aa7fb5a-3ec4-43fe-b256-6b75c808fe7e ✅ Language service inside of notebook cells ![image](https://github.com/microsoft/qsharp/assets/12157751/b710cc71-ce7d-4732-8acb-f315778239a1) ✅ QIR Generation https://github.com/microsoft/qsharp/assets/12157751/eed50ca5-85c4-4434-9443-a6c67ef10337
nirajvenkat · Feb 26, 2024 · 7a91721 · 7a91721
1 parent 224219f
commit 7a91721
Show file tree

Hide file tree

Showing 77 changed files with 1,308 additions and 363 deletions.
diff --git a/Cargo.lock b/Cargo.lock
diff --git a/Cargo.toml b/Cargo.toml
@@ -54,7 +54,7 @@ probability = "0.20"
 indenter = "0.3"
 regex-lite = "0.1"
 rustc-hash = "1.1.0"
-serde = "1.0"
+serde = { version = "1.0", features = [ "derive" ] }
 serde-wasm-bindgen = "0.6"
 wasm-bindgen = "0.2"
 wasm-bindgen-futures = "0.4"

diff --git a/compiler/qsc/benches/eval.rs b/compiler/qsc/benches/eval.rs
@@ -4,6 +4,7 @@
 use criterion::{criterion_group, criterion_main, Criterion};
 use indoc::indoc;
 use qsc::{interpret::Interpreter, PackageType};
+use qsc_data_structures::language_features::LanguageFeatures;
 use qsc_eval::output::GenericReceiver;
 use qsc_frontend::compile::{RuntimeCapabilityFlags, SourceMap};
 
@@ -20,6 +21,7 @@ pub fn teleport(c: &mut Criterion) {
             sources,
             PackageType::Exe,
             RuntimeCapabilityFlags::all(),
+            LanguageFeatures::default(),
         )
         .expect("code should compile");
         b.iter(move || {
@@ -38,6 +40,7 @@ pub fn deutsch_jozsa(c: &mut Criterion) {
             sources,
             PackageType::Exe,
             RuntimeCapabilityFlags::all(),
+            LanguageFeatures::default(),
         )
         .expect("code should compile");
         b.iter(move || {
@@ -56,6 +59,7 @@ pub fn large_file(c: &mut Criterion) {
             sources,
             PackageType::Exe,
             RuntimeCapabilityFlags::all(),
+            LanguageFeatures::default(),
         )
         .expect("code should compile");
         b.iter(move || {
@@ -86,6 +90,7 @@ pub fn array_append(c: &mut Criterion) {
             sources,
             PackageType::Exe,
             RuntimeCapabilityFlags::all(),
+            LanguageFeatures::default(),
         )
         .expect("code should compile");
         b.iter(move || {
@@ -116,6 +121,7 @@ pub fn array_update(c: &mut Criterion) {
             sources,
             PackageType::Exe,
             RuntimeCapabilityFlags::all(),
+            LanguageFeatures::default(),
         )
         .expect("code should compile");
         b.iter(move || {
@@ -134,6 +140,7 @@ pub fn array_literal(c: &mut Criterion) {
             sources,
             PackageType::Exe,
             RuntimeCapabilityFlags::all(),
+            LanguageFeatures::default(),
         )
         .expect("code should compile");
         b.iter(move || {
@@ -169,6 +176,7 @@ pub fn large_nested_iteration(c: &mut Criterion) {
             sources,
             PackageType::Exe,
             RuntimeCapabilityFlags::all(),
+            LanguageFeatures::default(),
         )
         .expect("code should compile");
         b.iter(move || {

diff --git a/compiler/qsc/benches/large.rs b/compiler/qsc/benches/large.rs
@@ -3,6 +3,7 @@
 
 use criterion::{criterion_group, criterion_main, Criterion};
 use qsc::compile::{self, compile};
+use qsc_data_structures::language_features::LanguageFeatures;
 use qsc_frontend::compile::{PackageStore, RuntimeCapabilityFlags, SourceMap};
 use qsc_passes::PackageType;
 
@@ -20,6 +21,7 @@ pub fn large_file(c: &mut Criterion) {
                 sources,
                 PackageType::Exe,
                 RuntimeCapabilityFlags::all(),
+                LanguageFeatures::default(),
             );
             assert!(reports.is_empty());
         })

diff --git a/compiler/qsc/src/bin/qsc.rs b/compiler/qsc/src/bin/qsc.rs
@@ -9,6 +9,7 @@ use log::info;
 use miette::{Context, IntoDiagnostic, Report};
 use qsc::compile::compile;
 use qsc_codegen::qir_base;
+use qsc_data_structures::language_features::LanguageFeatures;
 use qsc_frontend::{
     compile::{PackageStore, RuntimeCapabilityFlags, SourceContents, SourceMap, SourceName},
     error::WithSource,
@@ -55,6 +56,10 @@ struct Cli {
     /// Path to a Q# manifest for a project
     #[arg(short, long)]
     qsharp_json: Option<PathBuf>,
+
+    /// Language features to compile with
+    #[arg(short, long)]
+    features: Vec<String>,
 }
 
 #[derive(Copy, Clone, Debug, PartialEq, Eq, PartialOrd, Ord, ValueEnum)]
@@ -79,6 +84,8 @@ fn main() -> miette::Result<ExitCode> {
         dependencies.push(store.insert(qsc::compile::std(&store, capabilities)));
     }
 
+    let mut features = LanguageFeatures::from_iter(cli.features);
+
     let mut sources = cli
         .sources
         .iter()
@@ -93,12 +100,23 @@ fn main() -> miette::Result<ExitCode> {
             let mut project_sources = project.sources;
 
             sources.append(&mut project_sources);
+
+            features.merge(LanguageFeatures::from_iter(
+                manifest.manifest.language_features,
+            ));
         }
     }
 
     let entry = cli.entry.unwrap_or_default();
     let sources = SourceMap::new(sources, Some(entry.into()));
-    let (unit, errors) = compile(&store, &dependencies, sources, package_type, capabilities);
+    let (unit, errors) = compile(
+        &store,
+        &dependencies,
+        sources,
+        package_type,
+        capabilities,
+        features,
+    );
     let package_id = store.insert(unit);
     let unit = store.get(package_id).expect("package should be in store");
 

diff --git a/compiler/qsc/src/bin/qsi.rs b/compiler/qsc/src/bin/qsi.rs
@@ -9,6 +9,7 @@ use miette::{Context, IntoDiagnostic, Report, Result};
 use num_bigint::BigUint;
 use num_complex::Complex64;
 use qsc::interpret::{self, InterpretResult, Interpreter};
+use qsc_data_structures::language_features::LanguageFeatures;
 use qsc_eval::{
     output::{self, Receiver},
     val::Value,
@@ -47,6 +48,10 @@ struct Cli {
     /// Path to a Q# manifest for a project
     #[arg(short, long)]
     qsharp_json: Option<PathBuf>,
+
+    /// Language features to compile with
+    #[arg(short, long)]
+    features: Vec<String>,
 }
 
 struct TerminalReceiver;
@@ -80,6 +85,8 @@ fn main() -> miette::Result<ExitCode> {
         .map(read_source)
         .collect::<miette::Result<Vec<_>>>()?;
 
+    let mut features = LanguageFeatures::from_iter(cli.features);
+
     if sources.is_empty() {
         let fs = StdFs;
         let manifest = Manifest::load(cli.qsharp_json)?;
@@ -88,6 +95,10 @@ fn main() -> miette::Result<ExitCode> {
             let mut project_sources = project.sources;
 
             sources.append(&mut project_sources);
+
+            features.merge(LanguageFeatures::from_iter(
+                manifest.manifest.language_features,
+            ));
         }
     }
     if cli.exec {
@@ -96,6 +107,7 @@ fn main() -> miette::Result<ExitCode> {
             SourceMap::new(sources, cli.entry.map(std::convert::Into::into)),
             PackageType::Exe,
             RuntimeCapabilityFlags::all(),
+            features,
         ) {
             Ok(interpreter) => interpreter,
             Err(errors) => {
@@ -115,6 +127,7 @@ fn main() -> miette::Result<ExitCode> {
         SourceMap::new(sources, None),
         PackageType::Lib,
         RuntimeCapabilityFlags::all(),
+        features,
     ) {
         Ok(interpreter) => interpreter,
         Err(errors) => {

diff --git a/compiler/qsc/src/compile.rs b/compiler/qsc/src/compile.rs
@@ -2,6 +2,7 @@
 // Licensed under the MIT License.
 
 use miette::{Diagnostic, Report};
+use qsc_data_structures::language_features::LanguageFeatures;
 use qsc_frontend::{
     compile::{CompileUnit, PackageStore, RuntimeCapabilityFlags, SourceMap},
     error::WithSource,
@@ -27,8 +28,15 @@ pub fn compile(
     sources: SourceMap,
     package_type: PackageType,
     capabilities: RuntimeCapabilityFlags,
+    language_features: LanguageFeatures,
 ) -> (CompileUnit, Vec<Error>) {
-    let mut unit = qsc_frontend::compile::compile(store, dependencies, sources, capabilities);
+    let mut unit = qsc_frontend::compile::compile(
+        store,
+        dependencies,
+        sources,
+        capabilities,
+        language_features,
+    );
     let mut errors = Vec::new();
     for error in unit.errors.drain(..) {
         errors.push(WithSource::from_map(&unit.sources, error.into()));

diff --git a/compiler/qsc/src/incremental.rs b/compiler/qsc/src/incremental.rs
@@ -3,6 +3,7 @@
 
 use crate::compile::{self, compile, core, std};
 use miette::Diagnostic;
+use qsc_data_structures::language_features::LanguageFeatures;
 use qsc_frontend::{
     compile::{OpenPackageStore, PackageStore, RuntimeCapabilityFlags, SourceMap},
     error::WithSource,
@@ -37,6 +38,7 @@ impl Compiler {
         sources: SourceMap,
         package_type: PackageType,
         capabilities: RuntimeCapabilityFlags,
+        language_features: LanguageFeatures,
     ) -> Result<Self, Errors> {
         let core = core();
         let mut store = PackageStore::new(core);
@@ -47,15 +49,27 @@ impl Compiler {
             dependencies.push(id);
         }
 
-        let (unit, errors) = compile(&store, &dependencies, sources, package_type, capabilities);
+        let (unit, errors) = compile(
+            &store,
+            &dependencies,
+            sources,
+            package_type,
+            capabilities,
+            language_features,
+        );
         if !errors.is_empty() {
             return Err(errors);
         }
 
         let source_package_id = store.insert(unit);
         dependencies.push(source_package_id);
 
-        let frontend = qsc_frontend::incremental::Compiler::new(&store, dependencies, capabilities);
+        let frontend = qsc_frontend::incremental::Compiler::new(
+            &store,
+            dependencies,
+            capabilities,
+            language_features,
+        );
         let store = store.open();
 
         Ok(Self {

diff --git a/compiler/qsc/src/interpret.rs b/compiler/qsc/src/interpret.rs
@@ -26,6 +26,7 @@ use num_bigint::BigUint;
 use num_complex::Complex;
 use qsc_codegen::qir_base::BaseProfSim;
 use qsc_data_structures::{
+    language_features::LanguageFeatures,
     line_column::{Encoding, Range},
     span::Span,
 };
@@ -123,12 +124,13 @@ impl Interpreter {
         sources: SourceMap,
         package_type: PackageType,
         capabilities: RuntimeCapabilityFlags,
+        language_features: LanguageFeatures,
     ) -> Result<Self, Vec<Error>> {
         let mut lowerer = qsc_eval::lower::Lowerer::new();
         let mut fir_store = fir::PackageStore::new();
 
-        let compiler =
-            Compiler::new(std, sources, package_type, capabilities).map_err(into_errors)?;
+        let compiler = Compiler::new(std, sources, package_type, capabilities, language_features)
+            .map_err(into_errors)?;
 
         for (id, unit) in compiler.package_store() {
             fir_store.insert(
@@ -359,8 +361,15 @@ impl Debugger {
         sources: SourceMap,
         capabilities: RuntimeCapabilityFlags,
         position_encoding: Encoding,
+        language_features: LanguageFeatures,
     ) -> Result<Self, Vec<Error>> {
-        let interpreter = Interpreter::new(true, sources, PackageType::Exe, capabilities)?;
+        let interpreter = Interpreter::new(
+            true,
+            sources,
+            PackageType::Exe,
+            capabilities,
+            language_features,
+        )?;
         let source_package_id = interpreter.source_package;
         Ok(Self {
             interpreter,

diff --git a/compiler/qsc/src/interpret/debug/tests.rs b/compiler/qsc/src/interpret/debug/tests.rs
@@ -5,6 +5,7 @@
 
 use indoc::indoc;
 use miette::Result;
+use qsc_data_structures::language_features::LanguageFeatures;
 use qsc_eval::{output::CursorReceiver, val::Value};
 use qsc_frontend::compile::{RuntimeCapabilityFlags, SourceMap};
 use qsc_passes::PackageType;
@@ -70,6 +71,7 @@ fn stack_traces_can_cross_eval_session_and_file_boundaries() {
         source_map,
         PackageType::Lib,
         RuntimeCapabilityFlags::all(),
+        LanguageFeatures::default(),
     )
     .expect("Failed to compile base environment.");
 
@@ -144,6 +146,7 @@ fn stack_traces_can_cross_file_and_entry_boundaries() {
         source_map,
         PackageType::Exe,
         RuntimeCapabilityFlags::all(),
+        LanguageFeatures::default(),
     )
     .expect("Failed to compile base environment.");