[doc] Add draft of Oil vs. Shell

Remove some "future" deprecations, since we implemented them!

Update Oil vs. Python.

[release] Update release note template

build/doc.sh

@@ -93,6 +93,7 @@ readonly MARKDOWN_DOCS=(
   qtt
   language-influences
   oil-vs-python
+  oil-vs-shell
   syntactic-concepts
   syntax-feelings
 

devtools/release-note.sh

@@ -177,13 +177,18 @@ OSH spec tests:
 
 [spec-test]: \$xref:spec-test
 
-- [OSH spec tests for $metric_prev](//www.oilshell.org/release/$metric_prev/test/spec.wwz/osh.html): **1260** tests, **1116** passing, **104** failing
-- [OSH spec tests for $OIL_VERSION](//www.oilshell.org/release/$OIL_VERSION/test/spec.wwz/osh.html): **1393** tests, **1230** passing, **78** failing
+- [OSH spec tests for $metric_prev](//www.oilshell.org/release/$metric_prev/test/spec.wwz/survey/osh.html): **1260** tests, **1116** passing, **104** failing
+- [OSH spec tests for $OIL_VERSION](//www.oilshell.org/release/$OIL_VERSION/test/spec.wwz/survey/osh.html): **1393** tests, **1230** passing, **78** failing
 
 Oil spec tests:
 
-- [Oil spec tests for $metric_prev](//www.oilshell.org/release/$metric_prev/test/spec.wwz/oil.html): **1260** tests, **1116** passing, **104** failing
-- [Oil spec tests for $OIL_VERSION](//www.oilshell.org/release/$OIL_VERSION/test/spec.wwz/oil.html): **1393** tests, **1230** passing, **78** failing
+- [Oil spec tests for $metric_prev](//www.oilshell.org/release/$metric_prev/test/spec.wwz/oil-language/oil.html): **1260** tests, **1116** passing, **104** failing
+- [Oil spec tests for $OIL_VERSION](//www.oilshell.org/release/$OIL_VERSION/test/spec.wwz/oil-language/oil.html): **1393** tests, **1230** passing, **78** failing
+
+Translation progress:
+
+- [OSH C++ spec tests for $metric_prev](//www.oilshell.org/release/$metric_prev/test/spec.wwz/cpp/osh-summary.html): **1260** tests, **1116** passing, **104** failing
+- [OSH C++ spec tests for $OIL_VERSION](//www.oilshell.org/release/$OIL_VERSION/test/spec.wwz/cpp/osh-summary.html): **1393** tests, **1230** passing, **78** failing
 
 We have X significant lines of code:
 
@@ -196,8 +201,10 @@ And X new lines of physical code:
 
 - [src for
   $metric_prev](//www.oilshell.org/release/$metric_prev/metrics.wwz/line-counts/src.txt):
-  **20,553** lines of Python
-- [src for $OIL_VERSION](//www.oilshell.org/release/$OIL_VERSION/metrics.wwz/line-counts/src.txt): **23,343** lines of Python
+  **20,553** in OSH and common libraries, **4,814** in Oil
+- [src for
+  $OIL_VERSION](//www.oilshell.org/release/$OIL_VERSION/metrics.wwz/line-counts/src.txt):
+  **20,553** in OSH and common libraries, **4,814** in Oil
 
 ### Benchmarks
 
@@ -218,8 +225,8 @@ $OIL_VERSION](//www.oilshell.org/release/$OIL_VERSION/benchmarks.wwz/osh-runtime
 
 The lines of native code was reduced:
 
-- [nativedeps for $metric_prev](//www.oilshell.org/release/$metric_prev/metrics.wwz/line-counts/nativedeps.txt): **133,291** lines
-- [nativedeps for $OIL_VERSION](//www.oilshell.org/release/$OIL_VERSION/metrics.wwz/line-counts/nativedeps.txt): **130,403** lines
+- [oil-cpp for $metric_prev](//www.oilshell.org/release/$metric_prev/metrics.wwz/line-counts/oil-cpp.txt): **133,291** lines of C++
+- [oil-cpp for $OIL_VERSION](//www.oilshell.org/release/$OIL_VERSION/metrics.wwz/line-counts/oil-cpp.txt): **130,403** lines of C++
 
 The binary size stayed almost the same:
 

doc/deprecations.md

@@ -128,4 +128,5 @@ Idioms](idioms.html).
 ## Related
 
 - [Ideas for Future Deprecations](future.html)
+- [Oil Language vs. Shell](oil-vs-shell.html)
 

doc/future.md

@@ -11,7 +11,6 @@ Deprecations](deprecations.html).
 These breakages may never happen, as they require a significant new lexer mode.
 If they do, you will want to avoid the following syntax:
 
-- `r'` and `c'` at the beginning of a word
 - the character sequences `$/`, `'''`, `"""` anywhere
 
 I expect that those sequences are rare, so this change would break few
@@ -21,61 +20,6 @@ programs.
 <div id="toc">
 </div>
 
-
-## Raw and C String Literals in Word Mode
-
-We might want to make string literals in the command/word context match Oil's
-expressions.  This means we have to disallow implicit concatenation, or certain
-instances of it.
-
-No:
-
-    var x = foo'bar'   # Not a valid Oil expression
-    echo foo'bar'      # Valid in shell, but discouraged
-
-Yes:
-
-    var x = 'foobar'
-    echo 'foobar'
-
-We don't want to break these, so = could be special
-
-    ls --foo='with spaces'
-    ls --foo="with $var"
-
-New raw and C strings (which technically conflict with shell):
-
-    echo r'\'   # equivalent to '\'
-    echo c'\n'  # equivalent to $'\t'
-
-
-## Multiline Strings and Redirects
-
-Instead of here docs:
-
-    cat <<EOF
-    hello
-    there, $name
-    EOF
-
-    cat <<'EOF'
-    $5.99  # no interpolation
-    'EOF'
-
-We could have multiline strings:
-
-    cat << """
-    hello
-    there, $name
-    """
-
-    cat << '''
-    $5.99  # no interpolation
-    '''
-
-Minor breakage: the `'''` and `"""` tokens become significant.  It may also be
-nice to change the meaning of `<<` slightly.
-
 ## First Class File Descriptors (`parse_amp`)
 
 We want to make redirection simpler and more consistent.  We can remove the
@@ -125,85 +69,5 @@ You can imagine:
 
 Minor breakage: making `$/` significant.
 
-## Bare Function Call
-
-Instead of
-
-    call f()
-    call g(x, y)
-
-You could do:
-
-    f()   # calls a func, doesn't begin a shell function declaration
-    g(x, y)
-
-- This change probably doesn't involve a lexer mode.
-- It would be a big breakage!
-
-<!--
-
-Idea: parse_square_brackets in lex_mode::OilCommand?
-
-Instead of
-
-    echo *.[ch]
-
-You have to do
-
-    echo @glob('*.[ch]')
-
-Or even:
-
-    echo @'*.[ch]'
-
-Which is a small price to pay to free up []
-
-Actually that's not bad... but
-
-    echo *.py 
-    still works
-
-So then you can distinguish sigil pairs
-
-EXPRESSION:
-
-    :(1 + 2)
-    &(1 + 2)
-
-COMMAND:
-
-    $[hostname] - bah!  conflicts   I guess the $(hostname) wart is OK?
-        $<1+2> or $<a[i]>  # too ugly?
-    @[seq 3]
-    x = ^[echo $PWD]
-
-    myarray = %[foo bar baz]
-
-Yes I like this.  Doh might take ahwile.
-
-Have to resolve $[1 + 2] and $[hostname].  We need another thing for expression
-substitution.  It can't be $(1 + 2)
-
-Maybe it's
-
-    $$(1 + 2)   
-    .(1 + 2)  
-    ~(1 + 2)
-
-    /(1 + 2)
-
-That's a syntax error?
-
-
-Globbing option:
-
-if (x ~ @'*.py') {
-}
-
-if (x ~~ '*.py') {
-}
-
-Hm the latter still reads better.
-
--->
+Note: this is probably possible with `shopt --set strict_dollar`.
 

doc/index.md

@@ -57,6 +57,10 @@ Reference:
 - [Language Influences](language-influences.html)
 - [Syntactic Concepts](syntactic-concepts.html)
   - [Command vs. Expression Mode](command-vs-expression-mode.html).
+- [Oil Language vs. Shell](oil-vs-shell.html).  How does Oil differ from
+  Bourne/POSIX shell?
+- [Oil vs. Python](oil-vs-python.html).  How do Oil's expressions differ from
+  Python?
 
 ### The Command Language
 
@@ -87,8 +91,6 @@ Oil has typed **expressions**, like Python and JavaScript.
 
 - [Expression Language](expression-language.html).  Types, literals, and
   operators.
-  - [Oil vs. Python](oil-vs-python.html).  How do Oil's expressions differ from
-    Python?
 - [Egg Expressions](eggex.html).  A new regex syntax, abbreviated *eggex*.
 
 ## Languages for Data (Interchange Formats)

doc/oil-vs-python.md

@@ -2,36 +2,45 @@
 in_progress: true
 ---
 
-
 Oil Expressions vs. Python
 ==========================
 
-Oil's expression language borrows heavily from Python.
+Oil's expression language borrows heavily from Python.  In fact, it literally
+started with Python's `Grammar/Grammar` file.
+
+This doc describes some differences, which may help Python users learn Oil.
 
-In fact, it literally started with Python's `Grammar/Grammar` file.
+If you don't know Python, see [A Tour of the Oil
+Language](oil-language-tour.html).
 
 <div id="toc">
 </div>
 
-## Literal Syntax
+## Literals for Data Types
 
-- String literals are like shell string literals.
+- String literals are like **shell** string literals, not like Python.
   - Single quoted - `r'c:\Program Files\'` or `$'line\n'`.
   - Double Quoted
-  - `\u{3bc}` instead of `\uhhhh` and `\UHHHHHHHH`
+  - Unicode literals are `\u{3bc}` instead of `\u03bc` and `\U000003bc`
 - Dicts: come from JavaScript, with unquoted keys, and "punning".
-- lists, ints, floats: the same
-- Tuples (TODO):
+- Lists: In addition to Python-like literals `['pea', 'nut']`, there are
+  shell-like literals `%(pea nut)`.
+- Booleans / null: `true`, `false`, and `null` are preferred, but `True`,
+  `False`, and `None` are accepted for compatibility.
+- Tuples (TODO): Does Oil have true tuples?
   - Singleton tuples like `42,` are disallowed, in favor of the more explicit
     `tup(42)`.
 
-### New Literals
+Other literals like ints and floats are the same as in Python.
+
+### New Literal Syntax
 
-- Lists with unquoted words: `%(one two three)`
 - `%symbol` (used in eggex now, but could also be used as interned strings)
 - Raw character literals like `\n` and `\u{03bc}`, and also `#'a'`
-- Block `^(ls | wc -l)`
-- Unevaluated expression: `^[1 +a[i] + f(x)]`
+- Unevaluated expressions
+  - Block `^(ls | wc -l)`
+  - Unevaluated expression: `^[1 + a[i] + f(x)]`
+  - Arg list: `^{42, verbose = true}`
 
 ## Operators
 
@@ -60,28 +69,27 @@ Oil doesn't overload operators as much, and does string <-> int conversion:
 - `a < b` does numeric comparison (with conversion).  `cmp()` could be for
   strings.
 
-No "accidentally quadratic"
-
-- No `in` for array/list membership.  Only dict membership.
-- The `++=` operator on strings doesn't exist
-
 Other:
 
 - I removed the `1:5:2` syntax because `0::2` conflicts with `module::name`.
   This might have been unnecessary.
-- Egg expressions and the `~` operator rather than the `re` module.
-- The `~~` glob match operator
 
 ### New Operators
 
 - `mydict->key` as an alias for `mydict['key']`
 - `++` mentioned above
+- Pattern Matching
+  - Egg expressions and the `~` operator rather than the `re` module.
+  - The `~~` glob match operator
 
 ## Semantic Differences
 
 - Iterating over a string yields code points, not one-character strings.
   - `s[i]` returns an integer code point ("rune").
   - TODO: maybe this should be `runeAt()` and `byteAt()`?
+- No "accidentally quadratic"
+  - No `in` for array/list membership.  Only dict membership.
+  - The `++=` operator on strings doesn't exist
 
 ## Related Links