Improvements related to ASF release process (apache#7539)

* [WiP] Improvements related to ASF release process

* Removing translation binaries (.mo)
* Working on 2 docker files to help package and validate releases
  * Dockerfile.from_tarball: takes a VERSION as input, downloads
    official source release fro svn and bakes it into a functional
    docker image that can be validated
  * Dockerfile.make_tarball: helps a maintainer to package and sign a
    release candidate or release

* Address COPYRIGHT + LICENSE issues

(cherry picked from commit 5ae2836)

* Add Roboto font to LICENSE, remove glyphicons files

(cherry picked from commit 9615f39)

* remove unused LICENSE entries

* Change babytux to open image in birth dashboard

(cherry picked from commit 2776d11)

* Improve instructions

* Docker tweaks

(cherry picked from commit df6e6462458ea8cff1482c142921b20f5607ad8e)

* Include image

.gitignore

@@ -81,3 +81,6 @@ ghostdriver.log
 testCSV.csv
 .terser-plugin-cache/
 apache-superset-*.tar.gz*
+
+# Translation binaries
+messages.mo

CONTRIBUTING.md

@@ -97,7 +97,12 @@ articles. See [Documentation](#documentation) for more details.
 
 ### Add Translations
 
-If you are proficient in a non-English language, you can help translate text strings from Superset's UI. You can jump in to the existing language dictionaries at `superset/translations/<language_code>/LC_MESSAGES/messages.po`, or even create a dictionary for a new language altogether. See [Translating](#translating) for more details.
+If you are proficient in a non-English language, you can help translate
+text strings from Superset's UI. You can jump in to the existing
+language dictionaries at
+`superset/translations/<language_code>/LC_MESSAGES/messages.po`, or
+even create a dictionary for a new language altogether.
+See [Translating](#translating) for more details.
 
 ### Ask Questions
 
@@ -578,16 +583,23 @@ See [`superset/assets/cypress_build.sh`](https://github.com/apache/incubator-sup
 
 ## Translating
 
-We use [Babel](http://babel.pocoo.org/en/latest/) to translate Superset. In Python files, we import the magic `_` function using:
+We use [Babel](http://babel.pocoo.org/en/latest/) to translate Superset.
+In Python files, we import the magic `_` function using:
 
 ```python
 from flask_babel import lazy_gettext as _
 ```
 
-then wrap our translatable strings with it, e.g. `_('Translate me')`. During extraction, string literals passed to `_` will be added to the generated `.po` file for each language for later translation.
-At runtime, the `_` function will return the translation of the given string for the current language, or the given string itself if no translation is available.
+then wrap our translatable strings with it, e.g. `_('Translate me')`.
+During extraction, string literals passed to `_` will be added to the
+generated `.po` file for each language for later translation.
 
-In JavaScript, the technique is similar: we import `t` (simple translation), `tn` (translation containing a number).
+At runtime, the `_` function will return the translation of the given
+string for the current language, or the given string itself
+if no translation is available.
+
+In JavaScript, the technique is similar:
+we import `t` (simple translation), `tn` (translation containing a number).
 
 ```javascript
 import { t, tn } from '@superset-ui/translation';

INSTALL.md

@@ -0,0 +1,23 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+  http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# INSTALL / BUILD instructions for Apache Superset (incubating)
+
+At this time, the docker file at RELEASING/Dockerfile.from_tarball
+constitutes the recipe on how to get to a working release from a source
+release tarball.

LICENSE.txt

@@ -208,26 +208,9 @@ limitations under the License.
    subcomponents is subject to the terms and conditions of the following
    licenses.
 
-========================================================================
-Third party Apache 2.0 licenses
-========================================================================
-
-========================================================================
-MIT licenses
-========================================================================
-
-    (MIT License) cal-heatmap v3.6.2 (https://github.com/wa0x6e/cal-heatmap)
 
 ========================================================================
-BSD 2-Clause licenses
-========================================================================
-
-========================================================================
-BSD 3-Clause licenses
+Third party Apache 2.0 licenses
 ========================================================================
 
-    (BSD 3 License) parallel-coordinates v0.7.0 (http://syntagmatic.github.com/parallel-coordinates/)
-
-========================================================================
-Creative Commons Attribution 4.0
-========================================================================
+(Apache 2.0 License) The Roboto family of fonts (https://github.com/google/roboto)

NOTICE

@@ -3,6 +3,3 @@ Copyright 2016-2019 The Apache Software Foundation
 
 This product includes software developed at The Apache Software
 Foundation (http://www.apache.org/).
-
-This product includes data licensed under a Creative Commons Attribution 4.0
-License (http://www.diva-gis.org/Data).

RELEASING/Dockerfile.from_tarball

@@ -0,0 +1,71 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+FROM python:3.6-jessie
+
+RUN useradd --user-group --create-home --no-log-init --shell /bin/bash superset
+
+# Configure environment
+ENV LANG=C.UTF-8 \
+    LC_ALL=C.UTF-8
+
+RUN apt-get update -y
+
+# Install dependencies to fix `curl https support error` and `elaying package configuration warning`
+RUN apt-get install -y apt-transport-https apt-utils
+
+# Install superset dependencies
+# https://superset.incubator.apache.org/installation.html#os-dependencies
+RUN apt-get install -y build-essential libssl-dev \
+    libffi-dev python3-dev libsasl2-dev libldap2-dev libxi-dev
+
+# Install nodejs for custom build
+# https://superset.incubator.apache.org/installation.html#making-your-own-build
+# https://nodejs.org/en/download/package-manager/
+RUN curl -sL https://deb.nodesource.com/setup_10.x | bash - \
+    && apt-get install -y nodejs
+
+RUN mkdir -p /home/superset
+RUN chown superset /home/superset
+
+WORKDIR /home/superset
+ARG VERSION
+
+RUN svn co https://dist.apache.org/repos/dist/dev/incubator/superset/$VERSION ./
+RUN tar -xvf *.tar.gz
+WORKDIR apache-superset-incubating-$VERSION
+
+RUN cd superset/assets \
+    && npm ci \
+    && npm run build \
+    && rm -rf node_modules
+
+
+WORKDIR /home/superset/apache-superset-incubating-$VERSION
+RUN pip install --upgrade setuptools pip \
+    && pip install -r requirements.txt -r requirements-dev.txt \
+    && pip install -e . \
+    && rm -rf /root/.cache/pip
+
+RUN fabmanager babel-compile --target superset/translations
+
+RUN pip install -e . \
+    && rm -rf /root/.cache/pip
+
+ENV PATH=/home/superset/superset/bin:$PATH \
+    PYTHONPATH=/home/superset/superset/:$PYTHONPATH
+COPY from_tarball_entrypoint.sh /entrypoint.sh
+ENTRYPOINT ["/entrypoint.sh"]

RELEASING/Dockerfile.make_tarball

@@ -0,0 +1,19 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+FROM python:3.6-jessie
+COPY make_tarball.sh /entrypoint.sh
+ENTRYPOINT ["/entrypoint.sh"]

RELEASING.md → RELEASING/README.md

@@ -16,6 +16,28 @@ KIND, either express or implied.  See the License for the
 specific language governing permissions and limitations
 under the License.
 -->
+# Apache Releases
+
+To make a fresh tarball of a git reference on apache/incubator-superset
+(push your tag first!)
+
+```bash
+docker build -t make_tarball -f Dockerfile.make_tarball . --build-arg VERSION=0.33.0rc1
+docker run make_tarball -f Dockerfile.make_tarball --env VERSION=0.33.0rc1
+```
+
+To make a working build given a tarball
+```bash
+# Building a docker from a tarball
+VERSION=0.33.0rc2 && \
+docker build -t apache-superset:$VERSION -f Dockerfile.from_tarball . --build-arg VERSION=$VERSION
+
+# testing the resulting docker
+docker run -p 5001:8088 apache-superset:0.33.0rc2
+# you should be able to access localhost:5001 on your browser
+# login using admin/admin
+```
+
 ## Refresh documentation website
 
 Every once in a while we want to compile the documentation and publish it.
@@ -57,14 +79,14 @@ First you need to setup a few things. This is a one-off and doesn't
 need to be done at every release.
 
 ```bash
-    # Create PGP Key
+    # Create PGP Key, and use your @apache.org email address
     gpg --gen-key
 
     # Checkout ASF dist repo
 
     svn checkout https://dist.apache.org/repos/dist/dev/incubator/superset/ ~/svn/superset_dev
 
-    svn checkout https://dist.apache.org/repos/dist/incubator/superset/ ~/svn/superset
+    svn checkout https://dist.apache.org/repos/dist/release/incubator/superset/ ~/svn/superset
     cd ~/svn/superset
 
 
@@ -84,6 +106,8 @@ Now let's craft a source release
     # Assuming these commands are executed from the root of the repo
     # Setting a VERSION var will be useful
     export VERSION=0.31.0rc18
+    export RELEASE=apache-superset-incubating-${VERSION}
+    export RELEASE_TARBAL=${RELEASE}-source.tar.gz
 
     # Let's create a git tag
     git tag -f ${VERSION}
@@ -98,11 +122,11 @@ Now let's craft a source release
     mkdir -p ~/svn/superset_dev/${VERSION}/
     git archive \
         --format=tar.gz ${VERSION} \
-        --prefix=apache-superset-${VERSION}/ \
-        -o ~/svn/superset_dev/${VERSION}/apache-superset-${VERSION}-source.tar.gz
+        --prefix=${RELEASE}/ \
+        -o ~/svn/superset_dev/${VERSION}/${RELEASE_TARBALL}
 
     cd ~/svn/superset_dev/
-    scripts/sign.sh apache-superset-${VERSION}-source.tar.gz
+    scripts/sign.sh ${RELEASE}-source.tar.gz
 ```
 
 ## Shipping to SVN
@@ -112,7 +136,7 @@ Now let's ship this RC into svn's dev folder
 ```bash
     # cp or mv the files over to the svn repo
     mkdir ~/svn/superset_dev/${VERSION}/
-    cp apache-superset-${VERSION}* ~/svn/superset_dev/${VERSION}/
+    cp ${RELEASE_TARBALL} ~/svn/superset_dev/${VERSION}/
     cd ~/svn/superset_dev/
     svn add ${VERSION}
     svn commit

RELEASING/from_tarball_entrypoint.sh

@@ -0,0 +1,42 @@
+#!/bin/bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+set -ex
+
+echo "[WARNING] this entrypoint creates an admin/admin user"
+echo "[WARNING] it should only be used for lightweight testing/validation"
+
+# Create an admin user (you will be prompted to set username, first and last name before setting a password)
+fabmanager create-admin \
+    --app superset \
+    --username admin \
+    --firstname admin \
+    --lastname admin \
+    --email [email protected] \
+    --password admin
+
+# Initialize the database
+superset db upgrade
+
+# Loading examples
+superset load_examples
+
+# Create default roles and permissions
+superset init
+
+FLASK_ENV=development FLASK_APP=superset:app \
+flask run -p 8088 --with-threads --reload --debugger --host=0.0.0.0