New blog post

2018/BuildYourOwnJavaScriptPlayground/post.html

@@ -0,0 +1,207 @@
+<p>Recently I spent some time working on my own JavaScript playground called <a href="https://krasimir.github.io/demoit/dist/">Demoit</a>. Something like <a href="https://codesandbox.io">CodeSandbox</a>, <a href="https://jsbin.com">JSBin</a> or <a href="http://codePen.io">Codepen</a>. I already <a href="http://krasimirtsonev.com/blog/article/my-new-and-shiny-too-for-live-demos-demoit">blogged</a> about why I did it but decided to write down some implementation details. Everything happens at runtime in the browser so it is pretty interesting project.</p>[STOP]
+<h2 id="the-goal">The goal</h2>
+<p>The JavaScript playground is a place where we can write JavaScript code and see the result of it. This means changes in a DOM tree or logs in the console. To accommodate those needs I created a pretty standard UI.</p>
+<p><img src="http://krasimirtsonev.com/blog/articles/demoit2/demoit.png" alt="demoit"></p>
+<p>We have two panels on the left side - one for the produced markup and one for the console logs. On the right side is the editor. Every time when we make changes to the code and <em>save</em> it we want to see the left panels updated. The tool must also support multiple files so we have a navigation bar with tabs for every file.</p>
+<h2 id="the-editor">The editor</h2>
+<p>Nope, I did not implement the editor by myself. That is a ton of work. What I used is <a href="https://codemirror.net/">CodeMirror</a>. It is a pretty decent editor for the web. The integration is pretty straightforward. In fact the code that I wrote for that part is just 25 lines.</p>
+<pre><code>// editor.js
+export const createEditor = function (settings, value, onSave, onChange) {
+  const container = el(&#39;.js-code-editor&#39;);
+  const editor = CodeMirror(container, {
+    value: value || &#39;&#39;,
+    mode:  &#39;jsx&#39;,
+    tabSize: 2,
+    lineNumbers: false,
+    autofocus: true,
+    foldGutter: false,
+    gutters: [],
+    styleSelectedText: true,
+    ...settings.editor
+  });
+  const save = () =&gt; onSave(editor.getValue());
+  const change = () =&gt; onChange(editor.getValue());
+
+  editor.on(&#39;change&#39;, change);
+  editor.setOption(&quot;extraKeys&quot;, { &#39;Ctrl-S&#39;: save, &#39;Cmd-S&#39;: save });
+  CodeMirror.normalizeKeyMap();
+  container.addEventListener(&#39;click&#39;, () =&gt; editor.focus());
+  editor.focus();
+
+  return editor;
+};</code></pre>
+<p>The constructor of CodeMirror accepts HTML element and a set of options. The rest is just listening for events, two keyboard shortcuts and focusing the editor.</p>
+<p>In the very first commits I put a lot of logic in here. Like for example the transpilation or reading the initial value from the local storage but later decided to extract those bits out. It is now a function that creates the editor and sends out whatever we type.</p>
+<h2 id="transpiling-the-code">Transpiling the code</h2>
+<p>I guess you will agree with me if I say that the majority of JavaScript that we write today requires transpilation. I decided to use <a href="https://babeljs.io">Babel</a>. Not because it is the most popular transpiler but because it offers a client-side standalone processing. This means that we can import <a href="https://unpkg.com/[email protected]/babel.js">babel.js</a> on our page and we will be able to transpile code on the fly. For example:</p>
+<pre><code>// transpile.js
+const babelOptions = {
+  presets: [ &quot;react&quot;, [&quot;es2015&quot;, { &quot;modules&quot;: false }]]
+}
+
+export default function preprocess(str) {
+  const { code } = Babel.transform(str, babelOptions);
+
+  return code;
+}</code></pre>
+<p>Using this code we can get the JavaScript from the editor and translate it to valid ES5 syntax that runs just fine in the browser. This is all good but what we have so far is just a string. We need to somehow convert that string to a working code.</p>
+<h2 id="using-javascript-to-run-javascript-generated-by-javascript">Using JavaScript to run JavaScript generated by JavaScript</h2>
+<p>There is a <code>Function</code> constructor which accepts code in the format of a string. It is not very popular because we almost never use it. If we want to run a function we just call it. However, it is really useful if we generate code at runtime. Which is exactly the case now. Here is a short example:</p>
+<pre><code>const func = new Function(&#39;var a = 42; console.log(a);&#39;);
+
+func(); // logs out 42</code></pre>
+<p>This is what I used to process the raw string. The code is send to the <code>Function</code> constructor and later executed:</p>
+<pre><code>// execute.js
+import transpile from &#39;./transpile&#39;;
+
+export default function executeCode(code) {
+  try {
+    (new Function(transpile(code)))();
+  } catch (error) {
+    console.error(error);
+  }
+}</code></pre>
+<p>The try-catch block here is necessary because we want to keep the app running even if there is an error. And it is absolutely fine to get some errors because this is a tool that we use for trying stuff. Notice that the script above catches syntax errors as well as runtime errors.</p>
+<h2 id="handling-import-statements">Handling import statements</h2>
+<p>At some point I added the ability to edit multiple files and I realized that <a href="https://github.com/krasimir/demoit">Demoit</a> may act like a real code editor. Which sometimes means exporting logic into a file and importing it in another. However, to support such behavior we have to handle <code>import</code> and <code>export</code> statements. This (same as many other things) is not built-in part of Babel. There is a plugin that understands that syntax and transpile it to the good old <code>require</code> and <code>exports</code> calls - <a href="https://www.npmjs.com/package/babel-plugin-transform-es2015-modules-commonjs">transform-es2015-modules-common</a>. Here is an example:</p>
+<pre><code>import test from &#39;test&#39;;
+export default function blah() {}</code></pre>
+<p>gets translated to:</p>
+<pre><code>Object.defineProperty(exports, &quot;__esModule&quot;, {
+  value: true
+});
+exports.default = blah;
+var _test = require(&#39;test&#39;);
+var _test2 = _interopRequireDefault(_test);
+function _interopRequireDefault(obj) {
+  return obj &amp;&amp; obj.__esModule ? obj : { default: obj };
+}
+function blah() {}</code></pre>
+<p>There is no result of that yey but at least the code gets transpiled correctly with no errors. The plugin helps to have the correct code as a string but has nothing to do with running it. The produced code didn&#39;t work because there is no <code>require</code> neither <code>exports</code> defined.</p>
+<p>Let&#39;s go back to our <code>executeCode</code> function and see what we have to change to make the importing/exporting possible. The good news are that everything happens in the browser so we do actually have the code of all the files in the <em>editor</em>. We know their content upfront. We are also controlling the code that gets executed because as we said it is just a string. Because of that we can dynamically add whatever we want. Including other functions or variables.</p>
+<p>Let&#39;s change the signature of <code>executeCode</code> a little bit. Instead of code as a string we will accept an index of the currently edited file and an array of all the available files:</p>
+<pre><code>export default function executeCode(index, files) {
+  // magic goes here
+}</code></pre>
+<p>Let&#39;s assume that we have the files of the editor in the following format:</p>
+<pre><code>const files = [
+  {
+    filename: &quot;A.js&quot;,
+    content: &quot;import B from &#39;B.js&#39;;\nconsole.log(B);&quot;
+  },
+  {
+    filename: &quot;B.js&quot;,
+    content: &quot;const answer = 42;\nexport default answer;&quot;
+  }
+]</code></pre>
+<p>If everything is ok and we run <code>A.js</code> we are suppose to see <code>42</code> in the console. Now, let&#39;s start constructing a new string that will be sent to the <code>Function</code> constructor:</p>
+<pre><code>const transpiledFiles = files.map(({ filename, content }) =&gt; `
+  {
+    filename: &quot;${ filename }&quot;,
+    func: function (require, exports) {
+      ${ transpile(content) }
+    },
+    exports: {}
+  }
+`);</code></pre>
+<p><code>transpiledFiles</code> is a new array that contains strings. Those strings are actually object literals that will be used later. We wrap the code into a closure so we avoid collisions with the other files and we also <em>define</em> where <code>require</code> and <code>exports</code> are coming from. We also have an empty object that will store whatever the file exports. In the case of <code>B.js</code> that&#39;s the number 42.</p>
+<p>The next steps are to implement the <code>require</code> function, and execute the code of the file at the correct <code>index</code> (remember how we pass the index of the currently edited file):</p>
+<pre><code>const code = `
+  const modules = [${ transpiledFiles.join(&#39;,&#39;) }];
+  const require = function(file) {
+    const module = modules.find(({ filename }) =&gt; filename === file);
+
+    if (!module) {
+      throw new Error(&#39;Demoit can not find &quot;&#39; + file + &#39;&quot; file.&#39;);
+    }
+    module.func(require, module.exports);
+    return module.exports;
+  };
+  modules[${ index }].func(require, modules[${ index }].exports);
+`;</code></pre>
+<p>The <code>modules</code> array is something like a bundle containing all of our code. The <code>require</code> function is basically looking into that bundle to find the file that we need and runs its closure. Notice how we pass the same <code>require</code> function and the <code>module.exports</code> object. This same object gets returned at the end.</p>
+<p>The last bit is executing the closure for the current file. The generated code for the example above is as follows:</p>
+<pre><code>const modules = [
+  {
+    filename: &quot;A.js&quot;,
+    func: function (require, exports) {
+      &#39;use strict&#39;;
+      var _B = require(&#39;B.js&#39;);
+      var _B2 = _interopRequireDefault(_B);
+      function _interopRequireDefault(obj) { return obj &amp;&amp; obj.__esModule ? obj : { default: obj }; }
+      console.log(_B2.default);
+    },
+    exports: {}
+  },
+  {
+    filename: &quot;B.js&quot;,
+    func: function (require, exports) {
+      &quot;use strict&quot;;
+      Object.defineProperty(exports, &quot;__esModule&quot;, { value: true });
+      var answer = 42;
+      exports.default = answer;
+    },
+    exports: {}
+  }
+];
+const require = function(file) {
+  const module = modules.find(({ filename }) =&gt; filename === file);
+
+  if (!module) {
+    throw new Error(&#39;Demoit can not find &quot;&#39; + file + &#39;&quot; file.&#39;);
+  }
+  module.func(require, module.exports);
+  return module.exports;
+};
+modules[0].func(require, modules[0].exports);</code></pre>
+<p>This code gets sent to the <code>Function</code> constructor. This is also close to how the bundlers nowadays work. They usually wrap our modules into closures and have similar code for resolving imports.</p>
+<h2 id="producing-markup-as-a-result">Producing markup as a result</h2>
+<p>There is no much of a work here. I just had to provide a DOM element and let the developer knows about it. In the case of <a href="https://github.com/krasimir/demoit">Demoit</a> I placed a <code>&lt;div&gt;</code> with class <code>&quot;output&quot;</code>.</p>
+<p>Look at the following screenshot. It illustrates how we can target the top-left panel:</p>
+<p><img src="http://krasimirtsonev.com/blog/articles/demoit2/demoit2.png" alt="demoit"></p>
+<p>The code that comes from CodeMirror is executed in the context of the same page where the app runs. So, the code has access to the same DOM tree.</p>
+<p>There was one problem that I had to solve though. It was about cleaning the <code>&lt;div&gt;</code> before running the code again. This was necessary because there may be some elements from the previous run. A simple <code>element.innerHTML = &#39;&#39;</code> didn&#39;t work properly with React so I ended up using the following:</p>
+<pre><code>async function teardown() {  
+  const output = el(&#39;.output&#39;);
+
+  if (typeof ReactDOM !== &#39;undefined&#39;) {
+    ReactDOM.unmountComponentAtNode(output);
+  }
+
+  output.innerHTML = &#39;&#39;;
+}</code></pre>
+<p>If the code uses <code>ReactDOM</code> package we make the assumption that the React app is rendered in the exact that <code>&lt;div&gt;</code> and we unmount it. If we don&#39;t do that we&#39;ll get a runtime error because we flushed the DOM elements that React is using. <code>unmountComponentAtNode</code> is pretty resilient and does not care if there is React in the passed element or not. It just does its job if it can.</p>
+<h2 id="catching-the-logs">Catching the logs</h2>
+<p>While we code we very often use <code>console.log</code>. I needed to catch those calls and show them on the lower left panel. I picked a little bit hacky solution - overwriting the console methods:</p>
+<pre><code>const add = something =&gt; {
+  // ... add a new element to the panel
+}
+const originalError = console.error;
+const originalLog = console.log;
+const originalWarning = console.warn;
+const originalInfo = console.info;
+const originalClear = console.clear;
+
+console.error = function (error) {
+  add(error.toString() + error.stack);
+  originalError.apply(console, arguments);
+};
+console.log = function (...args) {
+  args.forEach(add);
+  originalLog.apply(console, args);
+};
+console.warn = function (...args) {
+  args.forEach(add);
+  originalWarning.apply(console, args);
+};
+console.info = function (...args) {
+  args.forEach(add);
+  originalInfo.apply(console, args);
+};
+console.clear = function (...args) {
+  element.innerHTML = &#39;&#39;;
+  originalClear.apply(console, args);
+};</code></pre>
+<p>Notice that I kept the usual behavior so I didn&#39;t break the normal work of the <code>console</code> object. I also overwrote <code>.error</code>, <code>.warn</code>, <code>.info</code> and <code>.clear</code> so I provide a better developer experience. If everything is listed in the panel the developer doesn&#39;t have to use the browser&#39;s dev tools.</p>
+<h2 id="all-together">All together</h2>
+<p>There is also some glue code, some code for splitting the screen, some code that deals with the navigation and local storage. The bits above were the most interesting and tricky ones and probably the ones that you have to pay attention to. If you want to see the full source code of the playground go to <a href="https://github.com/krasimir/demoit">github.com/krasimir/demoit</a>. You can try a live demo <a href="https://krasimir.github.io/demoit/dist/">here</a>.</p>