Skip to content

Commit 8927dc5

Browse files
hjwphjwp
committed
draft blog post on layers
1 parent f910bfe commit 8927dc5

File tree

4 files changed

+279
-0
lines changed

4 files changed

+279
-0
lines changed

‎blog/2020-08-13-so-many-layers.html

Lines changed: 170 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,170 @@
1+
<!DOCTYPE html>
2+
<html lang="en">
3+
4+
<head>
5+
<title>cosmic_python</title>
6+
<meta charset="utf-8">
7+
<meta name="viewport" content="width=device-width, initial-scale=1">
8+
<meta name="author" content="Harry Percival and Bob Gregory">
9+
<meta name="description" content="">
10+
<meta property="og:title" content="cosmic_python" />
11+
<meta property="og:type" content="website" />
12+
<meta property="og:url" content="https://www.cosmicpython.com/blog/2020-08-13-so-many-layers.html" />
13+
<meta property="og:image" content="https://www.cosmicpython.com/images/lobster_nebula.jpg" />
14+
<link rel="stylesheet" href="//fonts.googleapis.com/css?family=Roboto:300,300italic,700,700italic">
15+
<link rel="stylesheet" href="//cdnjs.cloudflare.com/ajax/libs/normalize/5.0.0/normalize.css">
16+
<link rel="stylesheet" href="//cdnjs.cloudflare.com/ajax/libs/milligram/1.3.0/milligram.css">
17+
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/[email protected]/friendly.css">
18+
<link rel="stylesheet" href="/styles.css">
19+
20+
</head>
21+
22+
<body>
23+
<main class="wrapper">
24+
25+
<nav class="navigation">
26+
<section class="container">
27+
<a class="navigation-title" href="/">
28+
<h1 class="title">Cosmic Python</h1>
29+
</a>
30+
</section>
31+
</nav>
32+
33+
<section class="container">
34+
35+
<h1> So Many Layers! A Note of Caution.</h1>
36+
<p>by Harry, 2020-08-13</p>
37+
38+
39+
<div class="row">
40+
<div class="column">
41+
<img src="/images/slice_of_sagittarius.jpg" />
42+
43+
<p class="float-right">
44+
<em><small><a href="https://www.nasa.gov/image-feature/goddard/2017/hubbles-slice-of-sagittarius">
45+
find out more about this image
46+
</a></small></em></p>
47+
48+
</div>
49+
</div>
50+
51+
52+
<div class="content">
53+
<p>In the book we are at pains to point out that each pattern is a trade-off, and
54+
comes with costs. But just on the offchance that anyone was still missing the
55+
message, and thinking we were saying that all apps should be built like this,
56+
I thought I&rsquo;d write a small blog post just to reinforce the message about
57+
costs. Each time you add a layer, you buy yourself some decoupling, but it
58+
comes at the cost of an extra moving part. In the simplest terms, there&rsquo;s an
59+
extra file you have to maintain.</p>
60+
<p><a href="https://www.cosmicpython.com/book/appendix_ds1_table.html">
61+
<figure>
62+
<img src="/book/images/apwp_aa01.png" alt="a recap of all the layers + parts of our architecture" max-height="60%">
63+
<figcaption>Here&rsquo;s a recap of all the layers + parts of our architecture</figcaption>
64+
</figure>
65+
</a></p>
66+
<p>I remember having to make a simple change to an app that the buying team at
67+
MADE uses. We needed to record an extra piece of information for each shipment,
68+
an optional &ldquo;delay&rdquo; field to be used in some ETA calculations. This is a nice
69+
illustration of a trip all the way through the stack, because things have to
70+
change all the way from the frontend/UI, all the way down to the database.</p>
71+
<p>If you&rsquo;re using a framework like Django, you might be used to thinking of a
72+
change like this, in a perfect world, as a change you can make to just <strong>one</strong> file.
73+
We change <code>models.py</code>, and then our <code>ModelForm</code> will be updated automatically,
74+
and maybe even the frontend, if we&rsquo;re using the form&rsquo;s autogenerated HTML, will
75+
&ldquo;just work&rdquo; too. That&rsquo;s one of the reasons that Django is so good as a rapid
76+
application development framework; by closely coupling its various parts, it
77+
saves you a lot of messing about with database tables, html forms, validation,
78+
and so on. And if those are the main things you spend your time on, then Django
79+
is going to save you a lot of time.</p>
80+
<p>But in our world (at least in theory <a href="#in_theory">*</a>),
81+
database tables and html forms are not where we spend our time. Instead, we
82+
optimise for making it as easy as possible to capture and understand business
83+
logic, and as a result we want to <em>decouple</em> things.</p>
84+
<p>What does it cost? Well, let&rsquo;s take a trip through each file I had to change,
85+
when I was making my <em>very minor</em> change to the data model in our app.</p>
86+
<ul>
87+
<li>I started off with editing a selenium test of the frontend, plus a javascript
88+
frontend test, plus the frontend javascript itself, plus an html template.
89+
That&rsquo;s four files already, but they&rsquo;re not strictly relevant to the patterns
90+
and layers whose cost I want to account for, so I&rsquo;m going to say they don&rsquo;t
91+
count. Don&rsquo;t worry; there&rsquo;s plenty more to come.</li>
92+
</ul>
93+
<p>So:</p>
94+
<ol>
95+
<li>An <strong>end-to-end / API test</strong> for the create and edit commands for the objects in question.</li>
96+
<li>The <strong>Command classes</strong> that capture various <a href="https://www.cosmicpython.com/book/chapter_10_commands.html">write interactions</a> a user can have
97+
with this model.</li>
98+
<li>The <strong>command schema</strong> which we use to <a href="https://www.cosmicpython.com/book/appendix_validation.html">validate incoming requests</a>.</li>
99+
<li>The <strong>service-layer tests</strong> which instantiate those commands to test their
100+
handlers</li>
101+
<li>The <strong>handlers at the service-layer</strong> that orchestrate these use cases.</li>
102+
<li>The <strong>domain model tests</strong> that were affected. Although <a href="https://www.cosmicpython.com/book/chapter_05_high_gear_low_gear.html">not every domain model
103+
needs low-level unit tests as well as service-layer tests</a>, so if I was being indulgent I might
104+
not count this.</li>
105+
<li>The <strong>Domain Model</strong> itself.</li>
106+
<li>The <strong>Repository integration test</strong>
107+
https://www.cosmicpython.com/book/chapter_02_repository.html</li>
108+
<li>The <strong>Repository and ORM config</strong> https://www.cosmicpython.com/book/chapter_02_repository.html</li>
109+
<li>The <strong>database schema</strong></li>
110+
<li>A <strong>migration file</strong> (admittedly autogenerated by Alembic, but we like to just give them a bit of a tidy-up before committing).</li>
111+
<li>The <strong>Event classes</strong> that capture ongoing internal/external consequences
112+
of the various affected use cases</li>
113+
<li>The <strong>Event schema</strong> files we use for (outbound) validation.</li>
114+
<li>And that&rsquo;s not all! Because this app uses CQRS, the read-side is separate from the
115+
write side, so I also had to change some <strong>API JSON view tests</strong></li>
116+
<li>And the <strong>CQRS JSON views code</strong></li>
117+
</ol>
118+
<p>So that&rsquo;s fifteen files. Fifteen!</p>
119+
<p>Now I should add that each change was very simple. Most were a matter of
120+
copy-pasting a line and some find+replace. The whole job might have taken an hour
121+
or so. But if you&rsquo;re used to this sort of thing taking five minutes and happening
122+
in a single file, or at most a couple, then when first confronted with all these
123+
layers, you are definitely going to start questioning the sanity of the entire
124+
endeavour. I know I certainly did.</p>
125+
<p>We think the cost we impose on ourselves here is worth it, because we believe
126+
that the main thing we want to make easy is not adding database fields and html
127+
forms. We want to make it easy to capture complex and evolving business
128+
requirements in a <a href="https://www.cosmicpython.com/book/chapter_01_domain_model.html">domain model</a>. But, as we try to say in each chapter,
129+
your mileage may vary!</p>
130+
<div id="#in_theory"><small><i>
131+
OK, in theory. In practice I think this particular app was a _little_
132+
overengineered. It was one of the first ones that the team had complete
133+
freedom to try new patterns on, and they may have gone to town a little...
134+
But on the other hand, there is talk of converting that app to eventsourcing,
135+
and if we _had_ used Django, I don't think that would even be on the table,
136+
so...
137+
</i></small></div>
138+
</div>
139+
140+
<div id="disqus_thread" style="margin: 10px"></div>
141+
<script>
142+
143+
var disqus_config = function () {
144+
this.page.url = 'https://www.cosmicpython.com/blog/2020-08-13-so-many-layers.html';
145+
this.page.identifier = 'cosmic-python--blog-2020-08-13-so-many-layers';
146+
};
147+
148+
(function() { // DON'T EDIT BELOW THIS LINE
149+
var d = document, s = d.createElement('script');
150+
s.src = 'https://cosmic-python.disqus.com/embed.js';
151+
s.setAttribute('data-timestamp', +new Date());
152+
(d.head || d.body).appendChild(s);
153+
})();
154+
</script>
155+
<noscript>Please enable JavaScript to view the <a href="https://disqus.com/?ref_noscript">comments powered by Disqus.</a></noscript>
156+
157+
<!-- Global site tag (gtag.js) - Google Analytics -->
158+
<script async src="https://www.googletagmanager.com/gtag/js?id=UA-40928035-3"></script>
159+
<script>
160+
window.dataLayer = window.dataLayer || [];
161+
function gtag(){dataLayer.push(arguments);}
162+
gtag('js', new Date());
163+
164+
gtag('config', 'UA-40928035-3');
165+
</script>
166+
167+
</section>
168+
</main>
169+
</body>
170+
</html>

‎images/slice_of_sagittarius.jpg

368 KB
Loading

‎index.html

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -89,6 +89,12 @@ <h3>Recent posts</h3>
8989
<ul>
9090

9191

92+
<li>
93+
<a href="/blog/2020-08-13-so-many-layers.html">2020-08-13 So Many Layers! A Note of Caution.</a>
94+
</li>
95+
96+
97+
9298
<li>
9399
<a href="/blog/2020-05-12-ddia-review.html">2020-05-12 Book review: Designing Data-Intensive Applications, by Martin Kleppmann</a>
94100
</li>
@@ -123,6 +129,8 @@ <h3>Guest Posts By David</h3>
123129

124130

125131

132+
133+
126134
<li>
127135
<a href="/blog/2019-08-03-ioc-techniques.html">2019-08-03 Three Techniques for Inverting Control, in Python</a>
128136
</li>
@@ -158,6 +166,8 @@ <h3>Classic 2017 Episodes on Ports & Adapters, by Bob</h3>
158166

159167

160168

169+
170+
161171
<li>
162172
<a href="/blog/2017-09-19-why-use-domain-events.html">2017-09-19 Why use domain events?</a>
163173
</li>

‎posts/2020-08-13-so-many-layers.md

Lines changed: 99 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,99 @@
1+
title: So Many Layers! A Note of Caution.
2+
author: Harry
3+
image: slice_of_sagittarius.jpg
4+
image_credit: https://www.nasa.gov/image-feature/goddard/2017/hubbles-slice-of-sagittarius
5+
6+
In the book we are at pains to point out that each pattern is a trade-off, and
7+
comes with costs. But just on the offchance that anyone was still missing the
8+
message, and thinking we were saying that all apps should be built like this,
9+
I thought I'd write a small blog post just to reinforce the message about
10+
costs. Each time you add a layer, you buy yourself some decoupling, but it
11+
comes at the cost of an extra moving part. In the simplest terms, there's an
12+
extra file you have to maintain.
13+
14+
15+
<a href="https://www.cosmicpython.com/book/appendix_ds1_table.html">
16+
<figure>
17+
<img src="/book/images/apwp_aa01.png" alt="a recap of all the layers + parts of our architecture" max-height="60%">
18+
<figcaption>Here's a recap of all the layers + parts of our architecture</figcaption>
19+
</figure>
20+
</a>
21+
22+
I remember having to make a simple change to an app that the buying team at
23+
MADE uses. We needed to record an extra piece of information for each shipment,
24+
an optional "delay" field to be used in some ETA calculations. This is a nice
25+
illustration of a trip all the way through the stack, because things have to
26+
change all the way from the frontend/UI, all the way down to the database.
27+
28+
If you're using a framework like Django, you might be used to thinking of a
29+
change like this, in a perfect world, as a change you can make to just **one** file.
30+
We change `models.py`, and then our `ModelForm` will be updated automatically,
31+
and maybe even the frontend, if we're using the form's autogenerated HTML, will
32+
"just work" too. That's one of the reasons that Django is so good as a rapid
33+
application development framework; by closely coupling its various parts, it
34+
saves you a lot of messing about with database tables, html forms, validation,
35+
and so on. And if those are the main things you spend your time on, then Django
36+
is going to save you a lot of time.
37+
38+
But in our world (at least in theory [*](#in_theory)),
39+
database tables and html forms are not where we spend our time. Instead, we
40+
optimise for making it as easy as possible to capture and understand business
41+
logic, and as a result we want to _decouple_ things.
42+
43+
What does it cost? Well, let's take a trip through each file I had to change,
44+
when I was making my _very minor_ change to the data model in our app.
45+
46+
* I started off with editing a selenium test of the frontend, plus a javascript
47+
frontend test, plus the frontend javascript itself, plus an html template.
48+
That's four files already, but they're not strictly relevant to the patterns
49+
and layers whose cost I want to account for, so I'm going to say they don't
50+
count. Don't worry; there's plenty more to come.
51+
52+
So:
53+
54+
1. An **end-to-end / API test** for the create and edit commands for the objects in question.
55+
2. The **Command classes** that capture various [write interactions](https://www.cosmicpython.com/book/chapter_10_commands.html) a user can have
56+
with this model.
57+
3. The **command schema** which we use to [validate incoming requests](https://www.cosmicpython.com/book/appendix_validation.html).
58+
4. The **service-layer tests** which instantiate those commands to test their
59+
handlers
60+
5. The **handlers at the service-layer** that orchestrate these use cases.
61+
6. The **domain model tests** that were affected. Although [not every domain model
62+
needs low-level unit tests as well as service-layer tests](https://www.cosmicpython.com/book/chapter_05_high_gear_low_gear.html), so if I was being indulgent I might
63+
not count this.
64+
7. The **Domain Model** itself.
65+
8. The **Repository integration test**
66+
https://www.cosmicpython.com/book/chapter_02_repository.html
67+
9. The **Repository and ORM config** https://www.cosmicpython.com/book/chapter_02_repository.html
68+
10. The **database schema**
69+
11. A **migration file** (admittedly autogenerated by Alembic, but we like to just give them a bit of a tidy-up before committing).
70+
12. The **Event classes** that capture ongoing internal/external consequences
71+
of the various affected use cases
72+
13. The **Event schema** files we use for (outbound) validation.
73+
14. And that's not all! Because this app uses CQRS, the read-side is separate from the
74+
write side, so I also had to change some **API JSON view tests**
75+
15. And the **CQRS JSON views code**
76+
77+
So that's fifteen files. Fifteen!
78+
79+
Now I should add that each change was very simple. Most were a matter of
80+
copy-pasting a line and some find+replace. The whole job might have taken an hour
81+
or so. But if you're used to this sort of thing taking five minutes and happening
82+
in a single file, or at most a couple, then when first confronted with all these
83+
layers, you are definitely going to start questioning the sanity of the entire
84+
endeavour. I know I certainly did.
85+
86+
We think the cost we impose on ourselves here is worth it, because we believe
87+
that the main thing we want to make easy is not adding database fields and html
88+
forms. We want to make it easy to capture complex and evolving business
89+
requirements in a [domain model](https://www.cosmicpython.com/book/chapter_01_domain_model.html). But, as we try to say in each chapter,
90+
your mileage may vary!
91+
92+
<div id="#in_theory"><small><i>
93+
OK, in theory. In practice I think this particular app was a _little_
94+
overengineered. It was one of the first ones that the team had complete
95+
freedom to try new patterns on, and they may have gone to town a little...
96+
But on the other hand, there is talk of converting that app to eventsourcing,
97+
and if we _had_ used Django, I don't think that would even be on the table,
98+
so...
99+
</i></small></div>

0 commit comments

Comments
 (0)