| layout | title | nav_order | description | image | permalink |
|---|---|---|---|---|---|
default |
Difference-in-Difference (DiD) |
1 |
Welcome to the DiD revolution. |
/assets/images/DiD.jpg |
/ |
This repository tracks the recent developments in the Difference-in-Difference (DiD) literature. It serves two purposes. First, it is an organized dump of my bookmarks from Twitter, GitHub, YouTube etc. The aim here is to keep track of the DiD updates. The second is to make sense of the different packages from an end-user's perspective. This part has to do with how to apply these methods in our research work. On the theory side, several really useful notes already exist (see the Resources section).
I will keep adding content to this website. It might contain errors or the language might not be precise enough. This probably reflects my own poor understanding of what is going on. So please give feedback! The aim of this repository is to collectively build up notes and a code base that we can all use.
Some thoughts below from my own perspective (these are also subject to change over time):
Several DiD innovations came out simultaneously in 2020 and 2021. At the heart of this new DiD literature is the premise that the classic Twoway Fixed Effects (TWFE) model gives wrong treatment estimates. This is especially true if the treatments are heterogeneous (differential treatment timings, different treatment sizes, different treatment statuses over time) which can result in "negative weights" that dilute the true estimates. The negative weights can be derived by comparing combinations of late-treated with early-treated, and treated versus not-treated groups (Bacon decomposition). There is also some discussion that these biases still persist even if treatments are not staggered.
The different DiD packages, start with TWFE bashing and use different methodological innovations to "correct" for TWFE biases. Within these packages are different ways of handling different DiD aspects like parallel trends, negative weights, and covariates. This is done using various methods ranging from bootstrapping, inverse probability weights, matching, influence functions, and imputations.
Here also lies my own confusion in terms of which package to use for which problem. Based on various Twitter conversations, it seems like it all boils down the research question what really we are trying to estimate. This sort of makes the package choice for analysis a bit subjective. Hopefully, more will be written on comparing the utility of each package by the experts out there. But at the time of writing this, more innovations are expected to come out in the coming months. This also includes replicating already-published TWFE papers on which some literature is starting to emerge at the time of this update including some review studies (Jan 2022). See the literature section on more details.
Let's look at the methods we know already. Conducting randomized experiments (RCTs) is not possible in all circumstances, despite being methodologically clean. In contrast, applications of quasi-experimental methods like IVs, synthetic controls, and RDDs are hard to find. This makes DiDs, in terms of applicability, a very powerful tool. If one has access to good primary or even secondary data, it is relatively easy to find interventions that can be tested on the data. Furthermore, these treatements usually have differential timings. Very rarely we see uniform and homogenous roll-out of treatments. And this makes the new DiD methods useful for two reasons. First, they correct the TWFE issues that are likely to provide wrong estimates especially in a differential treatment setting. And second, the event study setup is just a very powerful visual tool that makes the results easy to understand. Additionally, going from data to results is fairly fast (as compared to RCTs). The fast part is from a technical standpoint and one should be, of course, very careful about theory, context, and other socioeconomic factors. The second part also recently (Jan 2022) raised some concerns on Twitter.
My hunch is that the methodological issues identified with TWFEs, that led to new DiD papers, might also spillover into IV and RDD papers in the coming years. So investing in the DiD literature now will provide a strong foundation for keeping track of the the upcoming innovations. But as we start 2022, more and more review papers that summarize the methods have started popping up.
This is a working document. If you want to report errors or contribute, just open an issue, or start a discussion, or e-mail at [email protected]. Since paths and links are also subject to change, please report them to keep this repository as up-to-date as possible. I might add a discussion section below posts to allow for direct comments.
I am slowly starting to build a Stata code repository here. This is to help me (and you) navigate the literature and figure out the code structure for different packages. If you can improve these, then please open a pull request (PR).