|
| 1 | +--- |
| 2 | +layout: post |
| 3 | +title: "Triage of my Cauchy kernel project" |
| 4 | +categories: research |
| 5 | +--- |
| 6 | + |
| 7 | +This is a post about my latest paper, ["Minimax separation of the Cauchy kernel"](https://arxiv.org/abs/1909.06911), |
| 8 | + which I have just posted to the arXiv and submitted to the |
| 9 | + [SIAM Journal on Numerical Analysis](https://www.siam.org/Publications/Journals/SIAM-Journal-on-Numerical-Analysis-SINUM). |
| 10 | +I have wanted to write and publish a math paper for a long time, to signify an increased emphasis in my research |
| 11 | + on the mathematical underpinnings of atomistic simulation. |
| 12 | +It has taken me longer than expected to reach this goal because it has been difficult to focus on mathematical work |
| 13 | + that is entirely unrelated to work obligations, |
| 14 | + and my attempts to align employment with this sort of research activity have repeatedly failed. |
| 15 | +However, I am an extremely stubborn person, |
| 16 | + and I will persist in my more mathematical work even if my pace is slower than I'd like. |
| 17 | +For example, I had to heavily triage my original plan to complete this already massively delayed paper in a timely manner |
| 18 | + by dropping a set of planned applications and saving them for follow-up papers. |
| 19 | + |
| 20 | +I have always enjoyed math a lot, since early in grade school. |
| 21 | +It was my favorite subject all throughout high school (I was underwhelmed by my science classes until college), |
| 22 | + and was even on several competitive high school math teams. |
| 23 | +I would have been a math major in college (rather than the hedge of a dual math/physics major), |
| 24 | + but I wanted to pursue academic research and was repeatedly told that a math degree could only lead to an actuarial career. |
| 25 | +While I strayed from a more mathematical career by going to graduate school for physics, |
| 26 | + my research interests in physics have always been of a highly mathematical nature. |
| 27 | +I like to view computational physics as a kind of experimental math, |
| 28 | + where approximations and algorithms are tested for their relevance to physical simulation. |
| 29 | +My overly simple grad-school perspective on quantum mechanics was as a physical manifestation of linear algebra, |
| 30 | + which meant that computational condensed matter physics was just a very complicated numerical linear algebra problem. |
| 31 | +However, working in quantum information theory for five years have given me a broader perspective |
| 32 | + on the statistical and complexity-theoretic aspects of quantum mechanics. |
| 33 | + |
| 34 | +I discussed the background on this project a bit in a [previous post]({% post_url 2019-05-12-linear-scaling-too-little %}). |
| 35 | +Around 2007, I was working on a banded Hermitian eigensolver that was inspired by the physics concept of Wannier functions |
| 36 | + (a.k.a. Boys orbitals in quantum chemistry). |
| 37 | +These functions demonstrated that simultaneous spatial and spectral locality was possible, |
| 38 | + which allowed for more flexibility in divide-and-conquer strategies for eigensolving beyond the |
| 39 | + [one that had already been established at the time](https://doi.org/10.1137/S0895479892241287). |
| 40 | +In the course of these numerical experiments, |
| 41 | +it became necessary to fit low-rank approximations of the Cauchy kernel with a general form |
| 42 | + |
| 43 | +$$ \frac{1}{x - y} \approx \sum_{i=1}^{r} \frac{f_i(y)}{x - y_i} $$ |
| 44 | + |
| 45 | +where I was free to choose both $$y_i$$ and $$f_i(y)$$. |
| 46 | +For a fixed value of $$y$$, this was a standard linear minimax optimization problem. |
| 47 | +I had previously messed around with linear and nonlinear minimax optimization problems |
| 48 | + that I thought would be useful in electronic structure |
| 49 | + (for example, [my paper on rational approximations of Fermi-Dirac functions](https://doi.org/10.1063/1.4965886) originated from earlier attempts during grad school), |
| 50 | + so I already had some experience with problems of this form. |
| 51 | +I proceeded as I had in the past, with an ad-hoc implementation of the Remez algorithm. |
| 52 | +This time, I could make use of the analytical expressions for Cauchy matrix inverses and determinants |
| 53 | + in fitting the approximant at a set of max-residual points. |
| 54 | +It turned out that the relative error was what would lead to the tightest error bounds in |
| 55 | + my intended application, so I adapted my solution to minimize relative error. |
| 56 | +I then noticed something very surprising: the location of the maximum residual points did not change when I changed $$y$$. |
| 57 | +Because of the Cauchy matrix inverse formula, |
| 58 | + this meant that the optimal $$f_i(y)$$ were rational functions rather than something more general and arbitrary. |
| 59 | +Also, I was observing much more regular trends in the solutions than in other problems that I had solved, |
| 60 | + such as the maximum residual points and optimal $$y_i$$ values for different $$r$$ values collapsing |
| 61 | + onto a common function just like with Chebyshev nodes. |
| 62 | + |
| 63 | +I don't keep detailed research records (I've mostly operated under the assumption that published papers are record enough), |
| 64 | + but at some point either in late grad school or early in my academic post-doc at UT Austin, |
| 65 | + I decided that this result was probably optimal with respect to maximum relative error from a basic dimension-counting argument |
| 66 | + (I couldn't see how a more general functional form could reduce the maximum error) and I wanted to prove it. |
| 67 | +I'm not really big into proving things because it's hard and you aren't really rewarded for doing so as a physicist. |
| 68 | +However, I do think electronic structure simulations need a more rigorous underpinning, |
| 69 | + and there is probably a small set of core results waiting to be discovered that are important enough for a formal proof. |
| 70 | +I decided that this was one such result. |
| 71 | +My basic proof strategy was to prove separate upper and lower bounds, whereby optimality was proven if the bounds were equal. |
| 72 | +I proved the upper bound very quickly because it was a very straightforward task of restricting a minimization to a simple subset of the original domain. |
| 73 | +However, the lower bound just kicked my ass and I got nowhere in explaining all the special structure that I observed, so I eventually gave up and put it aside. |
| 74 | + |
| 75 | +I finally put together the lower-bound proof while at Sandia, |
| 76 | + but only after returning to it multiple times for brief but intense bursts of activity. |
| 77 | +Around summer 2012, I finally learned about Zolotarev's work in applying elliptic integrals/functions to rational approximation. |
| 78 | +This was very exciting at the time, because it explained all of the special structure that I had observed numerically. |
| 79 | +I can't imagine how anyone could make the connection from those numerical observations to the theory of elliptical integrals, |
| 80 | + and indeed Zolotarev had worked in the other direction - his adviser, Chebyshev, had instructed him to look for useful applications of elliptic integrals. |
| 81 | +Unfortunately, my excitement died down when I realized that none of this made a lower-bound proof any easier, |
| 82 | + although I could now rule out the previously-unexplained structure as being relevant to a proof. |
| 83 | +I did eventually cobble together a lower-bound proof as I was exposed to the necessary mathematical ideas while at Sandia. |
| 84 | +I worked on an idea for [relaxing classical stat mech problems into linear programs](https://arxiv.org/abs/1603.05180), |
| 85 | + which was itself a catastrophic failure, but it increased my familiarity with convex optimization and linear programming (two linear programs being essential steps in my proof). |
| 86 | +Also, I was exposed to many lower-bound proofs of classical computational complexity in quantum information theory |
| 87 | + that were a part of complexity-theoretic separation proofs between classical and quantum computing power. |
| 88 | +While none of those proofs were directly relevant, I learned about the wide variety of clever tricks and strategies |
| 89 | + that are often employed in producing lower bounds for optimization problems. |
| 90 | +I finally had a serviceable draft of a proof in summer 2017, which I polished up a few months ago for this paper. |
| 91 | +I probably could have wrapped this up a lot sooner if Sandia was supportive of my research interests, but it was not, |
| 92 | + and research tends to go a lot slower when you are being paid to do something else as your full-time job. |
| 93 | + |
| 94 | +This is my oldest research project that has been brought to fruition, |
| 95 | + and hopefully it is a sign of maturity and a precursor to other results following suit (albeit very slowly). |
| 96 | +My initial proof in summer 2017 was tied to a paper on banded Hermitian eigensolvers, |
| 97 | + and it collapsed under the weight of trying to accomplish too much at once. |
| 98 | +Up until three weeks ago, this paper was tied to three applications that would have further delayed it by many months if I hadn't cut them. |
| 99 | +I have a natural tendency to make big plans for papers, |
| 100 | + and I need to actively fight that by breaking up projects into smaller publishable pieces. |
| 101 | +Having shorter, more numerous papers is advantageous for many reasons, |
| 102 | + and it is almost essential for surviving the rat race of modern academia. |
| 103 | + |
| 104 | +My immediate next research priority is finishing up the first delayed application of this paper to electronic structure calculations. |
| 105 | +However, I need to do a better job of mixing blog posts about active research papers |
| 106 | + with other research discussions about other interests that are in a more passive mode. |
| 107 | +I'd like to start discuss quantum computing on this blog soon, and I have two quantum computing papers lined up as #2 and #4 in my publication queue. |
| 108 | +I also continue to refine and develop my semiempirical electronic structure plans, |
| 109 | + and I will discuss that at some point as well. |
| 110 | +I am happy to have finally finished this long-delayed paper, |
| 111 | + but my research agenda and career are very deep in a hole |
| 112 | + and it will take a lot of work before I can escape that hole. |
| 113 | +There will be a lot more research triage in my future. |
0 commit comments