Science writing: What we know vs. what we don't know

Academic science writing (primarily journal articles) uses a rhetorical framework that lays out a portion of some scientific discipline or natural process, locates and describes the 'knowledge frontier' that bounds what is known and what is unknown, and zooms into that frontier to answer a tidy question and install a very small bit of knowledge infrastructure that extends the knowledge frontier incrementally out into the wilderness. The focus is generally right at the frontier, with just enough of the knowledge 'civilization' mapped out to orient other scientists, and just enough of the wilderness speculated upon to give some idea of where the new piece of knowledge infrastructure might lead without opening the scientist up to being slapped when the future reveals differently. Much of the writing, reviewing and general effort is in making sure the nuts and bolts of the small scientific contribution are solid, but the aims of the community are generally forward, looking out into the wilderness.

System constraints are largely responsible for the narrow focus of academic science writing. Peer review is at the top of the list: Many reviewers (myself included) don't always like to see too much speculation in a paper. Such speculation is too frequently poorly-reasoned justification for why the work done in the article is important, or how it 'proves' a theory, rather than a well-thought-out description of uncertainty and possibility. I also believe that most readers of a paper are not reading to understand the exact question that the authors of a paper are trying to answer; they may want to answer a related question, or get the data for a very different study from the same region. The data and results of a paper (and often methods) have a longer shelf-life than the discussion. A long discussion of what we don't know is very rarely appropriate in a typical paper. Review papers are different, of course.

However, the sheer difficulty of creating a new piece of knowledge also makes one focus narrowly on the task at hand, and rightly so. Much of the discipline required to do science is used to force yourself to walk, not to run, and to crawl when you have to over sharp and rough terrain. Mistakes can be subtle and you need to do work that others can rely on. It is hard to make the switch from slow, deliberate and careful work to unmoored, ungrounded intellectual exploration through thought experiments and so forth.

(Does this restrict the progress of science, by making us into incrementalist drones that aren't capable of high-risk, high-reward science as is often claimed? Probably, a little, but as claimed by Richard Hamming and others knowledge grows exponentially, as does the economy, and with science the downside tail risks of propagating false information are as asymmetrically large vs. the upside and as consequential as they are in investing. Let knowledge progress incrementally. A speedup of a few percent is a fine goal.)

Popular science writing, on the other hand, aims backwards, describing what the state of knowledge is at the frontier and maps it back to familiar civilization, where the audience resides. The best of it, such as work done by Quanta Magazine or in books (often written by scientists rather than journalists) does a good job of describing the frontier and the potential layout of the near wilderness, and may speculate at what may lie beyond. The majority, however, doesn't. It focuses on what we know, rather than what we don't know. It may be presenting new work in a context appropriate for a non-specialist audience, or giving some background for a current event.

There aren't many good fora for far forward-looking scientific discussion, or in-depth discussion of what we don't know. Much of it happens verbally, in seminars or conferences, and especially over beers after these events. Blogs could be a great venue, but a lot of science blogging is popular science writing, explainers of topics or current events by scientists and graduate students, or promotion of recent papers or datasets. Then there is a lot of blogging about the scientific experience: advice for grad students, stuff about conferences, minority representation/experience, etc. Finally, there are a number of blog posts by various folks that describe methods (particularly computational) for other practitioners, and these are extremely useful. All of this stuff is necessary and usually interesting, but it still focuses on what's known when it focuses on knowledge at all.

Discussion boards like Reddit and Hacker News are in principle good places to read and write about what's not known, but the median quality of a post is low and I don't think professional scientists use them very much for this. They're also probably seen as a waste of time and/or a guilty pleasure, quite correctly. Maybe there is good discussion on twitter, but that's a terrible forum for nuance and detail and I avoid it.

As scientists, we have to have some mental maps of what we know and what we don't in order to plan (and to write introducitons to proposals and articles). I think that most of these maps stay mental. As I skip around between timescales and methods in my exploration of faulting (the focus of my research), I lose track of a lot of things that are in my head on sub-topics that I haven't worked in a few years. I think that the benefits of instantiating thoughts on what we don't know are probably great, but it's obviously difficult to lay this out with specificity. I have started to keep detailed digital notes (currently using org-mode but it's a non-starter on iOS, a problem for note-taking on the go, and I need to be able to draw diagrams). I think I'm also going to use this blog to write stuff out more, which helps organize thoughts and 'trim branches from the logic tree' as we do in PSHA, as well as for archival purposes. No one reads my blog so restricting discussion to the comments is not effectively restricting.

I'll start, you know, tomorrow.