Reasoning Transparency
What do we mean by Reasoning Transparency?
Reasoning transparency as a research skill is your ability to make your research clear and explicit in its reasoning and conclusions, so that others can more easily understand what you did to reach your key takeaways and how to integrate those takeaways into their own thinking. Reasoning transparency, like archival research or data analysis, is a skill that academics and researchers can cultivate in order to contribute to their fields. In our opinion, reasoning transparency is an often overlooked skill that can improve the quality of academic writing as well as its usefulness to other researchers.
A research paper with excellent reasoning transparency goes beyond the minimum requirements for publishing in academic journals. In addition to including a methods section, mentioning related literature, and providing an abstract, a paper with exceptional reasoning transparency expresses degrees of confidence in the claims it makes, and makes all relevant contextual information available to the reader. It also states clearly what was not done as part of the research, and ideally why not (what would have been top priorities given more time and/or resources?) Reasoning transparency is a concept as well a set of skills that must be practiced and refined in order to be useful. We recommend working through the exercises at the end of this post if you’d like some practice sharpening your reasoning transparency skills.
Why do we think this is important?
When reading a research paper, you want to know how to relate to the information presented so that you can update your own views appropriately. Conversely, working on your own reasoning transparency may help your peers better understand, utilize, and critique your work. We believe that if you’re making a conscious effort to present your line of reasoning more clearly to others, you might notice issues with it you wouldn’t otherwise have spotted if you were only seeking to satisfy academic publishing standards.
If you want to maximise the positive impact your research has on your field and the world, transparently communicating the process and takeaways of your work should be a high priority. This isn’t always highly incentivised in academia, but by developing high reasoning transparency you can contribute to improving the knowledge and epistemic norms in your academic field.
What information contributes to reasoning transparency in your writing?
First, open with a brief and clear summary of the key takeaways of your research, with links to corresponding sections. This practice:
Communicates to the reader what you think is most important about your research.
Allows the reader to dive into each specific claim based on how relevant it is to them.
Helps readers understand how you understand your own work.
Indicate which considerations are most important to your key takeaways
Along with your sources, include brief commentary on what the key, pre-existing arguments and data are on which your conclusions rest.
If there is relevant research and data that you are familiar with but chose not to integrate into your research, make a note of it, and briefly explain why.
This practice allows other researchers to focus on sources you have considered less, and check if they disagree with or doubt your reasoning given the emphasis you put on different considerations.
This allows others to more accurately represent your insights.
Express degrees of confidence you have in different claims
You can do this with words of estimative probability (ie: “we think it is unlikely that ____ is true.”) or by providing a probability estimate (ie: “we think that there is only about a 5% chance of _______ being true.”)
You may also feel that it is important to describe why you chose not to pursue certain directions in your research. For example, “it is possible that _____ is true, but since we do not think it is very likely, we decided to prioritize other directions.”
Scholarship is cumulative. No scholar can be expected to have exhaustive knowledge about every piece of research or evidence they draw on in their own work. We think that being more open about this is more epistemologically sound than pretending to know everything.
Helping a reader understand how you assign weight to different claims may help them have a better sense of how much your reasoning aligns with or diverges from their own.
Indicate what kinds of support you have for key claims, whether they are yours or someone else’s and whether they are instrumental to your argument
Give the reader a window into how closely you examined a particular source.
Be transparent about your qualification to assess a given study or claim.
Specify whether a claim is one that you synthesized from various sources of information. Does it follow logically from another claim? Is it easy to verify?
If there is data that informs your claims, can the reader access that data for themselves?
This will help you have a better understanding of how much confidence you should have in the claims you are using in your research.
It will also help readers decide how much weight to give different claims, especially when referring to edge cases or speculations that may be worth mentioning even if you are less certain about them.
Let’s Practice! Exercises to make your reasoning more transparent
If you would like to practice your reasoning transparency skills, we recommend trying the following exercises. We think that they are most effective if you do them with a small group of your peers, but you can do them on your own, too. Each exercise focuses on one of the four main recommendations for improving your reasoning transparency.
Exercise 1: Key summaries
Choose either an essay or paper you have written or a paper that you have read while researching your current project.
Take about 6 minutes to write a brief 3-8 point key summary that you would add to the top of the paper.
If working with others, take about 5 minutes to look at their summaries and give feedback.
Does the summary give you a good sense of what this paper is about?
Does it seem to include key arguments?
What is not clear to you and why?
Do all of these seem like key takeaways?
Is the summary too long? Too short?
Exercise 2: Main considerations
Take a moment to reflect on your current project and think about how to make its main considerations clearer. Is there anything crucial that you could highlight more?
If you’re doing this exercise alone, write a short list for yourself. If you are working with peers, take a few minutes to share your reflections with the group.
Exercise 3: Expressing confidence in claims
Each partner takes 7 minutes to skim one section of a research paper they have written.
Take 7 more minutes to explain the key takeaways from your section to your partner.
On your own, take 5 minutes to write down the main claims of your section, the degree of confidence you have in each one, and what kind of support you have for them.
Share and discuss with your partner.
Action Item: Use this Reasoning transparency checklist
Set yourself a check-in date in the later half of writing your research paper to go through this checklist and see whether you could make any adjustments to your work to improve your reasoning transparency. One self-check in (or a check in with an accountability partner!) can go a long way.
Further reading
The post that inspired this article: Reasoning Transparency – Open Philanthropy
An interesting critique of Reasoning Transparency on the Effective Altruism forum: Limits to Legibility – EA Forum
This advice piece was published 5/06/23. Thanks to Dan Muro for writing this piece.
