The broken game of academic research: how journals are shifting the incentives of researchers and ruining science
Games are fundamental to the behavior of rational agents, and many biological, societal, and economic systems can be understood and analyzed using game theory. In game theory, a game is defined by a set of players, a set of rules, and a set of objectives. In large, the goal of game theory is to define and study games, and in doing so, it can help to define policies (rules) and actions that optimize the process of reaching an objective. Life itself can be construed as the most elemental of games, where living organisms are the players striving to survive, thrive, and procreating. Similarly, conflicts such as wars can be analyzed as strategic games, with goals like territorial conquest or population control. Naturally, tactics become important in games, and a large part of game theoretic endeavors lies in identifying the optimal tactics (actions) that result in the desired outcome of a game.
When breaking down a complex system into a set of games, it becomes possible to define policies that shift the incentives of players to favor a desired outcome. Here, I propose an exploration into the academic sphere as a set of games, illustrating how the misalignment of incentives and goals among key stakeholders has led to suboptimal outcomes that fail to serve the interests of humanity.
The game of academia
In academia and research, a familiar game unfolds, involving two primary players: researchers and the public, who are the ultimate beneficiaries. The overarching objectives of this game are twofold: to expand humanity’s understanding of the universe and its phenomena, and to propel technological advancements in ways that benefits humanity. To steer clear of moral and ethical debates, the notion of “benefitting humanity” is defined as the ethically optimal objective in a Platonic sense (which may also include benefitting other organisms). Ideally, the academic system should incentivize actions aligned with these objectives, fostering beneficial research, facilitating the dissemination of knowledge, and fostering technological innovation.
Adopting a game theoretic viewpoint to navigate our intricate reality necessitates acknowledging that individuals are often enmeshed in multiple simultaneous games, each characterized by potentially conflicting objectives. Researchers are human too, and harbor ambitions beyond the altruistic pursuit of knowledge. Alongside the noble aspirations outlined earlier, they are driven by more egoistic aims such as financial security and the acquisition of influence and power. Most researchers therefore also strive for academic fame, which is accompanied by a larger paycheck and greater influence. The researcher now has three goals: the two ideal ones mentioned in the previous section and to become a famous researcher.
Fame in academia is a product of recognition and esteem bestowed by both the public and one’s peers. In essence, if the broader public perceives an individual as prominent within their field, then they indeed attain fame. If we wish to keep the researcher’s goals aligned with the ideal goals, it is therefore pertinent that the public and peers share these goals. Moreover, it’s crucial that the public receives an impartial portrayal of the published research, thereby facilitating an unbiased evaluation of the researchers contributions. We have now introduced a new player into the game, namely the communicator between the researcher, their peers, and the public. In actuality, the communicators are the scientific journals.
The power journals hold in academia is that of selecting and spreading research findings. They therefore play an important part in furthering and extending the border of knowledge, but they also play a large role in governing the influence and fame of researchers. It is therefore crucial that journals uphold a level of integrity and that they are regulated so that they don’t sway from the goal of science and placate the egoistic needs of humans. Unfortunately, this is exactly what is currently happening in the field of academia.
Journals are highly efficient profit machines that dictate the success of researchers while exploiting their flaws as humans as well as their institutions for monetary gain. They play their own game, in which impact factors are a proxy for profit, and their tactics include utilising researchers as editors, reviewers, and publishers, creating a money-printing cycle where researchers run like hamsters in the pursuit of scientific glory.
As long as the publication system exists in its current state, there will always be a misalignment between the ideal goals and the incentives of the researchers. This results in bad research, where the goal is to publish in high-impact journals instead of doing good science, resulting in reporting bias, fraud, and hype trains. Although the downstream impact of such research is unclear, it is reasonable to believe that research without such systems in place would generate better science, which in turn would make the world a better place. The diversion from such a world is morally wrong.
How to align incentives with goals
While it is possible to imagine a scenario where the researchers are unaffected by the incentives of journals and uphold a moral standard that furthers the ideal goals of research, it seems like a highly unlikely one. This is because of the inherent imbalance within the system, wherein any deviation from this ideal stance instantly elevates the errant researcher to fame, disrupting the equilibrium of the game. It is not a steady equilibrium. It is therefore necessary to change the system to align the incentives with the ideal goals, which means altering the way in which research is evaluated and spread to the public and to peers.
But how can science be spread in an unbiased way that doesn’t incentivize the researchers to deviate from the desired outcome? This is a difficult question for which I have no definitive answer. However, two pragmatic actions, advocated by numerous voices in the academic community, stand out as compelling avenues for enhancing the publishing system:
-
Transition away from traditional journals and pre-publication peer review models toward embracing pre-print servers. These platforms would entail an initial screening to filter out blatantly flawed submissions, while allowing the bulk of research to be freely accessible without undue editorial influence.
-
Embrace transparency by implementing open post-publication peer review.
These actions would remove monetization from the publication system, reduce the time of publication, thereby increase the rate of scientific exchange, and reduce the level of gatekeeping currently upheld by monopolized publishing companies. Journals play an important role in identifying good research, but this can be done just as easily with post-publication peer review and other types of archives.
This model has already been adopted by some fields of research, prominently in machine learning (ML) research. Here, articles are often never published in any conventional peer reviewed journal, but instead servers like arXiv are the names of the game. Papers deemed noteworthy quickly gain traction and virality within the research community. Alternative decentralized ways of funding, doing, and reporting science utilizing the Web3 stack are on the rise. Referred to as DeSci (Decentralized Science), these alternative frameworks represent a potential avenue for reshaping the scientific enterprise. However, their widespread adoption remains uncertain. In general, I believe that the two points above are enough to fix the problems with journals. Further, they are actionable and available with current technology.
In summary
While acknowledging the valuable contributions of scientific journals, it’s essential to address a fundamental issue: the disconnect between researchers’ incentives and the overarching goals of scientific advancement. When publications are valued primarily for their placement in prestigious journals and their impact on authors’ h-indexes, rather than for their substantive content and contributions to their respective fields, it undermines the integrity of scientific research. In an era where the importance of science and technology has never been greater, we must recognize the imperative for change within academic norms. Only by reforming the current system can we ensure that scientific endeavors truly serve the greater good.
Enjoy Reading This Article?
Here are some more articles you might like to read next: