FREQUENTLY ASKED QUESTIONS
Robert Borofsky
Center for a Public Anthropology
1. Introduction
The Public Anthropology Metrics Project encourages anthropologists to produce
publications that reach beyond the normal academic confines of their discipline to address concerns of the broader society. Many find it hard to push against the insularity of traditional academic practice. And yet, it is the broader society that provides much of the funding for anthropological research. Few would openly claim that the discipline’s publications are produced primarily for the anthropologist’s own personal benefit in advancing their careers. Yet the benefits of the discipline’s publications for others beyond the discipline – whether in the broader society or for the people they studied among – tend to be comparatively rare.
Clearly some anthropologists produce publications that attract much attention beyond the discipline. But a decent percentage of the field tends to focus strictly on internal concerns that are primarily relevant to only a small coterie of colleagues. The Metrics Project challenges anthropologists to produce publications of broader significance that reach beyond the discipline to the wider world.
2. How Were the Project’s Data Collected?
First, using the National Research Council’s rankings of anthropology departments (see https://www.chronicle.com/article/doctoral-programs-by-the-numbers-124703/ as well as https://en.wikipedia.org/wiki/United_States_National_Research_Council_rankings), I compiled a list of the full-time faculty in the top fifty departments. I then ran these faculty through the Altmetric database to see which publications by these faculty were noted by the world’s media. 1,2
Rather than primarily focus on a few individual faculty in a department whose publications receive wide attention, the Project ranked university departments based on (a) the number of faculty in a department having publications with three or more mentions in the international media (as affirmed by the Altmetric database) divided by (b) the number of full-time members in that department. The Project seeks to do more than take note of faculty successful in reaching out to broader audiences. The goal is to draw the discipline more generally into doing so.
Altmetric monitors a range of online sources on a continuing basis. It searches for mentions of scholarly outputs (items with DOIs, arXiv, SSRN IDs, or unique URI identifiers) in order to present this information in a structured, coherent way. While the Metrics Project focuses on mainstream news outlets and policy documents, it should be noted Altmetric also tracks other data sources such as Wikipedia, Twitter, Reddit, Sina Weibo, and Mendeley (note https://www.altmetric.com/about-our-data/our-sources/).
Collecting data from the over 4,000 mainstream news outlets Altmetric examines, it receives a real-time feed of news stories via Moreover (part of LexisNexis). Each of the news stories is searched for links to scholarly outputs as well as text-mined (English language only) for mentions of an author’s name and journal title. These pieces of information are then cross-referenced with publications from CrossRef to determine if a publication fitting these criteria has been published within 15 days on either side of the news item. If a positive match is made, the reference is included in Altmetric’s details page for that publication.
With respect to public policy documents, Altmetric monitors a curated list of policy sources and searches for new documents daily (as they appear online) for references. These references are then added to a publication’s details page, usually within a day’s time.3
Readers interested in seeing the complete list of Altmetric media journals searched in collecting data on individual faculty members, can refer to (https://metrics.publicanthropology.org/files/all_journals.pdf). Readers interested in seeing which journals published articles by the anthropologists highlighted in the Altmetric database cited here, should refer to (https://metrics.publicanthropology.org/files/approved_articles_count_per_journal.xlsx). If they wish, readers can select a few of the highly cited anthropologists mentioned and peruse the media sources listed in the red block next to each of their publications to get a sense of the range as well as the sources noted for each of the publications.
The second link may be of particular interest for many readers. It indicates which of the above journals were noted most often by the international press. It suggests the world media not only pay particular attention to a few widely known journals such as Science and Nature, but that they also pay attention to the prominent anthropological journals, such as Current Anthropology and American Anthropologist. The table suggests it is not just the journal an article is published in but also the topic of the journal article that attracts attention. An examination of the topics covered suggests few of the highlighted publications are “sensationalist.” Rather they cover topics of broad interest to a range of audiences.
Before reviewing the following data in detail, I would strongly recommend you read “PLEASE READ THIS.” It will put the Metrics Project with its massive amount of data in perspective. The database is not perfect. But it contains a wealth of very valuable information – information anthropologists would do well to pay attention to if they wish to reverse the decline of their discipline.
1 The list of full-time departmental faculty is based on that department’s website. Adjunct and affiliate faculty were excluded. Faculty names used in word searches are based on their names as presented on their departmental websites.
2 Since the Altmetric journal database became operational in 2012, the Metrics Project only considers publications appearing in journals since then. Books were added to the database in 2016.
3 For further information, please refer to: https://help.altmetric.com/support/solutions (accessed September 2, 2024).
3. What Do the Displayed Results Refer to?
When you click on https://metrics.publicanthropology.org you will see a list of schools. These are the schools referred to in respect to the National Research Council’s top fifty rankings. As noted above, the schools are ranked by the percentage of faculty in their anthropology department which have three or more citations in the world’s media (as recorded in the Altmetric database). When you click on a school, you will see a list of the full-time anthropology faculty at that school. The upper set of faculty refers to those who have had at least one of their publications referenced three or more times by the global media. Below that list are those who, to date, have not reached this level, but hopefully will soon.
To see the publications cited in the world’s media for each of the faculty noted, please click on the blue block that states: “CLICK HERE TO SEE HOW THESE FACULTY WERE CITED IN THE WORLD’S MEDIA.” This will lead you to the faculty, their publications, and what about these publications the global media takes note of. Click on the red block to see what they world media not about the individual publication cited to the right of the red block.
May I make one request? Since departmental lists of full-time faculty are regularly changing, if your department’s list is out of date, please let me know at webmaster@publicanthropology.org and I will correct it.
For additional information about Altmetric with more specifics on its operation, readers might wish to visit: https://help.altmetric.com/support/solutions.
4. How Does Altmetric Compare with Google Scholar in Assessing a Publication’s Public Impact?
One should be cautious in assuming that because scholar A cites scholar B, scholar B has had a significant influence on scholar A’s work. As noted by Borofsky in An Anthropology of Anthropology (2019, 113):
Often a publication’s intellectual value is assessed by the degree to which others cite it. This is a flawed standard. Citing an author does not guarantee actual engagement with the author’s ideas. Rhode writes: “There is no guarantee that authors have actually read the sources cited. Indeed, with technological advances, they need not even trouble to type them; entire string citations can be electronically lifted from other publications. Nor does it follow that the sources listed establish the proposition for which they are cited. Even when someone checks the notes, it is generally to determine only whether particular authorities support the text, not whether they are reliable or respected among experts.” (Rhode 2006, 38)
As An Anthropology of Anthropology suggests,
Most of the citations to [prominent anthropologists’] key works [are] of the “bump and go” variety, to use an American football metaphor. Authors mostly refer to them to convey they are aware of the relevant literature related to the topic they are writing about. But few seek to systematically engage with the ideas in these figures’ key works for more than two sentences. (2019, 79)
Grafton, in The Footnote: A Curious History observes:
Only the relatively few readers who have trawled their nets through the same archival waters can identify the catch in any given set of [foot]notes. . . . For most readers, footnotes play a different role. In a modern, impersonal society, in which individuals must rely for vital services on others whom they do not know, credentials perform what used to be the function of guild membership or personal recommendations: they give legitimacy. (1997, 7–8)
Reinforcing this perspective, as just noted, is the fact that within anthropology, rarely is a famous author’s key book cited and discussed for more than two sentences in citations referencing that book. They are mostly referred to in passing.4
Google Scholar focuses on the number of times an author is cited in various academic publications. It does not examine the contexts of a citation, the number of sentences that deal with the citation in a publication, or its general significance in the publication referred to. It assumes more citations means more intellectual engagement. For an example of how this may go wrong see “How Easy is it to Fudge Your Scientific Rank? Meet Larry, the World’s Most Cited Cat.”5
Altmetric, in contrast to Google Scholar, offers a less introverted perspective – academics citing academics. Public impact is assessed in the Metrics Project by what those beyond the academy write in the world’s media about a particular publication. Readers can click on a citation and read the relevant passage (or passages) related to that citation to understand how and why it is cited. Interested readers might also refer to https://www.researchinformation.info/feature/measuring-context for further information.
4 See Borofsky (2019, 66-106) for specific details.
5 Christie Wilcox, “How Easy is it to Fudge Your Scientific Rank? Meet Larry, the World’s Most Cited Cat.,” Science, August 1,2024, accessed September 2, 2024, https://www.science.org/content/article/how-easy-it-fudge-your-scientific-rank-meet-larry-world-s-most-cited-cat.
5. Can the Metrics Project Benefit Anthropologists and Anthropology Departments?
Today, two outside forces are reshaping the discipline. First, there is an increasing demand for accountability: Are anthropologists producing work of public value that justifies their salaries and the costs of their research?
The call for increased academic accountability has been building for decades. Starting with the National Science Foundation Act in 1950, the National Defense Education Act in 1958, and the Higher Education Act in 1965, considerable funding has poured into universities. Today, this funding totals more than $117 billion. Not unreasonably, those providing this funding are concerned about how this money is being spent.
Second, in times past, reviewers of a promotion portfolio might informally assess a colleague’s publications. With advances on the internet, metrics now exist for measuring the status of the journals an author publishes in, as well as the number of colleagues that cite their work. This has shifted the center of power regarding control over accountability assessments. Administrators no longer need depend on faculty versed in a discipline’s intricacies to assess an individual’s productivity. They can rely on a host of numbers instead.
What the Metrics Project offers is a way to address the demand for public accountability without succumbing to the flaws of the statistical citation assessments – numbers without context – associated with Google Scholar and Academic Analytics. The Altmetric data allow readers to understand and assess the context and significance of a citation in a more open, transparent manner that provides a better sense of its public impact.
6. When I Click on an Article, the Cited Author is Not Always Listed. Why is That?
Generally, when one clicks on the publication listed, the individual referred to will be one of the publication’s authors. However, occasionally it is not. I found in the Altmetric database, for example, 67 references to Paul Farmer, dating back to 2011. For most references, he is listed as an author. In respect to the others, one cited him in the publication’s text, another in the publication’s references, and for one, he authored a forward. There were also two other cases. One publication involved a Paul Hamilton and a Madison Farmer as authors. Presumably, the search engine combined them into Paul Farmer. In “The Year That Ebola Virus Took Over West Africa” by Bausch in the American Journal of Tropical Medicine and Hygiene (2015) the article does not make specific mention of Paul Farmer. As its title suggests, the article is about the Ebola epidemic in West Africa. Paul Farmer, in his role as Chief Strategist for Partners in Health, was quite involved in treating Ebola in West Africa during that period. I am unsure how Paul Farmer’s name became associated with this article. But the error rate, if you include the name merger above, is three percent (3%).
For further clarification of how Altmetric collects its data, readers might wish to refer to: https://www.altmetric.com/about-us/our-data/how-does-it-work/.
7. Who Sponsors the Metrics Project?
The metrics project is sponsored by the Center for a Public Anthropology. Please refer to the website, publicanthropology.org, for further information on the Center.i
If you are interested in learning more about Altmetric and how it can be applied across various departments and schools, please go to altmetric.com and click on the “Who we serve” heading and then click on “Institutions.” Or, when you go to altmetric.com, click on the “Solutions” heading and then “Altmetric Explorer.”
REFERENCES
Bausch, Daniel G.
2015 The Year that Ebola Virus Took Over West Africa: Missed Opportunity for Prevention. The American Journal of Tropical Medicine and Hygiene 92 (2): 229-232, https://doi.org/10.4269%2Fajtmh.14-0818.
Borofsky, Robert
2019 An Anthropology of Anthropology: Is it Time to Shift Paradigms? Kailua, HI: Center for a Public Anthropology.
Grafton, Anthony
1997 The Footnote: A Curious History. Cambridge: Harvard University Press.
Miles, Rachel, and Robyn Price
2023 Using Altmetric Data Responsibly: A Guide to Interpretation and Good Practice. https://vtechworks.lib.vt.edu/server/api/core/bitstreams/3a2357c7-a644-4ace-a82b-8338052c2908/content (accessed September 2, 2024).
Rhode, Deborah L.
2006 In Pursuit of Knowledge: Scholars, Status, and Academic Culture. Stanford, CA: Stanford University Press.
Wilcox, Christie
2024 How Easy is it to Fudge Your Scientific Rank? Meet Larry, the World’s Most Cited Cat. Science.
https://www.science.org/content/article/how-easy-it-fudge-your-scientific-rank-meet-larry-world-s-most-cited-cat (accessed September 2, 2024).