A Novelty Metric for Evaluating Journal Articles and Authors


Scientists have lengthy regarded for methods to measure the influence and worth of their analysis. In this text we suggest a brand new metric that makes an attempt to measure the recency of the information mentioned in an article.

Many of the present bibliometrics stem from work of Eugene Garfield who was among the many first to trace citations, and use the counts of citations in numerous algorithms to measure the influence of each papers and scientists. Since his work many metrics have been developed based mostly on citations. These embrace these fashionable metrics (this isn’t an exhaustive record):

All of those are based mostly on how usually different authors discovered a piece value citing in subsequent articles. These are all good metrics.

Here I suggest a measure of novelty of an article – the variety of new information reported reported within the article. Elsevier’s RESNET has extracted out hundreds of thousands of organic relationships from article abstracts and full-text from literature from . In this examine these are the “facts” used to guage the novelty of a paper.

Here is an instance organic relationship, “citrate upregulates cell differentiation”, which is talked about in 17 articles.


The kinds of relationships extracted in RESNET embrace regulation, binding, antagonistic occasions, pathway-disease hyperlinks, ligand-protein binding and many others. Each is annotated with all the articles stating the very fact.

Journal articles usually talk about many identified relationships along with proposing novel ones. The weighting algorithm we’re experimenting with down-weights relationships by the variety of earlier papers that state the relation to give attention to papers that state novel relationships. Therefore the “novelty score” of a paper could be computed with the next method, based mostly on all of the relationships extracted from the paper:


A novel, by no means earlier than reported relationship contributes 1.0, whereas a relationship said in 100 earlier papers (earlier: identical or earlier 12 months because the paper being ranked) provides solely 0.01 to the rating. Therefore papers stating newer, much less reported, relationships can have increased scores. Relationships that seem within the introduction of an article explaining the background of the analysis instance might not add considerably to the novelty rating. Unlike quotation scores, this metric is static. Since it solely counts the present and earlier years for the article being scored the metric will not be modified by future articles.

We examined the idea with about 10,000 biology articles – all from 2015 in order that all of them have had equal alternative to be cited. This was to permit comparability of the novelty rating with quotation counts.

The graph beneath compares the novelty rating towards quotation depend since 2015 for every paper within the corpus. The graph exhibits that the novelty rating will not be associated to citations and subsequently is offering completely different data. It isn’t a surprise that that the rating is roughly associated to the whole variety of relationships extracted from a paper.


The prime papers of 2015, as ranked by the novelty rating, are proven beneath.

mclark5-1024x216-9918267 mclark6-2-7178141

The highest scoring paper for 2015 was “Proteomic analyses reveal distinct chromatin‐associated and soluble transcription factor complexes” This work recognized a lot of novel protein-protein interactions. The interactions first reported within the paper, in addition to some reported in just one different paper are proven beneath.

mclark7-1024x347-4892442 mclark8-2448394

One can then create creator scores by summing or different features based mostly on the novelty of the papers that they’ve revealed or co-authored. This desk exhibits the scores for the authors with the very best sum of novel information revealed. However, features apart from easy sum might higher signify an creator metric.

One can see that some kinds of papers report extra relationships, e.g. people who have tables with goal binding of compounds are inclined to have massive numbers of novel information. So the scores could also be most comparable amongst papers and authors when evaluating the identical sort of analysis in biology or medicinal chemistry.

This bibliometrics experiment relies on organic relationships extracted for the RESNET dataset in PathwayStudio, so it focuses on biology and associated life-science matters. One may think about comparable evaluation for different fields akin to chemistry or physics as a strategy to extra shortly establish work reporting novel information.

The Elsevier Services workforce can apply this, and many different, kinds of revolutionary analytics for you to assist establish novel discoveries and researchers who’re reporting essentially the most revolutionary science in fields of your curiosity. We are striving to search out new metrics and analytics to handle your wants.

R&D Solutions for Pharma & Life Sciences

We’re blissful to debate your wants and present you ways Elsevier’s Solution may also help.