Article-level impact for web native scholarship

Cali.org, Eric Molinsky

• 9,596 signatories
• Don't use JIF
• Value all research outputs

• Journals are more or less all online

  • Many companies collecting metrics on internet use
  • We love to share things on the web
  • CV's are online
  • Etc., etc.

• We can listen to these conversations

Altmetrics: new ways of measuring impact

Article-level metrics: altmetrics on a scholarly paper

[1]: Keith Brown, CSID Blog, 2013-05-22

• Usage: html, xml, pdf downloads
  
  
  
• Citations: Scopus, PubMed Central, Crossref
  
  
  
• Social media: Twitter, Mendeley, Facebook, etc.

• Scholarly: Citations, shares by scholars, etc.
  
  
  
• Public: News articles, shares by non-scholars, etc.

• Serve
  • Universities: Plum Analytics
  • Publishers: Altmetric.com, ImpactStory
  • Individuals: ImpactStory
  • Readers: PLOS ALM
• Vary in data openness
  • Open data: PLOS
  • Just opened up: Plum Analytics (see here)
  • Some open data: Altmetric.com

• html, xml, pdf downloads, collected by publishers

plot_density(outdf, source = c("counter_pdf", "counter_html"), color = c("#DBAC6A", 
    "#A06D34"), plot_type = "histogram")

• Scopus, PubMed Central, Crossref

plot_density(outdf, source = c("crossref_citations", "pubmed_citations", "scopus_citations"), 
    color = c("#EFA5A5", "#CFD470", "#B2C9E4"), plot_type = "histogram")

• Twitter, Mendeley, Facebook, etc., collected by altmetrics aggregators

plot_density(outdf, source = c("twitter_total", "mendeley_total", "facebook_total"), 
    color = c("#EFA5A5", "#CFD470", "#B2C9E4"), plot_type = "histogram")

[1]: Priem, Piwowar & Hemminger, arXiv:1203.4745, Fig. 8

[1]: Priem, Piwowar & Hemminger, arXiv:1203.4745, Fig. 9

• Institutional subscribers w/ altmetrics dashboards for their faculty/staff
• Don't know if they evaluate them on these though?

• Pushing their profiles as the new CV

• Publishers display metrics

• Academics reporting altmetrics on their CV's and say that it may have helped them get tenure
• Stories of use in hiring/tenure decisions - but no hard data (e.g., this)
• Some have used altmetrics on their application to be an editor for PeerJ
• Wellcome Trust has been thinking about using for evaluations
• Amy Brand @ Harvard on evaluations - don't look at journal name; do look at citations - other products "leaking in"
• SSRN - uses downloads as a metric for relative impact/use
• Wikipedia mentions can be particularly interesting - a textbook on the web

Quickly accumulating altmetrics can help filter articles after publication

link

link

7 of 10 most popular articles in 2012 as measured by altmetrics from Altmetric.com were open access - none were form Science/Nature - majority of chatter about them from non-scientists - wouldn't have been apparent from citations



[1]: Ross Mounce 2013, BAIST

• Largely unrealized - especially useful in mega OA journals
• Jevin West

library(alm)
alm(doi="10.1371/journal.pone.0029797")

Slot "summary":
  views shares bookmarks citations
1 29229    237        51         7

Slot "data":
                .id  pdf  html shares groups comments likes citations total
1         bloglines   NA    NA     NA     NA       NA    NA         0     0
2         citeulike   NA    NA      1     NA       NA    NA        NA     1
3          connotea   NA    NA     NA     NA       NA    NA         0     0
4          crossref   NA    NA     NA     NA       NA    NA         7     7
5            nature   NA    NA     NA     NA       NA    NA         4     4
...

[1]: Get the alm R package here

• Data consistency: aggregators don't collect data in the same way, e.g., Twitter => ImpactStory (Topsy), Altmetric (Twitter API), PlumAnalytics (Topsy), PLOS ALM (??)
• Data provenance: aggregators tracking this, but newcomers may not - some provenance we'll never know
• Long-term sustainability: Web resources are ephemeral - how do we ensure data persistence?
• Openness:
  • open data
  • transparent calculations

• NISO to the rescue!!!
• $207K grant from Sloan Foundation
• Standards
  • Granularity or data collected
  • Duration of metrics
  • Role of social media in altmetrics
  • Data sharing

[1]: NISO Altmetrics Standards Site

• Follow the #altmetrics hashtag on Twitter, Google+
• Blogs of PLOS ALM, Altmetric.com, ImpactStory, and PlumAnalytics
• Altmetrics.org
• R packages for programmatic access to altmetrics data