The Distant Librarian

Quick and Dirty Data Viz of Canadian Federal Candidates

A couple of months ago I posted about using Flourish to build a bar chart race data visualization. Shortly after that Flourish had another blog post highlighting all the different ways their site could be used to visualize election data, and I've had it bookmarked waiting for another opportunity to experiment with the service. Earlier this week I learned of a wonderful new data set, Canadian Federal Candidates: 1867-2017, by Semra Sevi, and I gave it a go.

This is admittedly very quick and dirty, but I'm pretty happy with it so far. I did some text manipulation with OpenRefine to take care of what was originally presented in CAPS, and I've actually done a bunch of normalization of the Occupation field, which deserves more exploration, but the original purpose of the data set seems to be to explore gender, so that's what I initially focused on too. Oh, and this is only showing those candidates who were elected, while the original data set includes candidates who didn't win election as well.

So you'll want to play around with the stuff in the pale yellow box to the right of the screen. The play button will move from the default of all data to showing data for each Parliament from 01 to 42. You'll want to choose Group by Gender and Shade by Gender, and then just play around with the other options to watch the data change. That's pretty much the whole reason I chose the Survey Response type visualization, because of all the flying changes ;-) It'll take a moment for the embed to load below...

 

I want to do some more data cleanup on the date fields, which may provide additional ways to show the data on a timeline. I also want to explore mapping the data - I think Flourish is expecting country names, but it should work at the Provincial level as well. TWT.

A couple of notes on the process with Flourish. I had a minor heart attack when I explored including a background image behind the visualization, and it seemed I couldn't get rid of it again. I ended up fixing it by uploading a new image (I had originally pointed to an online image), and once I'd done that I then was able to delete it from the box, but that wasn't an option when I was pointing to a web-hosted image.

I also wish there were some more options to customize the display, namely I couldn't figure out how to include a label on the slider, which is showing the Parliament number. I'd also love to insist that Flourish didn't show something when there was no data to show (like a Province that didn't exist before a certain year). Oh, and a way to set the default groupings and shadings would be nice too.

I might try to take some time to see how Tableau compares...

11 Best Web Scraping Tools to Extract Online Data

Hongkiat just posted a list of the 10 Best Web Scraping Tools to Extract Online Data (ht ResearchBuzz). I've tried a few of them, but now have more to explore. I was really surprised that Octoparse wasn't on their list. I've had some really good success with it before, and it definitely deserves to be on a list like this.

Thoughts on building my first bar chart race with Flourish

If you're on social media you've probably seen a bunch of bar chart races pop up in the past couple of weeks. The folks at Flourish have a good post chronicling the "new" phenomenon. Even better, Flourish created a template which makes it pretty easy to build your own, so that's what I did. Here are my thoughts on the process, along with some suggestions for making it even better.

First, here's the one I built (it may take a moment to load - if it never does you can view it directly here):

 

 

The first thing you'll notice is that it goes on an on. I wanted a long run of data, but didn't realize how much that would make the whole visualization drag. You can play with the speed of the animation, but that doesn't seem granular enough for me, by which I mean in fact it might sometimes be useful to slog along, and at other points speed ahead. So my first wish to the folks at Flourish is to include a play bar that would allow the user to scrub forwards and backwards along the timeline. Along with that should be a pause button. Both of those could be taken care of by recording the animation and popping it into YouTube. You'll immediately lose the interactivity of being able to filter Provinces though:

Next up on the wish list is the ability to annotate the race. This would be harder knowing what I know about how I got the data into Flourish (just a spreadsheet, but more on that later), but I think it would be incredibly useful to be able to pause the video at certain spots to pop in an annotation (and why not links to more info) to explain WHY something just occurred. In my example, Why did Victoria and Vancouver, both in British Columbia, both end up with the highest price? Pretty sure it has to do with additional Provincial and Municipal taxes, but it would be nice to be able to pause the video and include a note to that effect, along with a link to something like this page. Guess that information could also easily be added as an overall annotation though.

The final wish, and this is likely way beyond what the folks at Flourish would want to worry about, would be a data transformation tool. In order to create a bar chart race you have to provide Flourish with tidy data, and that's not usually how a dataset like this would normally be delivered. As delivered, I had all the data in columns, like this:

But for it to work as a Bar Chart Race, the date variables had to run across the top, like this:

It took some fiddling with pivot tables in Excel to get the data moved around like that. I could've also done this easily in R, but I don't use it often enough to remember how to do that :-( It would be awesome if Flourish would've recognized the data I had and made the transformation for me!

As an aside, I had to do a fair bit of data cleanup in OpenRefine to get the data the way I wanted it. I sure do love that tool!

Looks like you could also build a Bar Chart Race using Observable, but I didn't see a tutorial over there, and it didn't look as straightforward to me. Kudos to Flourish on that front!

UPDATE: Here's how to build an animated Bar Chart Race in Tableau Software.

Considering Apple News+? Don't forget to check your local library first!

I guess I'll wait to see what TV+ brings later before declaring yesterday's Apple event a total yawner, but it did remind me to check out Flipster and Browzine again before spending any time and money on News+. What are Flipster and Browzine, you ask? They're digital magazine and journal platforms that you might want to check before you shell out money to Apple, since you may already be able to access the content you want for free.

Flipster is available to me through my local public library, and offers only 51 titles, though what your library offers likely differs. You can browse all potentially-available titles on the EBSCO Flipster site.

Browzine is more oriented towards scholarly literature, and is thus likely offered through your local college or university. I couldn't find a list of individual titles, but here's a list of the publishers they cover.

Both offer apps for your iOS (and Android) devices, and both work quite nicely.

Oh, and for newspapers, poke around your public library web page for PressReader.

Here's a list of the titles available in Canada for News+.

UPDATE: Lifehacker has a similar post, and theirs clued me in to something called RBdigital Magazines, which was new to me. My Public Library has it, and it appears to offer 190 titles, so depending on what they all are, possibly my best bet. Will be exploring...

Review: A funder-imposed data publication requirement seldom inspired data sharing

We've been doing a lot of talking about Data Management Plans at MPOW, and I recently came across the following article from PLOS One: A funder-imposed data publication requirement seldom inspired data sharing.

In it, the authors take a look at 315 research projects that had been funded by the Exxon Valdez Oil Spill Trustee Council (EVOSTC) between 1989 - 2010 to see how many data sets could be obtained. This is an interesting funding agency, as it funded research from many different disciplines, including the Social Sciences. As you might guess, the number of retrieved data sets is pretty darn low (26%), but the single biggest reason was that the authors were unable to reach the original researcher. I guess that shouldn't surprise me as much as it did. The other thing I found surprising is that they found slightly lower retrievability for research that was conducted after the EVOSTC enacted a formal data policy in 1995. I guess that's the sound bite for the article title!

But the other thing we've been talking about at MPOW is what kind of reward might entice researchers to post their data, and this article addressed that as well:

"Inherent in all of the above hurdles to data recovery is the absence of a reward structure for sharing data. There is no reward for the time investment required to learn how to curate data or construct data packages (including well documented and formatted error-free data, with project and file level metadata) in cases where original data curation was insufficient. The focus in science has traditionally been on production and citation of publications, and as such, a process for identification and citation of manuscripts has been well developed. Similar attribution and recognition of data would incentivize the archiving and sharing of data in the scientific community. One way publications are tracked and cited is through the use of digital object identifiers (DOIs), a tool that is also increasingly being used to attribute data. DOIs are particularly important for data identification because unlike manuscripts, data can be regularly updated or exist in multiple formats or subsets, so identifying specific versions of the data via a DOI is key to data proper data attribution and use. Although the use of data DOIs is a relatively new practice, they should facilitate more routine data citation as compared to traditional methods. Incorporating data citations into a scientists’ overall research output alongside journal publications should further incentivize data sharing."

So they're making some claims that they can't back up with this research, but which certainly makes sense. The one thing that might get researchers to post their data sets is better recognition in the tenure and promotion process of the work that's involved in making this information available online.

As a snarky aside, I wonder if the EVOSTC website has seen a redesign since it was first launched? :-0

Archive of tweets from Semantic Web in Libraries (SWIB) 2018

Here is an archive of tweets from this year's Semantic Web in Libraries (SWIB) conference. You can search and filter the results here,  and explore nifty visualizations here. All this courtesy of the excellent (and free) TAGS project.

A disturbing story on academic freedom and predatory publishing

While I had actually read the article in question shortly after it first came out, I was unaware of the fallout until earlier this morning. This is a story you should be aware of.

How I learned of it: Investigation launched into alleged academic freedom violations at Thompson Rivers University

What I had missed: B.C. economist in grim battle against deceptive scholarship and Suspended B.C. prof who exposed fake journals welcomes investigation.

In a nutshell, Dr. Derek Pyne wrote a paper in early 2017 titled, The Rewards of Predatory Publications at a Small Business School, in which he looked at how many of his colleagues in the business school at Thompson Rivers University had published articles in predatory journals. Quite a few, as it turned out. From his conclusion (emphasis mine):

Predatory journals have become an increasing problem when it comes to assessing and rewarding researchers for the merit of their publishing records. In addition, the presence of predatory journals makes it difficult for non-experts to judge the quality and validity of published research. This paper finds that, at least at one university, there are few incentives not to publish in predatory journals. In addition, when the opportunity cost of forgone income from extra teaching is significant, publishing in ranked journals is costly.

Of course it's understandable that his colleagues and administrators would be embarrassed by this paper, but apparently he has been suspended from his position, and the university as a whole, for bringing this information to light. I say apparently, because so far Thompson Rivers University hasn't provided any actual reason. Well, they do say all this.... Despite what you'll find in the preceding link, this appears to be completely against the concepts of academic freedom and tenure, like seriously TRU?

So this post is just to make sure y'all are aware of this sorry situation, and I now have an alert on his name so will let you know when it's been resolved. Of course these are my opinions and not those of my employer, yadda yadda yadda.

November 7 Updates:

Here's a good 15-minute audio summary of the whole situation: 10/3 podcast: A B.C. economist is in hot water for exposing fake journals

And here's an article from The Economist I missed from this past summer which mentions Dr. Pyne's paper, but appears to have come out just before his suspension: Some science journals that claim to peer review papers do not do so 

 

First look at NVivo Transcription

Almost exactly a year ago I took a quick look at three automated transcription tools, and today there's another one to add to the mix, though this one's not free. NVivo has launched an automated transcription service and I'm impressed! I uploaded the same audio clip I used in last year's shootout, a 40-second snippet from the inauguration of George W. Bush, and here's what NVivo made of it:

Please raise your right hand and repeat after me. I George Walker Bush do solemnly swear. George Walker Bush do solemnly swear that I will faithfully execute the office of president of the United States that I will faithfully execute the office of president of the United States and will to the best of my ability and will to the best of my ability preserve protect and defend the Constitution of the United States preserve protect and defend the Constitution of the United States. So help me God. So help me God. Congratulations.

It didn't pick up the separate speakers, but other than that it was a perfect transcription of a somewhat noisy audio clip, which moves it to the top of the list of the one's I've looked at so far.

You'll get 15 minutes of free transcription when you sign up (and as far as I can tell, you could use this w/o having a license for NVivo), though you DO have to enter a credit card to get an account. You can access the tool from within NVivo, on the Create tab:

Or from the website of your account, which is what I did. You just browse to your audio file and hit "upload". NVivo claims transcription occurs at 2x speed, so this should've only taken 20-seconds to transcribe. In my case it was just going and going, so I refreshed my browser after two minutes and it was all done, so that may be a little bug, or more likely something with my browser.

Cost of the service is on a per/hour basis, as follows:

I honestly have no idea if that's a fair price, but it certainly seems like it is. If you needed quality transcription I suspect you could pay a lot more. A major bonus is that this service covers 28 different languages! Feel free to try out one you know and let me know if the quality is still this high. Oh, I chose "English - North American" when I tried the Bush file.

Pretty nifty overall, I think.