The Distant Librarian

First run on autoEdit 2, open source transcription tool

Having had this tab opened for what seems like forever, I finally got around to running some tests of autoEdit 2, an open-source automatic transcription tool (automatically creates a text file from a given audio or video file). This tool only runs on the Mac, and there are two flavours, if you will. One, Gentle STT, runs completely on your local machine and costs nothing to do transcriptions, but it's not the best at it. If you have very clean audio you can probably get by with this, but I found it pretty sketchy. The second makes use of IBM Watson's Bluemix service, and for me was both faster and much more accurate. With Bluemix you get 16 hours of transcription free per month, but anything more than that will cost $0.02/minute, which is still pretty darn reasonable, IMHO.

One of the things that slowed me down was a lack of troubleshooting documentation. The software itself is pretty straightforward and easy to use, but the first step of plugging in your Bluemix API key is a bit sparse. I put in the wrong credentials, and autoEdit took them without complaint, and pretended to work on the transcription for a long time before I decided it wasn't really doing anything. I couldn't find anywhere to re-enter credentials, so I ended up uninstalling and reinstalling the program.

I'll see if I can contribute to the documentation, but for now, once you've got your free Bluemix account, you want to scroll to the bottom right of the main account page and click on the Create button.

On the next screen, choose the 2nd "tab", "Service Credentials", and then View Credentials. That's where you'll find your API username and password. Be sure to enter them in the correct order within autoEdit; the username, at least in my case, was much longer, and is presented 2nd on the Bluemix page :-/

That's all from me. It does work quite well, and if you need to create a written transcript from a video or audio file, I think this is a pretty darn good way to go!

Before and After - Calgary and Ft. McMurray

The other day Google Earth released updated historical imagery, so I thought it'd be neat to take a peek at two spots I know have grown a lot since 1985, the date of the earliest imagery.

I also wanted a quick and dirty way to make a before/after image slider, as shown on the above post, so I found JustaposeJS, which does just the trick! There are other options if you do want to get fancy. 

So here's Calgary:

And here's Ft. McMurray:

From the original post, "Note that the new data is created by blending all Landsat/Sentinel 2 data for a whole year to remove clouds and snow cover. The result is that changes that happen on timescales less than a year, such as seasonal changes, will not be visible."

I could look at stuff like this all day!

Related articles

Google Timelapse shows how any location on Earth has changed over the past 32 years

Google Earth VR Lets You Take a Virtual Vacation

Google Earth brings its breathtaking views into VR

Grab Your Red Cape and Fly Around the World With Google Earth VR

Mapping Western Economic Diversification Grants

A few weeks ago two new data sets appeared on the Open Data Canada website, WD - Grants and Contributions over $25,000 and Project Geo-Information, which "provides geographical information on projects issued by or on behalf of Western Economic Diversification Canada." Not particularly well-named data sets, but that's another issue. I thought it would be a fun exercise to combine the two, since the geo information is kinda worthless without the actual project data to go with it. 

First I imported both files into Google Sheets to see if I could merge the data sets there. I found an add-in called Merge Sheets that should've done it, but perhaps the files were too large, so no dice. Knock yourself out if you want to try (note the two sheet tabs at the bottom).

So I ended up using VLOOKUP in Excel to combine the two files. I always have a hard time wrapping my head around VLOOKUP, but I got it done. Had to save the result as a .csv in order to import to other tools; if saved as .xlsx there are errors as the formula continues to reference a separate sheet. Maybe if I'd worked with them together rather than as separate files. Anyhoo, the .csv worked when I then brought that new file into Google Fusion Tables for mapping fun.

And the end result is:

Seeing it on a map immediately makes me wonder why some big money is being spent in Ontario. I could also immediately saw one record had an incorrect lat/lon assigned to it, putting a BC project in ON, but I fixed that, so it doesn't appear on the above map.

Next up, if I make the time, is to assign a different icon to different types of projects, or perhaps different sizes / colours for different grant amounts. And explore what other online mapping tools might make this easy to work with now that the full data set exists.

4 Easy & Secure Ways to Record Screen Videos on iPhone or iPad: A Comparative Guide

I received an email from JP letting me know that a link on an old post of mine, The definitive guide to iOS screencasting, was no longer good.  He knew that 'cause he'd just finished writing a much more comprehensive post called 4 Easy & Secure Ways to Record Screen Videos on iPhone or iPad: A Comparative Guide. Interestingly, no new tools in the last 4 years - guess the ones that are out there are good enough! 

Thanks for the note JP.

Anatomy of an Information Trap, Part II: Setting Up and Sharing Google Alerts

Tara Calishain at ResearchBuzz has posted part 2 of her 3 part series on creating and working with information traps, Setting up and sharing Google alerts. In this post I learned how to automatically create and populate columns in a Google Sheet using IFTTT. Off the top of my head I can think of at least two projects where this is going to immediately come in handy!

I've said it before, and I'll say it again, Tara's work is worth supporting financially! C'mon, go create an account at Patreon and become part of the sharing economy - it really will give you the warm fuzzies. :-0

12 Screencasting tools that were all new to me

Admittedly I haven't spent lots of time staying up-to-date on screencasting tools, but I have to admit I was surprised that a recent list on MakeUsOf, Record Your Desktop with These 12 Great Screencasting Apps, contained all new stuff to me. Take a look, maybe you'll find something useful there! (OK, obviously I know about PPT, but didn't realize the extent to which it can do screen recording...)

Even some tools for Linux!

 

13 Tips for Recording Your iPhone, iPad, or iPod

In the most recent TechSmith newsletter was a link to recent post on their site titled, 13 Tips for Recording Your iPhone, iPad, or iPod. In addition to the useful information, it was nice to see the author using screencasts of iOS devices to illustrate what was being discussed :-)

to wit:

Static vs. Dynamic Tutorials: Applying Usability Principles to Evaluate Online Point-of-Need Instruction

Turner, B., Fuchs, C., & Todman, A. (2015). Static vs. Dynamic Tutorials: Applying Usability Principles to Evaluate Online Point-of-Need Instruction. Information Technology & Libraries, 34(4), 30-54. doi:10.6017/ital.v34i4.5831

This article finds "that web usability testing was a useful tutorial-testing tool while discovering that participants learned most effectively from text-and-image tutorials, since both rounds of participants completed tasks more accurately and more quickly than those who received audio/video instruction or no instruction." Helpfully, there are links to the tutorials used in the appendices of the paper, so you can unscientifically see if you agree with the findings too! 

Open Refine - not the right tool for this job (twitter archiving and geolocation)

A week or so ago I came across a series of posts from Spencer Greenhalgh in which he described how he used R to take a large group of collected tweets around the terrorist attacks in Paris in order to geolocate them on a map. The trouble is that very few users actually include geolocation information in their tweets, so Spencer had to figure out a way to grab an approximation of that information by mining each user's twitter profile page for his/her stated home location. This wouldn't indicate where the users were when they actually tweeted, but it gave him a good idea where the people usually were when they tweeted (if the profile page could be trusted), which was good enough for his experiments.

I've been playing with Open Refine of late, so thought I'd take a stab at this same experiment with that hammer. I figured out all the pieces, but was unsuccessful in the end, and I can't tell you exactly why. I got Spencer's archive of tweets, and was easily able to import it into Open Refine. The problem came when I started to pull in the user profile pages - no matter how I sliced and diced the archive I would always time out on some section or another. The full data set contains just over 6,000 lines, so I tried breaking that in half by date, into smaller pieces by client used to send the tweet, and by unique vs duplicate users. I was able to get a full scrape of the user profile pages for about 600 users, but every other group I tried would simply time out (I tried all sorts of variations on the throttle delay, from 250 - 10,000 milliseconds, but my scrape would always hang, depending on the set I was trying, either around 67% or 89%. Maybe it was a memory issue? Because it simply hung, I never got an error message to work with. If you have an idea what might've been the problem I'd LOVE to know!

The GREL expression I did eventually use to dig out the user location from those I could get was as follows:

value.parseHtml().select("span.ProfileHeaderCard-locationText")[0].toString()

That searches through the scraped profile page HTML for the CSS class around the user-stated location and just returns that bit of info, which would look like this:

<span class="ProfileHeaderCard-locationText u-dir" dir="ltr"> Atlanta, GA </span>

It did take me a while to figure this bit out, so for future reference I found this page most useful.

So now we've got a column of messy data, which I cleaned with the following expression:

replace(value,/<\/?\w+((\s+\w+(\s*=\s*(?:".*?"|'.*?'|[^'">\s]+))?)+\s*|\s*)\/?>/,'')

I found that one at https://github.com/OpenRefine/OpenRefine/wiki/StrippingHTML and once run it leaves behind the nice clean string of text we're after:

Atlanta, GA

And from there it would be trivial to run that column through any number of free online mapping tools in order to generate a map of where users say they're from.

So the biggest headache was getting all the HTML from the user profile page. JSON would've been MUCH cleaner, but like Spencer I wanted to try to do this without using the Twitter API, so AFAIK I was stuck with HTML.

Fast forward to today, when I see Nick Ruest's post, A look at 14,939,154 #paris #Bataclan #parisattacks #porteouverte tweets. I was having trouble with 6,000 tweets, and Nick was churning through almost 15 million tweets with a far more appropriate tool called twarc.

twarc is a command line tool and Python library for archiving Twitter JSON data. Each tweet is represented as a JSON object that is exactly what was returned from the Twitter API. Tweets are stored as line-oriented JSON. It twarc runs in three modes: search, filter stream and hydrate. When running in each mode twarc will stop and resume activity in order to work within the Twitter API's rate limits.

I had tried to work with twarc  a few months ago, but couldn't get it running on the shared server I was on, but after seeing just what it's capable of I will get it up and running, hog frog or dog!

My takeaway from this is that while there may be many different ways to solve a problem, they're certainly not all equal. Spencer stuck with what he knew and got something usable out of it. I tried with what I knew and failed, and Nick used what he knew and kicked both our butts. Spencer, do look at twarc too! :-)

Upward and onward!

Article of interest: A Survey of Online Library Tutorials: Guiding Instructional Video Creation to Use in Flipped Classrooms

This article showed up in the most recent issue of JAL: A Survey of Online Library Tutorials: Guiding Instructional Video Creation to Use in Flipped Classrooms, The Journal of Academic Librarianship, Volume 41, Issue 6, November 2015, Pages 751-757.  It seems to be a good snapshot of the current use of screencasts in ARL and CARL libraries.

Abstract

There has been a steady increase in library literature on “flipping the classroom.” This teaching strategy requires students to review course material outside the classroom beforehand, thereby allowing more time during workshops to apply newly gained knowledge and techniques. The proliferation of literature on classroom flipping provoked an interest to determine if other academic libraries were making explicit reference on their websites to the preliminary viewing of videos in a classroom context. To ascertain the extent of this practice, and inform the development of instructional videos at McGill, the authors surveyed all Canadian Association of Research Libraries (CARL) and Association of Research Libraries (ARL) websites to document the production of instructional videos, determine the various types of videos, and take note of explicit pre-viewing instructions prior to a workshop. Of the 140 libraries examined, 107 (76%) provide instructional videos on their website. However, of this group, only 2 (2%) clearly instruct users to watch instructional videos before attending a library research workshop. A literature review documents this emerging trend and contextualizes the results.

And a nifty graphic to whet your appetite:

Fun with OpenDataSoft - Mapping Calgary City Amenities

Earlier this month I happened across a blog post from a company called OpenDataSoft (ODS) in which they described how they put together and mapped a list of over 1,600 Open Data portals around the world. I thought that was pretty cool, and did a little exploration of their web-based platform and decided to try my hand at some data enrichment and publishing. I trolled the City of Calgary Open Data website for something I thought might work well and settled upon their List of City Amenities (dog parks, EMS stations, arenas and such). While they offer all the data as a .csv file, the geographic / mapping file was completely separate, and I wanted to try to put it all together into one nifty application. 

For my purposes, the .csv provided by the city was juuuust short; it didn't include any postal codes with the addresses, so I couldn't automatically generate a map with ODS. I ended up throwing the addresses at http://geocoder.ca/ which allowed me to grab the postal codes. Then I realized that at this time, ODS only maps postal codes in France (they're based in Paris). SO I grabbed an API key from Google Maps (linked to from within ODS) and THEN I was able to generate the desired map. There were a few outliers that I had to manually correct, but here's the result. Note the filters on the left - that was added with an option with ODS, and then there's the separate map tab, which again is an enhancement over what the City provided. I did take a look at the City's .kml file in Google Earth - it wasn't very useful, or accurate, IMHO.

There are a number of share/embed options, including the ability to download / share only filtered results. Here's a map of Calgary's Outdoor Pools (we're not a big outdoor pool city, as you might imagine):

Anyhoo, do take a look at OpenDataSoft - all this was done for free, though if you're planning to work with lots of data sets you'll have to pay, I think.

How to Extract Data from Tables in PDFs with Tabula and OpenRefine

Rahul Saigal has an excellent step-by-step post on how to extract data from tables in PDFS with Tabula and OpenRefine. I've had great success with Tabula, and am really starting to get excited about OpenRefine as well, having just completed this free online course, which I highly recommend: Introduction to OpenRefine.

From the introduction to Rahul's post:

While you could manually extract data from a table in a PDF, it's far from simple. There's no reason to take that much time when there are tools to make our job simpler. In this tutorial, I’ll show you how to extract table from the PDF easily with Tabula, and then clean the messy data to a more usable form with OpenRefine.

Have fun!

Screencasting iOS

A few days ago TechSmith released a new Labs tool called AppShow which is intended to allow developers to record demos of their iOS apps to include in the App Store. Just about the same time, Amit at Digital Inspiration posted how to do the same thing using QuickTime player. I finally carved out a few minutes to see how this all worked.

AppShow provides quite a few options to guide the user in creating an intro and various scenes; I found them to be overkill for what I was trying to do, but someone really trying to make something polished for the App Store might find these options useful.

First a note that you must have the latest version of OSX installed, Yosemite. Second, you can edit recordings made via either method via Camtasia Studio, or probably any other video editing software, but other than adding a quick title slide, I didn't do any other editing or processing using CS. I also chose to use a game instead of a library-specific app because I wanted to see how well the recordings did with motion and audio. The recordings were done with an iPhone5 and a mid-2009 MacBook Pro.

And finally, it took me quite some time to figure out how to export from AppShow to Camtasia Studio; couldn't find anything in the documentation or the File menu item, so here's your hint:

Here are four videos I posted to YouTube so you can compare the quality.

tl;dr Quality of all is quite good, and for quick and dirty QuickTime Player is the way to go - throw the recording into Camtasia Studio if you need to polish it up after the fact.

QuickTime (Title added via Camtasia Studio)

 

QuickTime (Direct export):

 

AppShow (Title added via Camtasia Studio)

 

AppShow (Direct export):

 

Free Web-based Tools for Information Literacy Instruction

The latest issue of Library Hi Tech News has the following article, which contains a pretty good list of free and freemium tools that may be new to you as many were to me.

Christine Palma Forbes , (2014),"Free Web-based Tools for Information Literacy Instruction", Library Hi Tech News, Vol. 31 Iss 10

OS X Yosemite Makes it Easy to Screencast iOS Apps

Here's a nifty feature to look forward to in the fall - the ability to record screencasts of your iOS apps in order to incorporate them in to tutorials, etc...

Once you connect an iOS 8 device to your Mac running OS X 10.10, it will be automatically available as a video camera. You can then capture anything you’re doing on-screen directly to your Mac using QuickTime Player.

Source: iPhone in Canada