Live Tweeting the Civil War + 150

One of the projects we’re beginning to embark upon is the “live” tweeting of the Civil War, 150 years after the fact, utilizing the battles and skirmishes outlined in Dyer’s Compendium. This is inspired by Ed Summers’ work to create a bot that tweets headlines from and links to 100 year old newspapers in the Library of Congress Chronicling America newspaper collection (his source code is available here). I’m still figuring out a few details with the help of Ed and others, but the following (a work in progress!) is what I’m considering for sources and tools.  As we work it out, we’ll be sure to provide a how-to in case this is something you’d like to use for state Civil War Sesquicentennial commemorations.

1. Primary source: Dyer’s Compendium.  Surprisingly, I haven’t found a fully digitized copy of this online–maybe it’s still under copyright, or not many copies of the original 1908 version exist to be digitized?  Got one you’d like to lend? update: in 2011, Emory University stepped up and put the 1908 version up on Internet Archive and in the Public Domain.

2. Structured data: Tufts University’s Perseus Project has made the text of Dyer’s Compendium available as structured data, downloadable in XML. Problem: It’s not Open Data, but should work for our purposes. Would love to see that issued as Open Data though!

3. XML to CSV conversion.  We need to get this XML translated to CSV so we can put it into a relational database and break down the selected events by date.

4. Choose the level of detail we want to highlight.  There’s a broad range of events listed, including skirmishes, occupations, engagements, battles, etc.  We’ll need to figure out how many events we want to broadcast.

4. Python script to automatically create tweets on the appropriate day, utilizing Twitter APIs.

5. I’d like to add a link to each tweet, directing readers to more information about the battle, but where should we point them to?  The National Park Service, Wikipedia, Freebase?