A blog about things that interest me, politics, news, media, architecture, development, environment, local history, secularism, web, dublin ireland, tara

Contact me at expectationlost@gmail.com
Showing posts with label data. Show all posts
Showing posts with label data. Show all posts

Wednesday, 13 October 2010

Government aircraft travel of Irish ministers 2010 exhibited, mapped, timelined, listed and editable

This screen shot shows an exhibit of Irish Government Ministers travel on Government aircraft in 2010 using this data via Ken Foxe's FOI's

I made the 2008 and 2009 travel exhibits last year when Ken Foxe put up the Freedom of Information requests of the Ministers Air Travel it hasn't got many views although I haven't promoted it a lot, but it didn't get as much notice or use as I hoped,I thought it could be a prototype for visual data journalism in Ireland I'm not sure of displayed entirely correctly or usefully for people but the one thing that was incomplete is the purpose of the visits, its unfair to show the travel without context of why they travelled, I should have spent as much as much time searching for that information, it takes a lot of work to go find the reason for the trip, even when the St Patrick's day ministerial travel was put out it was only one line of information they gave out to news organisations to explain what the ministers were doing when the flew around the world for March 17th. I couldn't hope to find details of every last trip myself and left it to other people to help find it, but its been difficult to find a quick and easy way to allow people add info to the Exhibit on the web page itself as opposed the google doc.

Its pity there no way to batch search, so I could quickly find why Mary Harney was in Helsinki in 2009 and Michael Martin went to New York in July 2010, although google Timeline has been useful Brian Cowen New York

If there was a way to search for trip keywords and return the top and allow people 1 click to confirm the relevant result? To gamify data correction. This hasn't been possible with google docs, but maybe possible with google apps. The only place I've seen something like that is Freebase's, acre apps like matchmaker

They could drown you in information but they don't? Government communications is still stuck in the notion of giving info to the press rather then the public, or its hard to find on their websites, although merrionstreet.ie may now make it easier to find information on the government. I need regular engaging posts to get links that's hard to do with old information thats incomplete, I guess that's where the actual journalism of the Sunday Tribune's Ken Foxe's comes in. I could suggest that the amount of trips and therefore the costs has reduced significantly from 2008 to 2010, although Im missing the data from the last months of 2009 and 2010, so I can't compare definitively as yet.

The source
Ken Foxe of the Sunday Tribune has been requesting information (FOIs) about government minister's travel and particularity noticed the excessively costly travel arrangements of John O'Donoghue the Ceann Comhairle (also when he was Minister for Tourism). Because of his and others journalism and rather then justify the costs he resigned as chair of the Irish house of parliament (Dail) in October 2009.

The change
Ken Foxe in a very unusual move for a professional journalist has shared the raw information he got in his FOI's with the public. The information from the Ministerial Air Transport Service is voluminous so with more eyes and perhaps my exhibit of the information in a more visual and manipulatable manner we can comprehend more of what politicians are doing with government resources. He used FOI'd government aircraft travel data to write ...Living on our jet plane, at a cost of €10.6m (18 Oct 2009, Sunday Tribune).

TRANSPORTING government ministers aboard the government jet over the past four years has cost the taxpayer more than €10.6 million.

Figures from the Department of Defence show that the €7,890-an-hour Gulfstream IV jet has been in use for 1,214 hours since the beginning of 2006. The government's Learjet – the running costs of which now come in at almost €3,000 an hour – has been in the air for 826 hours at a cost of €1.89 million.

...continue reading at tribune.ie

Ken's FOI data
Using the spreadsheets he then re-published Ken Foxe's Gov aircraft use FOI's eg. Ministers Air Travel - 2010 Jan - Oct (Scribd doc).

Mats Stats 2010 Up to 30 Sept 2010 (2)

My Data
I rearranged it into this google doc file with labels so that the Exhibit script can read it as JSON. How to make an exhibit from data fed directly from a Google Spreadsheet


Use Define groupings and calculations to calculate personal totals
Used Batch geocode for the locations and Notepad ++ and jEdit some basic excel/google spreadsheet macros and functions. And converting dates to the right format.

Made with Exhibit from MIT which...

Exhibit enables you to create html pages with dynamic exhibits of data collections without resorting to complex database and server-side technologies. The collections can be searched and browsed using faceted browsing. Assorted views are provided including Timelines, tiles, maps, and more.

I modified the Presidents Births and Death example of which you can see it's simple JSON file. You can use MIT's Babel to convert your data to JSON or XML (create live XML with editgrid.com) or any google spreadsheets form. The exhibit
You can pick out any element or facet of the info from the list on the left, the central tag cloud, the timeline, map or Flight time slider below the map, deselect it by un-checking on the sidebar list or use your browser's back button. You can zoom into the map under Ireland or Europe or view the Details of the trips in a table.

Your help
I need to add the purpose of the trip and a contemporaneous news article to give them fair context. Anyone can help me to do that by going to the google spreadsheet and editing the 'purposeoftrip' or 'contexturl' and 'urllabel' columns for each row/trip. I also need fill in the unnamed Ministers of State in the person column.If there is a another way to arrange the data to make it more comprehendable by people please suggest it.

Added paths so you can see the routes was easier then i thought using werelate.org coder additions and added to my own, didn't even have to run it locally polylines extension of exhibit made by www.werelate.org code changes

To do

I'm also worked on a politicians constituency map and TD exhibit

Snout in the the Trough
Ken Foxe has written a book called Snout in the the Troughbased on his Tribune articles and previously unpublished research and FOI's. I bought it and am reading it by the index, I haven't read much of it yet but it seems to be mostly a exhaustive description of recent political expenses, it may be better used as index reference then to be read front to back. I disappointed he didn't go more behind the scenes into the civil service part in the carrying out the privileged operations of the government but I guess its up to the politicians to take the lead in use and reform, though I still think he could have gone into the operation in more detail on this. When I got the book I went straight to the issue of the london embassy car hire practice, politics.ie gets a mention in the book, or alteast via Gavin Sheridan..., who said posting a link to his thestory site and the detailed expenses document from his FOI work here helped raise the issue beyond discussion and into the national daily papers. Ken foxe wrote...

Tens of thousands of euros had been paid to a London-based chaffeur company Cartel Limousines. The costs seemed at times incredibly high often more then £1400 per day The owner of the firm was Terry Gallgher, the son of the late Fianna Fail Minister Denis Gallagher. The anonymous whispers of the internet forums screamed crony-ism but the reality was somewhat different. Terry Gallgher's firm had been doing business with the Irish Embassy in London for two decades during a series of governments, including Fine Gael led administrations.
disappointed with the view taken, he should know that establishment cronyism goes beyond party politics, the civil service will favour the establishment across parties and allow for such costly practices to go on for years untackled. The book spends most of first half detailing the exposure of John O'Donoghue's excessive spending and of the other ministers who also overspent but eventual resignation of the Ceann Chomhairle overshadowed the expenses of all members of the Dail and their resistance to reform and the resignation of Tom O'Higgins who was supposed to in charge of reform, the eventual reform when it came made things even more opaque

Thursday, 4 March 2010

Tableau Public, Visualisation Dashboard

Via readwriteweb a top notch blog about web tech, I read about Tableau Public Software Which provide you with free* interactive graphs, maps and dashboards for the web.

This video example about taking public data about graffiti in New York, graphing it, mapping it, finding trends in the data and then showing those trends on the Dashboard. If only we had access to this kind of data in Ireland.

Obligatory Haiti Dashboard Olympic Medals Dashboard

The free service which you download to you desktop gives you 100,000 rows of data, or 50mb which you can upload to its site to view and share.

This is the new manyeyes for me, particularly because it can do maps of Ireland and you can upload and manipulated large tables of data at once.

Im not sure of its geocoding will be useful for Ireland but can use the updgraded batchgeocode.com to get the lat lng and input them directly. Batchgeocode now specifically geocodes for and within Ireland.


Desktop program

You can use excel or csv files. You can make a graph, from one sheet a map from another, then put them together on dashboard, making the filters for them global to get a fuller nuanced picture.

I've tried a few here myself using the data from my TD's Exhibit. It nearly makes it my use of Exhibit defunct. It seems like it had more ways to total, average and display data automatically.

Data at TD google spreadsheet
Constituency editable spreadsheet.
Party editable spreadsheet.

Download these files as xls, strip the { and } from the headers and think of ways of displaying the data.

Need to think of ways to use data to fully utilize this. Figure out how to make the difference in size of icons logarithmic.

Sunday, 1 November 2009

Government aircraft travel of Irish ministers 2009 exhibited, mapped, timelined, listed and editable.

This screen shot shows an exhibit of Irish Government Ministers travel on Government aircraft in 2009 using this data using Ken Foxe's Freedom Of Information Requests.

2008 finished

Made with Exhibit from MIT which...

Exhibit enables you to create html pages with dynamic exhibits of data collections without resorting to complex database and server-side technologies. The collections can be searched and browsed using faceted browsing. Assorted views are provided including Timelines, tiles, maps, and more.
You can copy examples from the Exhibit wiki and simile mailing list.

                        ex:role="view"
                        ex:viewClass="Map"
                        ex:label="Birth Places with Photos"
                        ex:latlng=".birthLatLng"
                        ex:center="38.479394673276445, -115.361328125"
                        ex:zoom="3"
                        ex:bubbleWidth="200"
                        ex:icon=".imageURL"
                        ex:shapeWidth="60"
                        ex:shapeHeight="60"
                        ex:selectCoordinator="president"
                        

...calling the named columns from the google spreadsheets.

The source
Ken Foxe of the Sunday Tribune has been requesting information (FOIs) about government minister's travel and particularity noticed the excessively costly travel arrangements of John O'Donoghue the Ceann Comhairle (also when he was Minister for Tourism). Because of his and others journalism and rather then justify the costs he resigned as chair of the Irish house of parliament (Dail) in October 2009.

The change
Ken Foxe in a very unusual move for a professional journalist has shared the raw information he got in his FOI's with the public. The information from the Ministerial Air Transport Service is voluminous so with more eyes and perhaps my exhibit of the information in a more visual and manipulatable manner we can comprehend more of what politicians are doing with our government resources. He used FOI'd government aircraft travel data to write ...Living on our jet plane, at a cost of €10.6m (18 Oct 2009, Sunday Tribune).

TRANSPORTING government ministers aboard the government jet over the past four years has cost the taxpayer more than €10.6 million.

Figures from the Department of Defence show that the €7,890-an-hour Gulfstream IV jet has been in use for 1,214 hours since the beginning of 2006. The government's Learjet – the running costs of which now come in at almost €3,000 an hour – has been in the air for 826 hours at a cost of €1.89 million.

...continue reading at tribune.ie

MIT Exhibit
I modified the Presidents Births and Death example of which you can see it's simple JSON file. You can use MIT's Babel to convert your data to JSON or XML (create live XML with editgrid.com) or any exhibit friendly form.

Ken's FOI data
Using the spreadsheets he then re-published Ken Foxe's Gov aircraft use FOI's eg. Ministers Air Travel - 2009 Jan - Oct (Scribd doc).

Mats - 2009 Jan - Oct

My Data
I rearranged it into this google doc file with labels so that the Exhibit script can read it as JSON. How to make an exhibit from data fed directly from a Google Spreadsheet


Use Define groupings and calculations to calculate personal totals
Used Batch geocode for the locations and Notepad ++ and jEdit some basic excel/google spreadsheet macros and functions. And converting dates to the right format.

The exhibit
You can pick out any element or facet of the info from the list on the left, the central tag cloud, the timeline, map or Flight time slider below the map, deselect it by un-checking on the sidebar list or use your browser's back button. You can zoom into the map under Ireland or Europe or view the Details of the trips in a table.

Your help
I need to add the purpose of the trip and a contemporaneous news article to give them fair context. Anyone can help me to do that by going to the google spreadsheet and editing the 'purposeoftrip' or 'contexturl' and 'urllabel' columns for each row/trip. I also need fill in the unnamed Ministers of State in the person column.If there is a another way to arrange the data to make it more comprehendable by people please suggest it.

I need to display the helicopter trips better.Working on it here to add polylines extension of exhibit made by www.werelate.org code changes

Its pity there no way to batch search, so I could quickly find why Mary Harney was in Helsinki and Michael maritin went to New York in July.

Want to do 2008
2008 MATS spreadsheet FOI'd by Ken Foxe
via scribd.
JSON base file
MATS 2008 Exhibit

To do
I need to figure out functions to add up totals...

I'm also working on a politicians constituency map and TD/Dail diary similar to the above.

The Story
A couple of young (ish) journalists have set up thestory.ie to gather and release data about Irish politics for crowd sourcing and crowd reviewing. Gavinblog and Mark Coughlan have, as have I, been inspired by the work of projects like the UK's mysociety.org in opening up shared data resources for citizen journalism. They are working towards creating an Irish version of whatdotheyknow.com, which makes it easier to request and share 'Freedom of Information' requests. Another MySociety project created a more accessible and manipulatable parliamentary record called Theyworkforyou, programmer John Handalaar has already created a version called kildarestreet.com for the Irish parliamentary debates.

The visualisations
I have taken some of thestory's spreadsheets of data and made visualisations (I'll describe how in another blog post).

A visualisation of donations to parties by-industry using Tree Map's via Manyeyes and Google Docs. Visualised Liam Carroll's Corporate Structure, a more dynamic ZVTM version of LC companies
Minister for Finance for Ireland Brian Lenihan's Diary during the financial crisis, in google calendar form, in SIMILE WIDGET timeline form.

APIs, feeds and linked data
I had been using some of the above methods to follow the Howth Malahide council elections and hope improve with use of RDF and XML and geocoded feeds. With free collaboration and document sharing sites, RDF and XML and feeds and API's its becoming easier for non-programmers to serve, slice and link data.

Links
Another Irish Journalist Mark Tighe is releasing data for his Sunday times articles
Guardian Open Platform.
Batch geocoding addresses.
Sunlight Labs the tech development of open government US.
The sunlight centre for open politics UK.
Zoho: Document collaboration site To do
The coder behind werelate.org has created some extensions exhibit pedi maps to make connection icons on the map. I think you havve to put the whole system on your own host, which when I manage to add it I can use to connect flight.

Thursday, 17 July 2008

Radiohead 3D LIDAR music video

Radiohead have used live Lidar laser scanning and recording to make a video for the new single House of Cards watch the music video the making of and then wait for the load up of the 3d video google gadget. 3d DataWhich you can download and manipulate yourself if you know how.

Mind blowing.

Tuesday, 10 June 2008

Many Eyes Visualisations of Ireland

I've been looking for ways to display Ireland election or referendum results dynamically live interested in visualisations and mapping of info and web 2.0 here are the two combined, Many Eyes an Visual Communication Lab IBM research project.

Anybody can upload data sets from various sources and use the visualisation models to visualise the date in various ways from graph to word clouds to heat maps.

Somebody called Colm has upload and visualised many interesting data about population, houses, politics and GAA.

The Shifting of Ireland's Population to Dublin

A Data Set of Population by County

Compare Bertie election 2007 acceptance speech and his resignation speech a year later (switch between 1 or 2 words on the Many Eyes sites).

Its a pity it doesn't have Irish electoral constituencies of other administrative districts if any has the shapefiles for them please add them.

Google had visualisation API for heat maps now to but only maps and recognises individual countries or administrative areas at the moment as far as I can see.

Im going to see if i find or add Ireland administrative boundary geonames.