A blog about things that interest me, politics, news, media, architecture, development, environment, local history, secularism, web, dublin ireland, tara

Contact me at expectationlost@gmail.com
Showing posts with label google map. Show all posts
Showing posts with label google map. Show all posts

Wednesday, 13 October 2010

Government aircraft travel of Irish ministers 2010 exhibited, mapped, timelined, listed and editable

This screen shot shows an exhibit of Irish Government Ministers travel on Government aircraft in 2010 using this data via Ken Foxe's FOI's

I made the 2008 and 2009 travel exhibits last year when Ken Foxe put up the Freedom of Information requests of the Ministers Air Travel it hasn't got many views although I haven't promoted it a lot, but it didn't get as much notice or use as I hoped,I thought it could be a prototype for visual data journalism in Ireland I'm not sure of displayed entirely correctly or usefully for people but the one thing that was incomplete is the purpose of the visits, its unfair to show the travel without context of why they travelled, I should have spent as much as much time searching for that information, it takes a lot of work to go find the reason for the trip, even when the St Patrick's day ministerial travel was put out it was only one line of information they gave out to news organisations to explain what the ministers were doing when the flew around the world for March 17th. I couldn't hope to find details of every last trip myself and left it to other people to help find it, but its been difficult to find a quick and easy way to allow people add info to the Exhibit on the web page itself as opposed the google doc.

Its pity there no way to batch search, so I could quickly find why Mary Harney was in Helsinki in 2009 and Michael Martin went to New York in July 2010, although google Timeline has been useful Brian Cowen New York

If there was a way to search for trip keywords and return the top and allow people 1 click to confirm the relevant result? To gamify data correction. This hasn't been possible with google docs, but maybe possible with google apps. The only place I've seen something like that is Freebase's, acre apps like matchmaker

They could drown you in information but they don't? Government communications is still stuck in the notion of giving info to the press rather then the public, or its hard to find on their websites, although merrionstreet.ie may now make it easier to find information on the government. I need regular engaging posts to get links that's hard to do with old information thats incomplete, I guess that's where the actual journalism of the Sunday Tribune's Ken Foxe's comes in. I could suggest that the amount of trips and therefore the costs has reduced significantly from 2008 to 2010, although Im missing the data from the last months of 2009 and 2010, so I can't compare definitively as yet.

The source
Ken Foxe of the Sunday Tribune has been requesting information (FOIs) about government minister's travel and particularity noticed the excessively costly travel arrangements of John O'Donoghue the Ceann Comhairle (also when he was Minister for Tourism). Because of his and others journalism and rather then justify the costs he resigned as chair of the Irish house of parliament (Dail) in October 2009.

The change
Ken Foxe in a very unusual move for a professional journalist has shared the raw information he got in his FOI's with the public. The information from the Ministerial Air Transport Service is voluminous so with more eyes and perhaps my exhibit of the information in a more visual and manipulatable manner we can comprehend more of what politicians are doing with government resources. He used FOI'd government aircraft travel data to write ...Living on our jet plane, at a cost of €10.6m (18 Oct 2009, Sunday Tribune).

TRANSPORTING government ministers aboard the government jet over the past four years has cost the taxpayer more than €10.6 million.

Figures from the Department of Defence show that the €7,890-an-hour Gulfstream IV jet has been in use for 1,214 hours since the beginning of 2006. The government's Learjet – the running costs of which now come in at almost €3,000 an hour – has been in the air for 826 hours at a cost of €1.89 million.

...continue reading at tribune.ie

Ken's FOI data
Using the spreadsheets he then re-published Ken Foxe's Gov aircraft use FOI's eg. Ministers Air Travel - 2010 Jan - Oct (Scribd doc).

Mats Stats 2010 Up to 30 Sept 2010 (2)

My Data
I rearranged it into this google doc file with labels so that the Exhibit script can read it as JSON. How to make an exhibit from data fed directly from a Google Spreadsheet


Use Define groupings and calculations to calculate personal totals
Used Batch geocode for the locations and Notepad ++ and jEdit some basic excel/google spreadsheet macros and functions. And converting dates to the right format.

Made with Exhibit from MIT which...

Exhibit enables you to create html pages with dynamic exhibits of data collections without resorting to complex database and server-side technologies. The collections can be searched and browsed using faceted browsing. Assorted views are provided including Timelines, tiles, maps, and more.

I modified the Presidents Births and Death example of which you can see it's simple JSON file. You can use MIT's Babel to convert your data to JSON or XML (create live XML with editgrid.com) or any google spreadsheets form. The exhibit
You can pick out any element or facet of the info from the list on the left, the central tag cloud, the timeline, map or Flight time slider below the map, deselect it by un-checking on the sidebar list or use your browser's back button. You can zoom into the map under Ireland or Europe or view the Details of the trips in a table.

Your help
I need to add the purpose of the trip and a contemporaneous news article to give them fair context. Anyone can help me to do that by going to the google spreadsheet and editing the 'purposeoftrip' or 'contexturl' and 'urllabel' columns for each row/trip. I also need fill in the unnamed Ministers of State in the person column.If there is a another way to arrange the data to make it more comprehendable by people please suggest it.

Added paths so you can see the routes was easier then i thought using werelate.org coder additions and added to my own, didn't even have to run it locally polylines extension of exhibit made by www.werelate.org code changes

To do

I'm also worked on a politicians constituency map and TD exhibit

Snout in the the Trough
Ken Foxe has written a book called Snout in the the Troughbased on his Tribune articles and previously unpublished research and FOI's. I bought it and am reading it by the index, I haven't read much of it yet but it seems to be mostly a exhaustive description of recent political expenses, it may be better used as index reference then to be read front to back. I disappointed he didn't go more behind the scenes into the civil service part in the carrying out the privileged operations of the government but I guess its up to the politicians to take the lead in use and reform, though I still think he could have gone into the operation in more detail on this. When I got the book I went straight to the issue of the london embassy car hire practice, politics.ie gets a mention in the book, or alteast via Gavin Sheridan..., who said posting a link to his thestory site and the detailed expenses document from his FOI work here helped raise the issue beyond discussion and into the national daily papers. Ken foxe wrote...

Tens of thousands of euros had been paid to a London-based chaffeur company Cartel Limousines. The costs seemed at times incredibly high often more then £1400 per day The owner of the firm was Terry Gallgher, the son of the late Fianna Fail Minister Denis Gallagher. The anonymous whispers of the internet forums screamed crony-ism but the reality was somewhat different. Terry Gallgher's firm had been doing business with the Irish Embassy in London for two decades during a series of governments, including Fine Gael led administrations.
disappointed with the view taken, he should know that establishment cronyism goes beyond party politics, the civil service will favour the establishment across parties and allow for such costly practices to go on for years untackled. The book spends most of first half detailing the exposure of John O'Donoghue's excessive spending and of the other ministers who also overspent but eventual resignation of the Ceann Chomhairle overshadowed the expenses of all members of the Dail and their resistance to reform and the resignation of Tom O'Higgins who was supposed to in charge of reform, the eventual reform when it came made things even more opaque

Thursday, 16 September 2010

Irish openheatmaps

There many ways to make thematic maps openheatmap just made it even easier, as Pete Warden says he wants to make the "YouTube for maps", its been available since July.

I prodded him into adding up Irish geo-political boundaries Choropleth maps templates.

I searched for Irish boundaries and thanks to by Ben Raue for the KML 2007 constituencies, Councils from the CSO (councils to be added to openheatmap soon) and Counties (KML).

Its a JQuery plugin using either Flash or HTML5’s Canvas overlayed on openstreetmap. Its open source and free to copy and run the most of the setup off your own site.

For Ireland use these as the name columns...

ire_constituency (2010 redrawn constituencies)
ire_constituency_2007
ire_constituency_2007_code
ire_constituency_code
ire_county
ire_county_code

Councils to come...

and then the value column.
or tab_FF, tab_FG etc. for the each map tab.

You can use .csv,(comma separated files), saved from your spreadsheet program or load them from a public a Google Docs spreadsheet.

Upload your map, it will make your aware of any problems, and once you have those fixed, you can preview the map and changed the colours and range settings for you data and then share the link to the map or copy the embed code for your own site.

Pete Warden has made a helpful video to explain it all. The full detailed documentation is here with the source at github/openheatmap

I saw this UK election results tabbed maps example and I used Openheatmap to make an Irish version.

I used the 2007 Election results summary (pdf). You'll find more election results to try at environ.ie Typically for the Irish government it comes in messed up tables in pdf instead of a spreadsheet file. Google Doc version

Time by heat maps has been available from the site, by adding a time column, and you can use an address column to automatically place addresses on heatmap, these have been available for a while but its hard to find good data to use for Ireland, geocoding still presumes addresses are in American so I suggest using Batchgeo, which you can restrict results to Ireland and review and correct and hand place some of the erroneous results. Most info is done by constituency, county or council so this should make it easier to to find data to apply. Animated time by heat map example

Lisbon Openheatmap
Lisbon results google doc
Lisbon II 2009 results

Sunday, 5 September 2010

Dublin Bay Coastline Google Timeline Map

Click through to google earth plugin view
One of many half done things, should I finish it give it labels etc.


2013 update
spreadsheet of sources for dublin bay coastline
looking at this again to see if i can find a tool that put this much info on screen.
things that don't quite work Bridges time line

ver2
Dublin Bay coast polygon timeline


using qgis+time manager plugin and mapstory http://mapstory.org/maps/507/ http://mapstory.org/maps/508/ unfortunatly neither display info before 1900!...?



Dublin Bridges over Liffey by year

Thursday, 4 March 2010

Tableau Public, Visualisation Dashboard

Via readwriteweb a top notch blog about web tech, I read about Tableau Public Software Which provide you with free* interactive graphs, maps and dashboards for the web.

This video example about taking public data about graffiti in New York, graphing it, mapping it, finding trends in the data and then showing those trends on the Dashboard. If only we had access to this kind of data in Ireland.

Obligatory Haiti Dashboard Olympic Medals Dashboard

The free service which you download to you desktop gives you 100,000 rows of data, or 50mb which you can upload to its site to view and share.

This is the new manyeyes for me, particularly because it can do maps of Ireland and you can upload and manipulated large tables of data at once.

Im not sure of its geocoding will be useful for Ireland but can use the updgraded batchgeocode.com to get the lat lng and input them directly. Batchgeocode now specifically geocodes for and within Ireland.


Desktop program

You can use excel or csv files. You can make a graph, from one sheet a map from another, then put them together on dashboard, making the filters for them global to get a fuller nuanced picture.

I've tried a few here myself using the data from my TD's Exhibit. It nearly makes it my use of Exhibit defunct. It seems like it had more ways to total, average and display data automatically.

Data at TD google spreadsheet
Constituency editable spreadsheet.
Party editable spreadsheet.

Download these files as xls, strip the { and } from the headers and think of ways of displaying the data.

Need to think of ways to use data to fully utilize this. Figure out how to make the difference in size of icons logarithmic.

Sunday, 1 November 2009

Government aircraft travel of Irish ministers 2009 exhibited, mapped, timelined, listed and editable.

This screen shot shows an exhibit of Irish Government Ministers travel on Government aircraft in 2009 using this data using Ken Foxe's Freedom Of Information Requests.

2008 finished

Made with Exhibit from MIT which...

Exhibit enables you to create html pages with dynamic exhibits of data collections without resorting to complex database and server-side technologies. The collections can be searched and browsed using faceted browsing. Assorted views are provided including Timelines, tiles, maps, and more.
You can copy examples from the Exhibit wiki and simile mailing list.

                        ex:role="view"
                        ex:viewClass="Map"
                        ex:label="Birth Places with Photos"
                        ex:latlng=".birthLatLng"
                        ex:center="38.479394673276445, -115.361328125"
                        ex:zoom="3"
                        ex:bubbleWidth="200"
                        ex:icon=".imageURL"
                        ex:shapeWidth="60"
                        ex:shapeHeight="60"
                        ex:selectCoordinator="president"
                        

...calling the named columns from the google spreadsheets.

The source
Ken Foxe of the Sunday Tribune has been requesting information (FOIs) about government minister's travel and particularity noticed the excessively costly travel arrangements of John O'Donoghue the Ceann Comhairle (also when he was Minister for Tourism). Because of his and others journalism and rather then justify the costs he resigned as chair of the Irish house of parliament (Dail) in October 2009.

The change
Ken Foxe in a very unusual move for a professional journalist has shared the raw information he got in his FOI's with the public. The information from the Ministerial Air Transport Service is voluminous so with more eyes and perhaps my exhibit of the information in a more visual and manipulatable manner we can comprehend more of what politicians are doing with our government resources. He used FOI'd government aircraft travel data to write ...Living on our jet plane, at a cost of €10.6m (18 Oct 2009, Sunday Tribune).

TRANSPORTING government ministers aboard the government jet over the past four years has cost the taxpayer more than €10.6 million.

Figures from the Department of Defence show that the €7,890-an-hour Gulfstream IV jet has been in use for 1,214 hours since the beginning of 2006. The government's Learjet – the running costs of which now come in at almost €3,000 an hour – has been in the air for 826 hours at a cost of €1.89 million.

...continue reading at tribune.ie

MIT Exhibit
I modified the Presidents Births and Death example of which you can see it's simple JSON file. You can use MIT's Babel to convert your data to JSON or XML (create live XML with editgrid.com) or any exhibit friendly form.

Ken's FOI data
Using the spreadsheets he then re-published Ken Foxe's Gov aircraft use FOI's eg. Ministers Air Travel - 2009 Jan - Oct (Scribd doc).

Mats - 2009 Jan - Oct

My Data
I rearranged it into this google doc file with labels so that the Exhibit script can read it as JSON. How to make an exhibit from data fed directly from a Google Spreadsheet


Use Define groupings and calculations to calculate personal totals
Used Batch geocode for the locations and Notepad ++ and jEdit some basic excel/google spreadsheet macros and functions. And converting dates to the right format.

The exhibit
You can pick out any element or facet of the info from the list on the left, the central tag cloud, the timeline, map or Flight time slider below the map, deselect it by un-checking on the sidebar list or use your browser's back button. You can zoom into the map under Ireland or Europe or view the Details of the trips in a table.

Your help
I need to add the purpose of the trip and a contemporaneous news article to give them fair context. Anyone can help me to do that by going to the google spreadsheet and editing the 'purposeoftrip' or 'contexturl' and 'urllabel' columns for each row/trip. I also need fill in the unnamed Ministers of State in the person column.If there is a another way to arrange the data to make it more comprehendable by people please suggest it.

I need to display the helicopter trips better.Working on it here to add polylines extension of exhibit made by www.werelate.org code changes

Its pity there no way to batch search, so I could quickly find why Mary Harney was in Helsinki and Michael maritin went to New York in July.

Want to do 2008
2008 MATS spreadsheet FOI'd by Ken Foxe
via scribd.
JSON base file
MATS 2008 Exhibit

To do
I need to figure out functions to add up totals...

I'm also working on a politicians constituency map and TD/Dail diary similar to the above.

The Story
A couple of young (ish) journalists have set up thestory.ie to gather and release data about Irish politics for crowd sourcing and crowd reviewing. Gavinblog and Mark Coughlan have, as have I, been inspired by the work of projects like the UK's mysociety.org in opening up shared data resources for citizen journalism. They are working towards creating an Irish version of whatdotheyknow.com, which makes it easier to request and share 'Freedom of Information' requests. Another MySociety project created a more accessible and manipulatable parliamentary record called Theyworkforyou, programmer John Handalaar has already created a version called kildarestreet.com for the Irish parliamentary debates.

The visualisations
I have taken some of thestory's spreadsheets of data and made visualisations (I'll describe how in another blog post).

A visualisation of donations to parties by-industry using Tree Map's via Manyeyes and Google Docs. Visualised Liam Carroll's Corporate Structure, a more dynamic ZVTM version of LC companies
Minister for Finance for Ireland Brian Lenihan's Diary during the financial crisis, in google calendar form, in SIMILE WIDGET timeline form.

APIs, feeds and linked data
I had been using some of the above methods to follow the Howth Malahide council elections and hope improve with use of RDF and XML and geocoded feeds. With free collaboration and document sharing sites, RDF and XML and feeds and API's its becoming easier for non-programmers to serve, slice and link data.

Links
Another Irish Journalist Mark Tighe is releasing data for his Sunday times articles
Guardian Open Platform.
Batch geocoding addresses.
Sunlight Labs the tech development of open government US.
The sunlight centre for open politics UK.
Zoho: Document collaboration site To do
The coder behind werelate.org has created some extensions exhibit pedi maps to make connection icons on the map. I think you havve to put the whole system on your own host, which when I manage to add it I can use to connect flight.