A blog about things that interest me, politics, news, media, architecture, development, environment, local history, secularism, web, dublin ireland, tara

Contact me at expectationlost@gmail.com

Sunday 13 November 2011

My many administrative regions.

Work in progress

List of My regions
Maps of Regions
Eastern River Basin District
Local Health offices
HSE North East
Church of Ireland Diocese
global.mapit.mysociety Postcodes

Wednesday 26 October 2011

% of Expenditure Limit vs % of Vote 2011 Irish General Election

View fully on Tableau Public
2011 election expenditure - SIPO
Dept of Environment Voting documents

Displayed as percentage of spending limit and percentage of vote to normalize for the different sized constituencies.
* 3 seat constituency - €30,150.00
* 4 seat constituency - €37,650.00
* 5 seat constituency - €45,200.00

2011 Election guidelines from Standards in Public Office Commission.
Google Spreadsheet data
2011 Election results on wikipedia

In the spreadsheet I have the data on the public and party funding, and the percentage that makes up of the candidates expenses just looking for a good way to show that.
Also need a better way to show the average e16,790 spend by candidates.
A 2011 candidates donations and costs

Tuesday 19 July 2011

What does the Irish President spend their time doing. Work in progress

Updated with Tableau Public dashboard below for years 2010/11.
I did get some help from coders, via scraperwiki although the poor formatting on president.ie frustrated the ability to scrape past engagements,.. need a mass geocoder.
Rough presidential engagements data google spreadsheets data garbled after spreadsheet failed
Functions of the President of Ireland - Citizen Information Board of Ireland.

The President is the Irish Head of State, although he or she plays a mainly ceremonial role in the running of the State. The current President of Ireland is Mary McAleese, who commenced serving her second term as President on 11th November, 2004, and which ends in November 2011 with an election campaign beginning in Autumn 2011.
The Government must approve all Presidential communications, messages or addresses, both to the Oireachtas (Irish houses of Parliament) and to the public.
Many of the powers of the President can only be exercised on the advice of the Government. This means that if the President wishes to exercise a power, he or she must first obtain the advice and agreement of the Government.
The President also possesses some powers that he or she cannot exercise without first consulting with the Council of State.
The President's powers include
Appointing members of the Government, judges and other officials
Summoning and dissolving the Dáil
Convening the Oireachtas
Representing the people
Signing Bills and referring Bills to the Supreme Court
Acting as Supreme Commander of the Defence Forces.

Áras an Uachtaráin Wikipedia Commoms

Generally the Presidents role is split into three parts, upholding the constitution, representing Ireland internationally and meeting and speaking to communities, charities, services and industry.
There is a lot of skepticism in Ireland as to why a politician or professional who would have made great effect in the previous political life would agree to be so restricted to not be able to make further great change in the role of President separate to the policy of the government of the day. while hoping to show how genuinely busy the President with many many engagements each and week, it also seems strange to have someone so highly qualified and experienced spending so much time meeting people and opening buildings rather then spending their time publicly debating and drafting laws and legislation as opposed as to much more discretely as President in reviewing laws passed by the government in power. New candidates for the President are making pronouncements are how they will carry out the role while there's still little much awareness, scrutiny or critique of the details of the President's work. I think the President role is to shake the hands of other foreign heads of state, presidents and kings on deals made by their governments, the weight of the decisions is on the President, similarly with the domestic laws. So I thought I would visualise the President's official diary to see how much time they spend on each role and sub-categorize their acts and engagements.
The President is one of the few Irish politicians who with regularity publishes their diary in public ahead of time on publically accessible web site rather then only directed towards the press. I think the impression of the Presidents would be improved if there were more ways to find out and keep track of their work. The diary should be put in well formatted and reusable format (such as .ics calendar file) and the license that could be spread around to be reused in whole or in part to keep people informed.
I have tried to code a scraper puts its such a steep learning curve with so much presumed knowledge, I've previously failed to grasp the looping and syntax needed and there are few examples that you can apply generally. So I put in a request to scraperwiki for some help, a user finddb wrote a ruby scraper which scraped the 2011 engagements. This gave me a csv file I could download and import into a google spreadsheet Although grateful for somebody to write the code for me, I need to split up the time_place variable to possibly geocode the address via Scraperwiki and figure out how scrape all past and future dates and to output and receive the information as it updates daily and weekly, and find the ideal way to share and display the stream of data.

Two coders attempted to scrape the badly formatted engagements data from the president website, but it was too onerous a task and is incomplete. They are about 6,000 records.
Matt's irish_president_engagements scrapper.
Owl's ie_pres_events_details scrapper.

I have categorised the engagements as date, year, time, destination, region (destf), day, address, latlng, arasnotaras, details, place, act, issue, constitutional, month. This need further review, further accuracy in geocoding and further judgement on the types of engagement and category which should be applied, I could probably categories more of the foreign affairs as economics trips. I've tried to isolate the active verb, the location, the place, the parties met in each engagement and a more general category of the issue associated with the event. I have categories whether this was constitutional act in regard to reviewing laws, appointing members of government after elections, receiving and appointing ambassador and carrying out state visits or more general public engagements.

Entity Extraction

Entity extraction still isn't easy or perfect, web services come and go, I have used Alchemy API, using the importxml formula in my google spreadsheet. Have to find someway of using all these API's on my spreadsheets beyond the google import limit of 50.

Wordle: President of Ireland Engagements
Word clouds have a bad reputation as anything in chart junk, particularly with word length exaggerating or undermining the importance of a word, but the more I look at the above the more I think it might a useful way to see straight away the keywords of the engagements, school, community centre, Ambassador from Brazil, signed Justice act etc without having to drill down or match colours with issues. I have been looking at the following examples, making it in R or Prefuse seems be the best bet for it, if I can find anyone who done similar.

superbowl twitter geo cloud animation
text analysis election08 stump speeches Tableau word clouds example

geography of presidential campaign rhetoric

Maple – a Web Map Service for Verbal Visualisation Using Tag Clouds Generated from Map Feature Frequencies

timecloud by stef
carto junk
mapping londons surnames
tagmaps discontinued - yahoo
Building a better word cloud Vidi tagmap
tag map using engagement data ugh.

As I have used many, times, before again I used MIT's Simile Exhibit, the faceted browsing dashboard which allows you to map, list and timeline and drill down through data and which is relative easy to make. I used the calendar extension for the first time, make sure you get all the files it needs. Exhibit receives data in JSON form which you can output from a google spreadsheet so that you can edit it by hand.
Irish Presidential Engagements Exhibit. Unfortunately this doesn't work anymore I went over the formula limit in the spreadsheet and now can't revert the document. Google spreadsheets is both great and crap.

You can click on any of the word clouds or select any facet from the sidebar to focus on the engagements on the map or the timeline. Click on the calendar tab or the details tables for further info.
I know there too much info on the screen but I use these exhibit as working visualizations to sort through the information and add categories to chop up the data, and figure out how it works, I'm using the in-google spreadsheet geocoding which is restricted to 50 addresses geocoded at a time, I still need to review each address again.
TimeFlow by Flowingmedia is desktop program for viewing large amounts of information, it was bought by google and its public development stopped but you can download and use an alpha release by importing csv files.

Tableau Public
Used the freemium visualization dashboard
Tableau Public to show map and timeflow style overview of engagements with some interactivity.
Other visualization options Timemap D3 linear heatmap calendar Choosel GWT Visualisation Tableau Public President of Ireland President Engagements bitly bundle of relevant links Irish president engagement details

Saturday 5 February 2011

% of Expenditure Limit vs % of Vote 2007 Irish General Election

View fully on Tableau Public

2007 election expenditure - SIPO

Displayed as percentage of spending limit and percentage of vote to normalise for the different sized constituencies.

* 3 seat constituency - €30,150.00
* 4 seat constituency - €37,650.00
* 5 seat constituency - €45,200.00

2007 Election guidelines from Standards in Public Office Commission.

Google Spreadsheet data

2007 Election results on wikipedia

ElectedNot ElectedAllAll £Avg £ of ElectedAvg £ of Not ElectedAvg £ of All
Votes Avg16.84%4.86%9.19%
Spending Avg60.1%36.64%45.4%13,70722,24513,66416,790

In the spreadsheet I have the data on the public and party funding, and the percentage that makes up of the candidates expenses just looking for a good way to show that.

Also need a better way to show the average e16,790 spend by candidates.
A 2011 candidates donations and costs

Tuesday 1 February 2011

Dail Constituency Boundaries for Election 2011

Anyone coming to my blog looking for Dail constituencies, I don't have em.

The official Constituency Commission has images and detailed reports on the boundaries and changes.

Go here for the 2012 boundaries in google-kml http://www.tallyroom.com.au/maps (free if credited to Ben Raue ) or here for more detailed ones with Electoral (sub) Divisions http://icampaigned.com/blog/dail-constituency-maps/ (not free) or go Ordnance Survey Ireland for commercial use files.

Shapefiles from the CSO

Saturday 29 January 2011

A better word cloud with R: FF Party vs Country

Keep meaning to learn R an open source statical graphics language so I copied this example from Drew Conway about building a better word cloud. I took the Fianna Fail leadership contest speeches from the Irish Times as recent examples of comparable speeches. There was a lot of talk of how often the spoke of what was good for the party as opposed to what was good for the country.

Click to enlarge

As Drew says this only shows common words so here are wordle clouds, of all there words.

Sunday 2 January 2011

Irish Government Legislation Exhibit

Irish Legislation Exhibit

Trying to put together something that shows the progress and types of Oireachtas Bills through the Irish parliamentary and see how many they have fulfilled from the Government Legislation Programme, an attempt at a less good IBM Many Bills. It shows details of bills progress through the Dail, Seanad and committees from the life for the government from 2007 to 2011. As always its ugly, heavy and there too much information on screen, but just trying to figure it out, I still love the faceted browsing on timelines made with Smilie Exhibit its allow you drill down through so many options and its so easy to make. It atleast gave me something to find out what other information I would need for a better version later.

My rough Calculations give these totals and averages
254 total bills
105 total enacted
0 total opp enacted

278 avg days everything
104 avg days to enact
149 avg opp bills life days
89 total opp bills

Im still trying to learn how bills are made so try these two guides
How legistation is enacted
How a Bill is made Video.

Collected details of the oireachtas bills here onto a Google Docs spreadsheet based on many already gathered by The Story


1st First Stage 1st Second Stage 1st Third (Committee) Stage 1st Fourth (Report) Stage 1st Fifth (Final) Stage 2nd First Stage 2nd Second Stage 2nd Third (Committee) Stage 2nd Fourth (Report) Stage 2nd Fifth (Final) Stage Enacted

Bills go through one house then the other.

Displaying progress of bills
Spent ages trying to get a percentage completed bar to show in Smilie Exhibit, I got Timeline percentage extension partly working then broke it, but in the end just used different colours but its hard to have enough spread in the colours. Obviously there problems with contrast and too much text on screen. I just wanted to create a filter every differentiation I could find.

Simile Gannt Timeline

latest start earliest end

Had to scrape the info from the Oireachtas site.

Used a combination of the previously gathered info from the story, Google docs import html, laborious copy and paste and scraperwiki

Scraped the source code with the help of Alvin who made a ruby scraper and another with help from Julian Todd the creator of scraperwiki.
2010-statutory-instruments-ireland irish-legislation bii-p-url Need to make some auto updating running totals of the various stages which would be more useful for people and look better, using this scraperwiki example, but the Oireachtas sites is very changeable and not easy to pinpoint the stage it is at.

How a bill becomes law Some good examples of infographics to the explain the complicated process.
bill to law
how our laws are made

How A Bill Becomes Law

I'll have to make my own version of this, I started the very basic of it on a google doc drawing

Oireachtas Bills
Dail Record
Kildarestreet.com Dail transcripts
Dail Sessions
30th Dail Wikipedia
Citizeninfo legislation