Blog stopped at customs in bizarre case of mistaken identity

By Matthew Landauer | Published: October 28, 2010

In a bizarre case of mistaken identity the department of Customs and Border Protection appear to think that OpenAustralia.org‘s republishing of Federal parliamentary discussions constitutes a blog.

Last week we were told by a person who works for the Australian Customs and Border Protection Service that OpenAustralia.org was blocked by their IT department’s security policy. It happened that they wanted to use email alerts to stay informed on issues relating to customs policy discussed in the Federal parliament. So, clearly, they were wanting to use the site for a very sensible, work related activity.

A few minutes later, after prompting on Twitter, I fired off an email to the Australian Customs

From:”Matthew Landauer” <matthew@openaustralia.org>
Sent:Wednesday, 20 October 2010 12:08:31 PM
To: information@customs.gov.au
Cc: contact@openaustralia.org
Subject: Query about Customs access to openaustralia.org

Dear Sir / Madam,

We just were told by a person who works for Customs that one of our websites http://www.openaustralia.org, which republishes the Federal proceedings of Parliament, which is run by a charity, the OpenAustralia Foundation, has been blocked from internal use inside the department.

Firstly, is this true?

Secondly, if this is true, what is your reason for blocking access given that there are many legitimate work uses for openaustralia.org within your department?

Thank for your time.

All the best,
Matthew Landauer

This morning I received this reply

Hello Matthew

We have received the following response from our IT Security Section:

The website http://www.openaustralia.org and it’s charity foundation http://blog.openaustralia.org/foundation/ are classified by the filtering software in use by the Australian Customs and Border protection service as ‘blogs’.

The Australian Customs and Border Protection does not allow general access to websites classified as ‘blogs’ at the present time to due to the threat websites within this category can pose to the security of the Australian Customs and Border Protection network. It is important to note that the filter list is provided by a third party and the Australian Custom and Border Protection service simply consumes this list. We do not make decisions on what category a website should be placed in.

If a business requirement exists for a user or groups of users to access content that is unavailable, they can request an exemption which will be granted after the appropriate approvals have been sought.

The classification of ‘blog’ is defined as below

Sites which contain ‘blogs’ (an abridgment of the term ‘web logs’). Blogs are usually maintained by an individual with regular entries of commentary, descriptions of events, or other material such as graphics or video. Entries are commonly displayed in reverse chronological order. Examples include:
Commentary on particular subjects such as news or politics
Online diaries
Photo blogs
Audio and video blogs
Regards

[name redacted] | Senior Customs and Border Protection Officer |Customs Information and Support Centre | CE&CS
Australian Customs and Border Protection Service
Customs House, 10 Cooks River Ave, Mascot NSW 2020
Phone I 1300 363 263
Fax I (02) 8339 6714
Email: information@customs.gov.au
Website: www.customs.gov.au

This information is provided as a guide only and should be clarified either by lodging a formal advice request with the appropriate section of Customs or employing the services of a customs broker if appropriate.

Irrespective of whether you think government departments should be blocking blogs as a matter of “security” policy, anyone who has spent more than a passing minute looking at OpenAustralia.org will know that it is most definitely not a blog, but rather republishes the Federal Hansard, the official proceedings of the Australian parliament.

This website blog.openaustralia.org is a blog. So, block this if you must (not that anyone inside Customs can read this, of course) but don’t block www.openaustralia.org. That’s just silly.

Posted in OpenAustralia.org | 15 Responses

Clock has run out on Victorian election monitoring

By Henare Degan | Published: October 25, 2010

On Saturday the 27th of November 2010 Victorians are going to the polls with the very big job of electing their next state government. That is a little over a month away.

Two weeks ago, a few of us who had worked on ElectionLeaflets.org.au during the Federal election got together to talk about what we should do next with the site.

With the upcoming Victorian election it seemed clear. Let’s make ElectionLeaflets work for the Victorian election as well.

We set today as our deadline for launching it.

Well, to cut a long story short, the disappointing thing is that we haven’t made it. We won’t be launching that site today.

There are a few reasons for this.

Firstly, remember that despite ElectionLeaflets being run by a charity, don’t assume that anyone is being paid to work on this. Quite the contrary, all the work that has gone into the site, setting it up, designing it, publicising it and talking to the media about it has all been done by unpaid volunteers.

We were hoping that adding support for multiple elections to ElectionLeaflets would be relatively straightforward, as the original developers had already thought about this from the beginning. However, it turned out to not be completely trivial. To make the required changes would need a bit of work – probably a few more full days effort.

We’re pretty good at getting things done on the smell of an oily rag and very little time but in this case with only one weekend, a few short evenings, and only two developers available, we simply ran out of time.

Despite the widespread media coverage and praise of ElectionLeaflets during the Federal election campaign, nobody came forward to say “Here’s a donation to support the running of the site and to ensure that it can be around for future elections.”

If that had happened we could have sponsored a developer to focus on this full time. It wouldn’t have taken much.

Donations for the site would also help pay for the PO Box that we rent to collect posted leaflets and help pay for the hosting of the leaflet images on Amazon’s S3 service.

Secondly, beyond the original core team of a dozen or so volunteers, we haven’t had many other people approach us offering their help to make this happen.

This is where you come in – our next goal is to cover the New South Wales state election in March 2011. The last couple of weeks has reminded us all that projects like this don’t build themselves and we need people that are as passionate as we are about keeping our elections fair and honest.

You can help shape this exciting project, and feel good knowing you’ve made a difference to electioneering in Australia. You’ll also be supported by a talented and diverse group of volunteers that are keen to share their skills and experience.

So if you want to see ElectionLeaflets cover the next election you’re called to vote in, you can start by making a donation, telling your friends about ElectionLeaflets, asking us questions on the mailing list, checking the existing trouble tickets, or even downloading our open source code and diving straight in.

If you’re not sure how to get started, please let us know in the comments how we can better help you get involved.

Henare and Matthew
OpenAustralia Foundation volunteers

PS. Victorians can still send their physical leaflets to:

Riley & Ephemera Collection,
C/O Australian History & Literature Team Access and Information Division State Library of Victoria,
328 Swanston Street,
Melbourne VIC 3000 Australia

Donors should include their name and contact details if they would like an acknowledgment of their donation and they only accept hardcopy donations (not scanned copies, or photographs).

Posted in Development, ElectionLeaflets.org.au | Tagged , , , | 1 Response

A new way of seeing

By Matthew Landauer | Published: July 16, 2010

Today, we’re happy to announce a new way of seeing planning applications from PlanningAlerts.org.au for the iPhone 3GS and Android phones. It uses an augmented reality application called Layar to display recent planning applications in real time over a live image of what your phone is pointed at.

To install follow this link on your mobile phone. If you don’t have Layar installed it will point you at the right place and if you do it should just launch the layer.

This is a new way of seeing what’s changing in your neighbourhood. It uses the same information that gets sent to you via email when you sign up for email alerts.

We’ve also added some information about the layer on the home page.

We built it by integrating the layar code into the PlanningAlerts web app, but that wasn’t strictly necessary. Any developer could have built the service using our API (Application Programming Interface) which allows direct access to the data that drives the site.

This is another example of the power of open data and what can be achieved fairly easily if the access to the raw data is available. Local councils, who publish planning applications on their website, with but a very few exceptions don’t publish the underlying data. One of the biggest tasks that PlanningAlerts does is to get access to the underlying data by screen scraping web pages and then republishing that data for others to enjoy.

Posted in Announcement, PlanningAlerts.org.au | 6 Responses

Browse local council websites for 15 minutes and make a big difference to PlanningAlerts.org.au

By Matthew Landauer | Published: June 27, 2010

Photo by chezmichelle (flickr.com)

As you may know PlanningAlerts.org.au works by collecting development applications from many local council websites and republishes them in an easy to use way, notifying people of new applications in their area and in that way keeping you informed of things happening near you.

To do this we need to know where those applications are. So far, we know where around 60 local authorities keep their applications. There’s more than 500 local authorities that we don’t know about.

We’d like to know about a whole lot more so that PlanningAlerts can be useful to more Australians. That’s where you come in.

We’ve created a Google spreadsheet with all the local councils that we know about which we don’t yet support on PlanningAlerts.org.au and we’d like your help to fill it in.

https://spreadsheets.google.com/ccc?key=0AmvYMal8CGUsdG1tM0lEWUctR194eGN6bUh0VGFfc1E&hl=en

This is what you do. Go to the spreadsheet.

  1. Pick a council that hasn’t had all its details filled in yet (“DA’s online?” column is empty). The councils are ordered with the big councils (lots of people) at the top. So, start with those near the top if possible.
  2. Follow the link to the council website (“Website” column)
  3. Find where they store their development applications
  4. Fill in the details in “DA’s online?”, “as HTML”, “website for DA notice” columns. “DA’s online” just says whether the council publishes the development applications online. Hopefully that will be a yes. “as HTML” should be “Yes” when they publish it as a regular web page. Fill in “No” if they publish the information in PDF or Microsoft Word documents for instance. The “website for DA notice” is just a link to the page on the council website where the development applications are stored.
  5. Repeat for a few councils
  6. Add a comment to this blog post to let us and everyone else know you helped out
  7. Give yourself a big pat on the back.

Thank you in advance!

Posted in Development, PlanningAlerts.org.au | Tagged , , , | 15 Responses

Embed a PlanningAlerts Google Map in your blog

By Henare Degan | Published: June 17, 2010

Inspired by Matthew Somerville’s post over at the mySociety blog about Embedding a FixMyStreet Google map in a blog, here’s how to embed a PlanningAlerts Google Map in your blog, like this one for Newtown, NSW:


View Larger Map

From the instructions on the API page, we can work out how to get an RSS feed for our suburb (or any other searches if you’re feeling adventurous!), just change the [state] and [suburb] parts of this URL:

http://www.planningalerts.org.au/applications.rss?state=[state]&suburb=[suburb]

  • Then go to Google Maps, paste that URL into its search box, and click Search Maps
  • Click the Link link to the top right of the map, and copy the Paste HTML to embed in website code
  • Paste that code into your blog post, sidebar, or wherever (you can alter the code to change its size etc.)

Thanks to Matthew at mySociety for the inspiration (and the better part of this blog post!). If you need any help, please don’t hesitate to sing out in the comments.

Posted in Development, PlanningAlerts.org.au | Tagged , , , | 1 Response

Search engines are how people find out about stuff

By Matthew Landauer | Published: May 11, 2010

It’s an easy thing to forget when you’re building a website: people find stuff using search engines. Most of the the time they won’t remember the name of your website and decide to go there. They’ll look something up on Google or one of the miriad other search engines and it will haphazardly lead them to something on the web, useful or not.

We’d all like to think that the website we’re making is so important and so useful that people will remember the name of it, bookmark it and come back to it regularly, but chances are that just isn’t the case.

In my last post I described the early stages of building PlanningAlerts and how the website was focused purely on getting people to sign up for email alerts and how, surprise surprise, the traffic was pretty small after launch.

Since this post is about searching you probably have some ideas where this is going, but I’ll digress a little further.

There’s a lot to be learned from past experience. Looking at the statistics of visitors to OpenAustralia.org while parliament is in session we get about 75% of our traffic through search engines, and of that about 50% is coming from normal searches on Google and 25% from “paid” searches on Google (we get advertising donated by Google through their Google Grants programme) and then a small sliver from other search engines. From my understanding these numbers are fairly typical of a large site.

Another interesting thing about the OpenAustralia.org traffic is that it varies hugely between the times that parliament is in session and when it’s in recess. A large part of this is because people don’t receive email alerts when parliament is in recess. Email alerts are the way that people are reminded that there is something interesting to look at and so it’s one of the really important ways that people keep coming back to OpenAustralia.

So, basically, search is what largely brings people to OpenAustralia for the first time, but email alerts are what keep them coming back.

So, how can we apply this experience to PlanningAlerts?

The answer is surprisingly simple. PlanningAlerts has a database of a huge number of planning applications spread across the country. At the moment we kind of hide this information and only make it available to you if you sign up for an email alert. If we put that information on our website and make it searchable and browseable (and indexable by search engines) we’ll find a whole lot more people will discover PlanningAlerts.

So, how did we go about doing this?

I’ll leave the details to another post. The changes are still in progress but some of the early gains are visible now. For instance, when you get an email alert it used to be that when you wanted to find out more about the planning application it would give you a link back to the council website. Now, instead we give a link that points back to the PlanningAlerts website. Every planning application has its own page on PlanningAlerts. Here’s an example:

We’re now also actively telling search engines about those pages using sitemaps. So, every time new planning applications are found on council websites (which are usually not indexed by search engines) they’re indexed by the search engines as quickly as possible.

We’re already now seeing that the majority of searches that lead people to find PlanningAlerts are coming from people doing searches on specific street addresses. This shows that it’s working.

Posted in Development | Tagged , | Leave a comment

PlanningAlerts – going back to basics

By Matthew Landauer | Published: May 7, 2010

image by Adam Foster (CC by-nc-nd)

Back in December of last year, we launched PlanningAlerts. It was a massive rush getting the site up and running. We had only seven weeks to build the site, including the scrapers, the bits of software that get all the planning data from the local council websites. We had applied to the Government 2.0 Taskforce for funding for the project and only received notice of go-ahead towards the end of November. Now, for some reason, which to this day still doesn’t make a whole lot of sense to me, they wanted every single project associated with the Taskforce wrapped up by the end of the year. So, that’s how the short deadline happened.

So, I did whatever any self-respecting developer does. I borrowed as much code as I could from elsewhere. In this case, the UK version of PlanningAlerts which was licensed as an open-source project under the GNU Public License (GPL).

That approach, similar to the way OpenAustralia was built, worked quite well. It gave me a clear target for the data that I needed to get from the local council websites through the scrapers.

The website itself did need some changing. In the UK, postcodes are really specific. They narrow things down to just a few houses on a street. This makes postcodes perfect as a quick way of specifying a location without having to give a full long-winded street address. So, the UK PlanningAlerts site uses postcodes to locate a place you want to receive planning alerts for.

Australian postcodes aren’t as specific as the UK ones. The areas of postcodes can be quite large and strangely shaped. So, postcodes are really not that great of a solution for locating a specific smallish area in Australia.

So, that meant I had to update things so that the user would enter a street address instead of a postcode. Luckily that turned out to be fairly straightforward. If my memory serves me correctly that took a little over a day to change.

So, we launched the site at the end of December to a short burst of interest. However, this time of year, towards Christmas and New Year must be about the worst possible time you could imagine to launch a new website. Everyone has other things on their mind: People to see, parties to go to, barbies to light, presents to buy…

No worries.

Fast forward a month to the end of January. We have a couple of hundred people signed up for email alerts but almost no visibility on the wider web. Relatively few people have mentioned PlanningAlerts on blogs and the like.

Time to go back to basics. What’s the point of building a service like PlanningAlerts if only a relatively small number of people use it? How can we get the message out?

The point of PlanningAlerts is to get emails of planning applications near you. Stuff comes to you in your inbox. After all, who really checks their local council website regularly for new and interesting development applications? Well clearly some people, but definitely not the majority.

So, the focus of the site is squarely on getting people to sign up for alerts. But how do you know what you’re signing up for? Well, in its current form, you just don’t.

Definitely time to go back to basics and rethink the whole experience. What came next? I’ll pick that in my next post…

Posted in Development | Tagged , , , | 5 Responses

Blog definitely not forgotten

By Matthew Landauer | Published: May 6, 2010

It’s been way too long since I’ve written anything on this blog, which is very poor.

Time to turn that around. There’s lots to talk about. Because I haven’t written anything in ages I don’t really know where to start. Hmmm…

Posted in Announcement | Leave a comment

From the workshop: OpenAustralia Labs

By Henare Degan | Published: January 25, 2010

What do you do in your spare time during a busy, week-long conference?

A) Relax
B) Have a beer
C) Set up new OpenAustralia sites

If you’re one of our volunteers, Tim ‘mithro’ Ansell or your author, Henare Degan, you answered ‘C’ – well, OK, and a bit of ‘B’ too : )

During last weeks linux.conf.au 2010, held in Wellington, New Zealand, Tim and Henare setup a new site called OpenAustralia Labs.

Labs is a place to showcase some of our more crazy experiments, or just things that aren’t quite ready for prime-time.

We have two experiments already posted to labs with more coming soon:

Register of Interests Transcriber

The Register of Interests Transcriber is Tim’s tool for turning the scanned register we opened up about a year ago into searchable passages of text. It’s been looking for a home since Tim created it a few months ago.

This is a tool we’d really like to see emerge from the labs and appear on the live OpenAustralia.org site so if you think you could help make it more polished, please check out the code and start discussing it on our Community mailing list.

Words in Parliament

This is an interesting little experiment created by Stephen Bartlett during our Melbourne hackfest. It grabs the most recent house debate and analyses the most frequently spoken words in that debate.

If you have any experiments you’d like to try out and demonstrate on OpenAustralia Labs, please don’t hesitate to discuss it on our community mailing list or create a ticket against the new Labs component on our issue tracker.

Posted in Announcement, Development | Tagged , | 1 Response

Launching our new website: Planning Alerts

By Matthew Landauer | Published: December 21, 2009

You might have noticed we’ve been fairly quiet of late. Well, that’s because we’ve been hard at work putting together a new website called Planning Alerts which we’re launching today.

Planning Alerts: Email alerts of planning applications near you

We’re really excited and proud to share this with you!

You’d probably know if your next door neighbour was going to knock their house down (you’d get a letter through the door telling you they had applied for planning permission and asking you what you thought about it). But you’d probably never find out if the old cinema or pub 5 streets away is going to be converted into luxury flats until the bulldozers turned up.

PlanningAlerts.org.au is a free service which searches as many planning authority websites as it can find and emails you details of applications near you. The aim of this to enable shared scrutiny of what is being built (and knocked down) in peoples’ communities.

Planning Alerts website screenshot

PlanningAlerts.org.au is brought to you by the OpenAustralia Foundation with financial assistance from the Australian Government 2.0 Taskforce*. It was adapted for Australia by Matthew Landauer and Katherine Szuminska, and is based on the UK site PlanningAlerts.com, built by Richard Pope, Mikel Maron, Sam Smith, Duncan Parkes, Tom Hughes and Andy Armstrong.

We’re not covering all the planning authorities yet. We’re nowhere near actually. We’ve started with the 50 or so authorities with the larger populations so we’re actually covering a good percentage of the country’s population.

If you’re interested in adding a screen scraper for your local authority we’ve made it pretty easy for you. All the software that runs the site and all the screen scraper code is open-source, so please take it and have fun!

*Original link dead – archived view courtesy of The Wayback Machine <3

Posted in Announcement, PlanningAlerts.org.au | 8 Responses