Putting Councillors to Bed in PlanningAlerts

By Matthew Landauer | Published: September 13, 2021

This week we’re removing councillors, and the ability to write to councillors from PlanningAlerts. We disabled the feature in 2019. Now we’re removing the functionality from the code that runs the site and as a side-effect historical comments to councillors made between 2016 and 2019 will now no longer be visible on PlanningAlerts.

Why on earth would we do this?

I thought long and hard before doing this. It was in the front of my mind that hard work went into building the feature, it did what was intended and many people used it. So, why remove it?

The main reason is maintaining a complete and up-to-date list of all councillors for the hundreds of local councils in Australia was not a task that we were able to do with the way that things were set up. Regular council elections cause everything to change as well as big events like the forced merger of 40 odd local councils in NSW. In 2019 when we disabled the feature we were so behind on updates and the experience for people is that it says you can write to councillors who are no longer councillors which is terrible. Rather than show stuff that is plainly misleading we decided to remove councillors completely.

There were some other big technical issues as well. Write-it, a service (and open-source software) developed by a Chilean not-for-profit that we were using to send and receive the emails to councillors, was not being supported anymore. We were increasingly running into issues where we had to do things like redact comments and were having trouble contacting the administrators. In the intervening time the service has gone completely offline and the software is not being developed any further. So, we would have to either take over support and development of the Write-It software and host a service ourselves or develop an alternative approach, perhaps using Cuttlefish, the open-source transactional email service that we use for sending out email alerts from PlanningAlerts.

I wish we could have gotten back to this and fixed all this but the truth is we haven’t. So, rather than leaving this unfinished, not working thing around for us to feel guilty about, we’ve decided to grasp the nettle and excise it completely from the codebase. This has the added benefit of significantly simplifying the codebase which makes it easier for anyone new to pick up as well as simpler for us to support and add new features to. 

If we rebuild or recreate this feature in the future we will likely have to focus earlier on finding a sustainable way for the councillor information to be kept up-to-date. One passing idea is to only add councillors to a particular local council if a small group of local people nominate themselves as maintainers of the data for their council. They are the people most motivated to keep things up to date. Of course we don’t really know if that would work in practise but it might be one place to start in the future. If this is something that you would be interested in doing were it to come back, please do let us know.

What about the historical record?

It doesn’t feel great to be making changes to old applications visible on PlanningAlerts by removing comments to councillors (comments to planning authorities are unaffected). To try to be good citizens about all of this we’ve archived all the publicly visible information in a machine readable format (YAML) at https://github.com/openaustralia/planningalerts-councillor-comments-archive 

We’ve also made sure that snapshots of all pages with comments to councillors have been archived on the Wayback Machine from the Internet Archive. Hurrah for the Internet Archive. We wrote some code to do this all automatically too:

You can access the archived snapshots on the Internet Archive by taking the URL of the application on PlanningAlerts (e.g. https://www.planningalerts.org.au/applications/687643) and putting it into the Wayback Machine URL https://web.archive.org/web/20210909/<insert URL here> so that you get a full URL like https://web.archive.org/web/20210909/https://www.planningalerts.org.au/applications/687643

Thank you

Thank you to Luke Bacon, Henare Degan and Hisayo Horie for developing the features for writing to councillors. 

Posted in PlanningAlerts.org.au | 2 Responses

Right to Know Pro For Journalists, Researchers, Academics, Campaigners, and FOI professionals.

By Kat Szuminska | Published: March 30, 2021

A new way to make FOI requests, for professionals

Since we launched Right to Know we’ve regularly heard from journalists, who would love to use Right to Know, but need to keep Freedom of Information (FOI) requests under wraps until a story comes together.

We also know it’s not only journalists who need to work carefully through the issues of complex FOI requests, until a body of work is concluded.  We know how vital it is that you have the option to keep requests and responses private while you get the facts straight. And we know that takes time. 

So now, we’re launching Right to Know Pro, a fully featured toolkit, for journalists and researchers.

Use all the powerful tools of Right to Know to help save time, make requests more easily, and keep them safe until you’re ready to share your work with the world.

When you’re ready, you can then share requests and responses with your audience. 

Make FOI requests using Right to Know Pro, your requests and responses remain connected to your story, helping the public understand the substance, and verifying the source of truth with citations. 

You make and track multiple complex FOI requests, so Right to Know Pro provides a clean and powerful dashboard to simplify your FOI workflow across multiple and complex requests.

In the absence of easily traceable information, people can and do make up their own stories about the truth of your work, especially when it challenges their existing world view.

Reshare and refer people who want to see your FOI sources down the track. The presence of your hard work will also continue to be available, even if the Government agency later removes their disclosure log. No dead links when machinery of government or policy changes turn up dead links on a Government pr media website.

  • Keep requests and responses private while you work on your story
  • Send requests to multiple authorities with batch requests
  • A super-smart to-do list: follow the progress of your requests
  • Save as draft to get back to it later
  • Action alerts: know when it’s time to take the next step
  • Daily summary emails to keep your inbox clean
  • A powerful actionable private dashboard: track and manage all your FOI projects

Get Started with Right to Know Pro now

Posted in Announcement, RightToKnow.org.au | Leave a comment

Thank You for All Your Comments

By Kat Szuminska | Published: January 13, 2021

Today we removed the ability for people to comment on speeches published at openaustralia.org.au

We’re taking a pause on letting anyone comment on OpenAustralia because, in our work we want to make it a little easier for people who want to make a difference, to have their voices heard, for their voices to matter in the collective decision making we call democracy. We don’t think we’ve been successful in offering that here.

The commenting facility takes work to maintain, so we need to know that it’s adding value for the people who use OpenAustralia. The truth is, only about 350 people have commented on a speech since the site launched in June 2008. Maybe that says we’re not providing a facility people want, or that we’ve not provided quite the right way to do it. We just don’t know.

To everyone who has taken the time and effort to read, consider, and respond to speeches made in Parliament on OpenAustralia, a heartfelt THANK YOU. Listed by order of recent comments:

Tibor Majlath, Charlie Schroeder, Bill Thomson, Mark Duffett, Steven Azar, Warren Mccahill, Andrew Jackson, Romeo Alvaro, tony mckinnon, Alis Jacob, Isabel Storey, Michael Peters, Errol Nilson, Zhan Pintu, Dirk Bossard, Brett Hilder, Meredith Doig, Bernie Ramsay, Gill Caunce, Michael Dixon, KENNETH JURY, Johannes Kock, Robert Ashman, Carl HANICH, Adrian Menzel, John McWhirter-Whitlock, Leigh Wiggins, WARREN RODWELL, jules makk, Philip Hodges, Dwight Walker, Carlo Larocca, marcia flower, Stephen Caddaye, Madeleine Chapman, Patti Luskan, Bianco Salsano, Alcy Infinity, Kellie Nelson, Vicki Stebbins, Michael Rynn, Alison Keen, Tony Zegenhagen, David Simpson, Stephen Tittensor, Ray Bazouni, Jacob Mulquin, Grant Miller, Pete Gello, Harry Makris, Margaret Neal, Christopher Grey, Claire Bettington, William Boeder, Luke Walker, Mark C-F, Deirdre Ryan, Planecrashzone Planecrashzone, Andrew Simmons, Benton, Kay Spurr, Jane Russ, Anthony McIntyre, Brett Sanderson, Peter Rogers, Tom Peace, Arthur Ventham, Patti Farnell, Nathanael Coyne, Kathleen Minassian, Pat Patrick, Jennifer Jary, Shelley Travers, Kaye Schwartz, Ryan Brown, Glenn Kerswell, Steve Butel, Natalie Davis, David Hill, Chas Van Hulsentop, Rodney Ross, Donna Connors, Shay Holmes, Mary Alderson, Jim Gray, Carmel Connors, David Collins, dan diver, Thomas Nel, George Peterson, Ahmad Mostafa, Jenny Watson, Bill Macfarlane, Brian JAMES, K N, William McCann, robyn furci, David Redwood, Nerryl Brown, Henare Degan, Roger Helbig, tim leung, Mark Nicol, Mark Addinall, Munir Chagpar, 22 football, Jill trotter, Jason McClurg, John Goss, Pauline Zerbo, George Parker, James Walker, Keran Carsburg, John Griffiths, Roger Colclough, David Barratt, Bernie McGurgan, Chris Weir, Margaret Major, Antony (Tony) Amos, Elaine, carlos aparicio, al collick, Tim Handley, Steven Knowles, Eric Palmer, Maksim Stojanovic, Rona Goold, Brett Silich, Patricia Norman, Penelope Bell, Nathan Campbell, John Clapton, Margaret-Rose STRINGER, Chris Edwards, Hank D, Mark Addinall, Debra Lovi, Ernie Gimm, Michael Wells, kristopher kubique, Hari Guduru, Belinda Schneider, Renato Bright, Patrick Shanahan, Barbara Olsen, Jan de Wit, Robin Sharee Eastick, Pui Ho Lam, Free Mind, Melissa Stewart, Craig Bellamy, R Barron, G Gough, john jeayes, robert, David Gardiner, Ross Ulman, Delphine Stagg, Emily, michael spalding, abc 123, marlene huff, karan krishna, Daniel Kinsman, Dot Newbold, Margaret Makewell, Minnette Sheiles, Rebecca Dunstan, steve callaghan, al, Peter Demmery, Bill Realph, Sarah, treena day, Steve Wickenden, Nicolette Norris, James Kruithof, Julian Fidge, Dean Cottier, Ben Christy, jaime schultz, Barbara Johnstone, Verden Bell, Graham Martin, jim coleman, Jackie Bowden, Michael Cranny, Travis Broes, Mark Thorp, Ern Chang, Ray Borradale, cath blue, Andrew Black, Joy Butler, brian cooper, paul, Josh Cooper, Susan LakesNeedWater, Melissa Raven, Ken Jury, Stella Dyball, Dallas Beaufort, Maarten Vanderhaar, Colin Gourley, Philip Copeland, Lisa Hodgson, Richard Eden, Corey Irlam, Secret Ballot, christian borleis, Mele Fotu, Simon Dodds, Warren Rankin, Ralf Kluin, Deirdre Ryan, Iain Murchland, Margaret Muddle, Mike Male, Tony Huynh, Yodie Batzke, John Winterbotham, Donald Tate,Kevin Russell, Nish, jewel mahmud, Brian Armour, shirley williams, James, Greg Boyles, Peter Adnams, Brandon Chant, chris barrett, Philip Clark, J Jackman, Philip Lillingston, rachael dersch, Roger Coates, Rachel Teesdale-Smith, Rochelle Roberts, David Edgar, Shayne Cummin, Collin Van Uden, Adam P, Dale McClelland, Ian Rist, Lindsay Holmwood, Josef Tadich, John Rodda, Thomas Castiglione, Rod Chaffey, clive Osmon, Bruce Reyburn, David McNeil, Mick Loeckenhoff, Laszlo Kovassy, Dominic Gladheart, John Trigge, Anthony Moore, kerrie watherston, Andrew Smith, Harjeet Sing, MOrgs MOrgs, Bernie Glynn, Sherif Mansour, Peter Thornton, Debbie Bayliss, SYED HASSAN RAZA NAQVI, Shaun Lambert, Ernest Chamberlain, harry spicer, Geoff Mason, Kris Gesling, Caroline Cox, Sally Rose, Bill Ray, Dominique Quirke, Melissa McFarlane, Karen Fehring, Elly Bingham, Lloyd Stephen, Bronwyn Moir, Tonia Paroz, Nicole Carver, Kiera Pedle, Frank Baarda, connaught linton, Judie Gade, Hadley Baker, John Brown, David Leigh, James Fehon, Jonathon Singleton, Juiced Pixels, Ben Gran, Grendel, Trent Murray, Steve H, Ashton McAllan, Ben, Leonard Matthews, Joshua Moran, Joel Dignam, Graham Inskip, James Hay, Raymond J Warren, David Tangye, Mal Boyce, Paul Burns, Gary Jarrad, Matthew McDaniel, Delory, Paul Esson, Andrew Rossiter, Ben Rogers, Simon Rumble, Reng Ten, Brett Carnes, Daniel Snow, Amanda Kelly, David Elliott, Jason Geddes, Shelby de Piazza, Alexander Bakharev, Rasto Petrovic, Richard Jary, Mental Health Nurse, Robert Nelson, Philip Holder, Jaye, Kat Szuminska, Dan Oost, Patrick Terrett, Kieran Bennett, Michael O’Meara, Iain Murray, Ross Frove, Morgan Gobson, Noel Kelly, Peter Wood, Alan Howard, Sean Carmody, Cameron Reilly, Scott Bulfin, Darren Entwistle, Peter Daams

I want to personally thank you for speaking up for something you feel strongly enough about in this way.

For the time being, any and all comments you made before will remain visible, at least the next few months.

What role might commenting play a role in the future direction of OpenAustralia?

To help us decide what to do, if you’ve made a comment before, we would love to talk to you. Would you be willing to say a little about your experience of making a comment on OpenAustralia or have a short conversation by phone? Email us at hello@oaf.org.au

Posted in Announcement, OpenAustralia.org | Leave a comment

How we deal with moderation on Right to Know

By Ben Fairless | Published: November 4, 2020

There have recently been a few articles published in The Guardian about a request on Right to Know that we were asked to moderate. As one of the volunteers who spends the most time looking after Right to Know, I wanted to explain what happened in this case and explain how we moderate requests on Right to Know generally.

How this specific request was handled

In October 2019, the Australian Public Service Commission (APSC) reached out to us requesting the removal of a number of requests. You can find our full email chain with the APSC here.

The first thing we asked the APSC was to tell us what they were specifically concerned about. We then worked through each concern with the APSC.

Instead of just removing the requests (as they asked us to do), we agreed to hide the names of 3 public servants and keep the requests on the site. We felt this was the right balance between someone’s name being associated with an allegation of illegal conduct, and transparency and requests remaining public on our site.

Once we had made the changes, we then annotated the requests and contacted the requestors so they knew what had happened and why.

Investigation by the Commonwealth Ombudsman

The Guardian then reported a complaint had been made to the Commonwealth Ombudsman that the APSC had engaged in “legal misconduct”. The report went on to say that the APSC had been cleared of misconduct.

We provided the below statement to The Guardian as soon as we were aware of the story:

Right to Know was not contacted at any point during the investigation by the Commonwealth Ombudsman into allegations the APSC intimidated us.

When the APSC made us aware of potentially defamatory content, we worked with the APSC and took steps to redact as little of the request as possible to keep the request public.

We feel that it would have been difficult, if not impossible, for the Ombudsman to get the full story without contacting us first. We would have appreciated being advised of the complaint and the outcome before learning about it through the media.

You can read the follow up story from The Guardian here.

Transparency and Moderation – Getting the balance right

We are obsessed with transparency, and with giving you tools that you can use to effect change in our democracy. Moderation is not something we take lightly, however there are some things you can do to help us.

The Simple Stuff

We ask people who make requests to keep them focused. This means:

  1. No arguments about your case
  2. No statements that could defame or insult others

If you follow these rules, it’s likely you’ll never come into contact with moderation on Right to Know.

There are a few things that we will remove as soon as we are made aware of them:

The software we use to run Right to Know also attempts to automatically hide telephone numbers and email addresses, however it’s not 100% perfect so we may hide these manually.

More Complex Moderation

We sometimes receive requests from government authorities (like the APSC and the Australian Taxation Office) who are concerned about a request. They may ask us to remove a request saying that it’s defamatory or there are other safety or “public interest” concerns.

We have a published Takedown Policy, however each request is different and we review each request on it’s merits.

The first thing we ask any authority who wants us to take moderation action is to put in writing exactly what they want removed and why they want it removed. We may follow up with the authority if we have further questions or to get a better picture of what’s going on.

As the email contact happens via our contact email address, it’s not automatically put on Right to Know. We can provide you with a copy if the authority agrees, otherwise you can request the email trail via Freedom of Information (using Right to Know for full transparency if you want!).

Once we the information from the authority, we review the request, any other relevant information, and our policies. Our goal is to keep as much of a request as possible online. Once we’ve worked out what we can do, we then let the authority know. In some cases, we will refuse to take action on a request (if it doesn’t meet our guidelines).

If we make any changes to a request, the last step (and most important) is putting a note on the request to indicate what we changed and why. We will also reach out to the requestor privately to let them know so they can ask us any questions.

More Information or questions?

We have quite a bit of useful information in our help pages, and you can always reach out to us via email if you have any questions. Our team are here to help with anything FOI related.

Posted in RightToKnow.org.au, Uncategorized | Tagged , , , , | Comments closed

The story of a tiny, almost invisible change

By Matthew Landauer | Published: September 29, 2020

I don’t really like technical blog posts.

Instead this is a story of a tiny, almost invisible change we made to PlanningAlerts, what led us to make this change and the surprising amount of under-the-hood work it took to get there.

We made a “tiny” change to how comments are shown on PlanningAlerts. “delivered to the planning authority” is now a link (it’s underlined).

screenshot of comment

When you click on the link it gives you some more information

screenshot of comment after clicking

This should give you extra confidence that the comment was definitely delivered to the local authority.

How it all started

A kind and considerate soul emailed us to let us know of a strange comment that had been posted on PlanningAlerts.

Does this post have any truth in it?

Is this post damaging the Planning Alerts site?

Was the CAPS really necessary?

Here is the comment in full

”DONT” NOT LEAVE COMMENT HERE IF IN FAVOUR OR AGAINST APPLICATION. MANY IF NOT ALL DO NOT MAKE IT TO COUNCIL MOST COMMENTS HERE NEVER MAKE IT TO THE RELEVANT DEPARTMENTAL AUTHORITY!!

Ive been let down by this page thinking mine and many others are being heard here. It makes a difference. It does not. After contacting council in past, they had no clue about my rejection of development or from others on this page. It seems Acting authorities never know about what written here. This site should be taken down as everyone presumes it has direct or indirect authority to distribute to relevant departments.

*** PLEASE CONTACT COUNCIL OR RELEVANT AUTHORITY DIRECTLY ***

There were similar comments made by the same person on three other applications.

My first emotional response was “how dare this person say this, how rude!”.

That person didn’t even contact us directly to ever let us know there was a problem. We don’t read all the comments. There’s just too many for a tiny team doing too much. And then to throw around those blanket accusations.

Ouch! It really hurt to hear that. We have put so much time and effort over the years into ensuring that emails from PlanningAlerts get successfully delivered. We went as far as developing a new open-source transactional email server, Cuttlefish, to give us visibility and improve deliverability across the board. We made PlanningAlerts to give people access to the democratic process and getting their comment delivered is pretty core to that objective.

Then to my horror I realised that what the rude comment was saying was partly true. For the last couple of months 15 or so comments were not delivered to that one particular council. This was terrible!

We are human and our software is written by humans.

In 10 years PlanningAlerts had an amazing track record. We actually only know of one instance when emails were not delivered to a council In that case, a council changed their email address and didn’t forward emails to their old address. As soon as we were notified, we manually readdressed the emails and worked with the council to get them delivered in time.

So, what was going on? Down the rabbit hole of email sending

(Here’s where we get technical)

What was super weird at first was that only some emails to that one council were bouncing. The bounce message mentioned “DMARC” which was not something I knew about at that stage. So why were only some messages getting bounced and not others when all the comment emails are generated the same way? Nothing in the content of the comments looked like spam, the formatting of each email is identical. I found it really confusing.

Then, I noticed a pattern. All the comment emails that were bouncing were made by people with “yahoo.com” or “yahoo.com.au” email addresses. A quick search of “yahoo DMARC” led me to an explanation.

Yahoo had an unusually strict DMARC policy which meant that any problems with emails that look to originate from yahoo domains would get hard bounced. So, we had a DMARC problem with our comment emails but it had been unknown to us up to that point because almost nobody was enforcing it so strictly.

At this point I’ll just quickly backtrack a little. For a long time we’ve been ensuring that our emails pass SPF and DKIM tests. Cuttlefish helps enforce this and does it all pretty much automatically. However DMARC, does two main things. It allows an owner of a domain (e.g. yahoo.com) to say how email servers should handle mail from that domain when SPF and DKIM tests fail. So, in our case SPF and DKIM was fine, or so we thought, so why were the yahoo.com emails failing to get delivered?

Well it turned out it was to do with the second part of what DMARC does. It also enforces domain alignment; the domain in the “from” address of the email needs to be the same domain as used in SPF and DKIM. This is the problem that got us. It turned out that our comment emails had headers of the following form:

From: <email address of the person making the comment>
Sender: contact@planningalerts.org.au
To: <email address of council>

The Sender header is used in circumstances where emails are being sent on behalf of someone else. It’s defined in the email RFCs but generally has not been particularly well supported.

The purpose of all that was to ensure that when someone makes a comment via email that they receive replies from the council rather than us.

It turns out that DMARC essentially stops the Sender from ever being used. It appears that this was a conscious choice made by the DMARC designers because the Sender field is not shown to users in email clients.

So, we switched the email headers to:

From: contact@planningalerts.org.au
Reply-to: <email address of the person making the comment>
To: <email address of council>

And this magically solved our problem.

We resent all the recent comments that had previously bounced because of this, and our job was done.

Actually no. It would have been easy to stop there but actually we wanted to make sure that if something similar to this happens in the future we don’t find out about it because someone emails us asking about a comment on the site. We should be the first ones to know there is a problem.

After we fixed the way that comment emails are sent we started working on a whole host of other changes to improve the visibility and the impact of any potential future hard bounces for comment emails.

Improving visibility and reducing the impact of problems

In explaining earlier that only emails from the “yahoo.com” didn’t get delivered I was over-simplifying. What actually happened was that a yahoo.com email would hard bounce and then the email address of the council would get added to a “deny list” in Cuttlefish. Once an address gets added to the “deny list” any further emails sent to that address will not be sent. Addresses then get cleared out from the “deny list” after one week. This is an automatic feature of cuttlefish designed to reduce the chances that we send out bad emails which improves the deliverability of our emails across the board.

So the case of sending emails to local authorities is different from sending transactional emails (email alerts, etc..) to “random” people on the internet. We should be really confident that the email address of the council is correct and that emails to them generally should not bounce.

So, next we added a feature to cuttlefish.io to allow PlanningAlerts when it’s sending the comment emails to say “please ignore the deny list when sending this email”. This means that if an email hard bounces to a council, that cuttlefish will still attempt delivery for any subsequent emails to the same address. So now if hard bounces do happen the impact will be reduced.

Then we started working on the “visibility” part. If a hard bounce happens for a comment email how can we get notified of this urgent problem? One option would be that we get an email. In recent years we’ve been trying hard to limit the number of emails generated automatically for notifications and instead send those notifications to Slack. So, why not just try sending a notification directly to Slack instead?

How to do this? We don’t want Cuttlefish to have the knowledge to notify us of certain problems with certain emails but not others. It makes much more sense for PlanningAlerts itself to do that because it’s the one that knows the difference between the different kinds of emails. This is where we need to get some of the delivery information from Cuttlefish back into PlanningAlerts.

This finally gave me the reason and motivation to add a feature to Cuttlefish that I’ve been wanting to do since I first built it – which is to add webhooks where every time there is a successful or failed email delivery a custom URL (on another site of your choosing) is hit with information about the email. In our case we’ll be making a new URL endpoint on PlanningAlerts to accept those webhook callbacks.

So how do we identify if the email corresponds to a comment email or an alert email and which comment email? I think the most elegant solution to that is to allow the sender of an email to attach custom metadata.

Here’s an example of a comment email as seen in Cuttlefish showing the attached metadata (in this case comment-id):

screenshot of cuttlefish metadata

So then it was a relatively simple matter of hooking up a bit of logic in PlanningAlerts where on every “delivery event” it receives from cuttlefish, it checks whether it’s from a comment and whether it was a hard bounce and if so it sends a notification to our slack channel.

screenshot of slack notification

At the same time as we put this in we noticed another council was bouncing emails! This time for some reason they had just decided to specifically block our email server. Thanks for that. Thankfully they quickly fixed it when we asked them to and we again resent the emails.

Making it visible to people using PlanningAlerts

We’re confident now that all the changes we made to improve the delivery of emails to councils and to notify us of problems makes it unlikely that similar problems will reoccur in future.

However all this stuff is under the hood and invisible to users. Perhaps that’s how it should be?

However, it’s not uncommon that we get emails from people who just want to make sure that the council did in fact receive the comment they made through PlanningAlerts. The council tells them they never received it, so we check in Cuttlefish to find that the council absolutely received the email.

What if we could make this more clear for people?

Why don’t we do something similar to what is in RightToKnow.org.au where the message says whether it was successfully delivered?

screenshot of RightToKnow request

So that’s what we did. The final piece in a long chain.

Posted in Planning, PlanningAlerts.org.au | Tagged | 1 Response

Last week in Parliament

By Mackay Ash | Published: April 17, 2020

You may have been reading about the unsuccessful amendments to the Coronavirus Economic Response Package that opposition MPs and Senators attempted to pass last week.

The amendments related to expanding the Government’s JobKeeper payment to include more workers, such as casual workers, workers with major charities and working-visa holders who are still in the country. They also called on the Government to create a specific emergency response package for the Arts and Entertainment sector, as many workers in that sector are ineligible for the JobKeeper payment. (You can see how your representatives voted on expanding the JobKeeper payment and supporting the Arts sector on They Vote For You.)

The majority of these amendments were proposed by Labor Party MPs and, while there was broad agreement between Labor, the Greens and other opposition parties and independents, there was also disagreement. For example, Labor Party senators refused to support several substantive Greens amendments in the Senate, arguing that to do so would only delay the passage of the bills. In the words of ACT Senator Katy Gallagher (Labor):

We will not engage in a situation where the legislation could get blocked between chambers. That is not an option.

Though this raises the question: if the Labor Party were unwilling to risk any delay to the legislation, why did they propose any amendments at all?

The answer relates to the difference between substantive amendments – like what the Greens senators proposed – and what could be called unsubstantial amendments.

Substantive versus ‘unsubstantial’ amendments

A substantive amendment would be one that is introduced in order to change the actual wording of the bill. This can mean replacing words, deleting words or adding new words (for example, adding a new clause). On the other hand, an ‘unsubstantial’ amendment wouldn’t make any changes to the bill itself but would instead change the wording of, for example, one of the usual procedural motions.

What are the usual procedural motions?

When a bill goes through Parliament, there are certain motions that must be agreed to. One of these is the second reading motion, which is:

That this bill be now read a second time.

Reading a bill for a second time is parliamentary jargon for agreeing with the main idea of the bill. If this motion is successful, the bill will then be considered in greater detail. If it is unsuccessful, then the bill is said to have failed and will no longer be considered by either house of parliament.

Amendments to procedural motions don’t affect the content of the bill. Instead, they can be used to express an opinion towards the bill. For example, an MP may propose to add the text: “but the House believes the Government is doing a very bad job.” Even if this proposed amendment is successful, it doesn’t actually change anything: the bill will still be read a second time and will continue its passage through the House. The only change is that future readers of the parliamentary record will be able to see that the majority of the House was unhappy with the Government at that time.

Last week’s amendments

With the exception of one substantive amendment in the House, all of the Labor Party’s proposed amendments were of the unsubstantial kind. That is, they proposed to add text to the end of the usual second reading motion but didn’t actually make any changes to the bills.  For example, Rankin MP Jim Chalmers (Labor) introduced the following unsubstantial motion:

That all words after “That” be omitted with a view to substituting the following words:

” whilst not declining to give these bills a second reading, the House:

(1) notes that this legislation gives the Treasurer extraordinary powers to include those not currently eligible for the JobKeeper Payment; and

(2) calls on the Treasurer to use his power under this legislation to ensure more jobs are protected and that struggling, otherwise viable businesses and organisations are able to access the JobKeeper Payment”.

This unsubstantial amendment was unsuccessful, but even if it had passed, it would not have made any actual change to the bills. Instead, future readers of the parliamentary record would simply be able to read about how the majority of the House wanted the Treasurer to use his powers – the Treasurer himself wouldn’t be bound to follow their advice.

The Labor Party’s one substantive amendment was moved by MP for Watson Tony Burke (Labor) during the Consideration in Detail stage, which is when the House debates the bill in greater detail. That amendment proposed to add new words to a clause in the bill but was unsuccessful because it did not have the support of the Coalition MPs.

In the Senate, the opposition senators often have the numbers to pass amendments even when all the government senators vote against them. However, if the amendment is substantive and changes the wording of a bill, it needs to go back to the House of Representatives for our MPs’ approval before it can become part of the bill.

This is where bills can become stuck: the government majority in the House will refuse to agree with the Senate’s amendments and the opposition majority in the Senate will refuse to accept the bill without those amendments.

Because the Government had already made clear that they would not be agreeing with any amendments in the House, the Labor Party senators decided not to attempt any substantive amendments in the Senate. They also chose not to support the Greens Party senators’ substantive amendments, which only received the support of senators from the Greens and Centre Alliance parties.

What next for Parliament?

Currently, Parliament will not sit again until 11 August 2020, due to concerns about spreading COVID-19. But according to ABC News:

… Mr Albanese [Leader of the Opposition] doesn’t accept that argument. He maintains that parliaments in other nations affected by coronavirus continue to meet and scrutinise their governments’ policies.

Whether the Opposition will succeed in convincing the Government to re-open Parliament remains to be seen.

UPDATE: The Prime Minister Scott Morrison has announced that there may be a trial week of sitting in May.

Posted in OpenAustralia.org, They Vote For You, Uncategorized | Tagged , , , , , | Leave a comment

Coronavirus and our Federal Parliament

By Mackay Ash | Published: March 20, 2020
Sitting calendar for the first half of 2020

Our federal parliament is scheduled to sit next week but, due to the Coronavirus pandemic, it’s going to look a bit different. On Wednesday, the Prime Minister released a media release explaining the type of changes we should expect. In a nut shell, our parliament is battening down the hatches and reducing its business to the bare minimum required to keep the country going.

What does this mean for representation?

For those of us with representatives who belong to the two major parties – the Coalition and the Australian Labor Party – there won’t be any significant changes in terms of how you are being represented in parliament. This is because your representatives are already voting according to how their party whip tells them to, except on those rare occasions when they cross the floor. So whether they are in parliament next week, or simply paired with an opposition member, the result will most likely remain the same.

For those of us with representatives who are independent or belong to a minor party, the question is a little trickier. The main reference to these representatives in the Prime Minister’s media release was that “All crossbench and independent Members and Senators will be able to attend [parliament], should they want to.” In other words, they will not be prevented from attending parliament, but the voting arrangements being made are mostly between the two major parties, leaving the independent and minor parties to their own devices. We will have to wait and see how many of these representatives are able to attend parliament next week.

What does this mean for bills that have not yet been passed?

The media release makes clear that parliament is only meeting to consider the bills relevant to their planned stimulus package and “any other immediate business”. Once that business is dealt with, the parliament will adjourn and our representatives will return home. This means that all bills that are not immediately relevant to the current crisis or required for the continued running of the country will be put off until a later date.

How can I keep up to date with our parliament?

The most up to date information is generally available via the House of Representative and Senate twitter accounts. Follow them for the latest news.

Posted in They Vote For You, Uncategorized | Tagged , , , , , , | Leave a comment

Who will you vote for this election? A party or a person?

By Mackay Ash | Published: May 3, 2019

They Vote For You is based on the principle of holding individual representatives to account. You can find your representative’s report card and see how your electorate’s voting power is being used in Parliament. If you’re unhappy with what you find, you can contact your representative and ask for an explanation. And if you’re very unhappy, you can vote against them in the upcoming federal election.

That’s how They Vote For You works: you look up your representative and you assess their individual performance. But does this person-based approach reflect reality?

Party politics

The vast majority of our MPs and senators belong to a party. And most belong to either the Australian Labor Party or to the Coalition (which is actually an alliance of parties). This means that when our representatives vote in Parliament, they vote along party lines – a process that is helped along by party whips who make sure their members vote the way their party wants them to vote.

On very rare occasions, our politicians rebel and cross the floor to vote with the other side. But this usually leads to the rebellious representative leaving the party, as was the case with former Liberal and now Independent MP for Chisolm Julia Banks.

This all begs the question: when we go to the polls on 18 May 2019, who are we voting for? The person? Or the party?

The answer no doubt depends on each voter themselves, as many of us have strong party affiliations whether or not we’re actually card-carrying members. Only you can say whether you’re ticking that box because you think Person X will best represent your electorate or because you like their party and want it to have the most voting power in Parliament.

And thanks to our party-based political system, both of these approaches to voting are valid.

But the important thing to remember is this: your representative is only in Parliament because your electorate put them there. And that means you outrank the party whip when it comes to deciding how they should vote on your behalf.

So after this election is over and the business of government resumes, remember to keep an eye on how your representative is voting for you on the issues you care about and, if you don’t like what you see, tell them about it. Because ultimately, they answer to you.

Posted in They Vote For You | Tagged , , , , , , , | Leave a comment

Do They Vote For You?

By Mackay Ash | Published: April 23, 2019

Comparing words and actions

The federal election is less than a month away and, with some exceptions, our MPs are hot on the campaign trail seeking re-election. Now’s the time to decide whether you’re happy with your local MP’s work in representing your electorate, or whether you think it’s time for a change.

How can They Vote For You help?

MPs seeking re-election will be making many claims about how they’ve stood up for their electorate during their time in parliament. They Vote For You lets you check their claims against their actual voting record.

For example, to check Barnaby Joyce’s voting records on water rights, you can enter his name into the search field on theyvoteforyou.org.au and be taken to his profile page.

From here you can search within the page for “water” and see that Barnaby Joyce has voted very strongly for “Making more water from Murray-Darling Basin available to use” and very strongly against “Increasing protection of Australia’s fresh water“. Clicking on those policy names takes you to a detailed list of all the times when this MP is on record as voting on divisions (or formal votes) which relate to this policy. We also include a detailed breakdown of how we calculate the scoring to determine a stance.

How accurate is this voting record?

They Vote For You is as accurate as the available parliamentary data allows us to be. Unfortunately, our parliament does not keep a record of how our representatives vote ‘on the voices,’ which is when our MPs stay in their seats and shout ‘Aye!’ and No!’ so that the loudest side wins. This is unfortunate, because most votes in parliament take place on the voices, leading to an incomplete voting record.

It is also important to keep in mind that politics is a messy beast. Even if the way an MP has voted in parliament doesn’t line up with how their electorate would want them to vote on one particular issue, that doesn’t mean they aren’t doing their job and representing their electorate. It’s often the case that politics involves compromise and tradeoffs – voting against the electorate’s desires on a smaller issue in order to be able to deliver a bigger win for the electorate on a more important issue. Those trade-offs and bigger pictures aren’t captured in the raw voting data, but we’d expect that any MP should be able to explain those things to their electorate.

Posted in They Vote For You | Leave a comment

A new era for morph.io

By James Polley | Published: April 10, 2019

When morph.io was built, we leveraged a lot of awesome existing work already in the open-source ecosystem. In particular, we were able to heavily lean on work down by the Herokuish project had done to create a platform that can run scripts written in a variety of languages. The Herokuish project, in turn, built on the cedar-14 stack which had been open sourced by Heroku; and that in turn was built on top of Ubuntu 14.04, which builds on countless contributions from people all over the world.

Five years later, the cedar-14 stack is nearing the end of its life. As of this month (April 2019) Canonical will no longer support Ubuntu 14.04; Heroku will no longer support cedar-14 – and morph.io will no longer be able to support the platform we’ve relied on for so long.

Instead, we’re going to be rolling out a new platform. We aren’t changing too much though – the components we’ve been using are working well – we’re simply upgrading them. The morph.io scraper platform will now be based on the current version of Herokuish, which is based on the Heroku-18 stack, which is based on Ubuntu 18.04, which is based on even more contributions from even more people around the world.

To help smooth the transition, we’ve added multi-platform support to Morph. The new platform is available now for you to start testing, and to start using if you find that it works for you.

We will be changing the default platform in early May. At that time, any scrapers which haven’t explicitly chosen to stay on the existing platform will start to run on the new platform. We’ll still keep the old platform around for a while longer to help anyone who didn’t manage to update their scraper before the change.

What is changing?

Supported language versions

Heroku-18 supports  more recent language versions than were usable on Cedar-14. However, this means that many older versions aren’t supported any more. If you were relying on an older version of your language, this is likely to mean you need to check for compatibility with the new version.

Heroku-18 currently supports Ruby 2.4.6, 2.5.5, and 2.6.2, as well as a wider list of Ruby versions availabe through JRuby. Python versions 2.7.16, 3.6.8, and 3.7.3 are currently supported. PHP is currently supported for version 7.1, 7.2, and 7.3. A full list of the supported versions for every language is available here.

If your scraper specifies a different language version, it’s possible that Herokuish may still be able to install and run that version for you. However, we suggest that you upgrade to the supported versions wherever possible.

Operating system packages

The shift from Ubuntu 14.04 to Ubuntu 18.04 is a large one. Many of the packages on the base platform have been upgraded to a much newer version. However, in some cases, packages have been replaced by a newer equivalent package, or have gone away completely. In some cases, new packages have been added that add capabilities that weren’t present before.

For the most part, we expect that you probably won’t need to pay attention to the OS package versions. However, the full list of packages on both cedar-14 and heroku-18 is available from Heroku’s support site.

What do I need to do?

Choosing a platform

You can choose which platform you want to use by including a file named platform in the top level of your scraper’s repository. The contents of the file should match one of the tags in our buildstep container registry. Currently, the available tags are early_release for testing the new platform, and cedar-14 to keep your scraper using the old platform. If no platform file is found, we’ll use the default release.

You can see an example of a scraper being configured for the new platform here.

Learning more and reporting problems

We’ll be posting updates about the new platform on help.morph.io as we get closer to making the new platform the default.

If you find bugs or run into problems on the new platform, please let us know on help.morph.io/.

Posted in Announcement, Morph | Leave a comment