Category Archives: Morph

A new era for morph.io

When morph.io was built, we leveraged a lot of awesome existing work already in the open-source ecosystem. In particular, we were able to heavily lean on work down by the Herokuish project had done to create a platform that can run scripts written in a variety of languages. The Herokuish project, in turn, built on […]

Also posted in Announcement | Leave a comment

Matthew’s back: Improving the stability and scalability of the morph.io platform

I’m very happy to be back at the Foundation for a little stint now to get my hands dirty on some coding, something I haven’t had any opportunity to do over the last year. It’s funny how you can miss these things! As you might have noticed, I’ve been away from working on morph.io and […]

Also posted in Announcement, Development | 1 Response

Ruby web scraping tutorial on morph.io – Part 5, saving your data & running it on morph.io

This post is part of a series of posts that provide step-by-step instructions on how to write a simple web scraper using Ruby on morph.io. If you find any problems, let us know in the comments so we can improve these tutorials. In the last post we dealt with the site’s pagination and started scraping […]

Posted in Morph | Tagged , , | 2 Responses

Ruby web scraping tutorial on morph.io – Part 4, dealing with pagination

This post is part of a series of posts that provide step-by-step instructions on how to write a simple web scraper using Ruby on morph.io. If you find any problems, let us know in the comments so we can improve these tutorials. In the last post we finished collecting the data we want but discovered […]

Posted in Morph | Tagged , , | Leave a comment

Ruby web scraping tutorial on morph.io – Part 3, continue writing your scraper

This post is part of a series of posts that provide step-by-step instructions on how to write a simple web scraper using Ruby on morph.io. If you find any problems, let us know in the comments so we can improve these tutorials. In the last post we started writing our scraper and gathering some data. […]

Posted in Morph | Tagged , , | Leave a comment

Ruby web scraping tutorial on morph.io – Part 2, start writing your scraper

This post is part of a series of posts that provide step-by-step instructions on how to write a simple web scraper using Ruby on morph.io. If you find any problems, let us know in the comments so we can improve these tutorials. In the past post we set up our scraper. Now we’re going to […]

Posted in Morph | Tagged , , | Leave a comment

Ruby web scraping tutorial on morph.io – Part 1, setting up your scraper

This post is part of a series of posts that provide step-by-step instructions on how to write a simple web scraper using Ruby on morph.io. If you find any problems, let us know in the comments so we can improve these tutorials.   With just a few lines of code, you can write a scraper […]

Posted in Morph | Tagged , , | Leave a comment

The next 3 months are going to be really busy – here is our plan

Since the beginning of this year the core team at the OpenAustralia Foundation has been getting together for a day every quarter to make a plan for the upcoming 3 months. As our team grows (we’re now 3 full time people) these kinds of occasional but regular planning sessions help to keep us working well […]

Also posted in OpenAustralia Foundation, Planning, PlanningAlerts.org.au, RightToKnow.org.au | 1 Response

A little scraping goes a long way

Last night, about 10 of us got together in Sydney for a fun night of scraping and learning about morph.io. I organised the get together because I’m just really excited about writing scrapers and using data from morph.io at the moment. I’ve only been writing scrapers for the last few months as Matthew and I […]

Also posted in Event | Tagged , , , | Leave a comment

Scraping javascript sites with morph.io

Just a quick post to let you know that it’s now possible to scrape javascript heavy sites easily using our scraping platform morph.io. This is really useful with Microsoft .NET web sites that often use complicated states stored in javascript with links simulated via javascript posts. Also, we recently discovered another more worrying example. The […]

Also posted in Announcement | Leave a comment
  • Occasional News

    Stay in the loop with occasional news and notes from the OpenAustralia Foundation in your inbox.

  • Categories

  • Archives

    • [+]2019
    • [+]2018
    • [+]2017
    • [+]2016
    • [+]2015
    • [+]2014
    • [+]2013
    • [+]2012
    • [+]2011
    • [+]2010
    • [+]2009
    • [+]2008
    • [+]2007