more scrapers, fix column order, extract title from lengthy description
more scrapers, fix column order, extract title from lengthy description


Former-commit-id: 765a9aab684bd300c60e5c698b8df43404db1d32

file:b/.gitignore (new)
  /nbproject/private/
[submodule "sag"] [submodule "couchdb/couchdb-lucene"]
path = sag path = couchdb/couchdb-lucene
url = https://github.com/sbisbee/sag.git url = https://github.com/rnewson/couchdb-lucene.git
  [submodule "couchdb/settee"]
  path = couchdb/settee
  url = https://github.com/inadarei/settee.git
  [submodule "lib/php-diff"]
  path = lib/php-diff
  url = https://github.com/chrisboulton/php-diff.git
  [submodule "lib/Requests"]
  path = lib/Requests
  url = https://github.com/rmccue/Requests.git
  [submodule "js/flotr2"]
  path = js/flotr2
  url = https://github.com/HumbleSoftware/Flotr2.git
  [submodule "lib/phpquery"]
  path = lib/phpquery
  url = https://github.com/TobiaszCudnik/phpquery.git
  [submodule "js/sigma"]
  path = js/sigma
  url = https://github.com/jacomyal/sigma.js.git
  [submodule "js/bubbletree"]
  path = js/bubbletree
  url = https://github.com/okfn/bubbletree.git
  [submodule "lib/querypath"]
  path = lib/querypath
  url = https://github.com/technosophos/querypath.git
  [submodule "lib/amon-php"]
  path = lib/amon-php
  url = https://github.com/martinrusev/amon-php.git
  [submodule "documents/lib/parsedatetime"]
  path = documents/lib/parsedatetime
  url = git://github.com/bear/parsedatetime.git
   
file:b/about.php (new)
  <?php
  include_once('include/common.inc.php');
  include_header('About');
  ?>
  <div class="foundation-header">
  <h1><a href="about.php">About/FAQ</a></h1>
  <h4 class="subheader">Lorem ipsum.</h4>
  </div>
  <h2> What is this? </h2>
  Disclo.gs is a project to monitor Australian Federal Government agencies
  compliance with their <a href="http://www.oaic.gov.au/publications/other_operational/foi_policy_frequently_asked_questions.html#_Toc291837571">"proactive disclosure requirements" to make a transparency league table as suggested by gov2 taskforce http://gov2.net.au/blog/2009/09/19/a-league-ladder-of-psi-openness/</a>.
  <h2> Attributions </h2>
  National Archives of Australia, Australian Governments’ Interactive Functions Thesaurus, 2nd edition, September 2005, published at http://www.naa.gov.au/recordkeeping/thesaurus/index.htm <br/>
  data.gov.au http://data.gov.au/dataset/directory-gov-au-full-data-export/ <br/>
  directory.gov.au <br/>
  australia.gov.au http://australia.gov.au/about/copyright <br/>
  <h2> Open everything </h2>
  All documents released CC-BY 3 AU
  Open source git @
 
  <h2>Organisational Data Sources</h2>
 
  http://www.comlaw.gov.au/Browse/Results/ByTitle/AdministrativeArrangementsOrders/Current/Ad/0 defines departments
  Agencies can be found in the Schedule to an Appropriation Bill (budget), Schedule to FMA Regulations and/or Public Service Act.<br>
 
  http://www.finance.gov.au/publications/flipchart/docs/FMACACFlipchart.pdf summarises these. view-source:https://www.tenders.gov.au/?event=public.advancedsearch.home is great for the suspended/active status<br>
 
  Fraud in gov depts by Fairfax Media http://www.smh.com.au/national/public-service-keeps-fraud-cases-private-20110923-1kpdr.html
 
  When defining the hierachy, this system is designed towards monitoring accountablity. Thus large agencies that have registered their own ABN
  and have their own accountablity mechanisms/website receive a seperate record as a child of their department.
  Some small agencies will choose to simply rely on their parent department's accountablity measures.<br>
 
  This flows through to organisation name and other/past names. A department that completely accounts for an agency will list that agency as an other child name.
  As agencies themselves shift between departments, there may be scope for providing time ranges but typically the newest hierarchy will be the one recorded.
  A department/agency name will be the newest active name assigned to that ABN.<br>
 
  ABN information is derived from the ABR. This is the definitive umpire about which former name should be linked to which current name.
  For example "Department of Transport and Regional Services" became "Department of Infrastructure, Transport, Regional Development and Local Government" (same ABN)
  however it later split into "Department of Infrastructure and Transport" (same ABN)
  and "Department of Regional Australia, Regional Development and Local Government" (new ABN).<br>
 
  Statistical information from http://www.apsc.gov.au/stateoftheservice/1011/statsbulletin/section1.html#t2total https://www.apsedii.gov.au/apsedii/CustomQueryx33.shtml
  and individual annual reports.<br>
 
  <h2>Webpage Assessment</h2>
  Much due care has been put into correctly recording disclosure URLs. Typically the "About", "Co