more scrapers
more scrapers


Former-commit-id: b3ae23479c9f8a0c79c9181bdd1ce0f5b3ea0656

[submodule "couchdb/couchdb-lucene"] [submodule "couchdb/couchdb-lucene"]
path = couchdb/couchdb-lucene path = couchdb/couchdb-lucene
url = https://github.com/rnewson/couchdb-lucene.git url = https://github.com/rnewson/couchdb-lucene.git
[submodule "couchdb/settee"] [submodule "couchdb/settee"]
path = couchdb/settee path = couchdb/settee
url = https://github.com/inadarei/settee.git url = https://github.com/inadarei/settee.git
[submodule "lib/php-diff"] [submodule "lib/php-diff"]
path = lib/php-diff path = lib/php-diff
url = https://github.com/chrisboulton/php-diff.git url = https://github.com/chrisboulton/php-diff.git
[submodule "lib/Requests"] [submodule "lib/Requests"]
path = lib/Requests path = lib/Requests
url = https://github.com/rmccue/Requests.git url = https://github.com/rmccue/Requests.git
[submodule "javascripts/flotr2"] [submodule "js/flotr2"]
path = javascripts/flotr2 path = js/flotr2
url = https://github.com/HumbleSoftware/Flotr2.git url = https://github.com/HumbleSoftware/Flotr2.git
[submodule "lib/phpquery"] [submodule "lib/phpquery"]
path = lib/phpquery path = lib/phpquery
url = https://github.com/TobiaszCudnik/phpquery.git url = https://github.com/TobiaszCudnik/phpquery.git
[submodule "javascripts/sigma"] [submodule "js/sigma"]
path = javascripts/sigma path = js/sigma
url = https://github.com/jacomyal/sigma.js.git url = https://github.com/jacomyal/sigma.js.git
  [submodule "js/bubbletree"]
  path = js/bubbletree
  url = https://github.com/okfn/bubbletree.git
  [submodule "lib/querypath"]
  path = lib/querypath
  url = https://github.com/technosophos/querypath.git
  [submodule "lib/amon-php"]
  path = lib/amon-php
  url = https://github.com/martinrusev/amon-php.git
  [submodule "documents/lib/parsedatetime"]
  path = documents/lib/parsedatetime
  url = git://github.com/bear/parsedatetime.git
   
file:a/about.php -> file:b/about.php
<?php <?php
include_once('include/common.inc.php'); include_once('include/common.inc.php');
include_header(); include_header('About');
?> ?>
<div class="foundation-header"> <div class="foundation-header">
<h1><a href="about.php">About/FAQ</a></h1> <h1><a href="about.php">About/FAQ</a></h1>
<h4 class="subheader">Lorem ipsum.</h4> <h4 class="subheader">Lorem ipsum.</h4>
</div> </div>
<h2> What is this? </h2> <h2> What is this? </h2>
Disclosr is a project to monitor Australian Federal Government agencies Disclo.gs is a project to monitor Australian Federal Government agencies
compliance with their <a href="http://www.oaic.gov.au/publications/other_operational/foi_policy_frequently_asked_questions.html#_Toc291837571">"proactive disclosure requirements"</a>. compliance with their <a href="http://www.oaic.gov.au/publications/other_operational/foi_policy_frequently_asked_questions.html#_Toc291837571">"proactive disclosure requirements" to make a transparency league table as suggested by gov2 taskforce http://gov2.net.au/blog/2009/09/19/a-league-ladder-of-psi-openness/</a>.
OGRE (Open Government Realization Evaluation) is a ranking of compliance with these requirements. <h2> Attributions </h2>
Prometheus is the agent which polls agency websites to assess compliance. National Archives of Australia, Australian Governments’ Interactive Functions Thesaurus, 2nd edition, September 2005, published at http://www.naa.gov.au/recordkeeping/thesaurus/index.htm <br/>
  data.gov.au http://data.gov.au/dataset/directory-gov-au-full-data-export/ <br/>
  directory.gov.au <br/>
  australia.gov.au http://australia.gov.au/about/copyright <br/>
<h2> Open everything </h2> <h2> Open everything </h2>
All documents released CC-BY 3 AU All documents released CC-BY 3 AU
Open source git @ Open source git @
   
<h2>Organisational Data Sources</h2> <h2>Organisational Data Sources</h2>
   
http://www.comlaw.gov.au/Browse/Results/ByTitle/AdministrativeArrangementsOrders/Current/Ad/0 defines departments http://www.comlaw.gov.au/Browse/Results/ByTitle/AdministrativeArrangementsOrders/Current/Ad/0 defines departments
Agencies can be found in the Schedule to an Appropriation Bill (budget), Schedule to FMA Regulations and/or Public Service Act.<br> Agencies can be found in the Schedule to an Appropriation Bill (budget), Schedule to FMA Regulations and/or Public Service Act.<br>
   
http://www.finance.gov.au/publications/flipchart/docs/FMACACFlipchart.pdf summarises these. view-source:https://www.tenders.gov.au/?event=public.advancedsearch.home is great for the suspended/active status<br> http://www.finance.gov.au/publications/flipchart/docs/FMACACFlipchart.pdf summarises these. view-source:https://www.tenders.gov.au/?event=public.advancedsearch.home is great for the suspended/active status<br>
   
Fraud in gov depts by Fairfax Media http://www.smh.com.au/national/public-service-keeps-fraud-cases-private-20110923-1kpdr.html Fraud in gov depts by Fairfax Media http://www.smh.com.au/national/public-service-keeps-fraud-cases-private-20110923-1kpdr.html
   
When defining the hierachy, this system is designed towards monitoring accountablity. Thus large agencies that have registered their own ABN When defining the hierachy, this system is designed towards monitoring accountablity. Thus large agencies that have registered their own ABN
and have their own accountablity mechanisms/website receive a seperate record as a child of their department. and have their own accountablity mechanisms/website receive a seperate record as a child of their department.
Some small agencies will choose to simply rely on their parent department's accountablity measures.<br> Some small agencies will choose to simply rely on their parent department's accountablity measures.<br>
   
This flows through to organisation name and other/past names. A department that completely accounts for an agency will list that agency as an other child name. This flows through to organisation name and other/past names. A department that completely accounts for an agency will list that agency as an other child name.
As agencies themselves shift between departments, there may be scope for providing time ranges but typically the newest hierarchy will be the one recorded. As agencies themselves shift between departments, there may be scope for providing time ranges but typically the newest hierarchy will be the one recorded.
A department/agency name will be the newest active name assigned to that ABN.<br> A department/agency name will be the newest active name assigned to that ABN.<br>
   
ABN information is derived from the ABR. This is the definitive umpire about which former name should be linked to which current name. ABN information is derived from the ABR. This is the definitive umpire about which former name should be linked to which current name.
For example "Department of Transport and Regional Services" became "Department of Infrastructure, Transport, Regional Development and Local Government" (same ABN) For example "Department of Transport and Regional Services" became "Department of Infrastructure, Transport, Regional Development and Local Government" (same ABN)
however it later split into "Department of Infrastructure and Transport" (same ABN) however it later split into "Department of Infrastructure and Transport" (same ABN)
and "Department of Regional Australia, Regional Development and Local Government" (new ABN).<br> and "Department of Regional Australia, Regional Development and Local Government" (new ABN).<br>
   
Statistical information from http://www.apsc.gov.au/stateoftheservice/1011/statsbulletin/section1.html#t2total https://www.apsedii.gov.au/apsedii/CustomQueryx33.shtml Statistical information from http://www.apsc.gov.au/stateoftheservice/1011/statsbulletin/section1.html#t2total https://www.apsedii.gov.au/apsedii/CustomQueryx33.shtml
and individual annual reports.<br> and individual annual reports.<br>
   
<h2>Webpage Assessment</h2> <h2>Webpage Assessment</h2>
Much due care has been put into correctly recording disclosure URLs. Typically the "About", "Corporate", "Publications" and "Sitemap" sections are checked at the very least. Much due care has been put into correctly recording disclosure URLs. Typically the "About", "Corporate", "Publications" and "Sitemap" sections are checked at the very least.
Occasionally it is nessicary to use a site or Google search. In several rare cases, there is a secret "Disclosure" navigation menu you can find if you find one of the mandatory publishing obligations in that category (seriously).<br> Occasionally it is nessicary to use a site or Google search. In several rare cases, there is a secret "Disclosure" navigation menu you can find if you find one of the mandatory publishing obligations in that category (seriously).<br>
Some rules about leniency:<br> Some rules about leniency:<br>
<ul> <ul>
<li>An empty FOI disclosure log counts, a page outlining what the FOI Act is does not.</li> <li>An empty FOI disclosure log counts, a page outlining what the FOI Act is does not.</li>
<li>A disclosure log in PDF or Word format counts :(</li> <li>A disclosure log in PDF or Word format counts :(</li>
<li>An empty File/Record list counts (although that's very minimalistic that you have no files, electronic or paper)</li> <li>An empty File/Record list counts (although that's very minimalistic that you have no files, electronic or paper)</li>
<li>Only a current information publication scheme page counts, not a s.9 FOI Act page or an organisation chart.</li> <li>Only a current information publication scheme page counts, not a s.9 FOI Act page or an organisation chart.</li>
<li>If there isn't a page easily listing all current and past Annual Reports, the most current one (html, pdf) counts.</li> <li>If there isn't a page easily listing all current and past Annual Reports, the most current one (html, pdf) counts.</li>
<li>Consultancy contracts might not need it's own webpage (if in Annual Report), grants/appointments might not apply to all organisations but Legal Services Expenditure (and all other obligations) does need a webpage. </li> <li>Consultancy contracts might not need it's own webpage (if in Annual Report), grants/appointments might not apply to all organisations but Legal Services Expenditure (and all other obligations) does need a webpage. </li>
   
<h2>Open Government Scoring</h2> <h2>Open Government Scoring</h2>
+1 point for every true Has... attribute<br> +1 point for every true Has... attribute<br>
-1 point for every false Has... (ie. Has Not) attribute</br> -1 point for every false Has... (ie. Has Not) attribute</br>
   
Don't like this? Make your own score, suggest a better scoring mechanism.</br> Don't like this? Make your own score, suggest a better scoring mechanism.</br>
   
<?php <?php
include_footer(); include_footer();
?> ?>
  Report year,Quarter,Agency,Is agency active,Administrating agency name,Portfolio,Nil return,Last return by agency,Date created,Requests for other information received from applicant,Requests for personal information received from applicant,Requests for other information on transfer,Requests for personal information on transfer,Requests for other information on hand 1 July 2012 ,Requests for personal information on hand 1 July 2012,Requests for other information granted in full,Requests for personal information granted in full,Requests for other information granted in part,Requests for personal information granted in part,Requests for other information access refused,Requests for personal information access refused,Requests for other information transferred,Requests for personal information transferred,Requests for other information withdrawn,Requests for personal information withdrawn,Response time within statutory time period for requests for other information ,Response time within statutory time period for requests for personal information ,Response time up to 30 days after statutory period for requests for other information,Response time up to 30 days after statutory period for requests for personal information,Response time 31 to 60 days after statutory period for requests for other information,Response time 31 to 60 days after statutory period for requests for personal information,Response time 61 to 90 days after statutory period for requests for other information,Response time 61 to 90 days after statutory period for requests for personal information,Response time over 90 days after statutory period for requests for other information,Response time over 90 days after statutory period for requests for personal information,Charges notified for requests for other information AUD,Charges notified for requests for personal information AUD,Total charges notified for requests for other information AUD,Total charges notified for requests for personal information AUD,Total charges collected for requests for other information AUD,Total charges collected for requests for personal information AUD,Internal review applications for requests for other information,Internal review applications for requests for personal information,Internal review decisions affirmed for requests for other information,Internal review decisions affirmed for requests for personal information,Internal review access granted in full for requests for other information,Internal review access granted in full for requests for for personal information,Internal review access granted in part for requests for other information,Internal review access granted in part for requests for other information,Internal review access granted after deferment for requests for other information,Internal review access granted after deferment for requests for personal information,Internal review access granted in another form for request for other information,Internal review access granted in another form for request for personal information,Internal review charges reduced for requests for other information,Internal review charges reduced for requests for personal information,Internal review of request for other information withdrawn without concession,Internal review of request for personal information withdrawn without concession,s 48 requests received,s 48 response time within statutory period,s 48 response time up to 30 days after statutory period,s 48 response time over 30 days after statutory period,s 48 internal review applications,s 48 request granted record altered,s 48 request granted record notated,s 48 request granted record altered and notated,s 48 requests refused,s 48 internal review application granted record altered,s 48 internal review application granted record notated,s 48 internal review application granted record altered and notated,s 48 internal review applications refused,Practical refusal written notification of intention to refuse a request for personal information,Practical refusal written notification of intention to refuse a request for other information,Practical refusal request for personal information subsequently processed following notification,Practical refusal request for other information subsequently processed following notification,Exemptions not claimed for requests for personal information,Exemptions not claimed for requests for other information,"Exemption documents affecting national security, defence or international relations for requests for personal information","Exemption documents affecting national security, defence or international relations for requests for other information",Exemption cabinet documents for requests for personal information,Exemption cabinet documents for requests for other information,Exemption documents affecting enforcement of law and protection of public safety requests for personal information,Exemption documents affecting enforcement of law and protection of public safety requests for other information,Exemption documents to which secrecy provisions of enactments apply for requests for personal information,Exemption documents to which secrecy provisions of enactments apply for requests for other information,Exemption documents subject to legal professional privilege for requests for personal information,Exemption documents subject to legal professional privilege for requests for other information,Exemption for documents containing material obtained in confidence for requests for personal information,Exemption for documents containing material obtained in confidence for requests for other information,Exemption documents disclosure of which would be contempt of Parliament or contempt of court for requests for personal information,Exemption documents disclosure of which would be contempt of Parliament or contempt of court for requests for other information,Exemption documents disclosing trade secrets or commercially valuable information for requests for personal information,Exemption documents disclosing trade secrets or commercially valuable information for requests for other information,Exemption electoral rolls and related documents for requests for personal