made generic OAIC format table scraper class
made generic OAIC format table scraper class


Former-commit-id: bf7c06cfe5fa5d8877f7ec1c99c26a9cc6ad51ff

[submodule "couchdb/couchdb-lucene"] [submodule "couchdb/couchdb-lucene"]
path = couchdb/couchdb-lucene path = couchdb/couchdb-lucene
url = https://github.com/rnewson/couchdb-lucene.git url = https://github.com/rnewson/couchdb-lucene.git
[submodule "couchdb/settee"] [submodule "couchdb/settee"]
path = couchdb/settee path = couchdb/settee
url = https://github.com/inadarei/settee.git url = https://github.com/inadarei/settee.git
[submodule "lib/springy"]  
path = lib/springy  
url = https://github.com/dhotson/springy.git  
[submodule "lib/php-diff"] [submodule "lib/php-diff"]
path = lib/php-diff path = lib/php-diff
url = https://github.com/chrisboulton/php-diff.git url = https://github.com/chrisboulton/php-diff.git
  [submodule "lib/Requests"]
  path = lib/Requests
  url = https://github.com/rmccue/Requests.git
  [submodule "js/flotr2"]
  path = js/flotr2
  url = https://github.com/HumbleSoftware/Flotr2.git
  [submodule "lib/phpquery"]
  path = lib/phpquery
  url = https://github.com/TobiaszCudnik/phpquery.git
  [submodule "js/sigma"]
  path = js/sigma
  url = https://github.com/jacomyal/sigma.js.git
  [submodule "js/bubbletree"]
  path = js/bubbletree
  url = https://github.com/okfn/bubbletree.git
  [submodule "lib/querypath"]
  path = lib/querypath
  url = https://github.com/technosophos/querypath.git
  [submodule "lib/amon-php"]
  path = lib/amon-php
  url = https://github.com/martinrusev/amon-php.git
   
file:a/about.php -> file:b/about.php
<?php <?php
include_once('include/common.inc.php'); include_once('include/common.inc.php');
include_header(); include_header('About');
?> ?>
<div class="foundation-header"> <div class="foundation-header">
<h1><a href="about.php">About/FAQ</a></h1> <h1><a href="about.php">About/FAQ</a></h1>
<h4 class="subheader">Lorem ipsum.</h4> <h4 class="subheader">Lorem ipsum.</h4>
</div> </div>
<h2> What is this? </h2> <h2> What is this? </h2>
Disclosr is a project to monitor Australian Federal Government agencies Disclo.gs is a project to monitor Australian Federal Government agencies
compliance with their <a href="http://www.oaic.gov.au/publications/other_operational/foi_policy_frequently_asked_questions.html#_Toc291837571">"proactive disclosure requirements"</a>. compliance with their <a href="http://www.oaic.gov.au/publications/other_operational/foi_policy_frequently_asked_questions.html#_Toc291837571">"proactive disclosure requirements" to make a transparency league table as suggested by gov2 taskforce http://gov2.net.au/blog/2009/09/19/a-league-ladder-of-psi-openness/</a>.
OGRE (Open Government Realization Evaluation) is a ranking of compliance with these requirements. <h2> Attributions </h2>
Prometheus is the agent which polls agency websites to assess compliance. National Archives of Australia, Australian Governments’ Interactive Functions Thesaurus, 2nd edition, September 2005, published at http://www.naa.gov.au/recordkeeping/thesaurus/index.htm <br/>
  data.gov.au http://data.gov.au/dataset/directory-gov-au-full-data-export/ <br/>
  directory.gov.au <br/>
  australia.gov.au http://australia.gov.au/about/copyright <br/>
<h2> Open everything </h2> <h2> Open everything </h2>
all documents released CC-BY 3 AU All documents released CC-BY 3 AU
Open source git @ Open source git @
   
<h2>Organisational Data Sources</h2> <h2>Organisational Data Sources</h2>
   
http://www.comlaw.gov.au/Browse/Results/ByTitle/AdministrativeArrangementsOrders/Current/Ad/0 defines departments http://www.comlaw.gov.au/Browse/Results/ByTitle/AdministrativeArrangementsOrders/Current/Ad/0 defines departments
Agencies can be found in the Schedule to an Appropriation Bill (budget), Schedule to FMA Regulations and/or Public Service Act. Agencies can be found in the Schedule to an Appropriation Bill (budget), Schedule to FMA Regulations and/or Public Service Act.<br>
   
http://www.finance.gov.au/publications/flipchart/docs/FMACACFlipchart.pdf summarises these. view-source:https://www.tenders.gov.au/?event=public.advancedsearch.home is great for the suspended/active status http://www.finance.gov.au/publications/flipchart/docs/FMACACFlipchart.pdf summarises these. view-source:https://www.tenders.gov.au/?event=public.advancedsearch.home is great for the suspended/active status<br>
   
  Fraud in gov depts by Fairfax Media http://www.smh.com.au/national/public-service-keeps-fraud-cases-private-20110923-1kpdr.html
   
When defining the hierachy, this system is designed towards monitoring accountablity. Thus large agencies that have registered their own ABN When defining the hierachy, this system is designed towards monitoring accountablity. Thus large agencies that have registered their own ABN
and have their own accountablity mechanisms/website receive a seperate record as a child of their department. and have their own accountablity mechanisms/website receive a seperate record as a child of their department.
Some small agencies will choose to simply rely on their parent department's accountablity measures. Some small agencies will choose to simply rely on their parent department's accountablity measures.<br>
   
This flows through to organisation name and other/past names. A department that completely accounts for an agency will list that agency as an other child name. This flows through to organisation name and other/past names. A department that completely accounts for an agency will list that agency as an other child name.
As agencies themselves shift between departments, there may be scope for providing time ranges but typically the newest hierarchy will be the one recorded. As agencies themselves shift between departments, there may be scope for providing time ranges but typically the newest hierarchy will be the one recorded.
A department/agency name will be the newest active name assigned to that ABN. A department/agency name will be the newest active name assigned to that ABN.<br>
   
ABN information is derived from the ABR. This is the definitive umpire about which former name should be linked to which current name. ABN information is derived from the ABR. This is the definitive umpire about which former name should be linked to which current name.
For example "Department of Transport and Regional Services" became "Department of Infrastructure, Transport, Regional Development and Local Government" (same ABN) For example "Department of Transport and Regional Services" became "Department of Infrastructure, Transport, Regional Development and Local Government" (same ABN)
however it later split into "Department of Infrastructure and Transport" (same ABN) however it later split into "Department of Infrastructure and Transport" (same ABN)
and "Department of Regional Australia, Regional Development and Local Government" (new ABN). and "Department of Regional Australia, Regional Development and Local Government" (new ABN).<br>
   
Statistical information from http://www.apsc.gov.au/stateoftheservice/1011/statsbulletin/section1.html#t2total https://www.apsedii.gov.au/apsedii/CustomQueryx33.shtml Statistical information from http://www.apsc.gov.au/stateoftheservice/1011/statsbulletin/section1.html#t2total https://www.apsedii.gov.au/apsedii/CustomQueryx33.shtml
and individual annual reports. and individual annual reports.<br>
   
Webpage Assessment <h2>Webpage Assessment</h2>
Much due care has been put into correctly recording disclosure URLs. Typically the "About", "Corporate", "Publications" and "Sitemap" sections are checked at the very least. Much due care has been put into correctly recording disclosure URLs. Typically the "About", "Corporate", "Publications" and "Sitemap" sections are checked at the very least.
Occasionally it is nessicary to use a site or Google search. In several rare cases, there is a secret "Disclosure" navigation menu you can find if you find one of the mandatory publishing obligations in that category (seriously). Occasionally it is nessicary to use a site or Google search. In several rare cases, there is a secret "Disclosure" navigation menu you can find if you find one of the mandatory publishing obligations in that category (seriously).<br>
Some rules about leniency: Some rules about leniency:<br>
An empty FOI disclosure log counts, a page outlining what the FOI Act is does not. <ul>
A disclosure log in PDF or Word format counts :( <li>An empty FOI disclosure log counts, a page outlining what the FOI Act is does not.</li>
An empty File/Record list counts (although that's very minimalistic that you have no files, electronic or paper) <li>A disclosure log in PDF or Word format counts :(</li>
Only a current information publication scheme page counts, not a s.9 FOI Act page or an organisation chart. <li>An empty File/Record list counts (although that's very minimalistic that you have no files, electronic or paper)</li>
If there isn't a page easily listing all current and past Annual Reports, the most current one (html, pdf) counts. <li>Only a current information publication scheme page counts, not a s.9 FOI Act page or an organisation chart.</li>
Consultancy contracts might not need it's own webpage (if in Annual Report), grants/appointments might not apply to all organisations but Legal Services Expenditure (and all other obligations) does need a webpage. <li>If there isn't a page easily listing all current and past Annual Reports, the most current one (html, pdf) counts.</li>
  <li>Consultancy contracts might not need it's own webpage (if in Annual Report), grants/appointments might not apply to all organisations but Legal Services Expenditure (and all other obligations) does need a webpage. </li>
   
<h2>Open Government Scoring</h2> <h2>Open Government Scoring</h2>
+1 point for every true Has... attribute +1 point for every true Has... attribute<br>
-1 point for every false Has... (ie. Has Not) attribute -1 point for every false Has... (ie. Has Not) attribute</br>
   
Don't like this? Make your own score, suggest a better scoring mechanism. Don't like this? Make your own score, suggest a better scoring mechanism.</br>
   
<?php <?php
include_footer(); include_footer();
?> ?>
  "Agriculture, Fisheries & Forestry",3530
  Attorney-Generals,708
  Administrative Appeals Tribunal,108
  AUSTRAC,60
  Australian Crime Commission,225
  Australian Customs Service,4934
  Australian Protective Service,1124
  CrimTrac Agency,34
  Family Court of Australia,574
  Federal Court of Australia,221
  Federal Magistrates Service,46
  Federal Privacy Commissioner,32
  HREOC,79
  ITSA,227
  National Native Title Tribunal,241
  Office of the D.P.P.,403
  Office of Film & Literature Class,39
  Office of the Parl. Counsel,45
  Communications IT & the Arts,776
  Aust. Broadcasting Authority,114
  Aust. Communications Authority,391
  Aust. National Maritime Museum,90
  National Archives of Australia,363
  National Library of Australia,443
  National Museum of Australia,185
  NOIE,160
  Defence,18231
  Defence Housing Authority,661
  DEST,1464
  Australian Research Council,50
  DEWR,2055
  EOWA,11
  Australian Industrial Registry,185
  Comcare,300
  NOHSC,89
  Environment and Heritage,1271
  Australian Greenhouse Office,167
  Bureau of Meteorology,1403
  GBRMPA,153
  National Oceans Office,31
  Family and Community Services,4832
  Aust. Institute of Family Studies,39
  Centrelink,25147
  Finance and Administration,771
  Australian Electoral Comm.,729
  ComSuper,338
  Foreign Affairs & Trade,2763
  ACIAR,44
  Health & Ageing,5137
  Aust. Institute of Health & Welfare,152
  ARPANSA,126
  FSANZ,110
  PSR,25
  DIMIA,4089
  Aboriginal Hostels Ltd.,341
  ATSIC,1072
  AIATSIS,62
  Torres Strait Regional Authority,30
  Industry Tourism & Resources,2691
  Prime Minister & Cabinet,344
  ANAO,278
  Commonwealth Ombudsman,78
  Office of National Assessments,47
  APS Commission,164
  Transport & Regional Services,945
  National Capital Authority,76
  Treasury,906
  Australian Bureau of Statistics,3024
  ACCC,446
  ASIC,1048
  Australian Taxation Office,20238
  Productivity Commission,197
  Veterans Affairs,2335
  Australian War Memorial,215
 
  "Agriculture, Fisheries and Forestry",3928
  Attorney-General's,781
  Administrative Appeals Tribunal,117
  AUSTRAC,61
  Australian Crime Commission,315
  Australian Customs Service,4736
  Australian Protective Service,1234
  CrimTrac Agency,44
  Family Court of Australia,608
  Federal Court of Australia,238
  Federal Magistrates Service,56
  Federal Privacy Commissioner,29
  HREOC,84
  ITSA,240
  National Native Title Tribunal,243
  Office of the D.P.P.,416
  Office of Film & Literature Class.,37
  Office of Parliamentary Counsel,46
  Communications IT & the Arts,607
  Aust. Broadcasting Authority,114
  Aust. Communications Authority,407
  Australian Film Commission,176
  AGIMO,95
  Aust. National Maritime Museum,93
  National Archives of Australia,386
  National Library of Australia,438
  National Museum of Australia,186
  Defence,18301
  Defence Housing Authority,679
  DEST,1684
  Australian Research Council,53
  DEWR,2171
  Australian Industrial Registry,188
  Comcare,325
  EOWA,10
  NOHSC,89
  Environment and Heritage,1276
  Australian Greenhouse Office,166
  Bureau of Meteorology,1371
  GBRMPA,162
  National Oceans Office,39
  Family and Community Services,4818
  Aust. Institute of Family Studies,38
  Centrelink,24904
  Finance and Administration,895
  Australian Electoral Comm.,765
  ComSuper,329
  Foreign Affairs and Trade,2809
  ACIAR,40
  Health and Ageing,5365