gitphp 0.2.9.1 :: ckanext-ga-report.git/commitdiff

Commented the code in ga_auth

3 files changed: (show all)
README.rst
ckanext/ga_report/command.py
ckanext/ga_report/ga_auth.py

file:a/README.rst -> file:b/README.rst

ckanext-ga-report	ckanext-ga-report
=================	=================

Status: Development	Status: Development

CKAN Version: 1.7.1+	CKAN Version: 1.7.1+


Overview	Overview
--------	--------

For creating detailed reports of CKAN analytics, including totals per group.	For creating detailed reports of CKAN analytics, including totals per group.

Whereas ckanext-googleanalytics focusses on providing page view stats a recent period and for all time (aimed at end users), ckanext-ga-report is more interested in building regular periodic reports (more for site managers to monitor).	Whereas ckanext-googleanalytics focusses on providing page view stats a recent period and for all time (aimed at end users), ckanext-ga-report is more interested in building regular periodic reports (more for site managers to monitor).

Contents of this extension:	Contents of this extension:

* Use the CLI tool to download Google Analytics data for each time period into this extension's database tables	* Use the CLI tool to download Google Analytics data for each time period into this extension's database tables

* Users can view the data as web page reports	* Users can view the data as web page reports


Installation	Installation
------------	------------

1. Activate you CKAN python environment and install this extension's software::	1. Activate you CKAN python environment and install this extension's software::

$ pyenv/bin/activate	$ pyenv/bin/activate
$ pip install -e git+https://github.com/okfn/ckanext-ga-report.git#egg=ckanext-ga-report	$ pip install -e git+https://github.com/okfn/ckanext-ga-report.git#egg=ckanext-ga-report

2. Ensure you development.ini (or similar) contains the info about your Google Analytics account and configuration::	2. Ensure you development.ini (or similar) contains the info about your Google Analytics account and configuration::

googleanalytics.id = UA-1010101-1	googleanalytics.id = UA-1010101-1
googleanalytics.username = googleaccount@gmail.com	googleanalytics.username = googleaccount@gmail.com
googleanalytics.password = googlepassword	googleanalytics.password = googlepassword
ga-report.period = monthly	ga-report.period = monthly

Note that your password will be readable by system administrators on your server. Rather than use sensitive account details, it is suggested you give access to the GA account to a new Google account that you create just for this purpose.	Note that your password will be readable by system administrators on your server. Rather than use sensitive account details, it is suggested you give access to the GA account to a new Google account that you create just for this purpose.

3. Set up this extension's database tables using a paster command. (Ensure your CKAN pyenv is still activated, run the command from ``src/ckanext-ga-report``, alter the ``--config`` option to point to your site config file)::	3. Set up this extension's database tables using a paster command. (Ensure your CKAN pyenv is still activated, run the command from ``src/ckanext-ga-report``, alter the ``--config`` option to point to your site config file)::

$ paster initdb --config=../ckan/development.ini	$ paster initdb --config=../ckan/development.ini

4. Enable the extension in your CKAN config file by adding it to ``ckan.plugins``::	4. Enable the extension in your CKAN config file by adding it to ``ckan.plugins``::

ckan.plugins = ga-report	ckan.plugins = ga-report


Authorization	Authorization
--------------	--------------

Before you can access the data, you need to set up the OAUTH details which you can do by following the `instructions <https://developers.google.com/analytics/resources/tutorials/hello-analytics-api>`_ the outcome of which will be a file called credentials.json which should look like credentials.json.template with the relevant fields completed. These steps are below for convenience:	Before you can access the data, you need to set up the OAUTH details which you can do by following the `instructions <https://developers.google.com/analytics/resources/tutorials/hello-analytics-api>`_ the outcome of which will be a file called credentials.json which should look like credentials.json.template with the relevant fields completed. These steps are below for convenience:

1. Visit the `Google APIs Console <https://code.google.com/apis/console>`_	1. Visit the `Google APIs Console <https://code.google.com/apis/console>`_

2. Sign-in and create a project or use an existing project.	2. Sign-in and create a project or use an existing project.

3. In the `Services pane <https://code.google.com/apis/console#:services>`_ , activate Analytics API for your project. If prompted, read and accept the terms of service.	3. In the `Services pane <https://code.google.com/apis/console#:services>`_ , activate Analytics API for your project. If prompted, read and accept the terms of service.

4. Go to the `API Access pane <https://code.google.com/apis/console/#:access>`_	4. Go to the `API Access pane <https://code.google.com/apis/console/#:access>`_

5. Click Create an OAuth 2.0 client ID....	5. Click Create an OAuth 2.0 client ID....

6. Fill out the Branding Information fields and click Next.	6. Fill out the Branding Information fields and click Next.

7. In Client ID Settings, set Application type to Installed application.	7. In Client ID Settings, set Application type to Installed application.

8. Click Create client ID	8. Click Create client ID

9. The details you need below are Client ID, Client secret, and Redirect URIs	9. The details you need below are Client ID, Client secret, and Redirect URIs


Once you have set up your credentials.json file you can generate an oauth token file by using the	Once you have set up your credentials.json file you can generate an oauth token file by using the
following command, which will store your oauth token in a file called token.dat once you have finished	following command, which will store your oauth token in a file called token.dat once you have finished
giving permission in the browser::	giving permission in the browser::

$ paster getauthtoken --config=../ckan/development.ini	$ paster getauthtoken --config=../ckan/development.ini


Tutorial	Tutorial
--------	--------

Download some GA data and store it in CKAN's db. (Ensure your CKAN pyenv is still activated, run the command from ``src/ckanext-ga-report``, alter the ``--config`` option to point to your site config file) and specifying the name of your auth file (token.dat by default) from the previous step::	Download some GA data and store it in CKAN's db. (Ensure your CKAN pyenv is still activated, run the command from ``src/ckanext-ga-report``, alter the ``--config`` option to point to your site config file) and specifying the name of your auth file (token.dat by default) from the previous step::

$ paster loadanalytics token.dat latest --config=../ckan/development.ini	$ paster loadanalytics token.dat latest --config=../ckan/development.ini

	The value after the token file is how much data you want to retrieve, this can be

	* all - data for all time (since 2010)

	* latest - (default) just the 'latest' data

	* YYYY-MM-DD - just data for all time periods going back to (and including) this date



Software Licence	Software Licence
================	================

This software is developed by Cabinet Office. It is Crown Copyright and opened up under the Open Government Licence (OGL) (which is compatible with Creative Commons Attibution License).	This software is developed by Cabinet Office. It is Crown Copyright and opened up under the Open Government Licence (OGL) (which is compatible with Creative Commons Attibution License).

OGL terms: http://www.nationalarchives.gov.uk/doc/open-government-licence/	OGL terms: http://www.nationalarchives.gov.uk/doc/open-government-licence/

file:a/ckanext/ga_report/command.py -> file:b/ckanext/ga_report/command.py

import logging	import logging

from ckan.lib.cli import CkanCommand	from ckan.lib.cli import CkanCommand
# No other CKAN imports allowed until _load_config is run, or logging is disabled	# No other CKAN imports allowed until _load_config is run, or logging is disabled

class InitDB(CkanCommand):	class InitDB(CkanCommand):
"""Initialise the extension's database tables	"""Initialise the extension's database tables
"""	"""
summary = __doc__.split('\n')[0]	summary = __doc__.split('\n')[0]
usage = __doc__	usage = __doc__
max_args = 0	max_args = 0
min_args = 0	min_args = 0

def command(self):	def command(self):
self._load_config()	self._load_config()

import ckan.model as model	import ckan.model as model
model.Session.remove()	model.Session.remove()
model.Session.configure(bind=model.meta.engine)	model.Session.configure(bind=model.meta.engine)
log = logging.getLogger('ckanext.ga-report')	log = logging.getLogger('ckanext.ga-report')

import ga_model	import ga_model
ga_model.init_tables()	ga_model.init_tables()
log.info("DB tables are setup")	log.info("DB tables are setup")


class GetAuthToken(CkanCommand):	class GetAuthToken(CkanCommand):
""" Get's the Google auth token	""" Get's the Google auth token

	Usage: paster getauthtoken <credentials_file>

	Where <credentials_file> is the file name containing the details
	for the service (obtained from https://code.google.com/apis/console).
	By default this is set to credentials.json
"""	"""
summary = __doc__.split('\n')[0]	summary = __doc__.split('\n')[0]
usage = __doc__	usage = __doc__
max_args = 0	max_args = 0
min_args = 0	min_args = 0

def command(self):	def command(self):
from ga_auth import initialize_service	"""
	In this case we don't want a valid service, but rather just to
	force the user through the auth flow. We allow this to complete to
	act as a form of verification instead of just getting the token and
	assuming it is correct.
	"""
	from ga_auth import init_service
initialize_service('token.dat',	initialize_service('token.dat',
self.args[0] if self.args	self.args[0] if self.args
else 'credentials.json')	else 'credentials.json')

class LoadAnalytics(CkanCommand):	class LoadAnalytics(CkanCommand):
"""Get data from Google Analytics API and save it	"""Get data from Google Analytics API and save it
in the ga_model	in the ga_model

Usage: paster loadanalytics <tokenfile> <time-period>	Usage: paster loadanalytics <tokenfile> <time-period>

Where <tokenfile> is the name of the auth token file from	Where <tokenfile> is the name of the auth token file from
the getauthtoken step.	the getauthtoken step.

And where <time-period> is:	And where <time-period> is:
all - data for all time	all - data for all time
latest - (default) just the 'latest' data	latest - (default) just the 'latest' data
YYYY-MM-DD - just data for all time periods going	YYYY-MM-DD - just data for all time periods going
back to (and including) this date	back to (and including) this date
"""	"""
summary = __doc__.split('\n')[0]	summary = __doc__.split('\n')[0]
usage = __doc__	usage = __doc__
max_args = 2	max_args = 2
min_args = 1	min_args = 1

def command(self):	def command(self):
self._load_config()	self._load_config()

from ga_auth import initialize_service	from ga_auth import init_service
try:	try:
svc = initialize_service(self.args[0], None)	svc = init_service(self.args[0], None)
except TypeError:	except TypeError:
print 'Have you correctly run the getauthtoken task and specified the correct file here'	print 'Have you correctly run the getauthtoken task and specified the correct file here'
return	return

from download_analytics import DownloadAnalytics	from download_analytics import DownloadAnalytics
from ga_auth import get_profile_id	from ga_auth import get_profile_id
downloader = DownloadAnalytics(svc, profile_id=get_profile_id(svc))	downloader = DownloadAnalytics(svc, profile_id=get_profile_id(svc))

time_period = self.args[1] if self.args and len(self.args) > 1 else 'latest'	time_period = self.args[1] if self.args and len(self.args) > 1 else 'latest'
if time_period == 'all':	if time_period == 'all':
downloader.all_()	downloader.all_()
elif time_period == 'latest':	elif time_period == 'latest':
downloader.latest()	downloader.latest()
else:	else:
since_date = datetime.datetime.strptime(time_period, '%Y-%m-%d')	since_date = datetime.datetime.strptime(time_period, '%Y-%m-%d')
downloader.since_date(since_date)	downloader.since_date(since_date)

file:a/ckanext/ga_report/ga_auth.py -> file:b/ckanext/ga_report/ga_auth.py

import httplib2	import httplib2
from apiclient.discovery import build	from apiclient.discovery import build
from oauth2client.client import flow_from_clientsecrets	from oauth2client.client import flow_from_clientsecrets
from oauth2client.file import Storage	from oauth2client.file import Storage
from oauth2client.tools import run	from oauth2client.tools import run

from pylons import config	from pylons import config


def _prepare_credentials( token_filename, credentials_filename ):	def _prepare_credentials( token_filename, credentials_filename ):
	"""
	Either returns the user's oauth credentials or uses the credentials
	file to generate a token (by forcing the user to login in the browser)
	"""
storage = Storage( token_filename )	storage = Storage( token_filename )
credentials = storage.get()	credentials = storage.get()

if credentials is None or credentials.invalid:	if credentials is None or credentials.invalid:
flow = flow_from_clientsecrets(credentials_filename,	flow = flow_from_clientsecrets(credentials_filename,
scope='https://www.googleapis.com/auth/analytics.readonly',	scope='https://www.googleapis.com/auth/analytics.readonly',
message="Can't find the credentials file")	message="Can't find the credentials file")
credentials = run(flow, storage)	credentials = run(flow, storage)

return credentials	return credentials

def initialize_service( token_file, credentials_file ):	def init_service( token_file, credentials_file ):
	"""
	Given a file containing the user's oauth token (and another with
	credentials in case we need to generate the token) will return a
	service object representing the analytics API.
	"""
http = httplib2.Http()	http = httplib2.Http()

credentials = _prepare_credentials(token_file, credentials_file)	credentials = _prepare_credentials(token_file, credentials_file)
http = credentials.authorize(http) # authorize the http object	http = credentials.authorize(http) # authorize the http object

return build('analytics', 'v3', http=http)	return build('analytics', 'v3', http=http)


def get_profile_id(service):	def get_profile_id(service):
# Get a list of all Google Analytics accounts for this user	"""
	Get the profile ID for this user and the service specified by the
	'googleanalytics.id' configuration option.
	"""
accounts = service.management().accounts().list().execute()	accounts = service.management().accounts().list().execute()

if accounts.get('items'):	if not accounts.get('items'):
firstAccountId = accounts.get('items')[0].get('id')	return None
webPropertyId = config.get('googleanalytics.id')
profiles = service.management().profiles().list(
accountId=firstAccountId,
webPropertyId=webPropertyId).execute()

if profiles.get('items'):	accountId = accounts.get('items')[0].get('id')
# return the first Profile ID	webPropertyId = config.get('googleanalytics.id')
return profiles.get('items')[0].get('id')	profiles = service.management().profiles().list(
	accountId=accountId, webPropertyId=webPropertyId).execute()

	if profiles.get('items'):
	return profiles.get('items')[0].get('id')

return None	return None