I wrote this up in a short brain dump to share with some colleagues internally recently, but since our Learning Analytics Principles and Purposes policy has now been re-published with a CC license, I thought it might be worth sharing some of the back story of it’s development to aid anyone thinking about how they might re-use it locally. All the info about our Learning Analytics policies is now up on our Academic Services webpages too.
In order to develop institutional policy on learning analytics, in 2016 we convened a task group reporting to our Senate Learning and Teaching Committee, and our Knowledge Strategy Committee. The task group was convened by Professor Dragan Gasevic, Chair of Learning Analytics and Informatics. The group included Professor Sian Bayne, Assistant Principal Digital Education; representatives from academic Colleges; the Edinburgh University’s Students Association; and representatives from Student Systems and Information Services.
Our Director of Academic Services produced an initial draft of a Learning Analytics policy for review by our institutional task group. It was a relatively detailed policy which covered the following sorts of topics:
- Sources of data for learning analytics
- Sources of data for learning analytics
- Initiating learning analytics activities
- Transparency and consent
- Privacy and access to data
- Retention and disposal of data
- Validity and interpretation of data
- Supporting positive interventions
- Enabling students to reflect on their learning
- Supporting staff to make the most of learning analytics
- Oversight of Learning Analytics activities
- Other relevant policies
Ethical values, legal obligations and the reasons for engaging with learning analytics were all embedded in the policy, but as we worked on revisions, considered inputs from external sources, and planned how to consult on a draft it became clear that this detailed policy was likely to beg more questions than it answered without being more explicit about our values and our ethical position upfront. We also had to contend with periods of time where there was limited data protection resource available to the task group, and where the legal basis for processing under GDPR that would be available to us was still being debated in the House of Lords.
At the same time as we were developing local policy, colleagues at Edinburgh (Prof Dragan Gasevic and Dr Yi-Shan Tsai) were involved with the EU Sheila project, developing a learning analytics policy development framework for the EU. There were several key outputs from that project that we used in pre-print form to inform our work:
- Executive summary of the literature on LA adoption and policies in other HEIs.
- Slides presenting the results of the group concept mapping study from the SHEILA project.
- Slides presenting the results of the interviews of senior leaders in over 50 institutions across Europe from the SHEILA project.
In particular, the group concept mapping activity carried out by the Sheila project (surveying various European Universities) identified that defining objectives for learning analytics was very important, but also very hard (http://sheilaproject.eu/wp-content/uploads/2017/04/The-state-of-learning-analytics-in-Europe.pdf). As part of our local policy development, myself and Dragan Gasevic met and discussed what we felt were the 6 main purposes for learning analytics in an Edinburgh context, and these were written up into the policy as a means of tackling this issue head-on for Edinburgh.
The literature review on learning analytics adoption that the Sheila project produced also identified various challenges to adoption, and on further consideration I drafted a separate Purposes and Principles document which extracted various of the principles embedded in the detailed policy and responded to many of the challenges and concerns identified in the literature review. Given some of the challenges we were experiencing around clarity on new data protection legislation for resolving areas the more detailed policy, this was the point at which our task group decided to separate the two pieces and start with a consultation on Purposes and Principles only.
The Purposes and Principles were outlined and discussed at Senate in early 2017 and then taken to each School for discussion as part of the consultation plan that Academic Services devised for us. To support this consultation we also developed a webpage that outlined existing research and operational activities in learning analytics at Edinburgh (https://www.ed.ac.uk/information-services/learning-technology/learning-analytics).
This high-level values-first route proved to be an effective way to start, as consultation with many Schools identified that the level of knowledge and understanding of learning analytics was highly variable across the institution, and that there were significant pockets of concern about ethics and about support for staff and students to make more use of data.
The Sheila project also ran a student survey at Edinburgh during this time period and we were also able to finesse the Principles and Purposes to respond to student concerns and expectations.
In considering how to achieve oversight and governance in the absence of the more detailed policy, and in a potentially quite complex and changing area, we also proposed the establishment of a Learning Analytics Review Group. As we pursue more data-driven operational activities this helps close out an ethical review gap in our operational activities. This governance model is now of interest to colleagues working on institutional data governance activities more generally.
Once the Principles and Purposes were approved, with support from our Data Protection Officer, and more clarity on GDPR we were then able to tidy up the more detailed policy which defines the ‘mechanics’ of how activities can be initiated, what roles and responsibilities exist, what sources of data might be implicated etc. This policy was approved by our Senate Learning and Teaching Committee in May 2018. Importantly, this policy has also been able to link in to other work around data governance within the institution, and formally recognises the role that our institutional ‘Data Stewards’ have to play in the approvals process for learning analytics projects.
Important inputs to the development of policy (as well as the Sheila project inputs) included:
- The Open University learning analytics policy (http://www.open.ac.uk/students/charter/essential-documents/ethical-use-student-data-learning-analytics-policy)
- Previous involvement in the development of the JISC Code of Practice for Learning Analytics (https://analytics.jiscinvolve.org/wp/2015/06/04/code-of-practice-for-learning-analytics-launched/)
- Higher Education Commission “From Bricks to Clicks” report (http://www.policyconnect.org.uk/hec/research/report-bricks-clicks-potential-data-and-analytics-higher-education)
- “Learning Analytics: Ethical Issues and Dilemmas” Sharon Slade and Paul Prinsloo (2013) (http://journals.sagepub.com/doi/10.1177/0002764213479366)
- Discussions with peers and experts at sector groups like SHEEC
- Conversations with Edinburgh Data Protection and Information Security professionals
- JISC Effective Learning Analytics Co-Design project (https://www.jisc.ac.uk/rd/projects/effective-learning-analytics)
- JISC Learning Analytics Network meetings
- Outputs from the EU LACE project, in particular their DELICATE framework for ethics and privacy (http://www.laceproject.eu/ethics-privacy/)
- Outputs from the Edinburgh PTAS-funded LARC project (http://www.de.ed.ac.uk/project/learning-analytics-report-card)
- Experience from Edinburgh colleagues, notably Prof Gasevic, Dr Yi-Shan Tsai, colleagues in Digital Education, and colleagues in Medicine.
- The Task group who helped develop the policy and debate the various drafts.
(Don't let your ethics slip Copyright We are Neo via Flickr CC-BY 2.0)