7. The Identified Informal Learner: Recognizing Assessed Learning in the Open
© Patrina Law, CC BY 4.0 http://dx.doi.org/10.11647/OBP.0103.07
Badged open courses (BOCs) were piloted on the OpenLearn platform by the Open University (OU) in the UK in 2013. These are free online course upon the completion of which, digital badges are awarded. Based on the evaluation of their impact, they now form a key strand to the OU’s free learning provision, embracing Open Educational Practices at their core. The first permanent suite of BOCs was launched on OpenLearn in 2015 and evaluated for impact, both from an outreach and a business perspective. The application of a branded open digital badge, with associated assessment and feedback has provided a mechanism to motivate and reward informal learners whilst also generating a higher than expected click-through to make an enquiry about becoming a formal student.
Introduction
The Open University (OU) in the UK has long delivered a diverse range of courses to large numbers of people. The OU was established in 1969 with the aim of opening up higher education (HE) to all, regardless of circumstances, geographical location or qualifications. Currently the OU is serving some 200,000 students and is particularly concerned with reaching those who might not otherwise have access to higher education, ensuring that there are as few barriers as possible. As part of this commitment to access, the OU has freely released educational materials into the public realm. This helps to support the twin pillars of core OU activity:
- Social mission, that is, public awareness of, and easy access to life-long learning opportunities, including free, informal learning.
- Business mission, that is brand awareness, student registration, student preparedness, asset and archive exploitation/re-use in formal learning and income from fee-paying customers.
The OU has been providing free learning via its OpenLearn platform since 2006. It ensures that it provides about 5% of its course materials as free open educational resources (OER) every year on OpenLearn (www.open.edu/openlearn). It also serves as the platform through which the OU promotes its partnership with the BBC and the related free courses and articles that are created to support its co-productions with them. It does this because free learning is an interpretation of its Royal Charter which states that it will “promote the educational well-being of the community generally”.
This provision of free learning is also part of one of the OU’s strategic objectives: Journeys from Informal to Formal Learning (JIFL). Originally supported by a grant from the William and Flora Hewlett Foundation, the platform now hosts around 1000 free courses, short articles, activities, videos and ebooks all released under a Creative Commons license. OpenLearn is accessed by over five million users a year, of whom 100,000 are the university’s own students. OpenLearn also delivers a 13% click-through rate of learners wanting to know more about becoming an OU student. Existing metrics show that as a free learning platform, OpenLearn attracts a very balanced demographic that is, its learners are less qualified overall than those attending Massive Open Online Course (MOOC) platforms.
In an attempt to demonstrate an ongoing institutional commitment to new models of teaching, learning and assessment to serve both informal learners and students alike, the development of badged open courses (BOCs) were piloted by the OU on OpenLearn and evaluated in 2013. The BOC initiative built on ongoing research on the motivations and demographic profiles of learners using OpenLearn (Law, Perryman and Law, 2013; Perryman, Law and Law, 2013). Based on the evaluation of these pilot courses and key evidence from OpenLearn surveys it was found that 80% of informal learners strongly felt that they wanted to have their informal online learning achievements recognized through the availability of free certificates. Hence a suite of free BOCs awarding an OU-branded digital badge and certificate were developed in 2014 and launched in 2015 and their impact evaluated (Law, 2015, 2016).
This chapter reports on the evaluation of the 2015 BOCs and how they build on what we now know of the strategic importance of free learning recognition in an unsupported (non-tutored) online environment. Initial results reveal that the majority of respondents declare that BOCs provide a sense of achievement despite the absence of any tutor-led instruction and that they would be sharing their achievements with their employer. In terms of impact to OU business, metrics compare favorably with informal learning per se, with 26% of learners visiting the BOCs choosing to click through to the OU’s formal qualifications webpages. This is more than twice the percentage of the average OpenLearn learner.
Stacey (2012) identifies ten key benefits to institutions for supporting OER initiatives and provides useful criteria against which to develop and experiment with Open Educational Practices and learning design for free learning environments:
- OER increases access to education;
- Provides students with an opportunity to assess and plan their education choices;
- Showcases an institution’s intellectual outputs, promoting its profile and attracting students;
- Converts students into fee paying enrolments;
- Accelerates learning;
- Adds value to knowledge production;
- Reduces faculty preparation time;
- Generates cost savings;
- Enhances quality; and
- Generates innovation through collaboration.
This initial impact of the BOCs concurs with Stacey’s suggestion that OER can “lead to faster learning, greater learner success”, and supports his notion that OER may subsequently generate revenue, where BOCs in particular see such high motivation and formal course sign-up (Stacey, 2012).
A Background to Digital Badging
Digital badging in educational sectors offers a new way to reward and motivate learners, providing evidence of skills and achievements in classroom or online settings. As OERs across multiple platform types and formats have continued to diversify to match learners and educators’ preferences, so the notion of recognition for informal learning in these spheres has become accepted provision by some educators and philanthropic providers.
Hickey (2012) identifies three possible functions for digital badges:
- Summative functions, that is, assessment of learning.
- Formative functions for individuals, that is, assessment for learning.
- Transformative functions for systems, that is, assessment as learning.
Models of the educational use of digital badging are wide-ranging (Hickey and Willis, 2015) though invariably have as a common theme the expectation of a motivational tool and as a form of micro-credential, that is, associated with a short course or activity undertaken to develop a skill. Gibson et al. (2013) simply identify digital badging as an incentive for earners to identify progress and to signify achievement and learning. Clark, Howard and Early (2006) note that motivation is key to learning and that its application with the issuing of digital badges through BOCs supports this (Law, 2016). Abramovich, Schumn and Higashi (2013) state that: “[…] the potential benefit of an assessment is determined by its ability to both maintain learning motivation and accurately communicate a student’s learning”. By developing summative and formative assessments using Moodle quizzes in open courses in the way that Hickey identifies above, the OU is attempting to both communicate feedback and provide motivation to learners who lack any tutorial support.
Specifically, within the context of higher education, Bixler and Layng (2013) argued that digital badges would “hold great promise” but that at the time “policies on badges for higher education institutions” did not exist. More recently, McDaniel and Fanfarelli (2015) describe the use of digital badges woven into the undergraduate curriculum as a tool for both feedback and motivation, through the issuing of badges in two different ways: by a tutor in a classroom and separately via an internal online management system.
Early detractors of digital badging were largely seen in the educational blogging community noting the mainstreaming of badging into the digital world as disruptive, dangerous if poorly employed and unlikely to have any comparative value to formal qualifications (Crotty (2012); Halavais (2012); Jenkins (2012)).
Early case studies for offering digital badges in higher education focus on concepts around the characteristics of badges as rewards. Charleer, Klerkx, Santos and Duval (2013) suggest their use is a means to feedback, encourage motivation, catalysts for discussion and being socially sharable. The drivers for offering digital badges in higher education are described by Wu, Whiteley and Sass (2015) in other contexts: within a classroom setting; as co-curricular support; as a means to fulfil graduation requirements, and as part, or all of an outreach agenda. Where case studies of digital badges to date tend to focus on a specific educational sector, the application of open badges within BOCs embraces several of these characteristics and contexts and explores their value as a strategic activity that supports social mission through the delivery of OER.
Key Features of Badged Open Courses
All learners who study a BOC participate in a number of online assessments delivered through the deployment of Moodle quizzes. The courses are designed to be as robust as any of the OU’s modules in terms of quality and pedagogy: they follow strict learning design procedures, academic authoring, assessment and critical readership. Each course is structured into eight notional weeks covering twenty-four hours of learning, although a learner can take the course at his or her own pace. At the end of each week, practice quizzes are provided that count towards the assessment at the end of weeks four and eight.
Based on the evaluation of the 2013 pilots, the courses published in 2015 were largely in support of learner preparedness at an introductory level. They were entitled:
- Succeed with Maths — Part 1.
- Succeed with Maths — Part 2.
- Taking Your First Steps into Higher Education.
- Succeed with Learning.
- English: Skills for Learning.
- Succeed in the Workplace.
In order to be consistent, each BOC was developed with the same structural format in terms of the use of rich media, voice, learning design and assessment, so that any learner would come to understand what was expected of him or her when studying any BOC. This approach also supports the University to ensure quality enhancement when using the OU brand, especially in the absence of a framework for the classification of openly badged courses in higher education.
Each BOC starts with a video introducing the course content, as shown in Figure 1. This video is complemented by further, weekly videos each of which explains the learning outcomes for that section of the course and gives a “familiar face” to the learning. The pilot study showed that this “familiar face” — not necessarily the author of the content, but a subject expert or practitioner nonetheless — was appreciated by learners in the absence of any social group structure to the learning or real-time tutor.
Methods of Evaluation
In order to evaluate the BOCs delivered in 2015, mixed method surveys were made available at the start and end of each BOC, with participation optional. Surveys were based on those used in the pilot study and were delivered using the SurveyMonkey platform with each start of course survey and each end of course survey being identical across BOCs. Surveys comprised a combination of Likert scale, multiple choice and open questions. Data on the number of registrations and the onward journey of learners were gathered using Google Analytics and comScore Digital Analytix (DAX) software. The aim of evaluating the BOCs through surveys and data captured via platform data analytics was to examine the impact, both short and long term, of BOCs, with particular emphasis on:
- Examining demographics (in alignment with OU data collected about informal learners on OpenLearn overall);
- Tracking data to show movement of learners within the platform, where they were referred from into the platform and their onward journey;
- Gaining a profile of the types of learners who are more likely to convert to formal learning;
- Giving a picture of the types of learning methods and course elements most likely to encourage learners to progress in an open, unsupported environment; and
- Assessing the motivational aspects of badging and whether learners showed their achievements to an employer or prospective employer.
In addition, comparisons were made with data from studies undertaken by the author in 2013 and 2014 on OpenLearn (Law and Perryman, 2015) and again in 2015, in order to gain a deeper understanding of learner demographics and their motivations for study. For these studies, surveys were promoted via web-links embedded within OpenLearn and via the OpenLearn newsletter. The study included questions drawn from the OER Research Hub (OERRH, http://oerresearchhub.org) open research question base to allow for comparison with existing data collected through OER projects globally.
The OpenLearn survey received 1,177 responses in 2013, then 3,133 responses in 2014 and 1,299 responses in 2015. BOCs were not present on OpenLearn at the time the 2013 and 2014 surveys were live.
Results
Across all six BOCs during the evaluation period (March to October 2015) there were 2,804 responses to the start of course surveys and 786 responses to the end of course surveys. The project itself was reported internally through the university’s strategic priority of JIFL, with the requirement that it be evaluated against anticipated benefits. These benefits are summarized in Table 1 and draw on data from surveys and internal analytics.
The surprisingly high percentage of click-throughs (26%) to make an enquiry at the OU as a result of studying a BOC (shown in Table 1) could, in part, be explained by the more favorable demographic of BOC learners compared with OpenLearn learners who then signed up for a formal higher education qualification (see Table 2).
For example:
- The majority of respondents to the end of course survey for the Taking your First Steps in higher education BOC (53%) declare a “School leaving certificate” as their highest qualification compared to OpenLearn, where the majority of learners (44–50%) declare a “degree qualification” as their highest qualification.
- BOC learners are younger overall than OpenLearn learners. Table 2 shows that the majority of BOC learners are in the 26–45 age range; for OpenLearn they are in the 46–65 age range.
Data indicating that up to 57% of BOC learners would be sharing their achievement with an employer was of particular interest when considering the extension of the BOC courses and how these could be aligned to the formal curriculum. Hence, in September 2015, a further survey was issued to those respondents who said they would be sharing their digital badge with their employer or prospective employer and had agreed to be followed up for additional research. Initial data show that:
- 75% felt their employer valued the BOC that they had taken.
- 80% of those who had not shared their achievement still planned to do so.
- 98% felt the BOC had a positive impact on their work.
Challenges
An initial concern of the project team in defining the assessment criteria for BOCs was that the bar was being set too high — that is, applying assessments and a requirement to view each page of a BOC could act as a deterrent to, rather than a motivator for completion. Where scant research and almost no empirical data existed with regard to the impact of BOCs in an open environment at the time the BOCs were developed, developing them with such prescribed assessment criteria was considered a risk. With robust assessment becoming a key element to obtaining any university-branded digital badge, open or otherwise, the current wave of interest inspired the first conference on “Badging in Higher Education” which took place in the UK in 2016, during Open Education Week, which may have generated much-needed empirical data around the use of digital badges in education.
Another challenge was that, of the 2015 BOCs, half were developed using repurposed content; much of this had already been released on OpenLearn as OER, usually in the form of large chunks of text. This content itself was originally adapted from OU modules that had been produced for short introductory courses and withdrawn from the OU curriculum. It was felt that this approach to adapting existing content would be less time consuming than writing from scratch. However, the reality was that after developing a robust learning design and editorial approach for each BOC, there was little reduction in time spent on repurposing existing content compared with writing it anew. This can partly be explained by the proportionately large amount of time required to develop meaningful formative assessments for Moodle.
To explain the endeavor to write formative assessments for Moodle, each week of a BOC contains either a practice assessment (weeks 1–3 and 5–7) or a marked assessment that counts toward the final badge (weeks four and 8). For weeks four and 8, the author was required to write 45 questions to enable learners to repeat attempts and receive a random selection of questions and answers, up to three times (in order to form a question bank for the fiftheen quiz questions for those weeks). Each author was also required to write a further five questions for non-assessed practice evaluations required for the remaining six weeks.
Impact
The social and institutional impact on the University of delivering BOCs is being widely communicated internally and through international badging networks. Based on the impact of the 2015 BOCs, a second wave of courses is in development in 2016, ostensibly to widen the range of introductory-level BOCs. In addition, the University has decided to extend the curriculum to postgraduate level and career and professional development (CPD) courses. This notion of open badges playing a role in support of CPD is reflected in four areas of higher education that are being discussed in the author’s badging networks:
- As a method of encouraging first year students to complete.
- As a preparedness activity between enrolment and qualification start.
- For engaging in skills/employability courses.
- For internal staff development.
For the OU, BOCs produced in 2016 will be in support of the higher education sector as a whole in providing a common core of CPD subjects (the courses can be re-purposed, re-badged and re-used by any institution within the terms of the Creative Commons license), to support and prepare its own students and to further extend BOCs as a vehicle for outreach. They are likely to cover the following subjects, which have been endorsed by the OU’s Careers Advisory Service and are in support of the Journeys from Informal to Formal Learning strategy:
- Working in the voluntary sector.
- Working in science, technology engineering and maths (STEM).
- Digital literacy.
- Succeeding in postgraduate study.
- Digital scholarship.
- Resilience and flexibility.
- Commercial communication and negotiation.
- Leadership and followership.
- Understanding business structures.
As the OU seeks to improve the economic split-decision between new course development and repurposing existing content when it comes to building new CPD materials, it has embraced a new approach to mainstreaming the development of open content that achieves better efficiencies. Open Educational Practices are not widely embraced at the OU unless someone has been involved in the creation of an OpenLearn course or MOOC. Hence, the development of new BOCs and open courses on OpenLearn will require authors, editors and instructional designers to embed the design approach of BOCs in all open course design. Non-badged OpenLearn courses that are adapted from formal modules will now require learning design to take place at the very inception of formal module design, including the development of a BOC where this is a strategic fit.
Gaining a clearer understanding of what works best for open course design at the OU has been driven by the evaluation of the OpenLearn surveys and the BOC evaluation data overall. Based on this, the following guiding principles have been developed for University staff to consider when designing and developing OpenLearn courses:
- Learners value recognition of their achievement (in the form of a free statement of participation and digital badge) in passing tests and completing a course of study.
- Within all the rich media presented throughout each BOC, learners most value quizzes that include feedback.
- Closed environments with a start and finish date — that is, MOOCs — have lower completion rates than open courses with no start and finish date.
- The use of activities and video (especially that of a tutor, or “face” of a course) are especially valued.
- Forced social activity encourages high drop-out rates.
Outreach
In terms of outreach to disabled learners, OpenLearn evaluations in 2013–15 have shown that the number of learners who declare a disability has been reported at around 23%. In order to put this into context, the UK national average of people of working age who declare a disability is reported at 16% (UK Department for Work and Pensions, 2014).
The 2015 OpenLearn surveys indicated that for 59% of respondents with a disability, materials on OpenLearn had improved their confidence in their ability to study. This compares well with those without a disability, for whom 58% said the materials had also improved their confidence.
Demographic data on accessibility was also gathered in both the start and end of course surveys for BOCs to ascertain completion by this group. Table 3 shows that the percentage of disabled learners varied across the BOCs:
- For all but two BOCs (Succeed with Maths — Part 1 and English: Skills for Learning) there is a slight drop in the percentage of disabled learners completing a BOC.
- Half of the BOCs show a higher than average percentage overall of learners declaring a disability in their start of course surveys compared to the OpenLearn overall rate.
Where this second point varies most is seen in the BOC English: Skills for Learning where only 15% declare a disability. This may be explained by a higher proportion of non-UK learners studying the course compared with the other BOCs, where figures for declaration and descriptions of disability vary compared with those figures that people in the UK have grown used to. Conversely, for Succeed with Learning, the figure for those declaring a disability (37%) is far higher than for OpenLearn overall, where the majority of learners are UK-based.
One of the key comments provided by disabled learners using OpenLearn is the request to have content available in multiple formats. In response to this, and after tackling some technical difficulties in making global updates to more than 850 courses during 2015, it has now been possible to provide the following formats for all OpenLearn courses, including BOCs:
- Ebook (epub)
- IMS common cartridge
- Kindle ebook
- OU XML file
- RSS
- Word
- HTML
- Interactive ebook (epub)
- OU XML package
- SCORM
Many of these format types are also published for free by the OU on its channels on iTunes U, Google Play and Amazon (for use on Kindle devices) and are released under a Creative Commons license, along with any new video and audio created for the courses (for iTunes U, AudioBoom and YouTube). Previously, content providers developed syndication practices as a way to make web feeds available from a website in order to provide a summary of the recently added content (such as the latest news or forum posts). In recent years at the OU, the term “syndication” has come to be used for the republishing of assets and courses — whether individually or as collections — via feeds, embedded codes or the uploading of content to third-party platforms and applications.
The term open syndication is used internally at the OU to define the distribution of OER that carries a Creative Commons license. Within it is also the activity of disaggregating a course for its parts to maximize the use of assets. For example, a BOC will be developed containing new video, audio or animations to explain key concepts. These assets are themselves released to the appropriate platform that specializes in that particular media type e.g. YouTube for video, audio and animation. In addition, the whole course will be released as an ebook. Hence, from the perspective of the University’s commitment to free learning and outreach, the BOC project represents a coming together of several strands of openness in order to maximize the social and business missions of providing open learning.
Future Implications
With digital badging becoming established as a trend across educational sectors, it has been shown to recognize and motivate learners, providing evidence for achievements and learning in a variety of formal and non-formal settings. As the diversification of OER across multiple platform types and formats has evolved to suit different learners and educators alike, so the notion of recognition for informal learning in these spheres has become accepted provision by some educators and philanthropic providers, where it can be achieved at scale.
For MOOC providers, this recognition of participation is provided across a range of criteria (for example, passing tests or viewing part or all of a course) through the sale of certificates that carry the MOOC provider’s brand. For OpenLearn, a recognition of learning is provided for free through the issuing of the open badge and OU-branded statement of participation (a PDF) in support of open principles and practice. Badge recipients progress at their own pace and not in a cohort, but have to view each page of the course and pass the assessments. Whether learners value a university-branded provision more or less than something they have paid for from a privately operating MOOC platform is yet to be evaluated.
When it comes to learning for outreach, the issuing of an open digital badge (or overarching, recognition for informal learning) for free may become an important element in the pursuit of open principles in education. This may yet also have a positive impact on employability and — as seen in the case of BOCs — reap a financial reward in the form of new student registrations that are higher than for other forms of informal learning provision thus far developed.
The use of the OpenLearn platform as a test bed for innovation in eLearning has provided some surprising data with positive implications for both the social and business missions of the OU. In addition, understanding media mix in terms of what makes an impactful and engaging OpenLearn course will have positive financial implications and enable better planning and development in an environment where around sixty new (non-badged) OpenLearn courses are being produced each year alongside formal module production. The awarding of a digital badge will also be relevant to the OU’s formal students, who will see this University recognition for non-formal study on their student record and in the future, on their Higher Education Achievement Report. These particular resources could support student success in retention and completion, employability and academic excellence and, with the application of the Creative Commons license, will give other higher education providers the opportunity to share, re-badge or republish.
This last point is most likely to resonate with non-distance higher education providers more generally — especially those that do not have easy access to a production and publishing mechanism for OER but that may prefer to find a home for digital badging in support of undergraduates in their critical first year.
Conclusions
As BOCs become a business-as-usual activity for the OU and the strategy that underpins them extends from introductory to induction and from postgraduate to CPD, new goals will be set to extend their support in emerging areas across curricula. The theoretical frameworks underpinning openness in education have shown themselves to have extensive practical application: open badging is another arrow in the quiver of open applications and practices that support the goal of democratizing higher education.
The early detractors writing about digital badging, discussed earlier in this chapter, described it as dangerous if poorly employed and unlikely to have any comparative value to formal qualifications due to the fact that anyone could (and still can), issue a digital badge. What is known from the evaluation of BOCs is that learners are keen to display their achievements — to be recognized informal learners — but that branding is key to this desire.
Not surprisingly then, there is a move to address this notion of a lack of credibility, which is currently being spearheaded by the US-based Instructional Management Systems (IMS) organization through a working group called “Open Badge Extensions for Education (OBEE)”. The group is attempting to improve and implement a consistent approach to badge taxonomy and description to:
- Augment badge metadata to provide valuable information about the credentialing institution, criteria, assessment and evidence for the awarding of an open badge
- Embed data and analytics by imposing meaningful metrics to improve badge “currency”
- Determine how badge consumers e.g. employers, will quickly discern compliant badges and therefore trust what is being represented
- Implement a “conformance certification process” to certify compliance with open badges and OBEE extensions.
Now, the spotlight of interest in digital badges is being shared by the notion that they might be a way in to educational accreditation and that this might overturn educational institutions’ hold on formal credentials (Jacobs, 2012). Badges may also find themselves in the center of new developments around micro-credentialing — as a set of non-formal learning achievements verifiable to an individual to demonstrate a commitment to professional and skills development. Rather than this being interpreted as a threat to formal credit-awarding bodies, it offers a new opportunity for those developing quality-assured OERs, open badges and practices to offer an alternative route into formal education.
References
Abramovich, S., Schumn, C. and Higashi, R. M. (2013), Are Badges Useful in Education? It Depends Upon the Type of Badge and Expertise of Learner, Educational Technology Research and Development, 61(2), pp. 217–232, http://www.lrdc.pitt.edu/schunn/research/papers/Abramovich-Schunn-Higashi.pdf
Bixler, B. and Layng, K. (2013), Digital Badges in Higher Education: An Overview, http://tinyurl.com/PSUDBWhitePaper
Charleer, S., Klerkx, J., Santos, J. and Duval, E. (2013), Improving Awareness and Reflection Through Collaborative, Interactive Visualizations of Badges, in Kravcik, M., Krogstie, B., Moore, A., Pammer, V., Pannese, L., Prilla, M., Reinhardt, W. and Ullmann, T. (Eds.), ARTEL13, Proceedings of the 3rd Workshop on Awareness and Reflection in Technology-Enhanced Learning, 1103, pp. 69–81 (Paphos).
Clark, R. E., Howard, K. and Early, S. (2006), Motivational Challenges Experienced in Highly Complex Learning Environments, in Elen, J. and Clark, R. E. (Eds.), Handling Complexity in Learning Environments: Theory and Research, Amsterdam: Elsevier, pp. 27–43.
Crotty, J. M. (2012), Why Get a Pricey Diploma When a Bleepin’ Badge Will Do? http://www.forbes.com/sites/jamesmarshallcrotty/2012/01/26/the-end-of-the-diploma-as-we-know-it/#4da9638b23f0
Department for Work and Pensions (2014), Disability Facts and Figures, https://www.gov.uk/government/publications/disability-facts-and-figures/disability-facts-and-figures
Halavais, A. (2012). Badges: The Skeptical Evangelist, http://alex.halavais.net/badges-the-skeptical-evangelist
Hickey, D. (2012), Some Things about Assessment that Badge Developers Might Find Helpful, Remediating Assessment, http://remediatingassessment.blogspot.co.uk/2012/03/some-things-about-assessment-that-badge.html
Hickey, D. T. and Willis, J. E. (2015), Research Designs for Studying Individual and Collaborative Learning with Digital Badges, in D. Hickey, J. Jovanovic, S. Lonn, J. E. Willis, III (Eds.), Proceedings of the Open Badges in Education (OBIE 2015) Workshop, New York: Poughkeepsie, 16 March 2015, http://ceur-ws.org
Jacobs, J. (2012), Digital Badges Threaten Colleges’ Monopoly on Credentials, US News, http://www.usnews.com/education/best-colleges/articles/2012/01/20/digital-badges-threaten-colleges-monopoly-on-credentials
Jenkins, H. (2012), How to Earn Your Skeptic “Badge”, http://henryjenkins.org/2012/03/how_to_earn_your_skeptic_badge.html
Law, P., Perryman, L-A. and Law, A. (2013), Open Educational Resources for All? Comparing User Motivations and Characteristics across the Open University’s iTunes U Channel and OpenLearn Platform, Open and Flexible Higher Education Conference 2013, 23–25 October 2013, Paris, European Association of Distance Teaching Universities (EADTU), pp. 204–219.
Law, P. and Perryman, L-A. (2015), Internal Responses to Informal Learning Data: Testing a Rapid Commissioning Approach, European Journal of Open, Distance and E-Learning, pp. 76–84.
Law, P. (2015), Recognising Informal Elearning with Digital Badging: Evidence for a Sustainable Business Model, Open Praxis, 7(4), pp. 299–310, http://dx.doi.org/10.5944/openpraxis.7.4.247
Law, P. (2016), Digital Badging at The Open University: Recognition for Informal Learning, Open Learning: The Journal of Open, Distance and E-Learning, http://dx.doi.org/10.1080/02680513.2015.1104500
Perryman, L.-A., Law, P. and Law, A. (2013), Developing Sustainable Business Models for Institutions’ Provision of Open Educational Resources: Learning from OpenLearn Users’ Motivations and Experiences, Open and Flexible Higher Education Conference 2013, 23–25 October 2013, Paris, European Association of Distance Teaching Universities (EADTU), pp. 204–219.
McDaniel, R. and Fanfarelli, J. R. (2015), A Digital Badging Dataset Focused on Performance, Engagement and Behavior-related Variables from Observations in Web-based University Courses, British Journal of Educational Technology, 46(5), pp. 937–941, http://dx.doi.org/10.1111/bjet.12272
Stacey, P. (2012), The Economics of Open, https://edtechfrontier.com/2012/03
Wu, M., Whiteley, D. and Sass, M. (2015), From Girl Scout to Grown Up: Applications of Digital Badges in Higher Education, The Online Journal of Distance Education and e-Learning, 3(2), pp. 48–52.