Learning Analytics Possibilities and Challenges of Using Data Science

Priscila Zigunovas
08/02/18
Adjust the
text size

Quick take: Learning analytics is trending and it seems to offer endless possibilities. Institutions and education professionals can benefit from this powerful technology, as long as they remember that working with data requires human wisdom.

Still a young field, learning analytics is being increasingly seen by higher education institutions as a powerful resource to inform decisions and achieve better learning results.

“What gets measured gets managed.” This quote, attributed to Peter Drucker, the father of modern management, is often used by analytics researchers and thinkers in reference to the power of data to inform decisions, a practice that many higher education institutions could benefit from.

“In order to effectively manage our systems, our processes and the success of students, we need to have measurement, because with that we have access to information and knowledge that we can use to control particular outcomes,” says Dr. Timothy Harfield, Senior Product Marketing Manager for Blackboard Analytics.

The biggest change in education in recent years has been the increase in scale, which not only impacts the number of people institutions are able to reach, but also limits their ability to engage students face-to-face. Learning analytics can play an important role in this challenge.

“As education becomes increasingly scaled and asynchronous, analytics becomes more important as a tool in support of high-quality teaching and learning practices that are responsive to meet the needs of students in a timely manner,” says Harfield, who has a background in philosophy and sociology and has published extensively on how learning analytics can be used to promote student success along with humanistic values.

However, most institutions are still far from exploring all the possibilities that learning analytics can offer.

Photo Timothy Harfield, Senior Product Marketing Manager for Blackboard Analytics
Timothy Harfield, Senior Product Marketing Manager for Blackboard Analytics

Great Expectations

Learning analytics, as a field and as a discipline, is still very young – about six years old. Institutions and researchers are still working with a great deal of experimentation.

Over the past few years, some institutions have invested in learning analytics with great success, while other early adopters were disappointed and are now skeptical about what the field can offer, according to Harfield.

“One day we woke up and we had access to all kinds of data. Everybody got really excited. There was a lot of promise, but not a whole lot of clarity around what was really possible and how this access to data could actually benefit institutions,” explains Harfield.

The novelty of the field and the lack of experience by practitioners, institutions, vendors and researchers caused them not to realize which types of policies and practices would be necessary to make a successful analytics initiative.

Over time, since the innovation rate was not as rapid as institutions were expecting, the initial excitement turned into disappointment. As it turns out, data alone isn’t sufficient to get the work done and developing impactful data-informed initiatives requires a great amount of dedication from institutions.

Now that this first stage has passed, the future of learning analytics looks quite promising. “Right now, because institutions, media and vendors have developed much more realistic expectations, we know more and can start really getting the work done. We have a level of maturity that is necessary to be realistic about what we need to do in order to see real results,” says Harfield.

Explore tools and resources for implementing learning analytics at your institution.

Human Judgment is Still Essential

Analytics is nothing more and nothing less than the visual display of quantitated information, according to Harfield. However, capturing activity in the form of data and transforming that data into visual displays of information, such as tables, charts and graphs, involves human judgment, and institutions have to take that into account.

In the expert’s experience, the institutions that are most effective at working with learning analytics are those with experienced and prudent practitioners who carefully consider the data in the context of deep knowledge about students, institutional practices and cultural factors.

“Learning analytics is not an opportunity for us to stop thinking, assessing and making decisions, but it is an important artifact that needs to be considered along with a variety of other sources of knowledge, including the human wisdom that comes with experience in order to solve particular problems,” Harfield says.

“Analytics is not a replacement for human judgment. It’s a form of information, but it still needs to be interpreted, and that interpretation requires human wisdom.”

Identifying Institutional Barriers

Over time, institutions will be using increasingly more data as a base for their decisions and strategies. A major challenge that they have to face is not to be overwhelmed with the amount of data. That’s why it’s important to invest in structures like data warehouses, so that they have access to data when they need it.

“However, once they have invested in that infrastructure, they need to forget about the data and focus on the questions, such as ‘What are the problems we need to solve as an institution?,’” says Harfield.

He suggests beginning with those inquiries and then thinking about how to translate them into questions that can be answered using the available data, and finally translating that data back into strategies that can actually inform and improve the specific outcomes institutions are looking to achieve.

“Let’s not forget that the university is a very old institution and, as a result, it’s incredibly complex. It can be really challenging to navigate if you are a student,” says Harfield.

The result of that is often a paradox: universities and colleges want students to be successful; however, because of the complexity of these institutions, they often end up including systematic barriers to the success of the very students they want to see succeed.

Analytics can provide the data that institutions need to identify those barriers, and it can positively impact student success but also institutional success and efficiency as well.

“Analytics allows institutions to see themselves almost like in a mirror. It allows them to gain access to how the institution is functioning as a whole and to identify the way in which students are being systematically and disproportionately advantaged or disadvantaged because of how that institution functions,” explains Harfield.

Possibilities for Instructors

A way in which instructors are using analytics to improve their courses is by creating solutions to identify students at risk. This could be automated by using predictive analytics, through Blackboard Predict, and also through tools like the Retention Center in Blackboard Learn 9.1. The later allows faculty to establish a threshold based on factors that they consider important, and also to monitor students as needed.

“This type of access to information about student activity and the consequences of that activity for their success are less important if you are in a small face-to-face class, but it becomes more important as you enter into online courses and large classes, where that face-to-face interaction is lost,” says Harfield.

Proactive advising using predictive analytics is a trend that will be even more present in the future, as it allows instructors and institutions to identify students at risk before they go off-track.

Interviews

Ethical Concerns With Learning Analytics: What You Should Be Aware of

Harfield recalls that the reason why traditional approaches to academic advising increasingly don’t work is because students that are in most need of that advising — usually low income, first generation and minority students — are exactly the ones that are the least likely to actively seek out help from support systems on their own.

“By using predictive analytics, we are able to identify students at risk early, before they fail the class, before they have dropped out,” suggests Harfield. These students can be invited for a conversation with professors, student success professionals, academic advisors or coaches in order for them to understand the potential barriers that students are facing and to develop strategies to help them overcome these difficulties.

Another way in which teachers can use analytics is giving students access to their own information, which fosters a sense of self-regulated learning. Research has found interesting results in that area. John Fritz, from University of Maryland, Baltimore County (UMBC), has found that students who used a feedback tool called “Check my Activity” were 1.92 times more likely to earn a grade C or higher compared to students who didn’t use the tool. 1

“Also, I have seen through research that we have done with the University of Michigan that the benefits received by having access to this kind of student-facing analytics disproportionately affects lower performing students. So, it’s actually helping exactly the kind of students we want to help keep on track,” tells Harfield.

From a pedagogical perspective, he explains, learning analytics gives instructors an opportunity to create interesting assignments that require students to, for example, reflect upon the analytics that they are seeing.

“We know that simply presenting information to students doesn’t make a difference in their behavior, but what does have an impact are opportunities to actively reflect on that data and what it means for them,” says Harfield.

Future perspectives

Although there are still many questions to be answered regarding the use of data in education, there are also several opportunities for pedagogical innovation in the use of learning analytics that instructors, professors, teachers and coaches have yet to explore.

In the future, Harfield says he would like to see more reflection and research done on the most effective way to leverage these new analytic technologies.

“I’m really looking forward to seeing, as technology advances, how we will be able to adapt our strategies, our approaches and our thinking about pedagogy to make the most effective use of those technologies in support of students.”

Understand How Analytics Technology Works

Blackboard’s solution for using data to drive innovation in education is called Analytics for Learn, and it is designed to help administrators, instructional designers and faculty to identify students at risk and optimize university learning environments for greater rates of student success.

Analytics for Learn offers three important features: data aggregation, the ability to perform longitudinal analysis, and the ability to combine data from multiple sources. Gain a better understanding of each concept below.

Data Aggregation – In the learning management system (LMS), every student click and behavior is recorded. “These produce huge complex tables that are extremely challenging to write queries against. Also, as you write these complex queries, you run the risk of putting an enormous burden or strain on that operational system,” explains Harfield. When Analytics for Learn is used, that stream of data is pulled into another system and then transformed, so that information can be easily accessed.

Longitudinal Analysis – Since retaining data requires a great deal of storage, most learning management systems have a tendency to hold information for a short period, usually one or two years. “But by storing this information in a separate warehouse or system, such as Blackboard Analytics for Learn, it’s possible to capture information and analyze trends across multiple years,” explains Harfield.

Combining Data from Multiple Sources – Analytics for Learn makes it possible to combine information not only from the learning management system but also from the student information system. “That allows us to do really interesting things, to understand learning behaviors in light of demographic factors and other information that is only stored within the student information system,” says Harfield. Some institutions are integrating data from other learning tools and systems as well.

Timothy Harfield, Senior Product Marketing Manager for Blackboard Analytics

Photos:

AFP Tami Chappell

Source:

1Fritz, J. (2013, April 30). Using Analytics at UMBC: Encouraging Student Responsibility and Identifying Effective Course Designs. Retrieved November 8, 2017, from https://www.educause.edu/ir/library/pdf/ERB1304.pdf.

 

End of Comments