Should online educational platforms offer courses following a schedule or release them on demand?

Researchers from Carnegie Mellon University and University of Pennsylvania have published a new Journal of Marketing study that examines online educational platforms and the question of whether they should release content through a scheduled format that resembles a traditional university course or use an on-demand release strategy.

The study is titled “More Likely to Pay but Less Engaged: The Effects of Switching Online Courses from Scheduled to On-Demand Release on User Behavior” and is authored by Joy Lu, Eric T. Bradlow, and J. Wesley Hutchinson.

In 2011, the online education industry catered to around 300,000 consumers. In 2021, it served 220 million, thanks in part to increased enrollment during the COVID-19 pandemic. Traditional universities and institutions are increasingly adopting hybrid course formats. For example, the number of full-time online MBA students surpassed in-person MBA students for the first time in the 2020-21 academic year.

Today, online educational platforms like Coursera and edX offer a range of flexible course content, but these firms are faced with a tricky question: Should they release content through a scheduled format that resembles a traditional university course with a subset of lectures and quizzes available at the start of each week, or should they follow in the footsteps of Netflix and Hulu with an on-demand release strategy where all the material is immediately available upon registration?

This new article finds that the choice of format for content release not only impacts overall user engagement and firm revenue but also user performance and learning outcomes.

The researchers studied over 67,000 users taking an introductory marketing course on Coursera consisting of 32 short lecture videos and four quizzes. The study took advantage of a natural experiment policy change where the platform switched the course from a scheduled format to an on-demand release format while keeping the actual content the same.

The scheduled format closely resembled a traditional university course, with some of the study material available at the beginning of each week for four weeks. In the on-demand format, all four weeks of content was made available upon registration. All users could take the course for free or opt into paying for a completion certificate, either as a one-time fee in the scheduled format or a monthly subscription in the on-demand format.

More users, less engagement

The study’s findings show that the switch to on-demand content doubled the percentage of paying users from 14% to 28%. Lu explains that “the on-demand format was successful in increasing short-term firm revenue by bringing in more paying users. On the downside, the switch resulted in significantly lower lecture completion rates and lower quiz performance.”

The on-demand format also negatively impacted downstream platform engagement. The marketing course was promoted in a “Business Foundations” set with three other courses on operations, accounting, and finance.

“Compared to users in the scheduled format, those in the on-demand format ended up taking one or two fewer additional courses six months after the focal marketing course,” says Bradlow.

Analysis of user activity reveals two new learning patterns:

• A subset (13%) of users in the on-demand format continued to return and take quizzes well beyond the recommended four-week course period. The greater flexibility in the on-demand content release and payment structure likely enabled these users to “stretch out” their consumption.
• The on-demand format increased the practice of binging—with user activity being clumped together (i.e., more binging) as compared to being evenly spaced out (i.e., less binging). In the scheduled format, binging was negatively related to course performance, which is consistent with the intuition that binging reflects procrastination or cramming. However, in the on-demand format, binging was positively related to performance, suggesting that on-demand users may binge as a form of strategic time management by setting aside time to consume in spurts.

Real-world implications

This study offers vital lessons for chief marketing officers in the online education space:

• The switch to the on-demand format attracted a set of users who were more likely to pay, but were less engaged in the course. On-demand content is potentially helpful at bringing in a new user segment or expanding the current user base, similar to universities offering concurrent hybrid MBAs that cater to busy students with full-time jobs. Managers must consider the trade-off between offering structure versus flexibility and may even consider offering different content release options simultaneously but at different price points by emphasizing their unique features.
• Platforms may need to adapt their content to account for users who binge on content and others who space it out over time. For example, firms can include more recaps or reviews to reduce frustration resulting from users forgetting content. It may even be a viable strategy to embrace the prevalence of binging among users by highlighting or designing sets of lectures that are “bingeable” versus more modular.
• Many online platforms offer episodic content that may be released in installments and thus need to make decisions regarding the content release format.

“Our study provides insights that help managers anticipate the potential consequences of such decisions,” says Hutchinson. “On-demand content offers clear short-term benefits in terms of increased revenue but potentially long-term costs in terms of decreased engagement and new challenges in maintaining user engagement.”