Telegraph Media Group increases syndication opportunities with Elastic on Google Cloud

The Telegraph Media Group (TMG) is one of the UK's best recognized print and digital news publishers. It includes The Telegraph newspaper that was founded in 1855 and today comprises The Telegraph website and app, The Daily Telegraph and The Sunday Telegraph print titles, and The Telegraph Edition app. Together they deliver award-winning journalism to a monthly audience of more than 20 million people, including more than half a million digital subscribers.

Like most news publishers today, TMG has a diverse business model based on subscriptions, print circulation, advertising, events and content syndication. It also has a fast-growing e-commerce division, which includes a popular travel section where readers can book exclusive vacation offers including hotels, rental homes, and cruises.

To remain competitive against traditional news sources and increased competition from the likes of social media, TMG needs to fully understand how its customers engage with content across all its channels. This is especially important for subscriber growth. For example, TMG can use this data to understand which articles are driving subscriptions and which are the most engaging for readers.

Another challenge to staying competitive TMG faced was its content syndication architecture. TMG has several content management systems that host more than 2 million articles. The effort required to integrate the multiple systems prevented TMG from maximizing the revenue opportunities and impeded distribution to media channel partners.

Armed with a plan to meet these challenges head-on and make the best possible use of its content, TMG reached out to Elastic.

Toby Wright, is the CTO at TMG. He tells us, "We'd been using Elastic's Logstash to ingest and transform data for a number of years. When we spoke to the team at Elastic about our goals, they demonstrated the Elastic Enterprise Search and analytics engine, and it was clear that it was the technology we needed to help meet our growth and distribution objectives."

Now, when a reader browses a TMG website, the visit generates several data attributes including page views, dwell time, and the all-important subscription conversion rate. This real-time data, up to 40GB in volume per day, is stored in Elastic where it is held for eight days.

Ciro Rizzo, Head of Engineering, Platforms with TMG, says, "At any time Elastic is storing about 400GB of data, which is around 91 years of photos taken by an average mobile phone user. We use this information to display the top performing articles on a 30-meter-wide screen in our newsroom so that journalists, editors, visitors, and advertisers can see the most read articles, how many readers have subscribed that day, and much more," he says. "This real-time analytics platform is called Pulse, it's also available to journalists and other employees via their laptops and a soon-to-be-launched mobile application."

Elastic is also at the center of TMG's content management strategy. To overcome the challenge of having millions of articles stored in multiple content management systems, Rizzo and his team came up with the concept of a content API that pulls all the documents into Elastic for syndication and distribution. TMG calls this content API the Unified Content Model (UCM), and it currently stores about 2 million articles dating back to 2001. In the future, TMG plans to delve even further into its archive and bring more articles into the system.

TMG also made the important decision to deploy Elastic Cloud on Google Cloud Platform, which now hosts the majority of the organization's systems. Rizzo says, "The great advantage for us is that Elastic manages the cluster on Google Cloud Platform. They make sure that their systems are always aligned with our needs, that they are always available, and that everything is backed up automatically. It really reduces the time we spend on upkeep and maintenance. Instead, we can focus on the deployment of new features to drive subscriptions and other growth areas."

The Travel Section of the website also harnesses the power of Elastic, managing all the data that relates to location dates, availability, and pricing. This enables TMG to organize and present offers that are relevant to the article being viewed and so generate revenues and commissions from these sales.

Traffic driven to TMG websites in search of news about TV programs and schedules is another potential source of subscribers. All TV listings data from a TMG partner is consumed by an API into Elastic from which information about a show can be displayed on the website including the start time, duration, channel, and even the cast.

TMG can now respond in real-time to changes in customer behavior and set up alerts based on predetermined targets, with Elastic playing a key role in this improvement.

Justyna Owczarek, Head of Data, Delivery at TMG, says, "Thanks to the Elastic Observability features in Pulse, website editors rarely miss an opportunity to boost engagement and conversions — for example, by promoting a trending article from the second page to the home page where it is exposed to even more visitors and potential subscribers."

As well as helping TMG work towards its content syndication goals, Elastic also reduces the burden on internal analytics teams. Owczarek says, "In the past we had to spend a lot of time pulling together manual reports. If you were in editorial or a journalist, you also needed a background in SQL or other programming languages to analyze article data. Now all that information is available with just a click of a mouse or a touch on a mobile screen."

The Pulse tool has already found favor with TMG employees, with up to 900 daily visitors and 5,000 page views. Amir Zareian is a Senior Data Engineer at TMG and uses Elastic daily. Zareian says, "Elastic delivers real-time data ingestion and real-time analysis, alongside a sophisticated alert system with none of the complicated set up typically associated with these features. It’s great to see so many employees taking advantage of it."

Elastic also helps to speed up and streamline the setup of syndication of TMG content with new media partners. In addition to existing collaborations with Bloomberg, the NLA, Yahoo MSN, Apple News, and Facebook.

Rizzo says, "Because we had already built the content API, it was much faster to launch on the Facebook channel. There’s a real sense that we can now send and syndicate our content with much less effort than before."

Looking to the future, TMG hopes to make wider use of new Elastic features, including the growing opportunities for machine learning to test headlines, or find the topics that most interest readers. It also plans to deploy Pulse as a mobile application with even more data available for analysis, and to use Elastic as the platform for its website search in addition to travel and TV listings.

"It all boils down to a better understanding of our audience," says Wright. "We have a target of 10 million registered users, and a million subscribers by 2023. We are excited about how quickly Elastic is evolving and how they will play a central role in helping us reach these goals — and continue to publish world-leading journalism."