Netflix, the video-on-demand streaming company, has seen a surge in its growing number of users every day as well as in the viewership of its TV shows. It is constantly striving to provide an enriching experience to its viewers. To keep pace with the ever-increasing demands of user experience, Netflix is introducing a collection of tools and algorithms to make its content more audience relevant.
AVA( Aesthetic Visual Analysis)- analyses large volumes of images obtained from video frames of a particular TV show to set as the title image for that show. Netflix understands that a more visually appealing title image plays an incredibly important role assisting a viewer find new shows and movies to watch.
Usually, content editors had to go through tens of thousands of video frames for a show, to select a good title image. To give you a gist of the effort required- a single one-hour episode of ‘Stranger Things’, consists of nearly 86,000 static video frames. Imagine sieving through each one of these frames painstakingly to find the perfect title image that will not only connect with the viewers, but also give them a gist of the storyline.
To top it all up, the number of frames can go up to a million depending on the number of episodes in a show. This task of manually screening the frames is almost impossible and labor intensive, if not ineffective.
Additionally, the editors choosing the image stills require an in-depth expertise of the source content that they’re intended to represent. Considering Netflix has an exponentially increasing catalog of shows, this presents a very challenging expectation for the editors to surface meaningful images from videos.
Enter AVA, using its image classification algorithms for sorting the right image at the right time.
The ever-growing number of images on the internet space has led to challenges in its processing and classification. To address this concern, a research team from University of Barcelona, Spain in collaboration with Xerox corporation has developed a method called Aesthetic Visual Analysis (AVA) as a research project. The project contains a vast database of over 2.5 lakh images combined with metadata such as
Using statistical concepts like standard deviation, mean score and variance, AVA rates images. Based on the distributions computed from these statistics, they assess the semantic challenges and choose the right images for the database. AVA primarily alleviates the issues of extensive benchmarking and trains more images. They also enable images to get a better aesthetic appeal. Computing performance can be significantly optimised to have lesser impact on the hardware.
You can get more insights by reading the Research paper.
The process takes place in 3 steps:
The ‘best’ image is chosen considering three important aspects– the lead actors, visual range and sensitivity filters.
Source: Netflix Blog
Netflix’s decision to use AVA will not only save manual labour, but also reduce the cost involved in having manpower source through millions of images in order to get that one perfect shot.
This unique approach will help in obtaining meaningful images from video and thus enable creative teams to invest time in designing stunning artwork .
As for its users, a good title image means establishing a deeper connection to the show’s characters and storyline, thus improving their overall experience.
To understand the intricate workings of AVA, you can read Netflix engineering team’s original post on this topic on Medium.
Netflix bring in Verna Myers as new VP of Inclusion strategy to boost cultural diversity
I remember deciding to pursue my first IT certification, the CompTIA A+. I had signed…
Key takeaways The transformer architecture has proved to be revolutionary in outperforming the classical RNN…
Once we learn how to deploy an Ubuntu server, how to manage users, and how…
Key-takeaways: Clean code isn’t just a nice thing to have or a luxury in software projects; it's a necessity. If we…
While developing a web application, or setting dynamic pages and meta tags we need to deal with…
Software architecture is one of the most discussed topics in the software industry today, and…