The second MDN Workshop focusing on Artificial Intelligence (AI) for metadata extraction and use in the broadcast environment was held at the EBU 5-7 June 2018. The Metadata Developer Network is a technical group organized by the European Broadcasting Union's Strategic Programme on Media Information Management and AI (MIM_AI). The MDN is an active community for developers for knowledge sharing and collaboration. The annual MDN Workshop events are open to the public.
The EBU operates a reflector and for several years they have been working on AI to do automatic metadata extraction. to help people understand the importance of micro services and service architectures. Speech to text, translation, facial recognition are three of the most obvious use cases.
This year's even started with a unique "hands-on" day, providing those attending with the chance to see, close up, a great many demonstrations from a diverse group of players ranging from public broadcasters to start-ups.
On day two and three, one-hour in-depth presentations went deeper into the available technologies and how they are being tested and will be potentially put to use. Presentations included those from public broadcasters YLE, NRK, NHK, TV2, RTS, RAI, NPO, SRF, as well as the Olympic Broadcasting Service. Subtitling company Telestream gave a compelling presentation that featured automatic live subtitling right as the presentation was made. It is not yet 100% perfect and it was a relief to see some mistakes in the text, but not very many! Other start-ups included Valossa, and several EU-funded projects as well.
A few lessons from the hands-on day demonstrated that the Nordic participants are very much leading the way. The general feeling that amazing things are happening, for example the Speech to Text Recommendation Engine, and the Video Frame Fingerprinting technologies, both presented by YLE. The Nordic players are definitely leading the charge and showing all of us the way forward.
Joanneum Research demonstrated a FIMS-AME implementation in practice, showing that a lot of challenges still lie ahead ahead for the EBU group to make it more easy to integrate the AME services within the FIMS framework. Joanneum also demonstrated its approach to the evaluation of AI metrics, which is difficult since most cloud service providers that work on AME technologies, but most are "black boxes" making it problematic to evaluate them well.
Key presentations from the hands on day also included:
VRT (BE) : Automatic Keyword spotting on the VRT broadcast radio archive
Valossa (FI) : Video Recognition and Intelligence Platform
RAI (IT) : Integrated Cloud Based Services Platform - which was able to be trained to detect specific landmarks from an entire archive of content.
RTS (CH) : demonstrated its in-house-AME extraction solutions focused on facial recognition.
Fraunhofer (DE) : this presentation focused on detection of re-purposed content - and detecting clip copies within an archive.
NPO (NL) : presented its Automated Subtitling Cloud Service
AUTH (GR) : an EC-funded project conducted by the Artistotle University of Thessaloniki examining Multi-Drone Based Media Production - in terms of coordinated and autonomous flight of multiple drones and possible measures to increase security standards. synchronising and automating them.
Limecraft - another EC-funded project, presented how to harness AI for the radical automation of closed captioning and localisation. They are searching for help with use cases together with Finnish pubilc broadcaster YLE.
NRK (NO) - prsented a stellar example of how they have fully handed over their Origo search and production framework to the operational people. This is a good example of how it has been possibly to go fully operational using this innovative technology.
There was consensus amongst the participants that the hands-on day was a very good way to get close to what is happening.
Highlights of the sessions from day two and three included:
YLE's presention of its experiences integrating Olympic Broadcasting Services (OBS) and Gracenote metadata into the online experience, offering a detailed outline of its approach and the lessons learned. OBS separately presented its workflow for metadata processing and distribution of the Olympics broadcast data feed.
The Nordig Metadata Group - lead by Randi Volle (NRK) and Peter Mølsted of Danmarks Radio) presented the event metadata exchange format they developed for live and on-demand services. On the Nordic front NRK also presented its approach to using algorithms for automation, YLE demonstrated use cases related to tagging, and Nordic commercial broadcaster TV2 presented its analytics platform, "Sumo". NRK also presented its totally integrated project, "Origo" the last stage of this very advanced metadata project in terms of connecting commercial products like MAM to their modern metadata & production infrastructure.
NHK (JP) presented its novel approach to linking TV viewing and user behavior on a smartphone. And subtitling was very much in the spotlight, as NPO (NL) and Telestream (US) demonstrated, with an obviously and immediately relevant use case.
In future events, it is planned to add three new focus areas:
- a sports metadata stream with the participation of a number of public broadcasters. - a stream focused on FIMS MCMA (AI) - integrating AI tools into the workflow, and - a Wikidata stream focused on the management of a common vocabulary.
The full presentations from the 2018 MDN Workshop are available only to workshop participants and EBU members. You can access further information about joining the group on this link.