Integrated Proof of Concept

Goals

The TOSCA-MP Proof of Concept (PoC) comprises a unified platform that integrates components developed separately by different partners. To reduce the complexity of the system and to enable easier integration, the paradigm of Service Oriented Architectures (SOA) has been adopted. SOA also allows a better scalability of the overall system and many vendors in the broadcast industry already offer service-oriented interfaces for their systems

The TOSCA-MP PoC has been designed such that it supports the implementation of the TOSCA-MP business goals and the respective workflows.

Logical System Design

The Logical System Design of TOSCA-MP provides a high-level definition of the platform: it shows an overview of the four main components in TOSCA-MP, how they are connected and which kind of data is exchanged between them:

• The Metadata Production Management Framework (MPMF) builds the core of the internal communication architecture and is responsible for the mediation between components in the overall system. The exchanged information can be metadata coming from the components or control-/configuration-data needed by the components.
• The Distributed Repository Framework (DRF) allows the realization of application specific Network based distributed repositories for essence and metadata.
• Services in TOSCA-MP are defined as “components that provide their functionality over the network by a remote well-defined interface”. Several services have been developed for and/or integrated into the TOSCA-MP PoC. There are services for automatic feature extraction and according metadata generation, semantic enrichment and linking, benchmarking methods, etc.
• Graphical User Interfaces (GUI) for the user / administrator to communicate with the system. In the context of TOSCA-MP the GUIs provide functions to the user to perform queries, review search results and to control the overall system.

A comprehensive Version including all details of the TOSCA-MP System Design can be found in deliverable D6.3.

The Proof of Concept realisation
The workflow envisaged by TOSCA-MP globally consists of three main steps, each of which is supported by specific components within the PoC, aiming at different types of users, see figure below.

In the first step, the automated part of the PoC uses feature extraction services to automatically generate metadata for audiovisual content. This part of the PoC implements the appropriate processing chain and can typically be operated by an (IT-) administrator via an appropriate Graphical User Interface (GUI).

The automatically extracted features can be inspected interactively in the next step. Appropriate applications visualise the generated metadata from the first step and allow an expert user to verify and / or interactively revise them as annotations to the processed content.

In the final step, the TOSCA-MP components aim at end users such as editors and journalists, allowing them to flexibly search for (pieces of) content that fit their needs. The available indexing service, search engine and search result presentation application take into account the metadata and annotations generated in the previous steps.

Automatic feature extraction

The system that realizes the automatic feature extraction is based on the paradigms of Service Oriented Architecture (SOA), which are increasingly being adopted in media production environments. The controlling entity in the PoC that orchestrates the automatic feature extraction on AV essence (based on a BPMN 2.0 process) is the MPMF. It interfaces with administrator GUIs on one hand and with various feature analysis services and storage (Distributed Repository Framework, DRF, see D5.1) on the other. It includes a service/workflow configuration framework and extended process error handling. The various components were integrated by means of appropriate (REST- or SOAP-based) web service interfaces (see D6.3, D6.5 and D6.6).

Along with the Control & Configuration application (depicted as Control&Config GUI in the figure above) the MPMF provides two main functionalities:

1. Configuration of a workflow (Control&Config GUI – Workflow Configuration) for flexible adjustment of all process and service parameters;
2. Process execution and monitoring (Control&Config GUI – jBPM5 Process Engine – TOSCA Integration Layer).

Interactive annotation and analysis result visualisation
For the realization of the 2nd step following components are available in the TOSCA-MP PoC:

• AVDP viewer, which presents timeline-based metadata of one metadata document (XML document conforming to the MPEG-7 Audiovisual Description Profile (AVDP)) containing analysis results.
• GUI for interactive annotation, which allows visualizing as well as editing of results from the automated feature extraction.

Search and visualisation

The MPMF, based on the features and annotations that have been automatically created, accesses an Indexing service to trigger the process of indexing all new annotations generated during the automatic feature extraction. This index can consecutively be used by the Search service to find and retrieve appropriate AV content from the DRF. Every time a new indexing is required, the index is updated and the changes are reflected by the Search service.

The Search application and visualisation GUIs are also part of the PoC. Please refer to D3.3 (chapter 3.1) for further information on Semantic content annotation and indexing and the Semantic search engine. Following elements have been made available in the PoC:

• Exploratory based semantic search application;
• New search result visualization paradigms: visual clustering methods.

Standardisation of Service Interfaces

For the realisation of the PoC current SOA-based approaches in the media and entertainment sector have been considered as well as implementations in line with the joint EBU-AMWA FIMS (Framework for Interoperable Media Services) project which targets at standardising interfaces for common services in media production. The TOSCA-MP results have been reversely communicated to FIMS as well.