tocsca-logo FP7 EU

Pilot field trials to assess quality metrics

 benchmarking, quality  Comments Off on Pilot field trials to assess quality metrics
Feb 222013
 

Prior to the 1st EC review of TOSCA-MP in Feb. 2012 in Brussels, a pilot field trial was performed in order to assess prototypes of quality analysis tools developed in the project. The field trial was hosted by the Flemish broadcasting corporation (VRT), and 28 of their experts volunteered to participate in the two-part experiment.
pilotfieldtrial201302
First they were asked to rate the perceived sharpness of several viewed videos without any reference and then they compared two consecutively watched videos on a discrete comparison scale. Additionally, half of all tests were accomplished with an eye tracking system. For a better understanding on how people perceive the quality of a video and which parts are prioritized for judging the quality of a video sequence, we analyze the eye-tracking data and the resulting regions of interest obtained during scoring image quality. The resulting mean opinion scores of human perception are used to validate the sharpness algorithm which is one of several objective quality metrics that are developed within TOSCA-MP. Many thanks to all staff members of the Flemish public broadcasting company VRT who kindly participated in the subjective experiments!

Successful Video Browser Showdown 2013

 benchmarking  Comments Off on Successful Video Browser Showdown 2013
Jan 122013
 

Like in 2012, TOSCA-MP co-organised the Video Browser Showdown as a session of the MMM 2013 conference, held from Jan. 7-9 in Huangshan, China. The Video Browser Showdown is a live video browsing competition where international researchers, working in the field of interactive video search, evaluate and demonstrate the efficiency of their tools in presence of the audience. The aim of the Video Browser Showdown is to evaluate video browsing tools for their efficiency at “Known Item Search” (KIS) tasks with a well-defined data set in direct comparison with other tools. For each KIS task the searchers of the six participating teams from Europe and Asia need to interactively find a short video clip in a one-hour video file within a specific time limit.

Like in 2012, TOSCA-MP provided content collected by its archive partners on the MAMMIE platform and helped organising the competition. There were two rounds of the competition, one with expert users, the other one with novice users from the audience. The analysis of the search and some system logs of the 2012 competition was presented as a paper in the MMM 2013 main conference, and we are confident, that the analysis of the results of the 2013 competition again provides interesting insights.

vbs13_scoreboard

Scoreboard of one of the rounds (photo by Klaus Schöffmann from AAU Klagenfurt).

TOSCA-MP co-organises Video Browser Showdown 2013

 benchmarking  Comments Off on TOSCA-MP co-organises Video Browser Showdown 2013
May 082012
 

After a successful first event at MMM 2012 in Klagenfurt, Austria, TOSCA-MP is again co-organising this live video browsing competition in 2013. The aim of the Video Browser Showdown is to evaluate video browsing tools for their efficiency at “Known Item Search” (KIS) tasks with a well-defined data set in direct comparison with other tools. For each KIS task the searchers need to interactively find a short video clip in a one-hour video file within a specific time limit.

Draft guidelines based on those from 2012 are published and we request feedback until June 15, 2012. The call for submissions can be found on the MMM 2013 pages.

TOSCA-MP contributes 3 use cases to FIMS Phase 2

 benchmarking  Comments Off on TOSCA-MP contributes 3 use cases to FIMS Phase 2
Jan 312012
 

FIMS (Framework for Interoperable Media Service) is a joint effort of AMWA and the EBU to define an interoperable service oriented architecture for media production. Recently, FIMS has asked for submissions of use cases for phase 2 of FIMS. TOSCA-MP has contributed three use cases:

  • Automated metadata extraction services
  • Media query and search services
  • Benchmarking services for automated metadata extraction and search

The documents submitted to FIMS can be found here.

TOSCA-MP co-organised Video Browser Showdown 2012

 benchmarking  Comments Off on TOSCA-MP co-organised Video Browser Showdown 2012
Jan 152012
 

DSC_0859-e1330957216890The Video Browser Showdown (held as a separate session of MMM 2012 in Klagenfurt, Austria) is a live video browsing competition where international researchers, working in the field of interactive video search, evaluate and demonstrate the efficiency of their tools in presence of the audience. The aim of the Video Browser Showdown is to evaluate video browsing tools for their efficiency at “Known Item Search” (KIS) tasks with a well-defined data set in direct comparison with other tools. For each KIS task the searchers need to interactively find a short video clip in a one-hour video file within a specific time limit.

TOSCA-MP provided content collected by its archive partners on the MAMMIE platform and helped running the competition. The two rounds of the competition – one with expert users, the other one with novice users from the audience – brought interesting insights in which types of browsing and interactive search approaches work for which type of content and user.

© 2020 TOSCA-MP - Task-Oriented Search and Content Annotation for Media Production
The research leading to the presented results has received funding from the European Union's
Seventh Framework Programme (FP7/2007-2013) under grant agreement n° 287532. - Imprint