Call for evaluations proposals is now over, but is still available for a reference here.

The evaluation chairs have selected the following four proposals to be considered for the FG2015 special session on evaluations (final acceptance will be based on the submitted evaluation papers):

Video Person Recognition Evaluation

Organizers: Ross Beveridge, Bruce Draper, Patrick J. Flynn, Jonathon Phillips


Summary: This evaluation will address the problem of recognizing individuals in videos. The individuals in the videos are carrying out actions such as picking up an object or blowing bubbles; they are observed by the camera but the camera is not the individuals’ center of attention. The evaluation emphasizes complicating factors in video taken by people using common handheld devices in everyday settings. As a control, it also includes video acquired from a tripod mounted high quality camera. It is assumed most approaches will emphasize face recognition, but in general all or most of the people are in full view and innovative approaches may use visual cues beyond just the face.

Important reference: J.R. Beveridge, P.J. Phillips, D. Bolme, B. A. Draper, G.H. Givens, Y.M. Lui, M.N. Teli, H. Zhang, W.T. Scruggs, K.W. Bowyer, P.J. Flynn, S. Cheng: The Challenge of Face Recognition from Digital Point-and-Shoot Cameras, Proceedings of BTAS 2013, September 2013, pp. 1-8.

The Kinship Verification in the Wild Evaluation

Organizers: Jiwen Lu


Summary: Over the past two decades, many face image analysis problems have been investigated in computer vision. Representative examples include face alignment, face recognition, age estimation, facial behaviour analysis, gender classification and ethnicity recognition. Recent advances in face analysis have shown that it is possible to infer the kin relation of persons from their facial images. While kinship verification from facial images is an interesting and challenging problem, the performance of existing kinship verification approaches are still far from satisfying, especially when face images are captured in the wild. In this evaluation, we are interested in going a step further and evaluate the performance of possible solutions for facial kinship verification in the wild. There are many potential applications for kinship verification such as family album organization, missing parent/child search, and social media analysis.

Important reference: Jiwen Lu, Xiuzhuang Zhou, Yap-Peng Tan, Yuanyuan Shang, and Jie Zhou, Neighborhood Repulsed Metric Learning for Kinship Verification, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 36, no. 2, pp. 331-345, 2014.

Age Progression Competition

Organizers: Andreas Lanitis, Nicolas Tsapatsoulis


Summary: The topic of face-aging received increased attention by the computer vision community during the recent years. This interest is motivated by the important real life applications where accurate age progression algorithms can be used. Such applications include the development of person identification systems robust to aging variation and the generation of accurate predictions of the current appearance of missing persons. However, age progression methodologies may only be used in real applications provided that they have the ability to produce accurate age progressed images. The evaluation will involve the use of a purpose built dataset showing pairs of age separated face images of different individuals. Perspective participants in the competition will be provided with one image of a subject per pair and the age difference between the two images in the pair. Based on this information participants will be asked to predict the appearance of the face at the target age.

Important reference: A.Lanitis, Comparative Evaluation of Automatic Age Progression Methodologies. 8th IEEE International Conference on Automatic Face & Gesture Recognition, 2008. FG’08, 2008.

Fine-grained Evaluation on Face Detection in the Wild

Organizers: Junjie Yan, Bin Yang, Zhen Lei, Stan Z. Li


Summary: We provide a dataset to evaluate state-of-the-art academic and commercial face detection algorithms. It now has 4668 faces from 2350 real world images and keeps improving. Besides the bounding box annotations, we annotate landmarks, pose and many attributes. The additional annotations makes the fine-grained analysis of face detection results possible. For example, to evaluate the influence of glass in face detection, we can only take the faces labeled with glass into consideration in comparing different methods. In our benchmark, besides measuring the overall performance, we also report the specific performance with regard to occlusion, expression, gender, glasses, resolution and pose. In this way, we can clearly observe the advantages and disadvantages of face detection algorithm in different aspects. In this way, our dataset not only evaluates algorithms, but also provides a direction on how to improve it.