September 27, 2017

Vanessa el Khoury : Personalized Video Adaptation Framework (PIAF): constraint instantiation based user profile


Universal Multimedia Experience (UME) is the notion that a user should receive in-
formative adapted content anytime and anywhere. Personalization of videos, which adapts
their content according to user preferences, is a vital aspect of achieving the UME vision.
User preferences can be translated into several types of constraints that must be con-
sidered by the adaptation process, including semantic constraints directly related to the
content of the video. To deal with these semantic constraints, a ne-grained adaptation,
which can go down to the level of video objects, is necessary. The overall goal of this
adaptation process is to provide users with adapted content that maximizes their Quality
of Experience (QoE). This QoE depends at the same time on the level of the user’s sat-
isfaction in perceiving the adapted content, the amount of knowledge assimilated by the
user, and the adaptation execution time.
In video adaptation frameworks, the Adaptation Decision Taking Engine (ADTE),
which can be considered as the “brain” of the adaptation engine, is responsible for achiev-
ing this goal. The task of the ADTE is challenging as many adaptation operations can
satisfy the same semantic constraint, and thus arising in several feasible adaptation plans.
Indeed, for each entity undergoing the adaptation process, the ADTE must decide on
the adequate adaptation operator that satises the user’s preferences while maximizing
his/her quality of experience. The rst challenge to achieve in this is to objectively mea-
sure the quality of the adapted video, taking into consideration the multiple aspects of
the QoE. The second challenge is to assess beforehand this quality in order to choose
the most appropriate adaptation plan among all possible plans. The third challenge is to
resolve conicting or overlapping semantic constraints, in particular conicts arising from
constraints expressed by intellectual property rights owner about the modication of the
In this thesis, we tackled the aforementioned challenges by proposing a Utility Func-
tion (UF), which integrates semantic concerns with user’s perceptual considerations. This
UF models the relationships among adaptation operations, user preferences, and the qual-
ity of the video content. We integrated this UF into an ADTE. This ADTE performs a
multi-level piecewise reasoning to choose the adaptation plan that maximizes the user-
perceived quality. Furthermore, we included intellectual property rights in the adaptation
process. Thereby, we modeled content owner constraints, and proposed a heuristic to re-
solve conicting user and owner constraints. Moreover, we developed the Semantic Video
Content Annotation Tool (SVCAT), which produces structural and high-level semantic
annotation according to an original object-based video content model. We modeled as well
the user’s preferences proposing extensions to Moving Picture Expert Group (MPEG)-7
and MPEG-21. All the developed contributions were carried out as part of a coherent
framework called PIAF. PIAF is a complete modular MPEG standard compliant frame-
work that covers the whole process of semantic video adaptation.
We validated this research with qualitative and quantitative evaluations, which assess
the performance and the eciency of the proposed adaptation decision-taking engine
within Personalized vIdeo Adaptation Framework (PIAF). The experimental results show
that UF has a high correlation with subjective video quality evaluation.