Learning reduced models for motion estimation on ocean satellite images
The paper describes a learning method on sliding windows for estimating apparent motion on long temporal satellite sequences acquired over oceans. A «full model», which is defined on the pixel grid, is chosen to describe the dynamics of motion fields and images, based on heuristics of divergence-...
Gespeichert in:
| Datum: | 2011 |
|---|---|
| Hauptverfasser: | , , , |
| Format: | Artikel |
| Sprache: | English |
| Veröffentlicht: |
Морський гідрофізичний інститут НАН України
2011
|
| Schriftenreihe: | Екологічна безпека прибережної та шельфової зон та комплексне використання ресурсів шельфу |
| Schlagworte: | |
| Online Zugang: | http://dspace.nbuv.gov.ua/handle/123456789/112619 |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| Назва журналу: | Digital Library of Periodicals of National Academy of Sciences of Ukraine |
| Zitieren: | Learning reduced models for motion estimation on ocean satellite images / I. Herlin, D. Béréziat, K. Drifi, S. Zhuk // Екологічна безпека прибережної та шельфової зон та комплексне використання ресурсів шельфу: Зб. наук. пр. — Севастополь, 2011. — Вип. 25, т. 2. — С. 66-78. — Бібліогр.: 12 назв. — англ. |
Institution
Digital Library of Periodicals of National Academy of Sciences of Ukraine| id |
irk-123456789-112619 |
|---|---|
| record_format |
dspace |
| spelling |
irk-123456789-1126192017-01-25T03:02:21Z Learning reduced models for motion estimation on ocean satellite images Herlin, I. Bereziat, D. Drifi, K. Zhuk, S. Моделирование процессов в Мировом океане The paper describes a learning method on sliding windows for estimating apparent motion on long temporal satellite sequences acquired over oceans. A «full model», which is defined on the pixel grid, is chosen to describe the dynamics of motion fields and images, based on heuristics of divergence-free motion and advection of image brightness by the velocity. The image sequence is split into small temporal windows that half overlap in time. Image assimilation in the full model is applied on the first window to retrieve its motion field. This makes it possible to define subspaces of motion fields and images and a «reduced model» is defined by applying the Galerkin projection of the full model on these subspaces. Data assimilation in the reduced model is applied on this second window. The process is iterated for the next window until the end of the whole image sequence. Each reduced model is then learned from the previous one. The main advantage of the approach is the small computational requirements of the assimilation in the reduced models that make it feasible to process in quasi-real time image acquisitions. Twin experiments have been designed to quantify the full model and the learning method on sliding windows and demonstrate the quality of the motion fields estimated by the approach. У статті описується метод вкладених вікон, використовуваний для розрахунку параметрів руху при обробці зображень океану, отриманих за допомогою супутникових систем. «Повна модель», яка використовується для опису динаміки полів, заснована на рівнянні бездівергентного руху рідини і перенесення яскравості зображення швидкістю. Послідовність зображень розбивається на невеликі тимчасові вікна, з половинною перекриттям у часі. Асиміляція зображення в повній моделі проводиться для першого вікна. Це дозволяє визначити підпростори полів руху і зображень та побудувати «редуцiровану модель» проектуванням на ці підпростори методом Гальоркіна. Асиміляція даних в «скороченої моделі» застосовується для другого вікна. Цей процес повторюється для всієї послідовності вікон. Основною перевагою такого підходу є прискорення обробки, що дозволяє використовувати його при обробці зображень у темпі, близькому до реального часу. Переваги «скороченої моделі» продемонстровані чисельними експериментами використовуючи метод близнюків. В статье описывается метод вложенных окон, используемый для расчета параметров движения при обработке изображений океана, полученных с помощью спутниковых систем. «Полная модель», которая используется для описания динамики полей, основана на уравнении бездивергентного движения жидкости и переносе яркости изображения скоростью. Последовательность изображений разбивается на небольшие временные окна, с половинным перекрытием во времени. Ассимиляция изображения в полной модели проводится для первого окна. Это позволяет определить подпространства полей движения и изображений и построить «редуцированную модель» проектированием на эти подпространства методом Галеркина. Ассимиляция данных в «редуцированной модели» применяется для второго окна. Этот процесс повторяется для всей последовательности окон. Основным преимуществом такого подхода является ускорение обработки, что позволяет использовать его при обработке изображений в темпе, близком к реальному времени. Преимущества «редуцированной модели» продемонстрированы численными экспериментами используя метод близнецов. 2011 Article Learning reduced models for motion estimation on ocean satellite images / I. Herlin, D. Béréziat, K. Drifi, S. Zhuk // Екологічна безпека прибережної та шельфової зон та комплексне використання ресурсів шельфу: Зб. наук. пр. — Севастополь, 2011. — Вип. 25, т. 2. — С. 66-78. — Бібліогр.: 12 назв. — англ. 1726-9903 http://dspace.nbuv.gov.ua/handle/123456789/112619 551.465 en Екологічна безпека прибережної та шельфової зон та комплексне використання ресурсів шельфу Морський гідрофізичний інститут НАН України |
| institution |
Digital Library of Periodicals of National Academy of Sciences of Ukraine |
| collection |
DSpace DC |
| language |
English |
| topic |
Моделирование процессов в Мировом океане Моделирование процессов в Мировом океане |
| spellingShingle |
Моделирование процессов в Мировом океане Моделирование процессов в Мировом океане Herlin, I. Bereziat, D. Drifi, K. Zhuk, S. Learning reduced models for motion estimation on ocean satellite images Екологічна безпека прибережної та шельфової зон та комплексне використання ресурсів шельфу |
| description |
The paper describes a learning method on sliding windows for estimating apparent
motion on long temporal satellite sequences acquired over oceans. A «full model», which is
defined on the pixel grid, is chosen to describe the dynamics of motion fields and images,
based on heuristics of divergence-free motion and advection of image brightness by the
velocity. The image sequence is split into small temporal windows that half overlap in time.
Image assimilation in the full model is applied on the first window to retrieve its motion field.
This makes it possible to define subspaces of motion fields and images and a «reduced model»
is defined by applying the Galerkin projection of the full model on these subspaces. Data
assimilation in the reduced model is applied on this second window. The process is iterated for
the next window until the end of the whole image sequence. Each reduced model is then
learned from the previous one. The main advantage of the approach is the small computational
requirements of the assimilation in the reduced models that make it feasible to process in
quasi-real time image acquisitions. Twin experiments have been designed to quantify the full
model and the learning method on sliding windows and demonstrate the quality of the motion
fields estimated by the approach. |
| format |
Article |
| author |
Herlin, I. Bereziat, D. Drifi, K. Zhuk, S. |
| author_facet |
Herlin, I. Bereziat, D. Drifi, K. Zhuk, S. |
| author_sort |
Herlin, I. |
| title |
Learning reduced models for motion estimation on ocean satellite images |
| title_short |
Learning reduced models for motion estimation on ocean satellite images |
| title_full |
Learning reduced models for motion estimation on ocean satellite images |
| title_fullStr |
Learning reduced models for motion estimation on ocean satellite images |
| title_full_unstemmed |
Learning reduced models for motion estimation on ocean satellite images |
| title_sort |
learning reduced models for motion estimation on ocean satellite images |
| publisher |
Морський гідрофізичний інститут НАН України |
| publishDate |
2011 |
| topic_facet |
Моделирование процессов в Мировом океане |
| url |
http://dspace.nbuv.gov.ua/handle/123456789/112619 |
| citation_txt |
Learning reduced models for motion estimation on ocean satellite images / I. Herlin, D. Béréziat, K. Drifi, S. Zhuk // Екологічна безпека прибережної та шельфової зон та комплексне використання ресурсів шельфу: Зб. наук. пр. — Севастополь, 2011. — Вип. 25, т. 2. — С. 66-78. — Бібліогр.: 12 назв. — англ. |
| series |
Екологічна безпека прибережної та шельфової зон та комплексне використання ресурсів шельфу |
| work_keys_str_mv |
AT herlini learningreducedmodelsformotionestimationonoceansatelliteimages AT bereziatd learningreducedmodelsformotionestimationonoceansatelliteimages AT drifik learningreducedmodelsformotionestimationonoceansatelliteimages AT zhuks learningreducedmodelsformotionestimationonoceansatelliteimages |
| first_indexed |
2025-07-08T04:18:10Z |
| last_indexed |
2025-07-08T04:18:10Z |
| _version_ |
1837050933050081280 |
| fulltext |
66
© Isabelle Herlin, Dominique Béréziat, Karim Drifi, Sergiy Zhuk, 2011
УДК 551 .465
Isabelle Herlin 1,2, Dominique Béréziat 3,
Karim Drifi 1,2, Sergiy Zhuk 4
1 INRIA – B.P. 105, 78153 Le Chesnay Cedex, France.
2 CEREA, joint laboratory ENPC – EDF R&D, Université Paris-Est – Cité Descartes,
Champs-sur-Marne, 77455 Marne la Vallée, France.
3 Université Pierre et Marie Curie – 4 place Jussieu, 75005 Paris, France.
4 CWI – P.O. Box 94079, NL-1090 GB Amsterdam, Netherlands.
LEARNING REDUCED MODELS FOR MOTION ESTIMATION
ON OCEAN SATELLITE IMAGES
The paper describes a learning method on sliding windows for estimating apparent
motion on long temporal satellite sequences acquired over oceans. A «full model», which is
defined on the pixel grid, is chosen to describe the dynamics of motion fields and images,
based on heuristics of divergence-free motion and advection of image brightness by the
velocity. The image sequence is split into small temporal windows that half overlap in time.
Image assimilation in the full model is applied on the first window to retrieve its motion field.
This makes it possible to define subspaces of motion fields and images and a «reduced model»
is defined by applying the Galerkin projection of the full model on these subspaces. Data
assimilation in the reduced model is applied on this second window. The process is iterated for
the next window until the end of the whole image sequence. Each reduced model is then
learned from the previous one. The main advantage of the approach is the small computational
requirements of the assimilation in the reduced models that make it feasible to process in
quasi-real time image acquisitions. Twin experiments have been designed to quantify the full
model and the learning method on sliding windows and demonstrate the quality of the motion
fields estimated by the approach.
KEYWORDS: Motion Estimation, Data Assimilation, Model Reduction, Galerkin
projection
1. Introduction
Motion estimation from an image sequence has been intensively studied since
the beginning of image processing [1, 2]. The aim is to retrieve the velocity field
),( txw visualised by a discrete image sequence Z=zzZ=z
z tx,I=I=I …… 11 )( }{}{ .
The application of data assimilation techniques to motion estimation also
emerged a few years ago [3 – 5]. In the case of motion estimation, these
techniques aim to find the optimal solution to the equations describing the
temporal evolution of motion fields and images and to the observation equation,
which links the motion field to the observed image data. Their major drawbacks
are the memory and computer resources required that do not allow to process
long temporal sequences of large size images. To get round this problem,
reduction methods are required to apply the data assimilation on subspaces. In [6]
such reduced model has been proposed. Coefficients characterizing image
observations in the image subspace are assimilated in the reduced model to
estimate those characterizing the motion field.
67
In this paper, we focus on the estimation of motion on long temporal
windows of satellite images acquired over oceans. The image sequence is split
into small windows that half overlap in time. A «full model» is chosen in order to
approximately describe the dynamics of motion fields and images. Image
assimilation in the full model is applied on the first window to retrieve its motion
field. A learning process is designed that defines a «reduced model» from the full
model in the second window. This learning defines the subspaces used to
characterize motion and images and applies the Galerkin projection of the full
model on these subspaces. Data assimilation in the reduced model is then applied
for this second window. The learning method is iterated on the next window until
the whole image sequence has been processed.
The paper describes the two main components of the learning method on
sliding windows: the full model and its image assimilation process, the learning
of reduced models and their data assimilation systems.
Oceans are incompressible fluids and the 2D incompressible hypothesis still
remains a good approximation for image sequences if no or small vertical motion
occurs (no upwelling or downwelling). If the motion field is divergence-free
( 0)( =wdiv ), it is then only characterized by its vorticity ξ , according to the
Helmholtz orthogonal decomposition [7]. An equation on the dynamics of vorti-
city ξ is then included in the full model. As temporal integration of the vorticity
requires the knowledge of the velocity value at each time step, the discrete
computation of w from ξ is performed, based on an algebraic decomposition of
vorticity. The transport of image brightness by velocity, which is the usual optical
flow equation, is chosen to describe the image dynamics.
Section 2 describes the divergence-free image model used for motion
estimation on an image sequence. The algebraic method that computes w from
its vorticity ξ is also given. Section 3 explains how the solution is obtained by
minimizing a cost function with a strong 4D-Var (no error on the dynamics) data
assimilation method. The derivation of a reduced model by the Galerkin
projection is provided in Section 4. The learning method used to process long
temporal image sequences is fully described in Section 5. Section 6 provides
results on synthetic data for the full model and Section 7 for the learning method
on a long temporal window.
2. Definition of the full model
This section describes the divergence-free model that is used to determine
velocity from images, on the pixel grid, on the first window of the long temporal
sequence.
2.1. Divergence-free model
Vorticity characterizes a rotational motion while divergence characterizes
sinks and sources in a flow. A fluid motion ( )Tvu=w is described by its
vorticity
y
u
x
v
=
∂
∂
∂
∂ −ξ , under the hypothesis of null divergence [7]. ξ is chosen
as the first component of the state vector X of the full model. Deriving the
68
evolution law for ξ requires heuristics on the velocity w . The Lagrangian
constancy hypothesis, 0=
dt
dw
, is considered in the paper that can be expanded as
( ) 0
∂
∂
=ww+
t
w ∇. , or:
0=
∂
∂+
∂
∂+
∂
∂
y
u
v
x
u
u
t
u
(1)
0=
∂
∂+
∂
∂+
∂
∂
y
v
v
x
v
u
t
v
(2)
Let us compute the y-derivative of Eq. (1) and subtract it from the x-de-
rivative of Eq. (2), replace the quantity
y
u
x
v
∂
∂
∂
∂ − by the vorticity ξ , and we
obtain:
0=
∂
∂+
∂
∂+
∂
∂+
∂
∂+
∂
∂
y
v
x
u
y
v
x
u
t
ξξξξ
(3)
This is rewritten in a conservative form as:
0)( =∇+
∂
∂
w
t
ξξ
(4)
The observations that are used for the data assimilation process are images
acquired by satellites. The second component of the state vector is chosen as a
pseudo-image sI , which has the same dynamics than the image observation. It is
included in the state vector in order to allow an easy comparison with the image
observations at each acquisition date: they have to be almost identical. The evolution
law chosen for sI verifies the heuristics for the transport of images by velocities: this
is the well known Optical Flow Constraint Equation [1] expressed as:
0=∇+
∂
∂
wI
t
I
s
s (5)
or with the divergence-free hypothesis:
0)( =∇+
∂
∂
wI
t
I
s
s (6)
The divergence-free model is then defined by the state vector ( )TsI=X ξ
and its evolution system:
0)( =∇+
∂
∂
w
t
ξξ
(7)
0)( =∇+
∂
∂
wI
t
I
s
s (8)
69
2.2. Algebraic computation of w
When the state vector is integrated in time from an initial condition, using
Eqs. (7,8), the knowledge of ξ , sI and w is required. The velocity field w
should then be computed from the scalar field ξ as follow. A stream function ϕ
is first defined as the solution of the Poisson equation:
ξϕ =∆− (9)
Then, w is derived from ϕ :
T
yy
w
∂
∂−
∂
∂= ϕϕ
(10)
In the literature, Eq. (9) is usually solved in Fourier domain, with periodic
boundary conditions. An algebraic solution is proposed in order to allow Dirichlet
boundary conditions. An eigenfunction,φ , of the linear operator ∆− has to
verify λ=φ∆− with λ the associated eigenvalue. Explicit solutions of this
eigenvalue problem are the family of bi-periodic functions
)sin()sin()( mynx=yx,mn, ππφ with the associated eigenvalues 2222 mπ+nπ=λ mn, .
These functions form an orthogonal basis of a subspace of )(Ω2L , space of square-
integrable functions defined on the spatial domain Ω . Let )( mn,a be the coefficients
of ξ in the basis )( mn,φ . We have ∑
mn,
mn,mn, yx,a=yx, )()( φξ . It comes:
),(),(
, ,
, yx
a
yx
mn mn
mn ϕ
λ
ϕ ∑= (11)
We verify:
ξφλ
λ
φ
λ
ϕ ==∆−=∆− ∑∑ ),(),(),( ,
, ,
,
, ,
, yx
a
yx
a
yx mn
mn mn
mn
mn mn
mn
At each time step, having knowledge of ξ and )( mn,φ , the values of )( mn,a
are first computed. Then φ is derived by Eq. (11), using the )( mn,λ values, and w
by Eq. (10).
3. Strong 4D-Var Data Assimilation
Image assimilation is applied on the first window of the long sequence with
the full model described in Section 2.
We consider the state vector ( )Ts tyxItyx=y,tx,X ),,(),,()( ξ defined on
the space-time domain ],t[ N0×Ω . In order to determineX on this domain, the
4D-Var framework considers a system of three equations to be solved.
The first equation describes the evolution in time of the state vector X . This is
given by Eqs. (7, 8). For sake of simplicity, we summarize the system and
introduce the evolution model M for the state vectorX :
70
0)( =+
∂
∂
XM
t
X
(12)
We consider having some knowledge of the state vector value at initial date 0
which is described by the background value )( yx,Xb . As this initial condition is
uncertain, the second equation of the system involves an error term:
),(),()0,,( yxyxXyxX Bb ε+= (13)
The error )( yx,Bε is supposed Gaussian and characterized by its covariance
matrix )( yx,B .The last equation, named observation equation, links the state
vector to the image observations )( y,tx,I . It is expressed as:
),,()),,((),,( tyxtyxXHtyxI Rε+= (14)
with H the observation operator.
As the component sI is directly comparable to the observations, the operator
H reduces to a projection: sI=HX=XH )( . Image acquisitions are noisy and
their underlying dynamics could be different from the one described by Eq. (8).
An observation error, Rε , is used to model these uncertainties. It is supposed
Gaussian and characterized by its covariance matrix )( y,tx,R .
For discussing how Eqs. (12, 13, 14) are solved by the data assimilation
method, the state vector and its evolution equation are first discretized in time
with an Euler scheme. The space variables x and y are omitted for sake of
simplicity. Let dt be the time step, the state vector at discrete index k ,
tNk ≤≤0 , is denoted )()( dtkX=kX × . The discrete evolution equation is :
))(())(()()1( kXZkXdtMkXkX k=−=+ (15)
with ( )Tssk kξwkIdtkIkξwkξdtkξ=kXZ )))(()(()()))(()(()())(( .. ∇−∇− . We
assume that obsN image observations )( itI are acquired at indexes.
obsNi1 t<<t<<t LL . Looking for ))()0(( tNX,,X=X L solving Eqs.(15, 13,
14) is expressed as a constrained optimization problem: the cost function
+−−= ∫
Ω
− dydxXXBXXXJ b
T
b ))0()0(())0()0((
2
1
))0(( 1
dydxtItHXtRtItHX ii
N
i
i
T
ii
Obs
))()()(())()((
2
1
1
1 −−+ ∑ ∫
= Ω
−
has to be minimized under the constraint of Eq. (15). The first term of J comes
from Eq. (13). The second term of J comes from Eq. (14), which is valid at
observation indexes it .
(16)
71
The gradient of J is obtained from the directional derivative of J and from
the definition of an auxiliary variable λ that verifies the backward equation:
))()()(()1()( 1
*
kIkHXkRHK
X
Z
k Tk −++
∂
∂
= −λλ
with 0)( =Nλ t , the term ))()()(( kIkHXkRH T −−1 being only taken into
account at observation indexes it . It can be proven [8] that the gradient reduces to:
)0())0((1
)0( λ+−=∆ −
bX XXBJ
The cost function J is minimized using an iterative steepest descent method.
At each iteration, the forward time integration of X is performed which provides
J , then a backward integration of λ computes )0(λ and provides J∇ . An
efficient solver [9] is used to perform the steepest descent given J and J∇ .
4. Derivation of a reduced model
This section explains the derivation by Galerkin projection of a reduced
model from the full model described in Section 2.
We assume that we have knowledge of the background value bξ of vorticity
at the beginning of the studied temporal window. The first issue is to define
subspaces for vorticity fields and images, onto which the evolution equations (7)
and (8) are projected. These subspaces are defined by their respective orthogonal
basis ξΨ and IΨ . First, a Proper Orthogonal Decomposition transform (POD)
is applied to the image observations Zz
ZII ...1}{ == that defines ′IΨ . Second, bξ
is numerically integrated in time with Eq. (7). It provides snapshots, on which
POD is applied to obtain ′ξΨ . We keep the first K modes of ′ξΨ and the first L
modes of ′IΨ to obtain ξΨ and IΨ .
Let )(tai and )(tbj be the projection coefficients of )(x,tξ and )(x,tI s on
ξΨ and IΨ . )(x,tξ and )(x,tI s are then approximated by:
∑
=
≈
K
i
ii xΨtatx
1
, )()(),( ξξ (17)
∑
=
≈
L
j
jIjS xΨtbtxI
1
, )()(),( (18)
and replaced in Eqs. (7) and (8):
0)()()()()()(
1 1
,
1
,, =
∇⋅
+
∂
∂
∑ ∑∑
= ==
K
i
L
j
jj
K
i
iii
i xΨtaxΨtawxΨt
t
a
ξξξ (19)
0)()()()()()(
1 1
,
1
,, =
∇⋅
+
∂
∂
∑ ∑∑
= ==
L
i
L
j
iIj
K
i
iijI
i xΨtbxΨtawxΨt
t
b
ξ (20)
72
This system is projected on ξΨ and IΨ :
0,)()(,)( ,,
1
,
1
,, =
∇⋅
+
∂
∂
∑∑
==
ki
K
i
ii
K
i
ikk
k
ΨΨtaΨtawΨΨt
t
a
ξξξξξ (21)
0,)()(,)( ,,
1
,
1
,, =
∇⋅
+
∂
∂
∑∑
==
lIlI
L
j
ji
K
i
ilIlI
l
ΨΨtbΨtawΨΨt
t
b
ξ (22)
with , being the scalar product in the )(Ω2L space:
∫
Ω
= dxxgxFgf )()(, (23)
System (21, 22) is simplified to get:
KktakBtat
dt
da Tk ...1,0)()()()( ==+ (24)
LltblGtat
dt
db Tl ...1,0)()()()( ==+ (25)
with: T
K tata=ta ))()(()( …1 , T
L tbtb=tb ))()(()( …1 ,
B(k) a KK × matrix :
kξ,kξ,
kξ,jξ,iξ,
ji,
ψ,ψ
ψ,ψψw
=B(k)
∂)( ⋅
,
G(l) a LK × matrix :
lI,lI,
lI,jI,iξ,
ji,
ψ,ψ
ψ,ψψw
=G(l)
∂)( ⋅
.
Let T
R tbta=x,tX ))()(()( be the state vector of the reduced model. System
(24, 25) is rewritten as:
0)( =+ RR
R XM
dt
dX
(26)
RM being the Galerkin projection of the full model M on ξΨ and IΨ .
5. Learning reduced models on sliding windows
This section describes the learning method on sliding windows, with the full
model of Section 2 applied on the first window and the reduced models of
Section 4 applied on the following. This learning method allows to process long
temporal image sequences.
The discrete sequence Z1=z
zI=I …}{ is first split into short temporal
windows, with 4 to 6 images, that half overlap in time. These windows are
denoted mWi , with m the index. Images belonging to 1Wi are assimilated in the
73
divergence-free model described in Section 2. This allows the retrieval of the
vorticity on 1Wi .
The retrieved value at the beginning of 2Wi is taken as background vorticity
bξ required to learn the reduced model on 2Wi , as it has been explained in
Section 4. The coefficients of projection of images belonging to 2Wi are
assimilated in the reduced model to retrieve the vorticity coefficients and
compute the vorticity values and motion fields over 2Wi .This again provides bξ
for 3Wi and allows to learn the reduced model on 3Wi . The process is then
iterated until the whole sequence I has been analyzed.
The method is summarized in Figure 1.
Fig. 1. Learning reduced models on sliding windows.
The major advantage is that full assimilation is only applied on the first
temporal window 1Wi that has a short duration. It requires, at each iteration of the
optimisation process, a forward integration of M and a backward integration of
its adjoint [5]. The complexity is proportional to the image size multiplied by the
number of time steps in the assimilation window. On the next window mWi , the
complexity greatly decreases as the state vector involved in the reduced models
RM is of size L+K , which is less than 10 in the experiments.
6. Results of the full model
In order to quantify the method, it is applied on synthetic data produced by
twin experiments.
A sequence of five synthetic observations (see Figure 3) is obtained by time
integration of the divergence-free model from the initial conditions displayed in
Figure 2.
t1 t5 t7 t8 t9 t10 t11 t12t2 t3 t6
Assimilation Reduced Model
Assimilation Reduced Model
Assimilation Full Model
t4
T0 TF
Observation Dates tz
Assimilation Full Model
Assimilation Reduced Model
Assimilation Reduced Model
74
Fig. 2: Pseudo-image, vorticity (positive values are drawn in
white, negative ones in black) and motion field at t = 0.
Fig. 3. Observations.
For the assimilation experiment, the background of vorticity is set to zero and the
one of pseudo-image is the first observation. The result of the assimilation process is
the state vector ( )Ts kIk=kX )()()( ξ and its associated motion vector w(k) over
the discrete assimilation window. In Table 1, the error between the motion result and
the ground truth is given for our approach and four state-of-the-art image processing
methods: [1, 10 – 12] that use either a 2L regularization of motion [1] or a second
order regularization on the divergence [10 – 12].
This demonstrates that our approach is almost exact for this twin experiment.
7. Results of the learning method on sliding windows
Twin experiments were also designed to quantify the learning method on sliding
windows and its benefit for motion estimation on long temporal image sequences.
The full model was used, with initial conditions displayed in Figure 4.
Snapshots of sI were taken to create the observation images Z=z
zI=I …1}{ .
Assimilation of these data in the full and reduced models is then applied as desc-
75
Table 1: Error analysis: misfit between motion results and ground truth.
Method
Angular error (in deg.) Norm error (in %)
Mean
Std.
Dev.
Min Max Mean Min Max
[1] 15,26 9,65 0,33 67,12 24,98 0,85 93,10
[11] 12,54 9,49 0,17 68,49 20,03 0,51 87,74
[12] 10,41 5,34 0,06 35,58 18,07 0,09 92,31
[10] 10,61 6,92 0,00 56,62 18,01 0,01 97,74
Our
approach
0,18 0,10 0,00 0,572 0,41 0,00 19,47
described in Section 5 on six windows. Results on motion estimation are given in
Figure 5 and compared with the ground truth provided by the simulation creating
the observations. Each column corresponds to the first frame of one of the six
windows mWi .
Fig. 4. Initialisation for the twin experiment: a – )0(ξ ; b – )0(sI .
In order to demonstrate the potential of the learning method on sliding
windows, statistics on the retrieved vorticity are provided. The normalized root
mean square error (in percentage) ranges from 1,1 to 4,0% from the first to the
sixth window, while the correlation value between the retrieved vorticity and the
ground truth decreases from 0,99 to 0,96.
The computing time reduces from around 4 hours for the first window
processed by the full model to less than 1 minute for the next five one, processed
by reduced models.
8. Conclusions
In the paper, we proposed a learning method on sliding windows for
estimating motion on long temporal image sequences with data assimilation
techniques. This method couples full and reduced models obtained by Galerkin
projection and allows to process images in quasi-real time. The method has been
quantified with twin experiments to demonstrate its potential. First, the quality of
motion fields retrieved by the full model has been assessed. Second, statistics on
performances of the reduced models learned on the sliding windows have been
provided.
a b
76
Fig. 5. Estimated Motion (a – e) compared to the ground truth (g – k).
a b c d e
g h i k j
76
77
One perspective is to replace the POD bases ξΨ which were used to define
the reduced models by a fixed basis in order to even reduce the computational
requirements on the first part of the image sequence.
Acknowledgements
This research is partially supported by the Geo-FLUIDS project (ANR 09
SYSC 005 02).
RE FERE NCES
1. Horn B. and Schunk B. Determining optical flow // Art. Int. – 1981. – vol. 17.
– P. 185-203.
2. Isambert T., Berroir J. and Herlin I. A multiscale vector spline method for estimating
the fluids motion on satellite images / In: Forsyth, D., Torr, P., Zisserman A. (eds.).
– Springer, Heidelberg, 2008. – part IV. – vol. 5303. – P. 665-676.
3. Papadakis N., Corpetti T. and Mémin E. (2007). Dynamically consistent optical flow
estimation / In ICCV. 2007. – P. 1-7.
4. Titaud O., Vidard A., Souopgui I. and Dimet F.-X. L. Assimilation of image sequences in
numerical models // Tellus Series A: Dynamic Meteorology and Oceanography. – 2010.
– vol. 62, № 1. – P. 30-47.
5. Béréziat D. and Herlin I. Solving ill-posed image processing problems using data
assimilation // Numerical Algorithms. – 2011. – vol. 56, № 2. – P. 219-252.
6. Drifi K. and Herlin I. Assimilation d’images dans un modèle réduit pour l’estimation
du mouvement / In GRETSI 2011 – Groupe d’Etudes du Traitement du Signal et des
Images, Bordeaux, France. – 2011. – 4 p.
7. Deriaz E. and Perrier V. Divergence-free and curl-free wavelets in two dimensions
and three dimensions: application to turbulent flows // Journal of Turbulence. – 2006.
– vol. 7, № 3. – P. 1-37.
8. Le Dimet, F.-X. and Talagrand O. Variational algorithms for analysis and assimilation of
meteorological observations: Theoretical aspects // Tellus. – 1986. – vol. 38A, № 2.
– P. 97-110.
9. Zhu C., Byrd R., Lu P. and Nocedal J. L-BFGS-B: a limited memory FORTRAN
code for solving bound constrained optimization problems. Technical Report
NAM-11. EECS Department, Northwestern University. – 1994.
10. Isambert T., Herlin I., and Berroir J.-P. Fast and stable vector spline method for fluid
flow estimation / In ICIP. – 2007. – P. 505-508.
11. Corpetti T., Mémin E. and Pérez P. Dense estimation of fluid flows // Pat. Anal. and
Mach. Int. – 2002. – vol. 24, № 3. – P. 365-380.
12. Suter D. Motion estimation and vector splines / In CVPR. – 1994. – P. 939-942.
Rec eived 05 .11 .2011 ;
Материал поступил в редакцию 05 .11 .2011 г .
78
АНОТАЦ IЯ У статті описується метод вкладених вікон, використовуваний для
розрахунку параметрів руху при обробці зображень океану, отриманих за
допомогою супутникових систем. «Повна модель», яка використовується для опису
динаміки полів, заснована на рівнянні бездівергентного руху рідини і перенесення
яскравості зображення швидкістю. Послідовність зображень розбивається на
невеликі тимчасові вікна, з половинною перекриттям у часі. Асиміляція зображення
в повній моделі проводиться для першого вікна. Це дозволяє визначити підпрос-
тори полів руху і зображень та побудувати «редуцiровану модель» проектуванням
на ці підпростори методом Гальоркіна. Асиміляція даних в «скороченої моделі»
застосовується для другого вікна. Цей процес повторюється для всієї послідовності
вікон. Основною перевагою такого підходу є прискорення обробки, що дозволяє
використовувати його при обробці зображень у темпі, близькому до реального часу.
Переваги «скороченої моделі» продемонстровані чисельними експериментами
використовуючи метод близнюків.
АННОТАЦИЯ В статье описывается метод вложенных окон, используемый для
расчета параметров движения при обработке изображений океана, полученных с
помощью спутниковых систем. «Полная модель», которая используется для
описания динамики полей, основана на уравнении бездивергентного движения
жидкости и переносе яркости изображения скоростью. Последовательность
изображений разбивается на небольшие временные окна, с половинным пере-
крытием во времени. Ассимиляция изображения в полной модели проводится для
первого окна. Это позволяет определить подпространства полей движения и
изображений и построить «редуцированную модель» проектированием на эти
подпространства методом Галеркина. Ассимиляция данных в «редуцированной
модели» применяется для второго окна. Этот процесс повторяется для всей
последовательности окон. Основным преимуществом такого подхода является
ускорение обработки, что позволяет использовать его при обработке изображений в
темпе, близком к реальному времени. Преимущества «редуцированной модели»
продемонстрированы численными экспериментами используя метод близнецов.
|