del profesor Gabriel Cristobal (CSIC - ELAI-UPM

Multidimensional
processing and analysis
of biomedical images
Gabriel Cristóbal
Instituto de Optica (CSIC)
Serrano 121, 28006 Madrid (Spain)
gabriel@optica.csic.es
http://www.iv.optica.csic.es
1
Collaborators
z Noemí Carranza, Instituto de Optica, CSIC, Madrid, Spain
z Elena Gil, Instituto de Optica, CSIC, Madrid, Spain
z Salvador Gabarda, Instituto de Optica, CSIC, Madrid, Spain
z Peter Schelkens, Vrije Univ. Brussels
z Heiko Neumann, Univ. Ulm, Germany
z Boris Escalante, UNAM, Mexico
z Filip Sroubek, Academy of Sciences Czech Rep, Prague
z Rodrigo Nava, UNAM, Mexico
z Mathias Keil, Centro Visión por Computador, Barcelona, Spain
z Leon Cohen, Hunter College CUNY, USA
z Doug Nelson, Dept. of Defense, USA
z Patrick Loughlin, Univ. Pittsburgh, USA
z Ted Maddess, Australian National University, Australia
z Manuel Forero, Univ. Birmingham, UK
2
Outline
z Motivation
z Time-frequency analysis in signal
processing.
{ Image fusion. Deconvolution
{ Overcomplete representations
{ Motion estimation
{ Image quality assessment
z Superresolution
zComplementary actions
zConclusions. Further research
3
Outline
z Motivation
z Time-frequency analysis in signal
processing.
{ Image fusion. Deconvolution
{ Quality estimation
{ Motion estimation
z Superresolution
zComplementary actions
zConclusions. Further research
4
Time-frequency representation moto
z Time-frequency representations can be considered
as the“Holy Grail” i.e. they constitute a keystone in the
area of signal Processing/Analysis (Jelena Kovacevik,
Univ. Maryland)
z Understanding the significance of the blob-like
structure of time-frequency representations
Musical score analogy
5
Which are the foundations?
z WD is the most prominent quadratic TF
representation
zOriginally it was defined by Wigner in 1932
in Quantum Mechanics
zLater (1948) Ville derived the same WD in
signal analysis
zIntroduced in Optics by Bastiaans in 1978
6
TF-based multidimensional SP
z TF representations can be extended beyond 1D, scalar or deterministic signals
1D
complexity
2D
3D
3D+t
ND?
• 3D texture • CMR analysis
• Phonocardiograms • Texture
• Optical flow
• High-order texture
• ECG
estimation
• Image fusion
• Epileptic seizures
• Earthquake precursor • Image deconvolution
• Image analysis
waves
(crack detection)
• Speech
• Optics
• Audio/acoustics
• Machine diagnosis/ • Radar imaging
•…
fault detection
• Source localization
• Quantum Mechanics
•…
• Mathematics
7
Wavelets vs joint representations
8
WD computation of 2D signals
9
Hybrid optical-digital system
Displacement unit
Object
Fourier
WD
lens
mirror
Fourier
lens
10
Texture segmentation using PWD+PCA
15x15
31x31
11
Patt. Rec. Letters, 20, pp. 481-487,1999
Recent 2D Applications
zMultitemporal
image fusion
12
Multitemporal image fusion
13
Recent 2D Applications
zBlind deconvolution
14
Recent 2D Applications
z
Multifocus image fusion
15
1D Applications
Detection of events in biological
signals
z
control
arrythmia
5th Int. Workshop on Information Optics,
AIP Proc. Vol XX, 2006
16
1D Applications
z(n)
Biomedical
signal
Steps
n
1) Discretize signal
2) Calculate a Time-frequency
representation (TFR) of the signal
3) Measure Rényi entropy
Time-frequency
representation
W(n,k)
W(n = ni , k)
n
Rényi entropy
Entropy is measured in
a instant-wise scheme.
The PWD corresponding
to instant n is considered
to perform the entropy
measure
R(n)
Rα =
1
⎛
⎞
log 2 ⎜ ∑∑ Pα (n, k ) ⎟
1−α
⎝ n k
⎠
n
17
1D Applications- Results
control
Murmur (aortic stenosis)
5th Int. Workshop on Information Optics,
AIP Proc. Vol XX, 2006
18
Image quality enhancement 1/2
19
Image fusion
20
Image quality enhancement 2/2
z In the case of heavily degraded hazy images, a new element can be added to the
pixel-wise frequency distribution. This new element takes account of the gray-level
information of the pixels and gives a different weight to each one.
⎛
z [ n] − B
λ = μ ⎜1+ A
⎜
C
⎝
⎞
⎟
⎟
⎠
21
5.Examples
Multisensor images (1)
a and b: Input images. c: Resulting fused image by applying a multiscale transform
method. d: Resulting fused image by applying the Wigner-Rényi method. The areas
within the white frame appear magnified in the figure below.
22
5.Examples
Multisensor images (2)
a and b: Input images. c: Resulting fused image by applying a multiscale transform
method. d: Resulting fused image by applying the Wigner-Rényi method. The areas
within the white frame appear magnified in the figure below.
23
5.Examples
Multisensor images (2)
24
Image fusion-feature level
Fused
pixel-level
contour-level
25
Outline
z Motivation
z Time-frequency analysis in signal
processing.
{ Image fusion. Deconvolution
{ Quality estimation
{ Motion estimation
z Variational methods
{Superresolution
zComplementary actions
zConclusions. Further research
26
Image quality evaluation
Una nueva herramienta para la evaluación de la calidad de imágenes
R. Nava, A. Gallego and G. Cristóbal
Congreso Asociación Española para la Ingeniería Biomédica (CASEIB), Pamplona, November 2006 (submitted)
27
Calidad de imagen
Conceptos básicos
Calidad de imagen
28
Todas las imágenes desenfocadas tienen MSE=200, pero su calidad es muy diferente!
Absolute entropy measures. From blur to noise
29
An ideal quality metric?
30
Entropy histogram noise index (EHNI)
⎛ 1 ⎞
1
EHNI = log ⎜⎜
⎟⎟
4
⎝ H (J ) ⎠
Left column:
A) Source image. B) Image plus 20% of
Gaussian noise. C) Image plus 40% of
Gaussian noise. D) Image plus 60% of
Gaussian noise.
Center column:
Pixel-wise entropy
Right column:
Histograms of the entropy of the images.
31
Denoising results: retina fundus
32
Outline
z Motivation
z Time-frequency analysis in signal
processing.
{ Image fusion. Deconvolution
{ Overcomplete representations
{ Motion estimation
z Variational methods
{Superresolution
zComplementary actions
zConclusions. Further research
33
e4
e3
e2
log-Gabor wavelets
Dictionary construction
e1
Simple cell model: Position,
orientation, scale and phase
selectivity, log-Gabor shape.
Gabor: optimal space-frequency
joint localization
Similar to independant components
of natural images [Olshausen97].
Multiscale - multiorientation
(inspired from V1).
Wavelets: convolution by a discrete
set of filters which are rotated /
scaled one another.
Å--- orientations --->
Å---- Scales -----> Å---- Scales ----->
3.A.
Phase 0°
Phase 90°
34
3.B.
Facilitation
Inhibition
Lateral inhibition
Complex
cells
inhibition: Edge localization by non-maxima suppression [Heitger92].
35
3.B.
Facilitation
Facilitation
Method for contour saliency:
- Thresholding on amplitude.
- Chain length [Yen98].
- Parent coefficients across scales.
complex cell
chain length
inhibition
36
3.B.
Multiscale edge extraction:
example
Complex
cells
Lateral facilitation
Lateral
inhibition
Fischer S., Perrinet L., Redondo R. and Cristóbal G.: “Sparse coding of images inspired by the
functional architecture of the primary visual areas”, EURASIP JASP Image Perception, in press.
37
Motion analysis
z Motion analysis is
based on disparity
information
z Disparity1 is derived
from sucesive frames
in time
z In contrast with stereo
disparity, motion
disparity defines
complex-valued
(vector field)
1Disparity
is a measure of the relative displacement
between ‘corresponding’ points of two images
38
Cardiac motion estimation
„
CMR is recognized as the best imaging
method for exploration cardiac function
z Analysis of heart wall deformation
{ Estimates the location of ischemic myocardial injury
{ Determine myocardial effects of coronary artery diseases
Example of
evaluation of
ischemic injury
39
Pre-processing stage
Filtering
+
Thresholding
40
Schematic Block Diagram
41
With CMR Sequences
z Ideal conditions are not met
{ Not guaranteed uniform
velocity
{ Deformable cardiac wall
First image
Last image
{
Correlation variable with time
{
Plane not detectable
Get uniform correlation with
a probabilistic approach
42
Optical Flow
(1 cycle diastole)
43
Outline
z Motivation
z Time-frequency analysis in signal
processing.
{ Image fusion. Deconvolution
{ Overcomplete representations
{ Motion estimation
z Superresolution
zComplementary actions
zConclusions. Further research
44
Qué es la superresolución?
Límite aumento:
X2-X2.5
45
Objetivos
zGenerar una imagen de alta resolución a
partir de diferentes imágenes de baja
resolución de la misma escena
z La imagen de alta resolución es aquella
que proporciona mayores detalles (no
simplemente más pixels)
46
Cómo se obtiene la SR?
Deconv
Registro de
imágenes
SR
Matriz de alta
resolución
Imágenes de
Baja resolución
47
Ejemplo de aplicación: identificación de
matrículas
48
Otros métodos de SR
Random Lens Imaging, Fergus, Torralba and
Freeeman, MIT, TR 2006-058
‘Image and depth from a conventional
Camera with a coded aperture’,
Levin, Fergus, Durand, Freeeman,
SIGGRAPH’07
‘Compressive sensing’
49
Microscopía automatizada
z Microscopio biológico MOTIC, modelo BA-400
Triocular, equipado con un sistema de objetivos de
contraste de fases de 10X, 20X, 40X y 100X. El
sistema de iluminación es Koehler con lámpara
halógena de 6V 30W centrable.
z Pletina motorizada en XYZ modelo H101 de Prior
Scientific, siendo la motorización en Z independiente y
adaptable a través del sistema de enfoque del
microscopio. La resolución (paso) de la motorización
en XY es de 0.1-5 micras dependiendo de la
configuración del controlador.
50
Microscopía automatizada
z Unidad controladora
mediante puerto RS232
de
los
motores
z Cámara Moticam refrigerada de alta
resolución MC 3000-C de 3.3 Mpixels con
refrigeración Peltier, conexión mediante IEEE
1394 o a través de tarjeta PCI con salida
IEEE 1394.
51
Microscopía automatizada
52
Detección e identificación de
micobacterias de tuberculosis (TB)
Ejemplo de imagen que contiene
micobacterias de TB
•
•
Automatización en Microscopía: Microscopio MOTIC, Pletina Prior,
Cámara Moticam, Fluorescencia
ImageJ (http://rsb.info.nih.gov/ij/)
53
Esquema general del
sistema de análisis de muestras
Autoenfoque Z
Escaneado X-Y
Análisis+
ROC
Captura
Prot. autom.
Umbralización
de color adaptativa
Extracción
canal verde
Validación
Clínica
Prot.
análisis
Validación
color
Detección bordes
+
operadores
morfológicos
Extracción de
características
de bordes +
“clustering”
Prot.
final
bacilo
Decisión a
través de la
teoría
Bayesiana
Imagen
RGB
no-bacilo
z Capacidad diagnóstica:
{ Especificidad: 98,1 (1,23%)
{ Sensibilidad: 92,9 (3,87%)
54
Autoenfoque para imágenes
con TB
log(histograma)
log(histograma)
55
Outline
z Motivation
z Time-frequency analysis in signal
processing.
{ Image fusion. Deconvolution
{ Overcomplete representations
{ Motion estimation
z Variational methods
{Superresolution
zComplementary actions
zConclusions. Further research
56
Complementary
actions
http://www.iv.optica.csic.es/Prisma/ES/Default.htm
57
58
59
Propuesta de proyectos
z “Métodos de evaluación de la calidad de imagen”
z“Automatización de un sistema de adquisición de
imágenes en microscopía”
z “Detección y análisis de eventos de interés en
señales fonocardiográficas y EEG”
z “Métodos de procesado en imagen forense y de
mejora de la resolución”
z “Implementaciones eficientes orientadas a FPGA”
60
References
z F. Sroubek, G. Cristóbal and J. Flusser, “A unified approach to
superresolution and multichannel blind deconvolution”, IEEE
Trans. On Image Processing, 16, pp. 2322-2332, 2007
z F. Sroubek, G. Cristóbal,E. Gil and J. Flusser, “Blind
superresolution imaging”, SPIE newsroom, 2007
z F. Sroubek, J. Flusser and G. Cristóbal, “Superresolution and
blind deconvolution for rational factors with and application to
color images”, Computer Journal, Oxford Univ. Press, 2007
z F. Sroubek, G. Cristóbal and J. Flusser, “Simultaneous superresolution and blind deconvolution”, AIP Inverse Problems
Conference, 2008 (in press)
61