Andolfi marco presentation
-
Upload
marco-andolfi -
Category
Documents
-
view
590 -
download
0
Transcript of Andolfi marco presentation
Automatic Modeling of 3D Human Face
Marco Andolfi
I. RagnemalmInstitute of Technology
of Linkoping
M. Schaerf“La Sapienza”
University of Rome
Supervisors:
Co-Supervisor:M. Fratarcangeli
“La Sapienza” University of Rome
The problem: Face modeling
Input Modeling System Output
Our goal
Input Modeling System Output
1 - Five Orthogonal Photos
2 – A “Generic” Mesh
No particular instrument
3D colored model of the subject’s head in photo
(no hair)3 – A Landmark set points
Our system:Input - Output - Features
The Input (1) - Five orthogonal Photos
Without expression
Without Gender Features
Without Race Features
The Input:(2) – A “generic” Mesh
(almost) MPEG-4 landmark points
The Input:(3) – The landmark set of
points
Modeling System
Morphing Sub-System
ColoringSub-System
Inside the system
Morphing sub system
Morphing Sub-System
Mesh Feature
Extraction
Photo Feature
ExtractionMorphing Execution
CameraError
Correction
2D – 3D Conversio
n
Morphing sub system
Preprocessing operation
Manually picking
Mesh feature extraction
Morphing Sub-System
Mesh Feature
Extraction
Photo Feature
ExtractionMorphing Execution
CameraError
Correction
2D – 3D Conversio
n
Morphing sub system
Noisy Data
Manually picking
Photo feature extraction
Morphing Sub-System
Mesh Feature
Extraction
Photo Feature
ExtractionMorphing Execution
CameraError
Correction
2D – 3D Conversio
n
Morphing sub system
Different distance Camera-subject
Scaling
Camera Errors: the corrected ones
Different distance Camera-subject
Camera orientation
Scaling
Translation
Camera Errors: the corrected ones
Different distance Camera-subject
Camera orientation
Wrong Head Position: Z axis
Scaling
Translation
Rotation
Camera Errors: the corrected ones
Camera Errors: the negligible ones
Rotation Angle?
Wrong Head Position: X axis
Camera Errors: the negligible ones
Rotation Angle?
Wrong Head Position: X axis
Rotation No info
Wrong Head Position: Y axis
Camera Errors: the negligible ones
Rotation Angle?
Wrong Head Position: X axis
Rotation No info
Wrong Head Position: Y axis
Negligible
Prospective error
Camera Errors: Summary
Rotation Angle?
Wrong Head Position: X axis
Rotation No info
Wrong Head Position: Y axis
Negligible
Prospective error
Different distance Camera-subject
Camera orientation
Wrong Head Position: Z axis
Scaling
Translation
Rotation
Corrected Ignored
Get reference point: Why?Rotation
Scaling
Get reference point: Why?Translation
Camera Errors: Get reference point
Camera Errors: Correction
Get scaling factor: example on Y
Fy Ly
FyScaling Factor = -----------
Ly
Noisy Data
Bad Result
Get scaling factor: example on Y
Fy1 Ly1
Fy1 + Fy2 + ... + FyNScaling Factor = ---------------------------------------
Ly1 + Ly2 + ... + LyN
Fy2 Ly2
Morphing Sub-System
Mesh Feature
Extraction
Photo Feature
ExtractionMorphing Execution
CameraError
Correction
2D – 3D Conversio
n
Morphing sub system
2D to 3D convertion
Several values for each coordinate
Weighted Average
More importance to front photo
Morphing Sub-System
Mesh Feature
Extraction
Photo Feature
ExtractionMorphing Execution
CameraError
Correction
2D – 3D Conversio
n
Morphing sub system
Single step morphing
RBF Morphing functionLow stiffness parameter(as interpolation)
Noisy Data
Bad Result
Interpolation
Single step morphing: why not...
High level quality points
Low level quality points
Two different level of quality
Double step morphingLow and High level quality
points
RBF Morphing functionHigh stiffness parameter(as approximation)
RBF Morphing functionLow stiffness parameter(as interpolation)
High level quality points
First step
Second step
c
Modeling System
Morphing Sub-System
ColoringSub-System
Inside the system
Coloring sub system
Coloring Sub-System
Texture coordinate computing
Bottom texture
generation
Weight computing
Texture mapping
Remove ghost effect
SolveOverlap artifact
Coloring sub-system
Texture coordinate computing
What about the camera correction error?
Not a simple orthogonal projection
Texture coordinate computing
Coloring Sub-System
Texture coordinate computing
Bottom texture
generation
Weight computing
Texture mapping
Remove ghost effect
SolveOcclusionProblem
Coloring sub-system
β
γ
α
NP
NF
NL
NT
Weight computingWhy? Avoiding overlapping
The weight as transparency level
Weight for vertex respect front photo
Proportional to α
WF = NF ∙ NP
Set NULL negative weights
NR
WR = NR ∙ NP < 0 → WR = 0
Coloring sub-system
Coloring Sub-System
Texture coordinate computing
Bottom texture
generation
Weight computing
Texture mapping
Remove ghost effect
SolveOverlapp artifact
Bottom texture generation
Little square
only skin
No mole, No scar
Repetitionof the little
square
Final bottom texture
Bad quality? (too many squares...)
AIM: obtain a texture containig only skin
Coloring Sub-System
Texture coordinate computing
Bottom texture
generation
Weight computing
Texture mapping
Remove ghost effect
SolveOverlapp artifact
Coloring sub-system
Texture MappingThe weight as transparency
level
Coloring Sub-System
Texture mapping
Remove ghost effect
SolveOverlapp artifact
Coloring sub-system
Texture coordinate computing
Bottom texture
generation
Weight computing
Ghost effect
Coloring Sub-System
Texture mapping
Remove ghost effect
SolveOverlapp artifact
Coloring sub-system
Texture coordinate computing
Bottom texture
generation
Weight computing
Solving occlusion problem
Solving occlusion problemDetect vertex between inner eye points
Giving weight to side photos
Removing weight to side photos
Let’s see some examples