Andolfi marco presentation
-
Upload
marco-andolfi -
Category
Documents
-
view
590 -
download
0
Transcript of Andolfi marco presentation
![Page 1: Andolfi marco presentation](https://reader036.fdocumento.com/reader036/viewer/2022062400/58a0fc791a28abbf248b594f/html5/thumbnails/1.jpg)
Automatic Modeling of 3D Human Face
Marco Andolfi
I. RagnemalmInstitute of Technology
of Linkoping
M. Schaerf“La Sapienza”
University of Rome
Supervisors:
Co-Supervisor:M. Fratarcangeli
“La Sapienza” University of Rome
![Page 2: Andolfi marco presentation](https://reader036.fdocumento.com/reader036/viewer/2022062400/58a0fc791a28abbf248b594f/html5/thumbnails/2.jpg)
The problem: Face modeling
Input Modeling System Output
Our goal
![Page 3: Andolfi marco presentation](https://reader036.fdocumento.com/reader036/viewer/2022062400/58a0fc791a28abbf248b594f/html5/thumbnails/3.jpg)
Input Modeling System Output
1 - Five Orthogonal Photos
2 – A “Generic” Mesh
No particular instrument
3D colored model of the subject’s head in photo
(no hair)3 – A Landmark set points
Our system:Input - Output - Features
![Page 4: Andolfi marco presentation](https://reader036.fdocumento.com/reader036/viewer/2022062400/58a0fc791a28abbf248b594f/html5/thumbnails/4.jpg)
The Input (1) - Five orthogonal Photos
![Page 5: Andolfi marco presentation](https://reader036.fdocumento.com/reader036/viewer/2022062400/58a0fc791a28abbf248b594f/html5/thumbnails/5.jpg)
Without expression
Without Gender Features
Without Race Features
The Input:(2) – A “generic” Mesh
![Page 6: Andolfi marco presentation](https://reader036.fdocumento.com/reader036/viewer/2022062400/58a0fc791a28abbf248b594f/html5/thumbnails/6.jpg)
(almost) MPEG-4 landmark points
The Input:(3) – The landmark set of
points
![Page 7: Andolfi marco presentation](https://reader036.fdocumento.com/reader036/viewer/2022062400/58a0fc791a28abbf248b594f/html5/thumbnails/7.jpg)
Modeling System
Morphing Sub-System
ColoringSub-System
Inside the system
![Page 8: Andolfi marco presentation](https://reader036.fdocumento.com/reader036/viewer/2022062400/58a0fc791a28abbf248b594f/html5/thumbnails/8.jpg)
Morphing sub system
![Page 9: Andolfi marco presentation](https://reader036.fdocumento.com/reader036/viewer/2022062400/58a0fc791a28abbf248b594f/html5/thumbnails/9.jpg)
Morphing Sub-System
Mesh Feature
Extraction
Photo Feature
ExtractionMorphing Execution
CameraError
Correction
2D – 3D Conversio
n
Morphing sub system
![Page 10: Andolfi marco presentation](https://reader036.fdocumento.com/reader036/viewer/2022062400/58a0fc791a28abbf248b594f/html5/thumbnails/10.jpg)
Preprocessing operation
Manually picking
Mesh feature extraction
![Page 11: Andolfi marco presentation](https://reader036.fdocumento.com/reader036/viewer/2022062400/58a0fc791a28abbf248b594f/html5/thumbnails/11.jpg)
Morphing Sub-System
Mesh Feature
Extraction
Photo Feature
ExtractionMorphing Execution
CameraError
Correction
2D – 3D Conversio
n
Morphing sub system
![Page 12: Andolfi marco presentation](https://reader036.fdocumento.com/reader036/viewer/2022062400/58a0fc791a28abbf248b594f/html5/thumbnails/12.jpg)
Noisy Data
Manually picking
Photo feature extraction
![Page 13: Andolfi marco presentation](https://reader036.fdocumento.com/reader036/viewer/2022062400/58a0fc791a28abbf248b594f/html5/thumbnails/13.jpg)
Morphing Sub-System
Mesh Feature
Extraction
Photo Feature
ExtractionMorphing Execution
CameraError
Correction
2D – 3D Conversio
n
Morphing sub system
![Page 14: Andolfi marco presentation](https://reader036.fdocumento.com/reader036/viewer/2022062400/58a0fc791a28abbf248b594f/html5/thumbnails/14.jpg)
Different distance Camera-subject
Scaling
Camera Errors: the corrected ones
![Page 15: Andolfi marco presentation](https://reader036.fdocumento.com/reader036/viewer/2022062400/58a0fc791a28abbf248b594f/html5/thumbnails/15.jpg)
Different distance Camera-subject
Camera orientation
Scaling
Translation
Camera Errors: the corrected ones
![Page 16: Andolfi marco presentation](https://reader036.fdocumento.com/reader036/viewer/2022062400/58a0fc791a28abbf248b594f/html5/thumbnails/16.jpg)
Different distance Camera-subject
Camera orientation
Wrong Head Position: Z axis
Scaling
Translation
Rotation
Camera Errors: the corrected ones
![Page 17: Andolfi marco presentation](https://reader036.fdocumento.com/reader036/viewer/2022062400/58a0fc791a28abbf248b594f/html5/thumbnails/17.jpg)
Camera Errors: the negligible ones
Rotation Angle?
Wrong Head Position: X axis
![Page 18: Andolfi marco presentation](https://reader036.fdocumento.com/reader036/viewer/2022062400/58a0fc791a28abbf248b594f/html5/thumbnails/18.jpg)
Camera Errors: the negligible ones
Rotation Angle?
Wrong Head Position: X axis
Rotation No info
Wrong Head Position: Y axis
![Page 19: Andolfi marco presentation](https://reader036.fdocumento.com/reader036/viewer/2022062400/58a0fc791a28abbf248b594f/html5/thumbnails/19.jpg)
Camera Errors: the negligible ones
Rotation Angle?
Wrong Head Position: X axis
Rotation No info
Wrong Head Position: Y axis
Negligible
Prospective error
![Page 20: Andolfi marco presentation](https://reader036.fdocumento.com/reader036/viewer/2022062400/58a0fc791a28abbf248b594f/html5/thumbnails/20.jpg)
Camera Errors: Summary
Rotation Angle?
Wrong Head Position: X axis
Rotation No info
Wrong Head Position: Y axis
Negligible
Prospective error
Different distance Camera-subject
Camera orientation
Wrong Head Position: Z axis
Scaling
Translation
Rotation
Corrected Ignored
![Page 21: Andolfi marco presentation](https://reader036.fdocumento.com/reader036/viewer/2022062400/58a0fc791a28abbf248b594f/html5/thumbnails/21.jpg)
Get reference point: Why?Rotation
Scaling
![Page 22: Andolfi marco presentation](https://reader036.fdocumento.com/reader036/viewer/2022062400/58a0fc791a28abbf248b594f/html5/thumbnails/22.jpg)
Get reference point: Why?Translation
![Page 23: Andolfi marco presentation](https://reader036.fdocumento.com/reader036/viewer/2022062400/58a0fc791a28abbf248b594f/html5/thumbnails/23.jpg)
Camera Errors: Get reference point
![Page 24: Andolfi marco presentation](https://reader036.fdocumento.com/reader036/viewer/2022062400/58a0fc791a28abbf248b594f/html5/thumbnails/24.jpg)
Camera Errors: Correction
![Page 25: Andolfi marco presentation](https://reader036.fdocumento.com/reader036/viewer/2022062400/58a0fc791a28abbf248b594f/html5/thumbnails/25.jpg)
Get scaling factor: example on Y
Fy Ly
FyScaling Factor = -----------
Ly
Noisy Data
Bad Result
![Page 26: Andolfi marco presentation](https://reader036.fdocumento.com/reader036/viewer/2022062400/58a0fc791a28abbf248b594f/html5/thumbnails/26.jpg)
Get scaling factor: example on Y
Fy1 Ly1
Fy1 + Fy2 + ... + FyNScaling Factor = ---------------------------------------
Ly1 + Ly2 + ... + LyN
Fy2 Ly2
![Page 27: Andolfi marco presentation](https://reader036.fdocumento.com/reader036/viewer/2022062400/58a0fc791a28abbf248b594f/html5/thumbnails/27.jpg)
Morphing Sub-System
Mesh Feature
Extraction
Photo Feature
ExtractionMorphing Execution
CameraError
Correction
2D – 3D Conversio
n
Morphing sub system
![Page 28: Andolfi marco presentation](https://reader036.fdocumento.com/reader036/viewer/2022062400/58a0fc791a28abbf248b594f/html5/thumbnails/28.jpg)
2D to 3D convertion
Several values for each coordinate
Weighted Average
More importance to front photo
![Page 29: Andolfi marco presentation](https://reader036.fdocumento.com/reader036/viewer/2022062400/58a0fc791a28abbf248b594f/html5/thumbnails/29.jpg)
Morphing Sub-System
Mesh Feature
Extraction
Photo Feature
ExtractionMorphing Execution
CameraError
Correction
2D – 3D Conversio
n
Morphing sub system
![Page 30: Andolfi marco presentation](https://reader036.fdocumento.com/reader036/viewer/2022062400/58a0fc791a28abbf248b594f/html5/thumbnails/30.jpg)
Single step morphing
RBF Morphing functionLow stiffness parameter(as interpolation)
Noisy Data
Bad Result
Interpolation
![Page 31: Andolfi marco presentation](https://reader036.fdocumento.com/reader036/viewer/2022062400/58a0fc791a28abbf248b594f/html5/thumbnails/31.jpg)
Single step morphing: why not...
![Page 32: Andolfi marco presentation](https://reader036.fdocumento.com/reader036/viewer/2022062400/58a0fc791a28abbf248b594f/html5/thumbnails/32.jpg)
High level quality points
Low level quality points
Two different level of quality
![Page 33: Andolfi marco presentation](https://reader036.fdocumento.com/reader036/viewer/2022062400/58a0fc791a28abbf248b594f/html5/thumbnails/33.jpg)
Double step morphingLow and High level quality
points
RBF Morphing functionHigh stiffness parameter(as approximation)
RBF Morphing functionLow stiffness parameter(as interpolation)
High level quality points
First step
Second step
c
![Page 34: Andolfi marco presentation](https://reader036.fdocumento.com/reader036/viewer/2022062400/58a0fc791a28abbf248b594f/html5/thumbnails/34.jpg)
Modeling System
Morphing Sub-System
ColoringSub-System
Inside the system
![Page 35: Andolfi marco presentation](https://reader036.fdocumento.com/reader036/viewer/2022062400/58a0fc791a28abbf248b594f/html5/thumbnails/35.jpg)
Coloring sub system
![Page 36: Andolfi marco presentation](https://reader036.fdocumento.com/reader036/viewer/2022062400/58a0fc791a28abbf248b594f/html5/thumbnails/36.jpg)
Coloring Sub-System
Texture coordinate computing
Bottom texture
generation
Weight computing
Texture mapping
Remove ghost effect
SolveOverlap artifact
Coloring sub-system
![Page 37: Andolfi marco presentation](https://reader036.fdocumento.com/reader036/viewer/2022062400/58a0fc791a28abbf248b594f/html5/thumbnails/37.jpg)
Texture coordinate computing
What about the camera correction error?
Not a simple orthogonal projection
![Page 38: Andolfi marco presentation](https://reader036.fdocumento.com/reader036/viewer/2022062400/58a0fc791a28abbf248b594f/html5/thumbnails/38.jpg)
Texture coordinate computing
![Page 39: Andolfi marco presentation](https://reader036.fdocumento.com/reader036/viewer/2022062400/58a0fc791a28abbf248b594f/html5/thumbnails/39.jpg)
Coloring Sub-System
Texture coordinate computing
Bottom texture
generation
Weight computing
Texture mapping
Remove ghost effect
SolveOcclusionProblem
Coloring sub-system
![Page 40: Andolfi marco presentation](https://reader036.fdocumento.com/reader036/viewer/2022062400/58a0fc791a28abbf248b594f/html5/thumbnails/40.jpg)
β
γ
α
NP
NF
NL
NT
Weight computingWhy? Avoiding overlapping
The weight as transparency level
Weight for vertex respect front photo
Proportional to α
WF = NF ∙ NP
Set NULL negative weights
NR
WR = NR ∙ NP < 0 → WR = 0
![Page 41: Andolfi marco presentation](https://reader036.fdocumento.com/reader036/viewer/2022062400/58a0fc791a28abbf248b594f/html5/thumbnails/41.jpg)
Coloring sub-system
Coloring Sub-System
Texture coordinate computing
Bottom texture
generation
Weight computing
Texture mapping
Remove ghost effect
SolveOverlapp artifact
![Page 42: Andolfi marco presentation](https://reader036.fdocumento.com/reader036/viewer/2022062400/58a0fc791a28abbf248b594f/html5/thumbnails/42.jpg)
Bottom texture generation
Little square
only skin
No mole, No scar
Repetitionof the little
square
Final bottom texture
Bad quality? (too many squares...)
AIM: obtain a texture containig only skin
![Page 43: Andolfi marco presentation](https://reader036.fdocumento.com/reader036/viewer/2022062400/58a0fc791a28abbf248b594f/html5/thumbnails/43.jpg)
Coloring Sub-System
Texture coordinate computing
Bottom texture
generation
Weight computing
Texture mapping
Remove ghost effect
SolveOverlapp artifact
Coloring sub-system
![Page 44: Andolfi marco presentation](https://reader036.fdocumento.com/reader036/viewer/2022062400/58a0fc791a28abbf248b594f/html5/thumbnails/44.jpg)
Texture MappingThe weight as transparency
level
![Page 45: Andolfi marco presentation](https://reader036.fdocumento.com/reader036/viewer/2022062400/58a0fc791a28abbf248b594f/html5/thumbnails/45.jpg)
Coloring Sub-System
Texture mapping
Remove ghost effect
SolveOverlapp artifact
Coloring sub-system
Texture coordinate computing
Bottom texture
generation
Weight computing
![Page 46: Andolfi marco presentation](https://reader036.fdocumento.com/reader036/viewer/2022062400/58a0fc791a28abbf248b594f/html5/thumbnails/46.jpg)
Ghost effect
![Page 47: Andolfi marco presentation](https://reader036.fdocumento.com/reader036/viewer/2022062400/58a0fc791a28abbf248b594f/html5/thumbnails/47.jpg)
Coloring Sub-System
Texture mapping
Remove ghost effect
SolveOverlapp artifact
Coloring sub-system
Texture coordinate computing
Bottom texture
generation
Weight computing
![Page 48: Andolfi marco presentation](https://reader036.fdocumento.com/reader036/viewer/2022062400/58a0fc791a28abbf248b594f/html5/thumbnails/48.jpg)
Solving occlusion problem
![Page 49: Andolfi marco presentation](https://reader036.fdocumento.com/reader036/viewer/2022062400/58a0fc791a28abbf248b594f/html5/thumbnails/49.jpg)
Solving occlusion problemDetect vertex between inner eye points
Giving weight to side photos
Removing weight to side photos
![Page 50: Andolfi marco presentation](https://reader036.fdocumento.com/reader036/viewer/2022062400/58a0fc791a28abbf248b594f/html5/thumbnails/50.jpg)
Let’s see some examples