3-D Reconstruction from a Single Still Image-学习《1》

Posted 2022-11-30 xiaobai_xiaokele

这个是自己感兴趣的领域之一，之前就已经了解过，但是没有坚持下去，所以这次只是重新拾起，而且要坚持下去，然后做出来。
这个是NG的课上了解到的一个项目，感觉很有趣。
主页http://ai.stanford.edu/~asaxena/reconstruction3d/。里面有很多资料。现在开始学习。

“
We consider the problem of estimating detailed 3-d structure from a single still image of an unstructured environment. Our goal is to create 3-d models which are both quantitatively accurate as well as visually pleasing.
For each small homogeneous patch in the image, we use a Markov Random Field (MRF) to infer a set of "plane parameters" that capture both the 3-d location and 3-d orientation of the patch. The MRF, trained via supervised learning, models both image depth cues as well as the relationships between different parts of the image. Other than assuming that the environment is made up of a number of small planes, our model makes no explicit assumptions about the structure of the scene; this enables the algorithm to capture much more detailed 3-d structure than does prior art, and also give a much richer experience in the 3-d flythroughs created using image-based rendering, even for scenes with significant non-vertical structure.
Using this approach, we have created qualitatively correct 3-d models for 64.9% of 588 images downloaded from the internet. We have also extended our model to produce large scale 3d models from a few images.
”
意思是“对于图像中每个同质块，我们使用MRF来预测一系列的既包含patch的3-d坐标以及3-d方向的“patch参数”，使用监督学习来训练MRF，对图像深度信息以及图像不同部分之间的关系进行建模，我们的模型不假设处理的环境是由一组小plane组成的，我们的模型不对场景结构做任何明确假设；这样就使算法能够比先前paper所用算法获得更多的3-d结构信息，并且观看基于图像渲染的3-d flythroughs 有更好的体验，即使是明显非垂直结构的场景。
”

可以看出来有几个关键点：MRF，监督学习，3D渲染flythrough

MRF: 马尔科夫随机场是干嘛用的？为什么这个地方要用MRF而不是别的方法？

监督学习： 用的哪种监督学习方法？怎么训练的？要得到什么样的模型？

3D渲染flythrough：这个是用来观看最后得到的3D模型效果的，需要怎么做才能看到效果呢？

上面这几点都是要弄清楚的。

所以首先就是要读这篇文章《Learning 3-D Scene Structure from a Single Still Image
》,Ashutosh Saxena, Min Sun, Andrew Y. Ng, In ICCV workshop on 3D Representation for Recognition (3dRR-07), 2007. (best paper) 

下一篇文章就是对这篇文章的解析。