Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
|
research:hflow:start [2015/08/12 13:15] aneufeld |
research:hflow:start [2015/09/21 11:17] aneufeld |
||
|---|---|---|---|
| Line 1: | Line 1: | ||
| - | ====== Monocular 3D Reconstruction of Traffic Scenes from Optical Flow ====== | + | ====== Estimating Vehicle Ego-Motion and Piecewise Planar Scene Structure from Optical Flow in a Continuous Framework ====== |
| We estimate a 3D scene structure and the vehicle egomotion given the forward and backward optical flow fields between two consecutive frames. | We estimate a 3D scene structure and the vehicle egomotion given the forward and backward optical flow fields between two consecutive frames. | ||
| Line 17: | Line 17: | ||
| Given rotation $R \in \text{SO}(3)$, translation $t \in S(2)$ and 3D point $X$, the point $X'$ relative to the second camera is given by $X' = R^\top (X - t)$. The translation is restricted to the unit sphere, since the translation norm cannot be estimated without additional information. | Given rotation $R \in \text{SO}(3)$, translation $t \in S(2)$ and 3D point $X$, the point $X'$ relative to the second camera is given by $X' = R^\top (X - t)$. The translation is restricted to the unit sphere, since the translation norm cannot be estimated without additional information. | ||
| Let $\pi$ denote the projection onto the image plane, | Let $\pi$ denote the projection onto the image plane, | ||
| - | $$ \pi\begin{pmatrix} x_1 \\ x_2 \\ x_3 \end{pmatrix} = \frac{1}{x_3} \begin{pmatrix} x_1 \\ x_2 \end{pmatrix}. $$ | + | $$ \pi\begin{pmatrix} x_1 \\ x_2 \\ x_3 \end{pmatrix} = \frac{1}{x_3} \begin{pmatrix} x_1 \\ x_2 \\ x_3 \end{pmatrix}. $$ |
| Let $z(x)$ denote the depth of pixel $x$ in the image plane, $X = z(x) x$. Depth can be calculated from the plane parameters, $z(x,v) = (v^\top x)^{-1}$. | Let $z(x)$ denote the depth of pixel $x$ in the image plane, $X = z(x) x$. Depth can be calculated from the plane parameters, $z(x,v) = (v^\top x)^{-1}$. | ||
| Optical flow is given by | Optical flow is given by | ||
| Line 54: | Line 54: | ||
| The proposed energy function is non-convex in rotation $R \in \text{SO}(3)$, $t \in S(2)$ and planes $v_i \in \mathbb{R}^3$. We choose the Levenberg-Marquardt algorithm for optimization. Note that the energy can be written in form | The proposed energy function is non-convex in rotation $R \in \text{SO}(3)$, $t \in S(2)$ and planes $v_i \in \mathbb{R}^3$. We choose the Levenberg-Marquardt algorithm for optimization. Note that the energy can be written in form | ||
| - | $$E(R,t,v) = \sum_j \left( f_j(R,t,v) \right)^2 = \lVert f(R,t,v) \rVert_2^2.$$ | + | $$E(R,t,v) = \sum_j \left( f_j(R,t,v) \right)^2 = \lVert f(R,t,v) \rVert_2^2,$$ |
| + | which can be optimized using Levenberg-Marquardt. We apply a damping factor, which is updated after each iteration. | ||
| ===== Results ===== | ===== Results ===== | ||
| - | We demonstrate the output of our algorithms on a few examples here, the paper contains a quantitative evaluation. From top to bottom, each example shows the reference frame, estimated depth and estimated normals. Depth is encoded as follows, the scale is derived from stereo data. | + | We demonstrate the output of our algorithms on a few examples here, the paper contains a quantitative evaluation. From top to bottom, each example shows the reference frame, estimated depth and normals and the flow error as measured in the paper (click for a full size view). Depth and depth error (in pixels) are encoded as follows, the scale is derived from ground truth data. |
| - | {{ :research:hflow:depth.png?nolink&200 |}} | + | {{ :research:hflow:depth2.png?nolink&200 |}} |
| The normal is color-encoded based on the approximately equidistant LAB colorspace. The next image shows the encoding after applying the Hammer-Aitov projection to a sphere and a simple ray-traced scene. | The normal is color-encoded based on the approximately equidistant LAB colorspace. The next image shows the encoding after applying the Hammer-Aitov projection to a sphere and a simple ray-traced scene. | ||
| Line 66: | Line 67: | ||
| {{ :research:hflow:planes.png?nolink&500 |}} | {{ :research:hflow:planes.png?nolink&500 |}} | ||
| + | The following examples show the first 30 KITTI stereo/flow training sequences. | ||
| | {{ :research:hflow:000000_10.png?direct&200 |}} {{ :research:hflow:depth_000000.png?direct&200 |}} {{ :research:hflow:normals_lab_000000.png?direct&200 |}} {{ :research:hflow:err_000000.png?direct&200 |}} | {{ :research:hflow:000001_10.png?direct&200 |}} {{ :research:hflow:depth_000001.png?direct&200 |}} {{ :research:hflow:normals_lab_000001.png?direct&200 |}} {{ :research:hflow:err_000001.png?direct&200 |}} | {{ :research:hflow:000002_10.png?direct&200 |}} {{ :research:hflow:depth_000002.png?direct&200 |}} {{ :research:hflow:normals_lab_000002.png?direct&200 |}} {{ :research:hflow:err_000002.png?direct&200 |}} | | | {{ :research:hflow:000000_10.png?direct&200 |}} {{ :research:hflow:depth_000000.png?direct&200 |}} {{ :research:hflow:normals_lab_000000.png?direct&200 |}} {{ :research:hflow:err_000000.png?direct&200 |}} | {{ :research:hflow:000001_10.png?direct&200 |}} {{ :research:hflow:depth_000001.png?direct&200 |}} {{ :research:hflow:normals_lab_000001.png?direct&200 |}} {{ :research:hflow:err_000001.png?direct&200 |}} | {{ :research:hflow:000002_10.png?direct&200 |}} {{ :research:hflow:depth_000002.png?direct&200 |}} {{ :research:hflow:normals_lab_000002.png?direct&200 |}} {{ :research:hflow:err_000002.png?direct&200 |}} | | ||
| - | | {{ :research:hflow:000009_10.png?direct&200 |}} {{ :research:hflow:depth_000009.png?direct&200 |}} {{ :research:hflow:normals_lab_000009.png?direct&200 |}} {{ :research:hflow:err_000009.png?direct&200 |}} | {{ :research:hflow:000019_10.png?direct&200 |}} {{ :research:hflow:depth_000019.png?direct&200 |}} {{ :research:hflow:normals_lab_000019.png?direct&200 |}} {{ :research:hflow:err_000019.png?direct&200 |}} | {{ :research:hflow:000023_10.png?direct&200 |}} {{ :research:hflow:depth_000023.png?direct&200 |}} {{ :research:hflow:normals_lab_000023.png?direct&200 |}} {{ :research:hflow:err_000023.png?direct&200 |}} | | + | | ||| |
| - | | {{ :research:hflow:000024_10.png?direct&200 |}} {{ :research:hflow:depth_000024.png?direct&200 |}} {{ :research:hflow:normals_lab_000024.png?direct&200 |}} {{ :research:hflow:err_000024.png?direct&200 |}} | | | | + | | {{ :research:hflow:000003_10.png?direct&200 |}} {{ :research:hflow:depth_000003.png?direct&200 |}} {{ :research:hflow:normals_lab_000003.png?direct&200 |}} {{ :research:hflow:err_000003.png?direct&200 |}} | {{ :research:hflow:000004_10.png?direct&200 |}} {{ :research:hflow:depth_000004.png?direct&200 |}} {{ :research:hflow:normals_lab_000004.png?direct&200 |}} {{ :research:hflow:err_000004.png?direct&200 |}} | {{ :research:hflow:000005_10.png?direct&200 |}} {{ :research:hflow:depth_000005.png?direct&200 |}} {{ :research:hflow:normals_lab_000005.png?direct&200 |}} {{ :research:hflow:err_000005.png?direct&200 |}} | |
| + | | ||| | ||
| + | | {{ :research:hflow:000006_10.png?direct&200 |}} {{ :research:hflow:depth_000006.png?direct&200 |}} {{ :research:hflow:normals_lab_000006.png?direct&200 |}} {{ :research:hflow:err_000006.png?direct&200 |}} | {{ :research:hflow:000007_10.png?direct&200 |}} {{ :research:hflow:depth_000007.png?direct&200 |}} {{ :research:hflow:normals_lab_000007.png?direct&200 |}} {{ :research:hflow:err_000007.png?direct&200 |}} | {{ :research:hflow:000008_10.png?direct&200 |}} {{ :research:hflow:depth_000008.png?direct&200 |}} {{ :research:hflow:normals_lab_000008.png?direct&200 |}} {{ :research:hflow:err_000008.png?direct&200 |}} | | ||
| + | | ||| | ||
| + | | {{ :research:hflow:000009_10.png?direct&200 |}} {{ :research:hflow:depth_000009.png?direct&200 |}} {{ :research:hflow:normals_lab_000009.png?direct&200 |}} {{ :research:hflow:err_000009.png?direct&200 |}} | {{ :research:hflow:000010_10.png?direct&200 |}} {{ :research:hflow:depth_000010.png?direct&200 |}} {{ :research:hflow:normals_lab_000010.png?direct&200 |}} {{ :research:hflow:err_000010.png?direct&200 |}} | {{ :research:hflow:000011_10.png?direct&200 |}} {{ :research:hflow:depth_000011.png?direct&200 |}} {{ :research:hflow:normals_lab_000011.png?direct&200 |}} {{ :research:hflow:err_000011.png?direct&200 |}} | | ||
| + | | ||| | ||
| + | | {{ :research:hflow:000012_10.png?direct&200 |}} {{ :research:hflow:depth_000012.png?direct&200 |}} {{ :research:hflow:normals_lab_000012.png?direct&200 |}} {{ :research:hflow:err_000012.png?direct&200 |}} | {{ :research:hflow:000013_10.png?direct&200 |}} {{ :research:hflow:depth_000013.png?direct&200 |}} {{ :research:hflow:normals_lab_000013.png?direct&200 |}} {{ :research:hflow:err_000013.png?direct&200 |}} | {{ :research:hflow:000014_10.png?direct&200 |}} {{ :research:hflow:depth_000014.png?direct&200 |}} {{ :research:hflow:normals_lab_000014.png?direct&200 |}} {{ :research:hflow:err_000014.png?direct&200 |}} | | ||
| + | | ||| | ||
| + | | {{ :research:hflow:000015_10.png?direct&200 |}} {{ :research:hflow:depth_000015.png?direct&200 |}} {{ :research:hflow:normals_lab_000015.png?direct&200 |}} {{ :research:hflow:err_000015.png?direct&200 |}} | {{ :research:hflow:000016_10.png?direct&200 |}} {{ :research:hflow:depth_000016.png?direct&200 |}} {{ :research:hflow:normals_lab_000016.png?direct&200 |}} {{ :research:hflow:err_000016.png?direct&200 |}} | {{ :research:hflow:000017_10.png?direct&200 |}} {{ :research:hflow:depth_000017.png?direct&200 |}} {{ :research:hflow:normals_lab_000017.png?direct&200 |}} {{ :research:hflow:err_000017.png?direct&200 |}} | | ||
| + | | ||| | ||
| + | | {{ :research:hflow:000018_10.png?direct&200 |}} {{ :research:hflow:depth_000018.png?direct&200 |}} {{ :research:hflow:normals_lab_000018.png?direct&200 |}} {{ :research:hflow:err_000018.png?direct&200 |}} | {{ :research:hflow:000019_10.png?direct&200 |}} {{ :research:hflow:depth_000019.png?direct&200 |}} {{ :research:hflow:normals_lab_000019.png?direct&200 |}} {{ :research:hflow:err_000019.png?direct&200 |}} | {{ :research:hflow:000020_10.png?direct&200 |}} {{ :research:hflow:depth_000020.png?direct&200 |}} {{ :research:hflow:normals_lab_000020.png?direct&200 |}} {{ :research:hflow:err_000020.png?direct&200 |}} | | ||
| + | | ||| | ||
| + | | {{ :research:hflow:000021_10.png?direct&200 |}} {{ :research:hflow:depth_000021.png?direct&200 |}} {{ :research:hflow:normals_lab_000021.png?direct&200 |}} {{ :research:hflow:err_000021.png?direct&200 |}} | {{ :research:hflow:000022_10.png?direct&200 |}} {{ :research:hflow:depth_000022.png?direct&200 |}} {{ :research:hflow:normals_lab_000022.png?direct&200 |}} {{ :research:hflow:err_000022.png?direct&200 |}} | {{ :research:hflow:000023_10.png?direct&200 |}} {{ :research:hflow:depth_000023.png?direct&200 |}} {{ :research:hflow:normals_lab_000023.png?direct&200 |}} {{ :research:hflow:err_000023.png?direct&200 |}} | | ||
| + | | ||| | ||
| + | | {{ :research:hflow:000024_10.png?direct&200 |}} {{ :research:hflow:depth_000024.png?direct&200 |}} {{ :research:hflow:normals_lab_000024.png?direct&200 |}} {{ :research:hflow:err_000024.png?direct&200 |}} | {{ :research:hflow:000025_10.png?direct&200 |}} {{ :research:hflow:depth_000025.png?direct&200 |}} {{ :research:hflow:normals_lab_000025.png?direct&200 |}} {{ :research:hflow:err_000025.png?direct&200 |}} | {{ :research:hflow:000026_10.png?direct&200 |}} {{ :research:hflow:depth_000026.png?direct&200 |}} {{ :research:hflow:normals_lab_000026.png?direct&200 |}} {{ :research:hflow:err_000026.png?direct&200 |}} | | ||
| + | | ||| | ||
| + | | {{ :research:hflow:000027_10.png?direct&200 |}} {{ :research:hflow:depth_000027.png?direct&200 |}} {{ :research:hflow:normals_lab_000027.png?direct&200 |}} {{ :research:hflow:err_000027.png?direct&200 |}} | {{ :research:hflow:000028_10.png?direct&200 |}} {{ :research:hflow:depth_000028.png?direct&200 |}} {{ :research:hflow:normals_lab_000028.png?direct&200 |}} {{ :research:hflow:err_000028.png?direct&200 |}} | {{ :research:hflow:000029_10.png?direct&200 |}} {{ :research:hflow:depth_000029.png?direct&200 |}} {{ :research:hflow:normals_lab_000029.png?direct&200 |}} {{ :research:hflow:err_000029.png?direct&200 |}} | | ||
| ===== References ===== | ===== References ===== | ||