Camera calibration can be done by the below mentioned four methods:

  1. 3D reference object based calibration
  2. 2D reference object based calibration
  3. 1D reference object based calibration
  4. Self-calibration

<aside> 💡 Self-calibration cannot usually achieve an accuracy comparable with that of pre-calibration because self-calibration needs to estimate a large number of variables.

</aside>

<aside> 💡 Highest accuracy can be achieved using 3D reference object, however 2D reference object calibration is the most widely used because of ease of use and good accuracy.

</aside>

$$ s \begin{bmatrix} u \\ v \\ 1 \end{bmatrix} =\begin{bmatrix}f_x & \gamma & x_0 \\0 & f_y & y_0 \\0 & 0 & 1\end{bmatrix}\begin{bmatrix}r_{11} & r_{12} & r_{13} & t_x \\r_{21} & r_{22} & r_{23} & t_y \\r_{31} & r_{32} & r_{33} & t_z\end{bmatrix}\begin{bmatrix}X \\Y \\Z \\1\end{bmatrix} $$

<aside> 💡 $\gamma$ - $\alpha*cot\theta$, theta is the angle between the two image axes.

</aside>

<aside> 💡 s - arbitrary scale factor - how big you want your image to be.

</aside>

3D reference object based calibration

  1. Capture Images: Take multiple photographs of the 3D reference object from different angles. This object has known dimensions and feature points (like a calibration grid or checkerboard).
  2. Identify Correspondences: Detect and match the feature points in the images with their known 3D coordinates.
  3. Compute Projection Matrix: The projection matrix P relates the camera's 3D point in homogeneous coordinates to its 2D image point also in homogeneous coordinates, via the equation x=Pâ‹…X. This matrix P can be initially estimated using a set of point correspondences.
  4. Decompose Projection Matrix: Assume P = [B|b] and since P = K*[R|t], B = KR, b = Kt. $Solving \; this \; equation - BB^T = KK^T$ we get the values of K. Once we have the values of K then $R = K^{-1}*B \; | \; t = K^{-1}*b$
  5. Non-Linear optimization: After this the maximum likelihood estimate can be obtained by minimizing the distances between the image points and their predicted positions using non-linear optimization.
  6. Calculating lens distortion
    1. $\hat{u} = u + (u - u_0)[k_1 (x^2 + y^2) + k_2 (x^2 + y^2)^2]\\\hat{v} = v + (v - v_0)[k_1 (x^2 + y^2) + k_2 (x^2 + y^2)^2]\\u,v - undistored\;image\;point\\u_0,v_0 - principal\;point$
    2. Similar to step 5, a non-linear optimization problem can be setup for calculating $k_1\;and\;k_2$

2D reference object based calibration

  1. Capture Images : Print a pattern and attach it to a planar surface;
  2. Identify Correspondences: Take a few images of the model plane under different orientations by moving either the plane or the camera. Detect the feature points in the images
  3. Get homography matrix H using DLT ( SVD )

$$ s \begin{bmatrix} u \\ v \\ 1 \end{bmatrix} = K \begin{bmatrix} r_1 & r_2 & r_3 & t \end{bmatrix} \begin{bmatrix} X \\ Y \\ 0 \\ 1 \end{bmatrix} = K \begin{bmatrix} r_1 & r_2 & t \end{bmatrix} \begin{bmatrix} X \\ Y \\ 1 \end{bmatrix}.

$$