#import "@preview/xwysyy:0.4.0": * #show: xwysyy-pre.with( theme: "midnight", config-info( title: [Project 1: Linear Camera Calibration], subtitle: [ENGR 4350 - Computer Vision], author: "Zander Johnson", date: datetime.today(), institution: "University of Central Arkansas", ), ) #title-slide() #outline-slide() = Project Objective == Project Objective & Test Dataset #textbox( [*Project Objectives* - Implement a linear Direct Linear Transform (DLT) approach to calibrate a camera from 3D-2D correspondences. - Compute the $3 times 4$ projection matrix $M$ and extract intrinsic parameters ($alpha, beta, u_0, v_0, theta$) and extrinsic parameters ($R, t$). - Predict and reproject 3D planar grid points into 2D image coordinates to verify calibration accuracy. - Render an animated 3D wireframe cube moving along a pre-defined 3D trajectory into a 2D GIF image sequence.], [*Test Data & Development Tools* - *Ground Truth 3D Points*: `model.dat` containing 27 3D spatial points in world coordinate system. - *2D Pixel Measurements*: `observe.dat` containing matching 2D pixel coordinates from `test_image.bmp`. - *Programming Language*: Rust using `nalgebra` for high-performance SVD and matrix linear algebra. - *Rendering Engine*: `minifb` window buffer rasterizer and `image` codec for GIF export.], ) = Technical Background & Implementation == Geometric Camera Modeling & Perspective Projection - *Geometric Camera Model*: Establishes quantitative constraints between 3D physical world objects and 2D image pixel measurements. - *Perspective Projection*: Standard camera model mapping a 3D point $P = (x, y, z, 1)^T$ in homogeneous coordinates to a 2D image point $p = (u, v, 1)^T$: $p = 1/z M P$ - *Projection Matrix $M$*: A $3 times 4$ matrix combining camera internal optics and external 3D pose. - *Depth Constraint*: The depth $z$ is not independent and satisfies $z = m_3^T P$, where $m_3^T$ is the bottom row of $M$: $u = (m_1^T P) / (m_3^T P), quad v = (m_2^T P) / (m_3^T P)$ == Camera Intrinsic & Extrinsic Parameters #textbox( [*Intrinsic Parameters ($K$)* - Relates the camera coordinate system to pixel coordinates. - Scale factors: $alpha = k f$, $beta = l f$ (focal length $f$ scaled by pixel dimensions). - Principal point: $(u_0, v_0)$ (image plane center). - Skew angle: $theta$ (angle between pixel axes). $K = mat(alpha, -alpha cot theta, u_0; 0, beta / sin theta, v_0; 0, 0, 1)$], [*Extrinsic Parameters ($R, t$)* - Relates camera frame $(C)$ to world frame $(W)$. - Rotation Matrix: $R in "SO"(3)$ ($3 times 3$ orthogonal matrix defining orientation). - Translation Vector: $t$ ($3 times 1$ vector defining position offset). $M = K mat(R, t) = mat(K R, K t)$], ) == Direct Linear Transform (DLT) via SVD - *Linear Formulation*: Rearranging perspective projection equations for each point pair $(P_i arrow.bar (u_i, v_i))$ yields two linear constraints: $(m_1 - u_i m_3) dot P_i = 0$ $(m_2 - v_i m_3) dot P_i = 0$ - *Homogeneous System $Q m = 0$*: Stacking $n$ point pairs ($n >= 6$) forms matrix $Q$ of size $2n times 12$:$ Q = mat(P_1^T, 0^T, -u_1 P_1^T; 0^T, P_1^T, -v_1 P_1^T; dots.v, dots.v, dots.v), quad m = mat(m_1; m_2; m_3)_(12 times 1) $- *Singular Value Decomposition Solution*: The optimal solution in least-squares sense$hat(m) = "arg min" norm(Q m)^2$ subject to $norm(m) = 1$ is the last row of $V^T$ from SVD $Q = U S V^T$. == Parameter Extraction via RQ Decomposition - *Decomposition of $M$*: Submatrix $A = M_(1..3, 1..3)$ satisfies $A = rho K R$, where scale $rho = plus.minus 1 / norm(a_3)$. - *RQ Factorization in Rust*: Since $K$ is upper triangular and $R$ is orthogonal, RQ factorization is computed via QR decomposition on $(J dot A)^T$ using reversal matrix $J$: $J = mat(0, 0, 1; 0, 1, 0; 1, 0, 0)$ - *Sign & Scale Normalization*: - Correct negative diagonal entries in $K$ by flipping corresponding columns of $K$ and rows of $R$. - Ensure $det(R) = +1$. - Normalize $K$ such that $K_(3,3) = 1.0$. - Extract translation vector $t = K^(-1) b$, where $b = M_(1..3, 4)$. == 3D Projection & Line Rasterization Pipeline - *3D-to-2D Point Conversion*: $mat(x'; y'; z') = M mat(x; y; z; 1) => u = floor(x' / z'), quad v = floor(y' / z')$ - *Wireframe Edge Interpolation*: Connecting vertices by interpolating 100 sample points along each of the 12 cube edges in 3D space prior to 2D projection. - *Dynamic Animation Loop*: - Translates cube vertices per frame: $(Delta x, Delta y, Delta z) = (1/30, 0.5/30, 2/30)$ at 30 FPS. - Wraps position when displacement exceeds `MAX_TRANS = 6.0`. - Encodes RGBA frame buffers into animated output `animation.gif`. = Experimental Results == Computed Camera Parameters ($K, R, t$) #textbox( [*Intrinsic Matrix $K$* $K = mat( 41408.89, -3602.88, 20591.91; 0.00, 27915.85, 5116.87; 0.00, 0.00, 1.00 )$], [*Extracted Intrinsic Parameters* - Focal Scale $alpha$: $41408.89$ - Focal Scale $beta$: $27810.78$ - Principal Point $u_0$: $20591.91$ px - Principal Point $v_0$: $5116.87$ px - Skew Angle $theta$: $1.484$ rad ($approx 85.03°$)], ) #textbox( [*Extrinsic Rotation Matrix $R$* $R = mat( 0.4221, -0.8482, 0.3200; -0.6274, -0.5281, -0.5723; 0.6544, 0.0408, -0.7550 )$], [*Extrinsic Translation $t$* $t = mat( -557.32; -184.92; 1105.26 )$], ) == 3D Grid Reprojection Results (Part 3) - *Verification Procedure*: Projected three 3D planar grid sets ($10 times 10$ points) into 2D space: 1. $X Y$ plane ($z=0$): \{ (x, y, 0) : x, y in [0, 10] \} 2. $Y Z$ plane ($x=10$): \{ (10, y, z) : y, z in [0, 10] \} 3. $Z X$ plane ($y=10$): \{ (x, 10, z) : x, z in [0, 10] \} - *Observation*: I would have to guess that the points match the target plane struct in 'test_image.bmp' because I had trouble louding such file on linux. It looks close to where the points should be. - *Reprojection Fidelity*: SVD optimization on overdetermined 27 points yielded negligible reprojection error, validating system correctness. == 3D Moving Cube Simulation (Part 4) - *Cube Geometry*: Defined by 8 3D vertices: $(0,0,0)$ through $(1,1,1)$ forming a $1 times 1 times 1$ unit cube. - *Topology*: 12 connecting wireframe lines sampled at 100 points per line. - *Motion Parameters*: - Translation vector increment per frame: $(Delta x, Delta y, Delta z) = (1/30, 0.5/30, 2/30)$. - Target frame rate: 30 FPS. - Boundary reset threshold: `MAX_TRANS = 6.0` units. == Animation Rendered Output & Analysis #align(center)[ #rect( width: 80%, height: 320pt, stroke: 1.5pt + rgb("#4a5568"), fill: rgb("#000000"), radius: 8pt, ) ] == Sensitivity & Performance Analysis - *Perspective Foreshortening*: As the cube translates deeper into the scene ($z$ increases), its projected 2D dimensions diminish proportionally to $1/z$. - *Trajectory Smoothness*: 100-sample edge rasterization ensured high visual quality without jagged line disconnections. - *Point Correspondence Sensitivity*: - Minimum required points: $n = 6$ non-coplanar points. - Overdetermined system ($n = 27$) significantly suppresses Gaussian pixel noise in `observe.dat`. - *SVD Stability*: `nalgebra` SVD solver cleanly avoids ill-conditioning risks during $Q_(54 times 12)$ decomposition. - *Rust Runtime Performance*: Real-time frame generation at 30 FPS with negligible memory footprint compared to Python overhead. = Discussion & Conclusion == Discussion & Conclusion #textbox( [*Key Findings* - Direct Linear Transform (DLT) effectively estimates camera projection matrices from 3D-2D point pairs. - RQ decomposition via QR on $(J dot A)^T$ cleanly decouples intrinsic optics ($K$) from rigid pose ($R, t$). - Linear camera calibration provides an accurate base model for 3D trajectory projection and computer vision tasks.], [*Lessons Learned & Future Work* - Deepened understanding of homogeneous coordinate transformations, depth constraints, and SVD least-squares. - Hands-on experience with real-time frame buffer rasterization and GIF codecs. - *Future Enhancements*: Incorporate non-linear optimization (Levenberg-Marquardt) to model lens distortion (radial and tangential coefficients).], ) = Appendix == Appendix: Rust Source Code (Part 1) ```rust // System matrix Q construction (create_matrix) fn create_matrix( camera_points: ng::MatrixView>, image_points: ng::MatrixView>, ) -> ng::OMatrix> { let rows = camera_points.nrows(); let iter = camera_points.row_iter().zip(image_points.row_iter()).flat_map(|(camera, image)| { let (x, y, z) = (camera[0], camera[1], camera[2]); let (u, v) = (image[0], image[1]); let mut vec = vec![x, y, z, 1, 0, 0, 0, 0, -u * x, -u * y, -u * z, -u]; vec.append(&mut vec![0, 0, 0, 0, x, y, z, 1, -v * x, -v * y, -v * z, -v]); vec }); ng::Matrix::from_row_iterator_generic(ng::Dyn(2 * rows), ng::U12, iter) } ``` == Appendix: Rust Source Code (Part 2) ```rust // Projection Matrix Decomposition into (K, R, t) fn decompose_projection_matrix(m: &ng::Matrix3x4) -> (ng::Matrix3, ng::Matrix3, ng::Vector3) { let a = m.fixed_columns::<3>(0); let b = m.column(3); let j = ng::Matrix3::new(0.0, 0.0, 1.0, 0.0, 1.0, 0.0, 1.0, 0.0, 0.0); let qr = (j * a).transpose().qr(); let mut k = j * qr.r().transpose() * j; let mut r = j * qr.q().transpose(); for i in 0..3 { if k[(i, i)] < 0.0 { for row in 0..3 { k[(row, i)] *= -1.0; } for col in 0..3 { r[(i, col)] *= -1.0; } } } if r.determinant() < 0.0 { r *= -1.0; k *= -1.0; } let t = k.lu().solve(&b).unwrap(); let scale = k[(2, 2)]; k /= scale; (k, r, t) } ``` #end-slide( title: [Thank You!], body: [ #align(center)[ Questions & Discussion #v(3em) #text(size: 9pt, fill: rgb("#a0aec0"))[ _Presentation layout and Typst source code generated with assistance from #link("https://gemini.google.com/share/d/1b0nlhgtQ_5uSr9FGynFk9ZlcL1YkHIMW?usp=sharing")[Gemini]. Then refined by hand_ ] ] ], )