cd /home/$USER/workspace/
git clone [email protected]:behnamasadi/robotic_notes.git
vcpkg is configured as a git submodule. Initialize it:
cd /home/$USER/workspace/robotic_notes
git submodule update --init --recursive
set the path:
export VCPKG_ROOT=$PWD/vcpkg
export PATH=$VCPKG_ROOT:$PATH
Setting VCPKG_ROOT tells vcpkg where your vcpkg instance is located.
Install required system dependencies for vcpkg (on Linux):
sudo apt-get install -y bison flex build-essential cmake autoconf autoconf-archive automake libtool libltdl-dev libx11-dev libxft-dev libxext-dev libxtst-dev libxrandr-dev ninja-build pkg-config
Now you can run:
cmake -S . -B build \
-DCMAKE_TOOLCHAIN_FILE=./vcpkg/scripts/buildsystems/vcpkg.cmake \
-DCMAKE_BUILD_TYPE=Release \
-DVCPKG_TARGET_TRIPLET=x64-linux-release
The VCPKG_TARGET_TRIPLET=x64-linux-release option ensures vcpkg only builds release packages, which significantly reduces build time (especially for large packages like OpenCV) and disk space usage. This is already configured in CMakeLists.txt, but you can explicitly set it as shown above.
The Rerun C++ SDK (and its Arrow dependency) is disabled by default because the SDK is downloaded and built via FetchContent, which is heavy. The Python rerun-sdk package covers the same use cases for the notebooks. To opt in and build the rerun-dependent C++ targets, add the following flags:
cmake -S . -B build \
-DCMAKE_TOOLCHAIN_FILE=./vcpkg/scripts/buildsystems/vcpkg.cmake \
-DCMAKE_BUILD_TYPE=Release \
-DVCPKG_TARGET_TRIPLET=x64-linux-release \
-DBUILD_RERUN_EXAMPLES=ON \
-DVCPKG_MANIFEST_FEATURES=rerun
cmake --build build --parallel
conda create -n robotic_notes python=3.12 -y
conda activate robotic_notes
cd /home/$USER/anaconda3/envs/robotic_notes/
Create this soft link.
ln -s /home/$USER/workspace/robotic_notes /home/$USER/anaconda3/envs/robotic_notes/src
Install the python packages:
conda install -c conda-forge opencv
pip install graphslam
conda install conda-forge::gtsam
conda install conda-forge::matplotlib
conda install conda-forge::plotly
conda install -c conda-forge jupyterlab
pip install ahrs
pip install pyceres
pip install liegroups
pip install rerun-sdk[notebook]==0.29.2
pip install "gradio_rerun==0.29.2"
pip install "gradio==6.5.1"
pip install ipykernel
pip install jupyterlab
-
Robotic System Design & Design Patterns
- 1. High-Level Architecture of a Humanoid Robot
- 2. Perception Pipeline: Consistency, Robustness, Synchronization
- 3. High-Frequency Sensor Pipelines in C++
- 4. Concurrency & Real-Time: SPMC Sensor Distribution
- 5. Task Execution: State Machine vs Behavior Tree
- 6. Debugging a Regressing Perception Model in Production
- 7. Time Synchronization
- 8. Safety & Fault Tolerance
- 9. Logging, Observability & Replay
-
Designing a Geometry & Imaging Library in C++
- 1. The three libraries, and what each one got right
- 2. The type razor: what earns a type
- 3. Storage: stack or heap, and who decides
- 4. The three-way split: owner, view, expression
- 5. Ownership:
unique_ptr,shared_ptr, raw, view - 6. Templates: what to parameterise on, and what not to
- 7. Inheritance: CRTP, virtual, and type erasure
- 8. Rotations: matrix, quaternion, angle-axis, Euler
- 9. Lie groups and Lie algebras: two types, not one
- 10. Inversion: when to transpose, and when to genuinely invert
- 11. Putting frames in the type system
- 12. The class map: hierarchy, composition, dependency
- 13. Images and volumes
- 14. Applying a transform to an image: why warping does invert
- 15. Point clouds: AoS vs SoA
- 16. Rotating fast, without copying
- 17. Algorithms and extensibility: plugging in a solver
- 18. Decision tables: which representation, when
- 19. Files, headers and build structure
- 20. Modern C++: what the language now does for you
- 21. Specification of a good design
- 22. Worked examples: what the calls actually look like
- 23. Anti-patterns: the mistakes people actually make
- 24. Do / Don't
- 25. The whole picture
- companion code:
src/geometry_library_design/—tiny_geometry.hpp+demo.cpp, dependency-free, build withg++ -std=c++17 -O2 demo.cpp -o demoor the CMake targettiny_geometry_demo
- Robot Configuration, Degrees of Freedom, and Topology
- Algebraic topology (playlist)
- Non-Holonomic Constraints, Pfaffian Constraints and Holonomic Constraints
- Kinematics of Differential Drive Robots and Wheel odometry
- 1. Velocity-based (dead reckoning)
- 2. Odometry-based Motion Model
- Nonlinear uncertainty model associated with a robot's position over time (The Banana Distribution is Gaussian)
- Coordinate Frame Conventions — ENU, NED, FRD, FLU, RDF, RUB
- Global frame conventions
- 1. Frames
- 2. What is stored in the ESKF state?
- 3. What does the IMU measure?
- Accelerometer model
- Gyroscope model
- What the accelerometer measures
- Rest (no motion) numeric example: why you see ±9.81
- “Add gravity” in the velocity update (numeric)
- Random walk
- Attitude estimation: two flavors
- Attitude from gravity (tilt sensing)
- Numerical Example
- Uniqueness of R_BW
- Quaternion integration
- Quaternion from a single accelerometer reading
- Euler-angle rates from body-axis rates
- Complementary filter
- Yaw from a magnetometer (tilt-compensated compass)
- 4. Orientation propagation (Body → World)
- 5. Why accelerometer must be rotated
- 6. Why gravity is added
- What does the IMU measure here?
- Why measured acceleration at IMU rest was not 9.8
- 7. Velocity propagation (world frame)
- 8. Position propagation (world frame)
- 9. Bias propagation (random walk)
- Kalibr-style IMU noise model
- Allan variance background
- EuRoC noise params → discrete-time numbers you use in code
- Further references
- Square binary fiducials (ArUco, AprilTag
- Composite calibration boards (ChArUco, AprilGrid)
- Which marker should I use?
- AprilTag — coordinate frame and measurements
- Generating AprilTags
- Detecting AprilTags (C++, Python, ROS)
- Common pitfalls
- AprilTag vs ArUco (and ChArUco / AprilGrid)
- Choosing a calibration target
- Calibration target files (AprilGrid PDF, YAMLs)
- Supported camera models and distortion
- Building the Kalibr Docker image
- Recording the calibration bag
- Camera-only calibration
- Camera–IMU calibration
- Common pitfalls
- 1. Names
- 2. Remapping Arguments
- NodeHandles
- Roslaunch
- URDF
- Publishing the State
- ROS best practices
- move_base
- ROS Odometery Model
- ROS State Estimation
- EKF Implementations
- Differential Drive Wheel Systems
- Choosing a distro
- Pull the image
- Run a persistent container
- Extending the image with a Dockerfile
- Docker Compose for multi-container setups
- ROS 2 ↔ ROS 1 bridge
- ROS 2 inside the container
- Colcon
- Packages
- Xacro
- Launch files
- Nav2
- teleop_twist_keyboard
- Useful container shortcuts
- Gazebo Versions
- Installation
- Building a model
- Building world
- Moving the robot
- Sensors
- Spawn URDF
- ROS 2 integration
- Distro / Gazebo mapping
- Dockerfile (Jazzy + Harmonic + ros_gz)
- Running the container
- GPU access (NVIDIA)
- Launching Gazebo
ros_gz_bridgeessentials- Demo recipes
- Troubleshooting
- Probability Review (Marginals/conditionals, probability vs likelihood, conditional Gaussians ) (foundational primer)
- Bayes Filter
- Maximum A-Posteriori (MAP) Estimation
- Information Form vs Covariance Form — which parameterization to carry, and why
- The underlying operation is multiplication
- The Kalman gain is the price of moment coordinates
- Then why does anyone use the covariance form?
- Why SLAM back-ends live in information space
- Why $H$ is literally an information matrix
- Marginalization: the bill information form eventually pays
- Decision table
- Kalman Filter
- Extended Kalman Filter
- Extended Kalman Filter for Differential Drive Robot
- Error State Extended Kalman Filter
- Error State Extended Kalman Filter(IMU, a GNSS, and a LiDAR)
- Multi-State Constraint Kalman Filter (MSCKF)
- Quaternion kinematics for the error-state Kalman filter
- Active Exposure Control for Robust Visual Odometry in HDR Environments
- Pose Graph SLAM from Scratch
- nano-pgo
- General Graph Optimization g2o explained
- Factor Graphs, Bayes Trees, and iSAM2 (with GTSAM)
- Resilient Autonomy in Perceptually-degraded Environments
- HBA Large-Scale LiDAR Mapping Module
- Hierarchical, multi-resolution volumetric mapping (wavemap)
- kiss-icp
- TagSLAM SLAM with tags
- OpenDroneMap
- Interactive SLAM
- Volumetric TSDF Fusion of Multiple Depth Maps
- Euclidean Signed Distance Field (ESDF)
- Lidar odometry smoothing using ES EKF and KissICP for Ouster sensors with IMUs
- Multisensor-aided Inertial Navigation System (MINS)
- GLOMAP explained
- Zero-Shot Point Cloud Registration
- Add Apriltag to loop closure
- Navtech Radar SLAM
- 1. Orthogonal Procrustes
- 2. Wahba's problem
- 3. Davenport's q-method and QUEST
- 4. Kabsch algorithm
- 5. Umeyama algorithm
- 6. Iterative Closest Point (ICP)
- Rotation Averaging — the same SVD/Procrustes trick applied to N estimates of one rotation
The vio_benchmark/ subproject is a self-contained
framework for comparing visual-inertial odometry estimators against
public datasets (EuRoC MAV, TUM-VIO, and others). It bundles OpenVINS
and VINS-Fusion as submodules, ships per-dataset calibration files
and ground-truth references, and includes pipeline scripts plus a
rerun.io visualisation that overlays multiple trajectories with the
live stereo camera projected inside each estimator's frustum. First
head-to-head result: OpenVINS 0.295 m APE vs VINS-Fusion 0.248 m on
EuRoC MH_01_easy.
- Trajectory analysis methodology and case studies
- Estimator consistency — NEES, NIS, and auditing the covariance
- Benchmark methodology — run-to-run variance and paired significance
- Runtime evaluation — latency, throughput, and tail costs
- Diagnostic procedure for broken VIO
- Estimator parameter reference
- Visual-Inertial Navigation Systems: An Introduction
- IMU Propagation Derivations (OpenVINS)
- Error State Kalman Filter VIO (ESKF-VIO)
- Overview
- Sensor setup
- Calibration files and projection matrices
- Ground-truth poses
- Displaying ground-truth poses in Rerun
- Stereo vision
- Sparse/dense reconstruction from known poses with COLMAP
- Understanding KITTI poses
- Dataset recipes — EuRoC, TUM-VIO, M2DGR, 4Seasons
- M3DGR — Multi-sensor, Multi-scenario, Massive-baseline SLAM Dataset for Ground Robots
- rvp group SLAM datasets
- DSEC — Stereo Event Camera Dataset for Driving Scenarios
- Hilti x Trimble SLAM Challenge 2026
- Benchmark Comparison of Monocular Visual-Inertial Odometry Algorithms for Flying Robots
- A Comparison of Modern General-Purpose Visual SLAM Approaches
- ETH3D
The lio_benchmark/ subproject is a self-contained
Gazebo simulator for LiDAR-inertial odometry development. It ships a
SubT-style rover with a 16-beam LiDAR + IMU + cameras, three
cave/tunnel worlds, and FAST-LIO wired into the rover by default.
One-line bring-up via docker compose. Unlike VIO, LIO works
correctly on this sim (FAST-LIO consistently produces 0.3–1.5 %
end-point error on rover recordings) — the architectural reason is in
the VIO diagnostic guide §3.
- lio_benchmark README and quick start
- Simulator GPU/rendering performance notes
- FAST-LIO + LiDAR tuning reference
- Why LIO is robust to bad IMU but VIO isn't
- Robust Real-time LiDAR-inertial Initialization
- Lidar SLAM for Automated Driving (MATLAB)
- FAST-LIO
- iG-LIO — incremental GICP tightly-coupled LIO
- DLIO — Direct LiDAR-Inertial Odometry with Continuous-Time Motion Correction
- CT-LIO — Continuous-Time LiDAR-Inertial Odometry
- LIO-SAM
- GLIM — versatile and extensible range-based 3D mapping framework
- LIMO — Lidar-Monocular Visual Odometry
- Structure from Motion from Scratch
- Rotation Averaging
- Single rotation averaging — chordal/SVD mean, quaternion (Markley) mean, Karcher mean, Weiszfeld median
- Multiple rotation averaging (rotation synchronization) — the global-SfM problem, Govindu → Martinec–Pajdla → Chatterjee–Govindu IRLS → Shonan
- Why outlier relative rotations, not the optimizer, are the hard part
- Robust Rotation Averaging (talk)
- Bundler
- Noah Snavely Reprojection Error
- Global Structure-from-Motion Revisited
- LightGlue
- XRefine: Attention-Guided Keypoint Match Refinement
- DenseSFM
- Pixel-Perfect Structure-from-Motion
- image-matching-webui
- Implementation tutorial
- Depth + pose training objective
- Supervised depth add-on
- Shapes, sizes and VRAM knobs
- Minimal architectures
- Training recipe
- Evaluation
- Baselines for a 4 GB GPU
- Multi-scale output
- Implementation tips
- Overview of the main families
- Pose regression heads — CNN vs ViT
- ViT-based VO architectures
- Temporal modeling
- Input and output representations
- Self-supervised VO
- DUSt3R & MASt3R notebook
- The paper landscape
- The 2026 role inversion
- VGGT-SLAM loop closure
- Persistent state as a learned filter
- The pointmap
- Pointmap head
- The canonical frame
- Global alignment, not bundle adjustment
- Matching — MASt3R's descriptor head
- Recovering intrinsics
- Recovering the relative pose
- Sparse global alignment (MASt3R-SfM)
- Speedy MASt3R
- MASt3R-SLAM with Rerun
- Gaussian Splatting
- instant-ngp
- Hierarchical 3D Gaussians for large datasets
- EvDeblurNeRF
- MegaScenes
- Morpheus
- Trajectory analysis
- Why naive comparison fails — the alignment problem
- Absolute Pose Error (APE / ATE)
- Relative Pose Error (RPE)
- Alignment — Umeyama Sim(3), SE(3), yaw-only, posyaw
- Sub-trajectory drift
- Which statistics to report
- Tools —
evoandrpg_trajectory_evaluation - Reporting recipe
- Evaluation metrics for poses and trajectories
- The one residual behind ATE and mAA
- Same Umeyama, different robustness
- The convention trap: $C = -R^\top t$
- Two different metrics are both called mAA
- image-matching-benchmark
- Estimator consistency
- Accuracy and consistency are different failure modes
- NEES — Normalised Estimation Error Squared
- The gauge trap — why absolute NEES is usually meaningless
- ANEES — and why one run tests almost nothing
- NIS — consistency without ground truth
- Diagnosis — FEJ, double-counting, sparsification
- Where P comes from — and the convention trap
- Reporting recipe
- Benchmark methodology
- "Deterministic" algorithms are not
- How many runs, and what to do with them
- Compare paired, not unpaired
- Bootstrap confidence intervals
- Aggregation traps — failures, weighting, normalisation
- Cherry-picking and leaderboard overfitting
- Reporting checklist
- Runtime evaluation
- Three numbers that get called "speed"
- The mean is the wrong statistic — report the tail
- Per-thread budgets
- Measurement traps — build type, playback rate, throttling
- Memory and energy
- Tooling — perf, Tracy, ros2 topic delay
- Reporting template
- Types of loss functions used in VO
- Rotation loss overview
- Quaternion loss
- Geodesic loss
- Full transformation loss $SE(3)$
- How to pick the weights
- Modern Robotics Mechanics, Planning, and Control (Kevin M. Lynch, Frank C. Park)
- Modern Robotics Mechanics, Planning, and Control (Instructor Solution Manual, Solutions )
- MODERN ROBOTICS MECHANICS, PLANNING, AND CONTROL (Practice Exercises)
- Basic Knowledge on Visual SLAM: From Theory to Practice, by Xiang Gao, Tao Zhang, Qinrui Yan and Yi Liu
- STATE ESTIMATION FOR ROBOTICS (Timothy D. Barfoot)
- SLAM for Dummies
- VSLAM Handbook
- SLAM Handbook
- Matrix Calculus (for Machine Learning and Beyond)
- Reinforcement Learning: A Comprehensive Overview
- Linear Algebra for Computer Vision, Robotics, and Machine Learning