HAND
Current Project — 2026 · In Progress

The Hand

Status
In Progress
Year
2026
Type
Mechanical · Electrical · Software
Stack
ROS2 · YOLOv8 · SolidWorks · Altium · STM32
// 01 — What It Is

Autonomous 6DOF Robotic Arm

The Hand is an autonomous 6DOF robotic arm that uses a camera module powered by a fine-tuned YOLOv8 model to detect, classify, and sort medication pills into daily dose compartments — fully by itself.

Medication errors are one of the leading causes of preventable harm in elderly care, where managing multiple pills across multiple daily schedules is a real and ongoing problem. The Hand is a low-cost, open-source approach to automating that process — detecting pills by shape and colour in real time, then picking and placing them into the correct tray slot using a full vision-to-motion pipeline.

This project was inspired by the countless 4DOF robotic arms on YouTube, but I wanted to combine my knowledge of computer vision and robotics to build something more technically impressive — and actually useful.

H
Image Coming Soon
// 02 — Objectives

Build Checklist

Checked off as I go.

// Phase 1 — Robot Modeling
✓
Generate xacro file — 6 links, 6 joints, joint limits, inertia values Design
✓
Add camera link to end-effector with Gazebo Harmonic sensor plugin Design
✓
Load and verify all 6 DOFs in Gazebo Harmonic Software
✓
Visualise TF frames and joint states in RViz2 Software
// Phase 2 — Arm Control + IK
✓
Set up MoveIt 2 — run setup assistant, generate SRDF and controller configs Software
✓
Test IK in RViz2 — drag end-effector to arbitrary poses Software
✓
Write Python MoveIt client using MoveGroupInterface API Software
✓
Test hardcoded pick-and-place motion in simulation Software
// Phase 3 — Vision (YOLOv8)
✓
Collect pill dataset — NIH Pillbox + own photos of proxy objects Vision
✓
Label dataset in Roboflow and fine-tune YOLOv8 model Vision
✓
Wrap YOLOv8 inference as a ROS 2 node publishing detections Software
✓
Test pill detection on simulated camera feed in Gazebo Vision
// Phase 4 — Full Simulation Integration
✓
Compute hand-eye calibration transform in simulation Software
✓
Build perception → planning bridge node (YOLO detections → 3D coords → MoveIt) Software
✓
Implement full control loop end-to-end in Gazebo Software
✓
Tune and stress-test — multiple pills, varied positions, edge cases Software
// Phase 5 — Physical Build
✓
Design full 6DOF arm CAD in SolidWorks using xacro dimensions as reference Design
✓
Source components — servos, ESP32, Raspberry Pi 4, camera, PSU Hardware
✓
Breadboard prototype — verify ESP32 ↔ Pi UART comms and servo response Hardware
✓
3D print all arm links and assemble Hardware
// Phase 5b — Firmware
✓
Set up micro-ROS on ESP32 — agent, transport layer, build system Firmware
✓
Write PWM control firmware using ESP32 LEDC hardware timer API Firmware
✓
Subscribe to /joint_commands topic and drive all 6 servo outputs Firmware
✓
Validate firmware end-to-end — Pi publishes, ESP32 moves servos correctly Firmware
// Phase 6 — PCB Design & Fabrication
✓
Design schematic in Altium — ESP32, PCA9685, servo headers, UART, power input Hardware
✓
Route 2-layer PCB layout — wide servo power traces, separated ground planes Hardware
✓
Run DRC, review Gerbers, export Excellon drill file for JLCPCB Hardware
✓
Order boards from JLCPCB, solder components, bring-up test Hardware
✓
Swap breadboard prototype for finished PCB, re-run servo tests Hardware
// Phase 7 — Real-World Deployment
✓
Install ROS 2 Jazzy on Raspberry Pi, transfer and verify all packages Software
✓
Real hand-eye calibration using checkerboard + easy_handeye2 Software
✓
Retune YOLOv8 on real photos under workspace lighting Vision
✓
End-to-end real-world test — detect, pick, and sort real pills Deployment

Progress updated by Ramy

// 03 — Skills I'll Get Out of It

What I'm Learning / Mastering

// Mechanical Design
CAD & 3D Printing

Designing a full 6DOF arm in SolidWorks with manufacturable joints, linkages, and mounting points — then printing and iterating on real hardware.

SolidWorks3D PrintingDFM
// Electrical Design
PCB Design

Designing a custom PCB in Altium Designer to power and control the arm's actuators, handle signal routing, and interface with the microcontroller.

AltiumSchematicsPCB Layout
// Software
ROS2 & MoveIt

Building a full ROS2 stack from URDF modelling to MoveIt motion planning, running forward and inverse kinematics for real-time arm control.

ROS2MoveItGazeboURDF
// Firmware
Embedded Firmware

Writing micro-ROS firmware for the ESP32 to subscribe to ROS 2 joint commands, drive servo PWM outputs via hardware timers, and communicate with the Raspberry Pi over UART.

ESP32micro-ROSC++PWM
// Computer Vision
Object Detection

Training and deploying a fine-tuned YOLOv8 model to detect and classify medication pills in real time, feeding localised coordinates into the arm's motion planning pipeline.

YOLOv8PythonOpenCVRoboflow
// Software
C++ & System Integration

Tying everything together — writing performant C++ ROS2 nodes, integrating vision with motion planning, and debugging a full hardware-software system end to end.

C++ROS2Integration