RIOS
2018–2026

GPU-accelerated YOLO (v8/v11) with PyTorch and TensorRT backends. Multi-camera multiplexing, Prometheus metrics, Kubernetes-native deployment. Production inference at <20ms across 30+ camera streams.

C++/Python observer architecture with pluggable algorithm pipeline. Three-stage depth processing (plane segmentation → corner estimation → full veneer estimation). 14 observer instances in production, 20+ diagnostic failure codes.

SAM2 and EfficientSAM extended for real-time video — camera predictors, streaming inference, multi-object tracking, mask hole filling. 4× speed improvement to 25ms/frame. ~4,000 LOC on top of the foundation models.

Domain-specific YOLO11-seg models — 10 object classes for wood products. mAP 0.778–0.923 across modal, amodal, and binary segmentation. Production defect detection and top-sheet segmentation against similar-colored backgrounds.

Real-time food defect classification on conveyor lines. MobileNetV3 / ResNet50 (6 architectures benchmarked). 3-class output with GradCAM debugging, 27ms inference, windowed majority voting. 50+ versioned datasets, deployed across 6 production lines.

YOLO detection wrapped as a first-class ROS node — published bounding boxes, batched multi-camera inference, parameterized model swap. Drives downstream tracking and grasp targeting.

Custom KAREL server and TCP/IP RMI client for direct FANUC controller access. Stream motion with spline trajectory generation. Joint and cartesian state streaming. Five-state robot status model (NOT_CONNECTED → CONNECTED → ENABLED → ACTIVE → FAULTED). Bypasses the standard teach-pendant workflow. Deployed across 8 FANUC R-2000iC / 220U robots.

Composable FSM architecture (20 states) for industrial automation. Base framework with launcher, commander, event synchronization. Domain-agnostic — works for any sequential automation process. Multi-robot coordination for pick / place / QC with PLC-synchronized fault recovery.

50Hz encoder-driven tracking with TF frame publishing. Five encoder strategies (VirtualCSI, Float, IO, PLC, IOGate). Enables picking from and placing onto a continuously moving conveyor without line stops. Validated at 8.5–10 BPM, >100 ft/min.

Piezoelectric tactile sensor (XTS-1) — 14×37 taxel array at 3–17kHz, picoCoulomb force measurement, 10 thermistors, 13 strain gauges. Three-layer driver stack with PIMPL pattern. TactileNet (dual-finger 3D CNN) classifies grasp events from 100ms windows. Integrated with Robotiq 2-finger gripper and custom Cypress FX3 + PSoC + FPGA firmware.

Containerized edge agent for industrial deployments — camera management, GPU inference (YOLO via Ultralytics / TensorRT), ROS2, MQTT, live streaming, telemetry. Two internal frameworks underneath: a worker-orchestration engine (15+ YAML-configured worker types wired into a channel-based pipeline) and a high-performance C++ data/inference engine. Runs on industrial-rugged NVIDIA Jetson Orin hardware (CERA) — ≥30 fps capture, up to a week of onboard offline storage, NVIDIA DeepStream / GStreamer real-time inference, Allen-Bradley / EtherNet-IP PLC integration, LTE fallback. Deployed under Helm-managed clusters across 25+ edge sites.

Visual builder where customers define regions of interest (box, polygon, line), set ML-prediction-based event triggers, and generate KPI reports — turning raw model predictions into structured semantic events without writing code, then deploying that logic straight to the edge. Output to PLC, ROS2, or HTTP. Customer-configured monitoring across wood products, metals, and construction materials.

Multi-tenant web and mobile platform organized as Company → Sites → Workcell Groups → Clusters → Camera Feeds, with role-based access, deployment history, drift indicators, configuration management, and reporting. Underneath: Helm-based GitOps with Rancher Fleet, a Kubernetes operator for edge CRDs, HashiCorp Vault for secrets, and RClone for data sync — keeping every cluster in sync with cloud-managed configuration across geographically distributed customer sites.

End-to-end pipeline from SAM3 foundation-model auto-annotation → pseudo-labeling → YOLO distillation to edge models, with roughly 100× speedup from teacher to edge student. A self-improving annotation loop (auto-annotate with fine-tuned checkpoints → human review/correction → fine-tune → iterate) runs as a weekly 7-step automated process: validate → annotate → curate → export → train → evaluate → report. Production YOLO11-seg models reach mAP 0.778 (modal), 0.651 (amodal), and 0.923 (2-class) across a 10-class wood/defect ontology, with versioned edge deployment and integrity verification. 50+ versioned datasets form a compounding data flywheel behind every production model.

Automated annotation pipeline for manufacturing video at TB scale. C++ frame extraction at 500+ fps across 64 cores, semantic deduplication, 246-dimension feature engineering, and GPU-accelerated interactive learning via a web UI for tool-assisted labeling (click / rect / polygon with live retraining). Intelligent stratified sampling compresses millions of frames at 1000:1, then generates YOLO training sets with teacher/student distillation.

Full-stack web app for reviewing ML-generated annotations. Accept / reject workflows, lasso selection, keypoint annotation, COCO export. Closes the human-in-the-loop for auto-annotation pipelines.

Model registry with versioned deployment to edge devices. S3 distribution with SHA256 integrity verification. TensorRT optimization for Jetson / CUDA targets. Continuous model updates to 25+ deployed edge clusters.

163 systematic experiments, 9 specialized detectors for robotic pick-and-place failures (double picks, partial picks, misalignment). Zero false positives on normal operations, 100% detection on confirmed failures. Interactive threshold tuning UI (SvelteKit + FastAPI). 97.7% false-positive reduction via state-conditional anomaly detection.

PLC event → synchronized multi-camera video (UniFi Protect NVR integration) for root-cause analysis. Structured failure categories, operator intervention tracking, cross-cluster pod management, Prometheus metrics, production database logging. Central to achieving 80% line availability.

Real-time HMI (Rails 7) for 8-robot operations — safety status, workcell control (start / stop / namespace cycling / go-home), recipe management (3PLY, 4PLY, 5PLY, Double 3PLY), diagnostics. WebSocket to ROS via rosbridge_suite. Primary operator interface for 4-shift, 24/7 production.

Camera FOV drift detection via multi-algorithm keypoint tracking, image quality assessment, critical-zone monitoring, ML-based template matching for baseline identification. Proactive issue detection across 25+ deployed edge devices.