ML Engineering • Data Systems • MLOps

Yohan Shanuka

ML Engineer

Engineering scalable data pipelines, MLOps workflows, and cloud-native ML systems.

pipeline-runner
RUNNING
pipeline.py
canary_monitor.log
18:45:21SYSexecuting...
Speed:
KafkaIngest
SparkProcess
AirflowSchedule
MLflowTrack
FastAPIServe
DockerPackage
KubernetesScale
PythonCore
KafkaIngest
SparkProcess
AirflowSchedule
MLflowTrack
FastAPIServe
DockerPackage
KubernetesScale
PythonCore
KafkaIngest
SparkProcess
AirflowSchedule
MLflowTrack
FastAPIServe
DockerPackage
KubernetesScale
PythonCore
KafkaIngest
SparkProcess
AirflowSchedule
MLflowTrack
FastAPIServe
DockerPackage
KubernetesScale
PythonCore
Yohan Shanuka

Engineering Mindset

Building Scalable ML & Data Systems.

I focus on building intelligent systems at the intersection of machine learning and data engineering, designing high-throughput distributed pipelines and deploying production-ready models that solve complex real-world challenges.

My goal is to develop production-ready machine learning workflows supported by reliable data infrastructure, modern backend systems, and scalable cloud architectures.

Particularly Interested In
MLOps & Automation
Data Engineering Pipelines
Machine Learning Systems
Cloud-Based Systems
Distributed Data Processing
Backend Infrastructure

Engineering Focus

Core Expertise

Three focused areas — each with a clear pipeline and the capabilities I bring to production systems.

Domain Focus

ML Engineering

Train, evaluate, and serve models with low-latency APIs.

System Lifecycle Flow
Features
Training
Serving
Core Capabilities
  • CNN & transfer learning
  • Model APIs & FastAPI
  • Prediction systems
  • Model optimization

Key Performance Index

Inference

< 100ms

Tools & Frameworks

Technology Ecosystem

A curated stack I use to build scalable ML systems, data pipelines, and cloud-native infrastructure.

ProficientCurrently Learning

Machine Learning

Ecosystem Focus

Developing and deploying deep learning, computer vision, and predictive models using modern frameworks.

Core Competencies

CNN & Transfer Learning
PyTorch & TensorFlow
Model Optimization
Average Skill86%
Verified Stack
90
TensorFlow
85
PyTorch
88
Scikit-learn
80
OpenCV
82
XGBoost
86
Keras

Case Studies

Production AI Systems

Deep-dive into the architecture, challenges, and metrics behind intelligent systems I've engineered.

01
Real-TimeDistributed · Stream Processing

Real-Time Streaming Pipeline

End-to-end streaming pipeline handling high-throughput event data with sub-500ms latency. Replaced legacy batch jobs that caused 24-hour reporting delays.

Core Challenge

24-hour delays in critical metric reporting due to batch processing.

Infrastructure:KafkaSpark StreamingMongoDBDocker
Languages & ML:PythonFastAPI
Real-TimeDistributedMLOps Enabled
02
Edge AIIoT · ML Prediction · Alerting

Cattle Health Monitoring System

IoT-driven pipeline combining streaming sensor ingestion with ML prediction to monitor livestock health patterns and trigger real-time alerts.

Core Challenge

Manual monitoring caused late disease detection and yield loss.

Infrastructure:MongoDBFastAPIKafkaDocker
Languages & ML:PythonTensorFlow
Edge AIAPI IntegratedComputer Vision
03
Computer VisionCNN · FastAPI · Docker Serving

Tomato Leaf Disease Classifier

Production-grade CNN microservice for automated crop disease diagnosis. Sub-100ms inference latency with a fully containerized deployment pipeline.

Core Challenge

Farmers needed reliable automated API for rapid field image diagnosis.

Infrastructure:FastAPIDockerReactOpenCV
Languages & ML:PythonTensorFlow
Computer VisionAPI IntegratedMLOps Enabled
Academic Contribution

Research & Publications

Peer-reviewed research at the intersection of IoT systems, edge AI, and real-world agricultural engineering challenges.

1Publications
IoT & AIDomain
DRR·VAURepository
AgriTechFocus Area
Research PaperApplied Research2024

Real-Time Cattle Monitoring Using Low-Cost IoT Smart Collars with LoRa Communication in Sri Lanka's Dry Zones

Authors

Yohan Shanuka, J.A.Det al.

Abstract

Explores the design and deployment of a low-cost IoT smart collar system using LoRa communication for real-time cattle health monitoring across remote dry-zone environments in Sri Lanka. The system captures biometric and behavioural data — including temperature, motion, and heart rate — and transmits it over long-range low-power networks to an ML-backed prediction and alert pipeline.

KeywordsIoTLoRaCattle MonitoringEdge AIReal-Time SystemsSri LankaSmart Agriculture
View PublicationAgriTech · Edge AI

More research in progress — stay tuned for upcoming publications.

System Design

Scalable Architecture

Blueprint-level architecture diagrams showcasing distributed workflows, ETL patterns, and containerized deployments.

Interactive Blueprint Map// 1.2M events/sec · 42ms avg latency
Kafkaevent streamSparkprocessingMongoDBstorageDashboardanalyticsSchema RegistryDQ Checks
Telemetry Console
Blueprint Profile

Streaming Data Architecture

High-throughput event pipeline

ThroughputPeak load scale
1.2M / s
Pipeline SLAService uptime
99.99%
Avg LatencyEvent execution duration
42ms
Interactive Map

Click any node in the blueprint to stream live logs & diagnostics.

Active Development

Currently Building

Real-time progress on active engineering projects — because great systems are always evolving.

In Progress
88%

CNN Tomato Disease Classifier

Transfer learning pipeline with automated retraining triggers and model versioning.

PyTorchMLflowDockerFastAPI
In Progress
55%

Churn Prediction Pipeline

End-to-end ML pipeline predicting customer churn with feature engineering, model serving, and drift monitoring.

XGBoostMLflowFastAPIRedis

Commit Activity

Last 7 months

CNN disease classifier
Streaming inference pipeline
View GitHub Profile →

How I Think

Engineering Philosophy

I focus on building scalable, production-ready AI systems that combine machine learning, distributed data pipelines, and efficient backend infrastructure — not just models, but intelligent systems.

Scalability

Systems that grow with demand. Every pipeline I build is designed to handle 10× the expected load from day one.

Reliability

99.9% uptime is the baseline. I engineer fault-tolerant architectures with graceful degradation and self-healing capabilities.

Observability

You can't improve what you can't measure. Every system ships with metrics, logs, and alerting baked in from the start.

ML EngineeringData EngineeringMLOpsCloud ArchitectureML SystemsDistributed Systems
Open to Opportunities

Let's Build Intelligent
Systems Together

Interested in AI engineering, data infrastructure, distributed systems, or ML platform architecture? I'm always open to discussing innovative projects and engineering challenges.

Usually responds within 24 hours