Bhagyesh Rathi
SOFTWAREDEVELOPER& AI/MLENGINEER
About Me
I am a Software Developer and AI/ML Engineer with a passion for building intelligent, scalable systems. Currently pursuing my Master's in Artificial Intelligence at San Jose State University, my focus lies at the intersection of robust backend engineering and cutting-edge machine learning.
At Rakuten, I engineered high-impact microservices, implemented secure OAuth 2.0 architectures, and orchestrated GCP deployments with Kubernetes. Whether it's developing interactive RAG pipelines, optimizing distributed systems, or training predictive models, I thrive on turning complex technical challenges into seamless user experiences.
Most recently, as a Forward Deployed Engineer Intern at Scalar Field (YC P25), I built a HubSpot MCP (Model Context Protocol) server that exposes CRM data and actions as tools for LLM agents, along with a RAG chatbot for GTM queries over internal data. I also contribute to GitLab's open-source monorepo, where my merged work includes a security fix hardening the import pipeline against DoS and a new REST endpoint in the Package Registry.
On the research side, I am co-first author of "A Comparative Evaluation of Retrieval Pipelines for Large-Scale Scientific Question Answering with Open-Weight LLMs," in press at IEEE AIxSET 2026, and I am currently exploring agent safety.
When I'm not writing code, you can find me exploring the latest advancements in LLMs or refining my problem-solving skills.
Experience
Forward Deployed Engineer Intern
Scalar Field (YC P25) · Seattle, WA
- Built a HubSpot MCP (Model Context Protocol) server exposing CRM data and actions as tools for LLM agents, enabling automated GTM workflows over the company's customer data
- Developed a RAG chatbot that retrieves lead information from a knowledge base to answer GTM queries, handling retrieval and structured responses over internal data
Open Source Contributor
GitLab · Remote
- Contributed a security fix to the GitLab monorepo enforcing streaming JSON validation limits in the import pipeline's NdjsonReader to mitigate DoS from malicious export archives (MR !224828)
- Added a REST DELETE endpoint to GitLab's Package Registry with authorization policies, request specs, and API docs, aligning behavior with existing upload/download endpoints (MR !242461)
Software Engineer Intern
Rakuten · San Mateo, CA
- Engineered Kotlin microservices for Social Authentication (Google, Apple, Facebook), driving a 40% adoption rate and a 37% increase in conversion rate.
- Developed robust REST APIs utilizing OpenID Connect and OAuth 2.0 to implement secure token-based authentication and authorization flows
- Architected scalable infrastructure on Google Cloud Platform (GCP), orchestrating Docker containers with Kubernetes via automated GitLab CI/CD pipelines
- Established comprehensive system observability by integrating OpenTelemetry with GCP Cloud Trace and structured Log4j logging for real-time performance insights
Software Engineer Co-op
Rakuten · San Mateo, CA
- Architected and deployed a core internal SDK that abstracted complex service-to-service communications, successfully published to Artifactory via GitLab CI/CD
- Engineered secure event listeners using Kotlin to intercept and process critical compliance signals (consent revocation, account deletion) from OAuth providers
- Implemented robust security protocols by validating and decoding JSON Web Tokens (JWT) to securely authenticate inter-service requests
Research & Publications
A Comparative Evaluation of Retrieval Pipelines for Large-Scale Scientific Question Answering with Open-Weight LLMs
Rathi, B.*, Chawla, E.*, Ershov, A., Andreopoulos, W. B. (* Equal contribution)
IEEE AIxSET 2026 (in press)
A reproducible, config-driven framework comparing six retrieval strategies for scientific question answering over a ~460K-paper arXiv corpus, evaluated with open-weight LLMs.
- Built a reproducible, config-driven pipeline comparing six retrieval strategies for scientific QA over a ~460K-paper arXiv corpus, taking a research prototype to a modular, release-ready codebase
- Implemented SPECTER2 domain embeddings with task-specific document/query adapters, a ChromaDB vector store, and an agentic tool-calling retriever via function-calling APIs
- Deployed and optimized the full stack on an NVIDIA DGX Spark (GB10, ARM64/CUDA 13), including getting ColBERT's PLAID late-interaction index compiling on a novel architecture and serving open-weight LLMs (Llama-3.1, Qwen2.5) locally via Ollama
- Designed an LLM-as-a-judge evaluation with an explicit answer/refusal gate and both conditional and unconditional scoring to fairly compare strategies with differing answer rates
- Parallelized generation and evaluation stages for a ~6× throughput improvement across ~20K queries
PathSafe Agent: Does an Agent's Safety Depend on Where a Harmful Instruction Comes From? (In Progress)
Ongoing research studying whether an LLM agent's safety behavior changes depending on the source of a harmful instruction — direct user input versus content encountered through tools, documents, or other agents.
Projects
Shipped a production-grade RAG chatbot serving real users on my live portfolio site, with a full ingestion pipeline and a streaming, guardrailed API
- Ingestion: Parsed and chunked the CV PDF into semantic segments (PDF → recursive chunking → 768-dim embeddings → Pinecone upsert)
- Embedding: Embedded text chunks using a transformer model to generate dense vector representations
- Storage & Retrieval: Stored vectors in a Pinecone vector database with top-K retrieval, score thresholding, and metadata filtering to minimize hallucination
- Generation: Built a streaming API endpoint using Vercel AI SDK with Gemini 2.5 Flash for token-by-token delivery, achieving sub-second TTFT
- Security: Hardened the system against prompt injection through input validation, system-prompt design, and topic-scoped guardrails
- Testing: Authored a 27-test integration suite covering embedding quality, retrieval accuracy, pipeline correctness, and injection defense
Built a ReAct tool-use agent that triages production incidents across 5 data sources (RAG over 768 runbooks, text-to-SQL, GCP status API vulnerability DB, web search) using dual Qwen 2.5 models (32B + 3B) on a single GPU via 4-bit quantization.
- Evaluated 4 prompting techniques across 80 benchmarks, finding self-reflection improved answer grounding by 125% (0.40→0.90) and actionability by 100% (0.50→1.0) over baseline, while prompt chaining achieved perfect tool selection on the 32B model but failed on 3B — revealing minimum capability thresholds for structured data
- Achieved ~4% prefill speedup on the 32B model by building KV cache reuse for system prompt acceleration.
- Implemented defense-in-depth security (SQL injection blocking, prompt hardening) passing 3/5 red-team attacks per model; deployed via Gradio with real-time model/technique comparison
End-to-end encoder-decoder image captioning models with attention mechanisms and interactive visualizations.
- Designed and trained three end-to-end encoder-decoder image captioning models (Show-and-Tell baseline, Show-Attend-and-Tell with soft attention, and Visual Sentinel adaptive attention) on MS COCO 2014, leveraging a fine-tuned ResNet-101 CNN encoder feeding a 7×7×2048 spatial feature grid into an LSTM language decoder
- Implemented Bahdanau soft attention with doubly stochastic regularization, producing dynamic per-word context vectors that boosted CIDEr from 1.023 → 1.040 and won all 8 evaluation metrics over the baseline; BLEU-4 of 0.329 materially exceeds the original 2015 paper (0.250)
- Engineered beam search decoding with a 1–10 beam-width ablation that lifted CIDEr from 1.013 (greedy) to 1.093 (beam=6), identifying optimal decoding configuration and quantifying the degradation from overly wide beams
- Built interpretable attention heatmap visualizations that trace each generated word back to the image region driving the prediction — turning a black-box captioner into a debuggable, explainable system
- Optimized training throughput by pre-extracting CNN features to HDF5, achieving full convergence in 30 epochs / ~3 hrs per model on an RTX 6000 with Adam + ReduceLROnPlateau scheduling
- Conducted CIDEr-vs-SPICE discrepancy analysis to expose blind spots in n-gram-only evaluation, demonstrating cases where models score well without semantic understanding
- Deployed as an interactive multi-model demo on Hugging Face Spaces via Docker + Gradio, allowing real-time image upload, model switching, and tunable beam width with live attention heatmaps
AutoML
FullStack AutoML platform and code generator
- Engineered an AutoML system using Scikit-Learn & Flask that automates preprocessing, task detection, and parallel model training/evaluation
- Designed a responsive React frontend with real-time interactive ROC/Scatter plots and confusion matrix for visualizations of top model
- Reduced training latency by 40% implementing Stratified Sampling and dynamic model switching to handle large datasets efficiently
- Developed a context manager to profile real-time CPU/RAM usage & a leaderboard to sort models by accuracy, time, or resource efficiency
- Built a transpiler engine to enable one-click downloads for both serialized models (.pkl) and their reproduction code
Bank Churn Data Analysis and Prediction using ML
Data analysis and prediction using ML
- Analyzed data of 10,000 account holders at a Multinational Bank by doing exploratory data analysis with Pandas
- Constructed a streamlined pipeline for training 5 machine learning models to predict customer churn using Scikit-Learn
- Implemented XGBoost, Random Forest, KNN, SVM, and Naive Bayes models and compared them
- Utilized N-fold cross-validation, F1-score, confusion matrix to evaluate the performance of each model
Skin Cancer Detection using CNN
CNN-based image classification for skin cancer detection
- Developed a CNN using Scikit-Learn to classify skin lesion images into 7 cancer categories, achieving 80% accuracy
- Preprocessed data with resizing, normalization, one-hot encoding, and oversampling to address class imbalance
- Optimized model performance using the Adam optimizer, learning rate annealing, and hyperparameter tuning
Skills & Technologies
Education
Masters of Science in Artificial Intelligence
Expected: May 2027San Jose State University · San Jose, CA
Relevant Coursework:
Bachelors of Science in Computer Science
August 2020 — May 2024San Jose State University · San Jose, CA
Relevant Coursework:
Get in Touch
I'm always open to discussing new opportunities, collaborations, or interesting projects.