Daksha Ladia
About
I build production-grade AI systems: LLM + retrieval applications, evaluation pipelines, backend algorithms and scalable data infrastructure. I hold an MS in CS from UMass Amherst and previously worked as a Software Engineer at Microsoft (Bing Ads), where I built ranking and simulation systems operating on large-scale traffic and data.
I’m strongest at the intersection of context engineering (retrieval, chunking, query synthesis, tool use), LLM system reliability (evaluation, regression, guardrails), and backend engineering (APIs, databases, deployment). I care about AI security and building systems that behave well under real-world abuse and edge cases.
- Retrieval quality: chunking strategies, query synthesis/expansion, hybrid retrieval, reranking, and grounded answers
- Reliability: eval sets, regression testing across prompts/models, and automated comparisons
- Tool use: structured outputs, guardrails, and feedback loops (e.g., execution-based validation for NL→SQL)
- Security: prompt injection awareness, data leakage prevention, and robust failure modes
Work Experience
Research Extern | Allen Institute for AI (AI2) | Jan 2026 - May 2026
- Researched semantic pre-training strategies to improve downstream generalization in language models
- Explored structured knowledge integration and contrastive learning for stronger language representations
-
Machine Learning Engineer Intern | System1 | May 2025 - Aug 2025
- Built a production-grade LLM system for natural language to SQL over business data (RAG + retrieval + feedback loops), achieving 98% execution success on real queries
- Designed evaluation pipelines to measure correctness and regression across the model (AWS Bedrock) and prompt changes
- Containerized the system with Docker, enabling reproducible experiments and rapid iteration
-
Contractor: Automatation of Healthcare Document Processing Platform| Freelance | June 2024 - Aug 2024
Built and sold enterprise healthcare document processing platform to medical insurance company. Architected end-to-end AI pipeline using GPT-4o and Vision API for document classification and OCR, reducing processing time by 99% and enabling thousands of daily document processing with high accuracy.
-
Software Engineer | Microsoft, Bing Ads | June 2020 - July 2024
- Optimized a large-scale ad simulation system to replay online auction traffic offline, enabling evaluation of ranking changes and ad performance across millions of users and 2TB daily production data pipelines
- Reduced compute cost by 9% and improved simulation efficiency by 15% through stratified sampling strategies, function-level optimizations, and evaluation restructuring for high-throughput data
- Designed and deployed a regression testing framework that automated parallel evaluation across feature-branches, replacing manual workflows; reduced manual effort by ~90% through standardized evaluation pipelines and automated comparisons
- Led A/B experimentation and rollout of ranking features tuning on production traffic, improving advertiser conversions by 4% through data-driven optimization and iterative model improvements
-
Data Science Intern | FarmGuide (Acquired by DeHaat) | Jan 2020 - May 2020
Developed ML models for crop identification and health monitoring with 96% accuracy. Built forecasting algorithms using ARIMA and Prophet for environmental insights and crop recommendations.
-
Software Engineering Intern | Microsoft, Bing Ads | May 2019 - July 2019
Analyzed seasonality patterns in advertiser campaigns using time-series modeling. Optimized bid pricing algorithms for peak holiday periods.
Selected Projects
-
Sweep — Generative Image Model Comparator
- Problem: comparing generative image models across prompts and parameters requires running each variation manually
- Approach: parameter sweep tool — mark any input field as swept, run variations in parallel, compare results in a labeled grid; Claude Sonnet expands prompts into stylistic variations along a named axis
- Stack: Python • FastAPI • HTMX • Claude Sonnet • Replicate API • SQLite • Docker • Fly.io
- Result: deployed app supporting 6 image models across 5 vendors with dynamic schema-driven forms
-
Semantic Pre-training Strategies for Language Models | Allen Institute Research Extern
- Problem: improve downstream generalization via better pre-training signals
- Approach: semantic-driven pre-training and structured knowledge integration
- Stack: Python • contrastive learning • representation learning
- Outcome: research strategies + experiments toward stronger language understanding
-
KYC Identity Verification System
- Problem: manual KYC verification was slow and non-scalable
- Approach: end-to-end pipeline (classification → vision extraction → validation → audit logging)
- Stack: Python • OCR • Fireworks AI • AWS
- Result: >95% field accuracy with a complete audit trail for compliance
-
Multi-Agent Browser Automation Platform
- Problem: web automation breaks under UI drift and hidden state
- Approach: vision + DOM signals, structured reasoning loops, and state-aware decision making
- Stack: Python • Gemini/vision LLMs • browser automation
- Result: improved task completion reliability in a controlled environment + automated dataset generation
-
Recomm AI - Privacy-Preserving Recommendation System
- Problem: build personalization without leaking user data
- Approach: differential privacy (gradient clipping + noise) + federated learning
- Stack: Python • privacy ML • evaluation
- Result: validated robustness using membership inference + adversarial testing
-
On-Device Privacy-First AI Companion
- Problem: deliver a helpful companion without sending sensitive data to servers
- Approach: on-device multimodal assistant + federated personalization
- Stack: on-device inference • voice + text
- Result: personalized adaptation to tone/mood without data leaving the device
-
Healthcare Document Processing Automation
- Problem: manual claim document processing was slow and costly
- Approach: OCR + classification + unstructured-to-structured extraction with validation
- Stack: GPT-4o • Google Vision API • Python
- Result: 99% time reduction in document processing
-
Enhanced Information Retrieval via Query Synthesis
- Problem: ambiguous queries reduce retrieval quality
- Approach: segmentation + pseudo-query generation + embedding search
- Stack: FAISS • vector search • NLP
- Result: 4% precision improvement over BM25 baseline
-
Question Answering Bot
- Problem: users needed fast, grounded answers from internal documents
- Approach: ingestion + retrieval + context-grounded generation
- Stack: LangChain • OpenAI API • PDFs/JSON
- Result: automated QA pair generation + context-aware responses
-
LLM Bias Analysis Research
- Problem: quantify and compare occupational/pronoun bias across LLMs
- Approach: statistical testing + controlled prompts + comparative analysis
- Stack: evaluation • statistics
- Result: published analysis + mitigation strategies (Submitted to COLM'25)
-
Real-Time GitHub Trending Prediction
- Problem: detect emerging repositories early from high-volume event streams
- Approach: streaming ingestion + feature generation + neural prediction
- Stack: Apache Kafka • Python
- Result: pipeline processing thousands of events/sec with trend forecasts
-
AI SlackBot
- Problem: teams lose context in long-running Slack threads
- Approach: summarization + retrieval over channel history + Q&A
- Stack: LLMs • Slack automation
- Result: faster onboarding + higher team throughput via searchable context
Publications
-
Virtual Community: An Open World for Humans, Robots, and Society
Multi-agent embodied AI simulation platform built with Unity and Python. Generated benchmarks for speed and resolution across robots, agents, and scenes. Accepted at ICLR 2026
-
Do Different Models Differ in their Bias Towards Occupational Preferences?
Comprehensive analysis of pronoun and occupational biases in LLMs using statistical tests. GPT-4o: <5% non-preferred pronoun selection; Qwen2.5: 100% positional bias. Submitted to COLM'25
Recognition
-
Innovation & Excellence Award
Microsoft (FY20-21) — Pipeline migration project improving operational efficiency
-
Judge & Mentor
MIT Hacks 2025 (Judge) • HackHarvard 2024 (Mentor) • She Hacks DTU 2021 (Mentor)
-
Reviewer
Grace Hopper Celebration 2025 — Application review across multiple verticals
-
MIT Policy Hackathon 2024
Proposed interpretable solution for SpO2 measurement error analysis
-
Top 1% National Exams
Inspire Scholarship from Government of India (2015)