Portfolio · Vol. 02Retrieval · Evals · Agents

Pulkit Singh.

MSc Computer Science, Southampton
B.E. Computer Engineering, NSIT, Delhi.
Tinkerer supreme.

01About

Engineer with a
product mindset.

Pulkit Singh

Computer Science grad from the University of Southampton (MSc), with a background in distributed systems and product management. I'm now focused on AI engineering: building retrieval, evaluation, and agent systems end to end.

Previously Associate PM at Original4Sure, where I led product analytics initiatives and an LMS platform serving 40,000+ users.

I write about what I'm learning on Substack.

Tech Stack
PythonPyTorchLangGraphRAGLLM EvaluationEmbedding Fine-tuningVector SearchChromaDBsentence-transformersHugging FaceOpenRouterLiteLLMFastAPIOpenTelemetryPostgreSQLNeo4jRedisBigQueryDockerSQLTypeScriptGit
02Selected Work

Things I've built.

Live on
Fly.io
Filing-Change Intelligence

Delta

A filing-change intelligence engine: type a ticker, get a five-year report on what actually changed across a company's 10-Ks, every claim traceable to a deterministic diff.

Delta fetches a company's last five 10-Ks, aligns them section-by-section through a stable anchor scheme, and computes a deterministic paragraph-level diff: cosine-similarity buckets classify each change, numeric deltas are pulled from XBRL structured data, and only the changed passages ever reach the LLM. The governing principle is that the model never finds the diff, it only explains it: every interpretation links back to a diff record with verbatim side-by-side quotes, which keeps the system cheap and structurally hard to hallucinate. Materiality is triaged (boilerplate → notable → material) so real changes surface above churn, and churn scores per section headline each report. Thresholds were tuned on a hand-labeled 50-pair sample; it's built on the v1 RAG eval harness, an 8-config chunking × embedding × rerank matrix where section-aware chunking beat fixed-size 28/56 vs 9/56, retained as a regression check. Server-rendered with FastAPI + Jinja2, deployed on Fly.io.

Live Report · NVDAWhat actually changed across five years of NVIDIA 10-Ks. Read the flagship report.Read →
Diff any ticker @ DeltaGitHub
Shipped ·
GitHub
LLM Gateway

Relay

A self-hosted LLM gateway that routes each request to the cheapest capable model, and proves the savings with a real benchmark, not a marketing number.

Sits between any OpenAI-SDK-compatible app and OpenRouter with a rules-based tiered router (cheap / smart / baseline), full request logging (tokens, cost, latency), and honest naive-vs-judge-calibrated savings accounting. The benchmark pipeline samples real traffic, runs routed vs baseline dual-path, scores with an LLM judge gated on Cohen's κ ≥ 0.7, then calibrates the savings formula from the judged results. Dogfooding: 412 requests, $4.80 saved, 62% routed to the cheap tier, −1.1% judged quality delta. 201 tests, 91% coverage, zero external infra required.

Visit Relay ↗GitHub
$4.80 saved
Dogfooding, 412 requests
62%
Routed to the cheap tier
91% coverage
201 tests
Shipped ·
GitHub
Agentic RAG

Blockchain RAG

A zero-cost, always-on Ethereum QA chatbot powered by a multi-tool ReAct agent.

The agent picks between five tools per question: vector retrieval over 35 Solidity/DeFi docs (Chroma, fine-tuned BGE embeddings), NL-to-SQL over 74K real mainnet transactions, NL-to-Cypher over a Neo4j graph of addresses and transfers, a sandboxed Slither static-analysis tool (Docker, network-isolated), and a live BigQuery fallback for on-chain data outside the local dataset. Fine-tuning the embedding model on synthetic QA pairs added +0.041 R@1 on a 74-query golden set (0.757 → 0.797). Runs entirely on free-tier LLMs via a LiteLLM gateway at $0/month, with guardrails on both input (rate limit, safety) and output (groundedness judge).

GitHub
0.797 R@1
+0.041 after fine-tuning
5 tools
Chosen per question by the agent
$0/mo
Free-tier LLMs only
03Experience

Where I've worked.

Associate Product Manager
Jul 2022 – Dec 2023
Original4Sure · Delhi, India

Led product analytics infrastructure across multiple services: designed the event tracking schema and implemented ETL pipelines processing 100,000+ daily events. Shipped a reporting API adopted by 3 enterprise clients (50K+ monthly reports), and led the technical architecture of an LMS MVP serving 40,000+ users with event-sourced progress tracking and automated certificate generation.

04Writing

Technical deep-dives.

05Education

Academic background.

2024 –
2025
University of Southampton

M.Sc. Computer Science

Merit.
Subjects studied: Deep Learning, NLP, Computational Finance, Cryptography.

Thesis: Towards Computationally Efficient Medical AI. Benchmarked CNN, transformer, and hybrid architectures for medical imaging on diagnostic accuracy vs compute, with a deployment-readiness score for models that run on real hospital hardware.

Read the thesis ↗
2018 –
2022
Netaji Subhas University of Technology

B.E. Computer Engineering

GPA 7.54.
Secretary, Finance & Economics Society of NSUT.
06Bookshelf

Books on repeat.

Live · BookmarkD embedTop 10
I built thisBookmarkD is my book-tracking app — rate what you read, build a list, then embed it on your own site.Build your list ↗How to embed
07Contact

Let's build something.