BiotechAlfa

The problem

Biotech companies publish their drug pipelines on their own websites, and no two pages look alike. One draws the stages as coloured bars, another is a filterable card grid, a third lists one row per trial. Comparing programs across companies, such as who is in Phase 2 against a given target, means reading every page by hand.

Earnings calls and investor events have the same problem. The replays sit on a handful of webcast platforms, usually behind a registration form, and most of them disappear after 30 to 90 days.

BiotechAlfa is one of my earliest projects. The first version had the same goal as this one, every company's pipeline in one place, but no event transcripts. I got the data with hardcoded CSS selectors and some reverse engineering of each site. The approach was close to what the pipeline does today, but back then every site needed its own hand-maintained scraper, and keeping them working was a pain. Eventually I stopped.

Fast forward to now, and AI is cheap, accessible and powerful. I restarted the project with more features. With a model doing the parsing, it's much easier to build a data pipeline that scales, and to pull data out of SEC filings, PDFs and images as well as web pages.

What I built

A research pipeline that runs on a schedule, and a website that publishes what it collects.

• Universe. A weekly screener picks the companies (US-listed healthcare, over $1B market cap), and a discovery step finds each one's investor events page and pipeline page.
• Pipelines. Each pipeline page becomes one row per program and indication, with stage, target, modality and partner on standard scales. Every row is tied to the saved copy of the page it was read from.
• Events. A daily watcher follows each company's events page. When a replay appears, the extractor for that webcast platform downloads the audio, and the call is transcribed with each speaker named from what's said on the call.
• Other sources. Pullers for ClinicalTrials.gov studies, SEC filings, and product revenue from the filings' XBRL data.
• The site. Next.js on Vercel, reading a Supabase copy of the data. Companies, programs, targets and events, with the date and source of every figure, and a reminder that a pipeline page is the company's own statement, not an independent source.
• Knowledge graph (in progress). A knowledge graph built from all of the above, so the site can give investors more meaningful insight.

Decisions worth explaining

Daily runs are plain code, and the agent writes the code

The daily work is Python on cron: watching events pages, downloading replays, transcribing them. There's no agent framework, and no model decides what happens to an event. The model calls that remain are transcription, naming speakers, and reading the odd events page the rules can't parse.

The agent comes in when the code runs out. When a replay is on a webcast platform no extractor knows, I run a Claude Code session whose job is to write a new extractor, and the download it was stuck on is the test. Investor webcasts sit on a small number of vendors, so each of those sessions covers every later event on that platform. Coverage only goes up, and the expensive part happens once per platform rather than once per event.

The agent reads the page, a script decides what goes in

Pipeline pages differ too much for one parser, so an agent with a skill reads each page and writes a result file. When a stage is only drawn as a bar, it reads a screenshot rather than guessing from the text.

The agent never writes to the database. An importer does, and it rejects any row whose program name (or a known alias) isn't in the visible text of the saved page, any value outside the standard scales, and any duplicate. A company the agent couldn't read cleanly ends up marked "needs review" with a note, instead of with a guess. The model handles the part that changes from company to company, and plain code decides whether a row is believable.