About Yahia Bakour
Yahia Bakour built Context.dev after running into the same wall that most AI product teams hit eventually: the model knows everything up until the moment it stopped learning, and nothing after that. His background is in developer tooling, and when he looked at how teams were solving the live web access problem, what he saw was a lot of redundant infrastructure being built and rebuilt across hundreds of companies that all needed the same thing. Context.dev is his answer to that. A single API covering web scraping, site crawling, brand and company intelligence, and transaction identification. Y Combinator backed it. More than 5,000 developers use it.
Something nobody wants to admit
Yahia Bakour gets asked a version of the same question often. An engineering team has spent weeks, sometimes months, building a scraping stack. They have handled the authentication, the rate limiting, the HTML parsing, the JavaScript rendering. It works. Then they find Context.dev and realise they could have had all of that with an API key and an afternoon. The question they ask is some variation of: should we switch?
His answer is always yes, but that is not the interesting part. The interesting part is the conversation that follows, about what the team built instead during those weeks, or rather, what they did not build. Features sitting in a backlog. A launch that slipped. A competitor who shipped something while the infrastructure was still being wired together.
The reason Context.dev exists is not that web scraping is technically hard. A working scraper is an afternoon project. The reason it exists is that a working scraper in January is not the same thing as a working scraper in June, and the gap between those two things is a maintenance burden that most teams do not price accurately when they decide to build.
1. The scraper you built in January
Writing a scraper feels straightforward because the first version is. You pick a URL, pull the HTML, parse what you need, and it works. The problem is that websites are not static documents. They are maintained by teams who update them, redesign them, add anti-bot measures, migrate to new frameworks, and change their URL structures on timelines that have nothing to do with your engineering schedule.
Cloudflare gets tightened and your requests start bouncing. A site switches from server-rendered HTML to client-side JavaScript and your parser gets back an empty div. A platform you scraped freely in January adds authentication in April. Every one of these breaks something, and the fix takes time that was supposed to go somewhere else.
Bakour is specific about this. He has talked to enough engineering teams to know that the initial build cost is rarely the number that matters. What matters is the accumulated time over the following twelve months: the on-call incidents, the dependency upgrades that broke rendering, the Monday mornings spent debugging a pipeline that was fine on Friday. That number is almost always larger than the one that appeared in the original estimate.
The vendor sprawl problem
Teams that recognise they should not build their own infrastructure often end up in a different trap. They sign up for a scraping service. Then a separate one for company logos. A third for brand enrichment. Possibly a fourth for transaction identification if the product touches financial data.
Each of those vendors has its own authentication scheme, its own error format, its own rate limits, its own billing cycle. When something breaks at 11pm, the first fifteen minutes are spent figuring out which vendor is responsible before the actual debugging starts. The overhead of managing four providers eats up a meaningful fraction of the time that was supposed to be saved by not building the infrastructure yourself.
Context.dev covers all of it under one key. Web scraping, full-site crawling, AI-powered structured extraction, brand and company intelligence, logo delivery, font and color detection, screenshot capture, transaction identification. One authentication flow. One dashboard. One billing line.
2. What the live web actually changes
The knowledge cutoff problem is not subtle. A language model trained on data through a certain date will answer questions about things that happened after that date with the same confidence it brings to things it actually knows. That is the model doing exactly what it was designed to do. The problem is that plausible and accurate are not the same thing when the question involves something that changed recently.
The practical consequence is that any AI product relying solely on a model’s training data gets quietly worse every day as the world moves on. Pricing changes. Companies pivot. Products launch and die. The fix Bakour describes is architectural: the agent makes a live request for the page it needs, gets back clean markdown, and reasons from that. It is not guessing what the pricing page says. It is reading it.
Why markdown specifically
Raw HTML is not a good input for a language model. A typical webpage might contain tens of thousands of tokens worth of noise: navigation, cookie notices, tracking scripts, sidebars, footers, and the structural markup connecting all of it. The actual content is often a small fraction of the total.
Converting that to clean markdown strips the noise and keeps the content. Headers stay headers. Lists stay lists. The model gets something it can reason from at a fraction of the token count. At scale, across hundreds or thousands of pages, that difference compounds into real cost savings and faster inference times.
Onboarding without forms
When someone signs up for a B2B product with a work email, the domain tells you a lot. Company name, logo, brand colors, industry, social profiles. Pre-filling that data during onboarding does not just save the user time. It changes the emotional experience of starting. Seeing your own brand rendered correctly inside a product in the first thirty seconds creates a different first impression than staring at empty fields.
DocsBot saw this directly. Aaron Edwards, their founder, reported measurably higher activation rates for free trials after adding brand-powered onboarding. Users who reach their first moment of value faster are more likely to stick around long enough to become paying customers.
Indices that reflect this week, not last quarter
Most RAG pipelines are built on indices that were populated once and refresh rarely, which means the answers they produce drift from reality over time. A crawl-based refresh pipeline using Context.dev changes that. Sitemap discovery pulls all pages on a domain. Each page gets scraped to clean markdown. The chunks go into the index. The index reflects what the site says now, not what it said when someone ran the pipeline last quarter.
Making transaction data legible
Merchant descriptors in financial data are notoriously hard to read. SQ *BLUE BOTTLE COF, AMZN MKTP US*2K8F7G1, GITHUB.COM 2FA8X91. Resolve them to real brand names with logos and categories and that work disappears. Context.dev’s transaction identification endpoint does that mapping and returns a brand profile. The result is transaction data a product can display and categorise without making the user decode anything.
3. Who is using it
The customer list covers a wider range of use cases than the API’s name might suggest. A scraping API sounds like a tool for data extraction. What it actually is, Bakour argues, is a data infrastructure layer, and that turns out to be useful in a lot of places.
Mintlify
Turns any GitHub repository URL into a fully branded documentation site. Brand data handles logos, colors, and visual identity automatically, without any manual setup by the user.
Daily.dev
Uses the company enrichment endpoint to keep their internal database accurate. No maintenance team required.
DocsBot
Pulls brand data at signup to pre-fill onboarding. Activation rates on free trials went up measurably after the integration.
Sunrun
Personalises affiliate landing pages using brand context pulled from domain data.
ION Design
Replaced a homegrown scraping system they had been maintaining for years with a single API call.
Comp AI
Generates on-brand sales decks by pulling logo, color, and company data at runtime.
Architect
Added brand data to their onboarding flow in an afternoon. No dedicated infrastructure required.
4. What getting started actually looks like
Bakour is particular about this. Self-serve means self-serve. You put in a work email, you get an API key. No sales call, no demo request, no form asking about your use case and team size. The free tier comes with 1,000 API credits and 10,000 Logo Link requests, no credit card required.
The ten-minute figure that customers keep mentioning is not a marketing claim. It reflects what the integration actually involves. There is one endpoint for scraping. The request takes a URL. The response comes back as markdown. Nick Khami at Mintlify described it as getting an API key immediately, finding thorough documentation, and starting integration within ten minutes. Luke Ramsden at Architect said the same thing. When multiple customers independently land on the same number, it is a reasonable number to trust.
How the pricing works
Scraping endpoints cost one credit per call. Brand and company data endpoints cost ten. Logo Link, which serves logos via CDN with a single image tag and no backend call on your end, runs on a separate quota and does not draw from your API credits. Failed calls are not billed.
Paid tiers increase the monthly credit allocation and the rate limit ceiling. Free gives 30 calls per minute. Scale gives 1,200. Logo Link has no rate limit at any tier, which matters for high-volume use cases where logos appear in feeds or tables. Early-stage companies can apply for up to 30% off.
On data freshness
Scraping endpoints always return what the page says at the moment of the request. Brand data is cached and refreshed quarterly, with anything older than three months pulled fresh when requested. Logo URLs stay valid for up to a year. If you need a specific domain updated before the automatic cycle, you can request it manually. There is also a Prefetch endpoint for teams that want fresh data ready before users trigger it.
5. The honest version of build vs. buy
Bakour is not trying to convince anyone that building your own infrastructure is always wrong. There are situations where it makes sense. His argument is narrower than that.
Most teams that choose to build their own web data stack are not doing it because of unusual requirements. They are doing it because it does not seem that hard, and because buying feels like adding a dependency. Both of those things are true. What gets left out of that calculation is the ongoing cost of ownership.
Sites change. Anti-bot measures improve. Rendering requirements shift. The engineer who built the original scraper might not be there anymore. The team that inherits it has to maintain something they did not design, for use cases that have evolved since it was written. That accumulated friction is the thing that does not appear on the whiteboard when the decision is made.
The teams that move fastest, in Bakour’s observation, are the ones who made the fewest decisions about infrastructure. They bought the data layer, bought the auth layer, bought the payments layer, and spent their engineering time on the thing users actually care about. That is not laziness. It is prioritisation.
A note on compliance
Context.dev is SOC 2 Type I certified, with Type II in progress. For teams selling into enterprise accounts, that certification reduces friction in security reviews. Context.dev also supports programmatic agent registration, meaning an AI agent can sign up for API access on its own without a human completing the flow. For fully automated pipelines, that matters.
About Context.dev
Context.dev is a web scraping and crawl API for AI agents and the teams building them. Founded by Yahia Bakour and backed by Y Combinator, the platform gives developers live access to web content, brand intelligence, company data, and transaction identification through a single API key. Over 5,000 developers use it today. SOC 2 Type I certified.
Leave a Reply