Scrape with AI
Write a sentence like "the product name, the price and whether it is in stock" and get exactly those details back, taken from what the page actually shows. Nothing to rewrite when the site changes its layout. You pay per use, with no subscription.
Run it online
Run this on our servers with your account. Free tools run in your browser; this one bills your KIT balance per the price above.
Instead of inspecting HTML and writing CSS selectors, you write a sentence — "the product name, price and whether it's in stock" — and a language model reads the rendered page to fill it in. It exists for the pages that break every selector-based scraper: inconsistent markup, A/B-tested layouts, or sites you've never seen before.
Describe the data instead of pointing at it
A classic scraper needs someone to open the page code, find where each value lives and write a selector for it. When the site ships a redesign or renames a class, the job quietly starts returning blanks. Here you write a sentence instead, such as the product name, the price and whether it is in stock, and a language model reads the finished page to fill it in. Until a few years ago models could not follow that kind of instruction reliably, so the only options were fragile selectors or hand-written patterns for every new site.
When describing beats selecting
It pays off when you try a source for the first time and want to know whether its data is worth the effort before mapping anything. It also suits listing pages where products are laid out slightly differently from one another, or where the site tests several layouts at once. Researchers use it to pull comparable details, such as headline, author and publish date, from dozens of unrelated news sites without writing rules for each one. If a page lacks something you asked for, that detail comes back empty rather than invented, and a failed attempt is retried automatically up to three times before you see an error.
Why natural-language extraction is now practical
Large language models only became reliable enough to follow structured-output instructions over ordinary text in the last few years; before that, the only options were brittle selectors or hand-written regular expressions for every new domain. Describing intent in a sentence — rather than encoding it as a CSS path — means the same request can survive a site redesign that would have silently broken a selector map.
What you can do with it
Prototyping a new scraping target
Point it at a site you've never scraped before and describe the fields you need, skipping the dev-tools work of mapping selectors before you even know if the source is worth it.
Sites with inconsistent or A/B-tested markup
Extract the same fields reliably from listing pages where different products render with slightly different HTML structures.
Long-tail research across many different domains
Pull comparable fields — author, publish date, headline — from dozens of unrelated news sites without writing a selector map for each one.
FAQ
How is AI web scraping different from selector-based scraping?
You describe the fields you want in plain language instead of writing CSS selectors, which makes it more resilient to layout changes at a somewhat higher per-URL cost.
Can it scrape any website?
It works on any publicly reachable page whose content renders in the response, including JavaScript-heavy pages, though sites that require login are out of scope.
Will it invent data that isn't on the page?
No — the model is grounded to the page's actual content and returns null for fields it genuinely can't find rather than guessing.
How do I write a good extraction description?
Name each field plainly and note its expected form when it matters, for example "price as a number without currency symbol" — specificity in your description improves match quality.
What happens if the page does not contain a detail I asked for?
That detail comes back empty instead of a guess. The model is told to use only what is actually on the page, so a missing price or author shows up as a gap you can see, never as an invented value.
Will my extraction keep working after the site is redesigned?
Usually, yes. Because you describe what you want in words rather than pointing at a spot in the page code, the same description can survive a redesign that would silently break a selector-based scraper.
Related tools
For developers — API access
Everything on this page is available programmatically. This section is for teams who want to wire it into their own systems; everyone else can just use the tool above.
API endpoint
Prefer to automate it? One authenticated POST creates the task; the result comes back by webhook or a signed link. The same capability also runs here on the web, by email and from Telegram — and soon from our app too.
Call it from your stack
curl -X POST https://api.kit.forhosting.com/web/scrape-ai \
-H "Authorization: Bearer $KIT_KEY" \
-H "Content-Type: application/json" \
-d '{"url":"https://example.com"}'const res = await fetch("https://api.kit.forhosting.com/web/scrape-ai", {
method: "POST",
headers: {
"Authorization": `Bearer ${process.env.KIT_KEY}`,
"Content-Type": "application/json"
},
body: JSON.stringify({
"url": "https://example.com"
})
});
const { task_id } = await res.json();import os, requests
res = requests.post(
"https://api.kit.forhosting.com/web/scrape-ai",
headers={"Authorization": f"Bearer {os.environ['KIT_KEY']}"},
json={
"url": "https://example.com"
},
)
task_id = res.json()["task_id"]<?php
$res = file_get_contents("https://api.kit.forhosting.com/web/scrape-ai", false, stream_context_create([
"http" => [
"method" => "POST",
"header" => "Authorization: Bearer " . getenv("KIT_KEY") . "\r\nContent-Type: application/json",
"content" => '{"url":"https://example.com"}',
],
]));
$task = json_decode($res, true);body := bytes.NewBufferString(`{"url":"https://example.com"}`)
req, _ := http.NewRequest("POST", "https://api.kit.forhosting.com/web/scrape-ai", body)
req.Header.Set("Authorization", "Bearer "+os.Getenv("KIT_KEY"))
req.Header.Set("Content-Type", "application/json")
res, _ := http.DefaultClient.Do(req)Example request
{
"url": "https://example.com"
}Example response
{
"task_id": "tsk_a1b2c3d4e5f6a1b2c3d4e5f6",
"type": "web.scrape_ai",
"status": "queued",
"_links": {
"result": "/tasks/tsk_…/result"
}
}The API is asynchronous: the call returns a task_id immediately and the result arrives by webhook. Polling is capped at 1 req/s per task.
Pricing
Published price — no tokens, no invented credits. A failed task is never charged.
Limits
timeout_sec | 30 |
max_crawl_pages | 25 |
Errors
| HTTP | Code | Meaning |
|---|---|---|
401 | unauthorized | Missing or invalid API key. |
402 | insufficient_balance | Your balance doesn't cover the task price. |
404 | unknown_type | That task type doesn't exist. |
429 | rate_limited | Too many requests. Use the webhook instead of polling. |
422 | task_failed | The task failed after 3 retries. You are never charged for it. |
Technical notes
The problem: layouts move, selectors break
Anyone who has run a selector-based scraper for more than a few months knows the routine — a site redesign ships, a class name changes, and a monitoring job silently starts returning nulls. This endpoint targets exactly that failure mode, along with the one-off case where you need data from a page today and don't want to spend twenty minutes in dev tools finding the right nesting to target.
How the extraction actually works
You call POST /web/scrape-ai with a URL and a plain-language description of the fields you want; the task_id comes back instantly while, behind it, the page is fetched, its rendered content is prepared for a language model, and that model is instructed to return only the fields you described, in JSON, grounded in what's actually present on the page rather than invented from general knowledge.
What to expect in the result and when things don't match
As with every task on the platform, a failed extraction retries automatically up to three times before returning a clear error, and you're only billed once a task completes successfully. The response is a JSON object keyed by the field names implied in your description; if the page genuinely lacks a field you asked for, it comes back null rather than a guessed value, because the model is grounded to the page content and instructed not to fabricate.
Where this fits alongside the other scraping endpoints
Teams typically reach for this endpoint first when prototyping a new data source, then migrate high-volume, stable pages to selector-based scraping once the layout is known and won't be worth re-describing every run. For cases where you need a strict, machine-checked shape — required fields, fixed types, enumerated values — the schema-scraping endpoint layers that guarantee on top of the same natural-language flexibility.
Integration use cases
Rescuing a broken selector job
Swap a monitoring pipeline over to the AI endpoint after a site redesign breaks its CSS selectors, buying time to decide whether to rebuild the selector map.
Technical questions
What does web.scrape_ai cost?
$0.046 per request plus $0.0145 per URL, charged only for tasks that complete successfully.
Is there a free trial for the AI scraping API?
There's no free tier — free tiers get abused and slow everyone down. Access runs on a prepaid ForHosting KIT balance: top up from $10.00 (it never expires) and each request is charged at its published price, so a call with no balance returns HTTP 402. No subscription, no tokens, no invented credits, and a failed task is never charged.
How do I receive the results?
Via a signed webhook when the task completes, or a signed link that stays valid for 24 hours if you'd rather fetch it yourself.
When should I use scrape_schema instead of scrape_ai?
Reach for scrape_schema when you need a strict, validated data shape with required fields and fixed types; use scrape_ai when a flexible plain-language description is enough.