Firecrawl

Point Penumbra at a URL and it crawls the site with Firecrawl and proposes a shape from what it finds.

Firecrawl turns a website into a starting domain model. Give Penumbra a URL, and it crawls the site (via Firecrawl) and proposes a shape from the content you can refine and materialize. It is the fastest way to bootstrap an ontology from a domain that already lives on the web.

Firecrawl is one example of a broader class. Anything that does structured output, structured input, memory capture, or document interpretation can consume a Penumbra shape — the Workbench’s shape_workbench_export_schema exports a shape as a provider-ready schema for Firecrawl, OpenAI, Anthropic, or Gemini. The crawl below runs through Penumbra’s own Firecrawl integration, so you do not need a Firecrawl key for it; to use Firecrawl in your own pipelines, get one at firecrawl.dev.

Crawl a site into a proposed shape

curl -X POST https://pnbr.io/v1/shapes/extract \
  -H "Authorization: Bearer $PENUMBRA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "source": { "type": "web", "url": "https://example.com" },
    "name": "Example domain model",
    "extractionPrompt": "Focus on products, pricing, and policies."
  }'

Field	Description
source.type	"web" to crawl a URL, or "file" to read a document.
source.url	The site to crawl (web source). Penumbra crawls up to 100 pages.
name	An optional name for the proposed shape.
extractionPrompt	Optional guidance for what to focus on while extracting.

The response is a shape proposal: a draft set of types, properties, and relationships derived from the crawled content. Nothing is committed yet.

From a file instead

The same endpoint accepts a document instead of a URL:

{ "source": { "type": "file", "content": "<your document text>", "format": "markdown" } }

Refine and materialize

A proposal is a starting point, not a finished shape. Open it in the Shapes Workbench to adjust types, properties, and relationships, compile and preview it, then materialize it so the runtime can use it.

This endpoint authenticates with your API key and consumes credits (a crawl spans many pages). It returns a draft shape; it does not write to your graph.