API Reference

The core of the library is the ScrapeClient class, which handles the initiation of scraping tasks and verification of webhook callbacks.

Constructor

new ScrapeClient(apiKey: string)

Creates a new instance of the ScrapeClient.

import { ScrapeClient } from "scrap-ai";

const client = new ScrapeClient("your-api-key");

Methods

scrape(url: string, prompt: string, callbackUrl: string): Promise<void>

Initiates a scraping operation. When the operation is complete, the results are sent via a POST request to the specified callbackUrl.

    await client.scrape(

    "https://example.com",
    "Extract all product titles and prices",
    "https://your-api.com/webhook"
    )
    

When your webhook endpoint receives a callback, the payload will follow this structure:

interface CallbackResponse {

status: "success" | "error";
data?: {
url: string;
results: any[];
timestamp: string;
};
error?: {
message: string;
code: string;
};
}

To ensure the authenticity and integrity of incoming webhook requests, the library provides a method to verify the webhook signature.

verifyWebhook(options: VerifyWebhookOptions): boolean

Validates a webhook request by comparing the provided signature against a generated HMAC.

const isValid = client.verifyWebhook({

body: JSON.stringify(req.body),
signature: req.headers["x-webhook-signature"],
timestamp: req.headers["x-webhook-timestamp"],
maxAge: 300000, // 5 minutes in milliseconds
});

if (!isValid) {
return res.status(401).json({ error: "Invalid webhook" });
}