> ## Documentation Index
> Fetch the complete documentation index at: https://api.livechatai.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Crawl websites to discover internal links

> **DEPRECATED:** This endpoint will be deprecated on August 1st, 2025. Please use the v1 API endpoints instead. Crawl websites to discover internal links



## OpenAPI

````yaml post /api/aibot/scrape
openapi: 3.0.1
info:
  title: LiveChatAi API Swagger
  description: >-
    Welcome to the developer hub for LiveChatAi. You'll find comprehensive
    guides and documentation to help you start working with LiveChatAi as
    quickly as possible. **NOTICE: All endpoints in this version are deprecated
    as of August 1st, 2025. Please migrate to v1 API endpoints.**
  version: '1.0'
servers:
  - url: https://app.livechatai.com/
security:
  - bearerAuth: []
paths:
  /api/aibot/scrape:
    post:
      tags:
        - Chatbot
      summary: Crawl websites to discover internal links
      description: >-
        **DEPRECATED:** This endpoint will be deprecated on August 1st, 2025.
        Please use the v1 API endpoints instead. Crawl websites to discover
        internal links
      requestBody:
        required: true
        description: Scraping configuration
        content:
          application/json:
            schema:
              type: object
              properties:
                url:
                  type: string
                  example: https://.....
                  description: >-
                    The URL of the website to crawl and fetch internal links
                    from
                sitemapUrl:
                  type: string
                  example: https://.....sitemap.xml
                  description: The sitemap URL to crawl and fetch internal links from
                streaming:
                  type: boolean
                  example: false
                  description: Enable streaming of the scraping results
                limit:
                  type: number
                  example: 100
                  description: Maximum number of links to scrape
      responses:
        '200':
          description: Successfully scraped website and retrieved internal links
        '500':
          description: Internal server error occurred during scraping
      deprecated: true
components:
  securitySchemes:
    bearerAuth:
      type: http
      scheme: bearer
      bearerFormat: JWT

````