Skip to content

Crawling overview

Resources can be crawled using the crawl command. Due to the nature of crawling data on the web, not all data will be present or correct. For that reason it's important to also check out the issues and limitations.

Cards

Depending on the crawler, the card resource will contain complete or partial information. If the card resource contains complete information, the isComplete property will be set to true.

When a card is crawled, it gets automatically analyzed (also see the analyze cards command). If the analyzer generates one or more errors, the isComplete property will be set to false. It's recommended to manually fix these errors and to run the analyze cards command until no more errors are generated. Afterwards (if applicable), the isComplete property can be manually set back to true.