Overview

Resources can be crawled using the crawl command. Due to the nature of crawling data on the web, not all data will be present or correct. For that reason it's important to also check out the issues and limitations.

Cards

Depending on the crawler, the card resource will contain complete or partial information. If the card resource contains complete information, the isComplete property will be set to true.

When a card is crawled, it gets automatically analyzed (also see the analyze cards command). If the analyzer generates one or more errors, the isComplete property will be set to false. It's recommended to manually fix these errors and to run the analyze cards command until no more errors are generated. Afterwards (if applicable), the isComplete property can be manually set back to true.

Bulbapedia

Command usage (options)

--bulbapedia-card-list-file-path

In general, Bulbapedia catalogues cards as one or more lists per expansion. However, for some expansions these lists are formatted in a non-standard way, meaning that they can't be properly crawled. In order to crawl these cards anyway, you can use a CSV file containing a custom card list with the following columns:

Column	Is value required	Description
tcg_region	Yes	Available values: International, Japan.
expansion_name	Yes
card_number	No	Can only be empty if the card has no number.
card_number_sorting_order	Yes
card_rarity	No	Can only be empty if the card has no rarity.
regulation_mark	No	Can only be empty if the card has no regulation mark.
card_url	Yes	The Bulbapedia card URL.

CSV headers are required to be present in the file.

CSV file example

tcg_region,expansion_name,card_number,card_number_sorting_order,card_rarity,regulation_mark,card_url
International,Base Set,1/102,1,Rare Holo,,https://bulbapedia.bulbagarden.net/wiki/Alakazam_(Base_Set_1)
International,Base Set,2/102,2,Rare Holo,,https://bulbapedia.bulbagarden.net/wiki/Blastoise_(Base_Set_2)