Skip to content

Crawling TCGplayer card variant prices

This document provides information for the TCGplayer crawler of the crawl card-variant-prices command.

TCGplayer has somewhat limited support for card variants. They have a fixed set of card variant types, which they call printings. Each product (card) on TCGplayer is assigned one or more of these printings, from which sellers can choose. The available printings are:

  • Normal
  • Holofoil
  • Reverse Holofoil
  • Unlimited
  • Unlimited Holofoil
  • 1st Edition
  • 1st Edition Holofoil

The Unlimited and Unlimited Holofoil printings are identical to the Normal and Holofoil printings. They're used exclusively for products that also have 1st Edition or 1st Edition Holofoil printings. This is to avoid confusion for buyers who might incorrectly assume Normal or Holofoil printings to be 1st Edition or 1st Edition Holofoil printings.

In the context of crawling card variant prices, this distinction for the Unlimited and Unlimited Holofoil printings has no real use. Therefore, we don't use them and automatically translate them to Normal and Holofoil internally.

TCGplayer has no option for card variants like Cracked Ice Holo or Cosmos Holo. This means that they're forced to list these card variants as separate products using a best matching printing (usually Normal or Holofoil). This makes it extremely difficult to automatically figure out which card variant they're referring to. Our solution is to use an external database that contains this mapping information.

A lot of card names, expansion names and card numbers are labeled in an inconsistent manner. For that reason, mapping information for these values is included in the external database as well.

Command usage (options)

--tcgplayer-product-mappings-dir-path

Because TCGplayer is quite inconsistent when it comes to card names, expansion names and card variants (as explained previously), an external database is needed that maps the values as they appear on TCGplayer to their corrected/sanitized values. This database consists out of a JSON file per product with the following properties:

Property Type Is required Description
productId int32 Yes The TCGplayer product ID.
productName string Yes The TCGplayer product name. It usually refers to the card name.
productGroupName string Yes The TCGplayer group name. It usually refers to the expansion name.
productNumber string|null Yes The TCGplayer product number. It usually refers to the card number.
productUrl string Yes The TCGplayer product URL.
cardId string|null No If this value is set, all other properties will be ignored when identifying the card. This can be useful when mapping card (variant) prices to an external database such as TCG Collector.
cardName string|null No Optionally can be used to help identify the card.
expansionName string Yes
cardNumber string|null Yes
normalCardVariantId string|null No See 1*.
holofoilCardVariantId string|null No See 1*.
reverseHolofoilCardVariantId string|null No See 1*.
firstEditionCardVariantId string|null No See 1*.
firstEditionHolofoilCardVariantId string|null No See 1*.
normalCardVariantType string Yes See 2*.
holofoilCardVariantType string Yes See 2*.
reverseHolofoilCardVariantType string Yes See 2*.
firstEditionCardVariantType string Yes See 2*.
firstEditionHolofoilCardVariantType string Yes See 2*.

This option is not required because no product mappings will exist yet on the first run. After the first run, the product mappings can be created by mapping the unhandled products output (see the --tcgplayer-unhandled-products-output-dir-path option).

Each JSON file is required to have a tcgplayer-product-mapping suffix, e.g., 158963.tcgplayer-product-mapping.json.

1*: If this value is set, all other properties will be ignored when identifying the card variant. This can be useful when mapping card (variant) prices to an external database such as TCG Collector.

2*: The *cardVariantType properties map the different TCGplayer printings to card variant types. All properties are required to have a value because TCGplayer doesn't always list the correct printings for a product.

--tcgplayer-skipped-products-dir-path

Sometimes, TCGplayer will list products that you don't want to track. In order to prevent these products from showing up in the unhandled products output (see the --tcgplayer-unhandled-products-output-dir-path option), a JSON file per product with the following properties can be used:

Property Type Is required Description
id int32 Yes The TCGplayer product ID.
name string Yes The TCGplayer product name. It usually refers to the card name.
groupName string Yes The TCGplayer group name. It usually refers to the expansion name.
number string|null Yes The TCGplayer product number. It usually refers to the card number.
url string Yes The TCGplayer product URL.
reasonForSkipping string|null No The reason for skipping the TCGplayer product.

Each JSON file is required to have a tcgplayer-skipped-product suffix, e.g., 158963.tcgplayer-skipped-product.json.

--tcgplayer-unhandled-products-output-dir-path (required)

Unhandled products are TCGplayer products for which no product mapping could be found during the crawling process. Each unhandled product is exported as a JSON file with the following properties:

Property Type Is required Description
id int32 Yes The TCGplayer product ID.
name string Yes The TCGplayer product name. It usually refers to the card name.
groupName string Yes The TCGplayer group name. It usually refers to the expansion name.
number string|null Yes The TCGplayer product number. It usually refers to the card number.
url string Yes The TCGplayer product URL.

This output should be mapped manually into product mappings or skipped products. If the TCG Collector database is used, the unhandled products can be semi-automatically mapped using the cms map tcgplayer-unhandled-products command.

To get an overview of all unhandled TCGplayer groups, the show tcgplayer-unhandled-groups command can be used.

Each JSON file is required to have a tcgplayer-unhandled-product suffix, e.g., 158963.tcgplayer-unhandled-product.json.

--tcgplayer-api-config-file-path (required)

This JSON file contains the configuration for the TCGplayer API.

Property Type Is required Description
publicKey string Yes
privateKey string Yes

How to map TCGplayer printings to card variants

For non-holographic products that are not a card variant listed as a separate product:

Property TCGplayer printing Card variant type Notes
normalCardVariantType Normal Standard Set
holofoilCardVariantType Holofoil Parallel Set: Reverse Foil See 1*.
reverseHolofoilCardVariantType Reverse Holofoil Parallel Set: Reverse Foil See 1*.
firstEditionCardVariantType 1st Edition Standard Set
firstEditionHolofoilCardVariantType 1st Edition Holofoil Parallel Set: Reverse Foil See 1*.

For holographic products that are not a card variant listed as a separate product:

Property TCGplayer printing Card variant type Notes
normalCardVariantType Normal Standard Set Foil
holofoilCardVariantType Holofoil Standard Set Foil
reverseHolofoilCardVariantType Reverse Holofoil Parallel Set: Reverse Foil See 1*.
firstEditionCardVariantType 1st Edition Standard Set Foil
firstEditionHolofoilCardVariantType 1st Edition Holofoil Standard Set Foil

For products that are a card variant listed as a separate product, the mappings will have to be adjusted according to the card variant type. Take Cosmos Holo for example:

Property TCGplayer printing Card variant type
normalCardVariantType Normal Standard Set
holofoilCardVariantType Holofoil Cosmos Holo
reverseHolofoilCardVariantType Reverse Holofoil Cosmos Holo
firstEditionCardVariantType 1st Edition Standard Set
firstEditionHolofoilCardVariantType 1st Edition Holofoil Cosmos Holo

1*: A card that doesn't have a Parallel Set: Reverse Foil variant might have an additional variant that is holographic, such as Cosmos Holo. If so, the TCGplayer printing should map to this card variant instead of Parallel Set: Reverse Foil. If the card has multiple holographic variants that are not Parallel Set: Reverse Foil, it will need to be investigated manually.

Automatic mapping

When using the cms map tcgplayer-unhandled-products or cms map tcgplayer-product-mappings command, there is a runtime option to attempt to automatically map the TCGplayer printings. However, this option is not completely foolproof and should not be used in the following cases:

  • The TCGplayer product is a separately listed card variant.

    Automatic mapping only works for "main" TCGplayer products and "main" products can unfortunately not be automatically distinguished from "card variant" products.

  • The TCG Collector card variants are incomplete.

    The mapping of the TCGplayer printings differs based on which TCG Collector card variants are available. If automatic mapping proceeds with an incomplete list of card variants, duplicate price errors are likely to occur during the crawling process (also see the Changes to product mappings over time section).

  • The TCG Collector card rarity is undetermined or incorrect.

    The TCG Collector card rarity is used to determine whether the card is holographic or not. This directly affects how the TCGplayer printings are mapped.

Changes to product mappings over time

Over time, manual maintenance to the product mappings might be required because of the following reasons:

  1. A new product has been added on TCGplayer.

    See the --tcgplayer-unhandled-products-output-dir-path option.

  2. A new printing has been added to an existing TCGplayer product.

    A new printing will cause the existing product mapping to incorrectly map to the same card variant. The crawler will log an error about a duplicate card variant that will need to be corrected manually. Although rare, it's also possible a seller on TCGplayer specified an incorrect printing. In that case, the error can be ignored.

    If the TCG Collector database is used, existing product mappings (including skipped products) can be semi-automatically remapped using the cms map tcgplayer-product-mappings command.

  3. A product has been deleted on TCGplayer.

    This deleted product should remain in the product mappings, because there is a chance of the product becoming available again. The crawler doesn't log any warnings or errors about deleted products.

  4. A new card (variant) has been added to an externally mapped database.

    If the card (variant) doesn't exist on TCGplayer, no action is required. If TCGplayer adds this card (variant) at one point, see item 1 and 2.

  5. One or more of the values of the mappings have gone out of sync with an externally mapped database.

    Unfortunately, the crawler has no way of detecting values that have gone out of sync with an externally mapped database. However, it's advised that the process of uploading the card (variant) prices to externally mapped databases should log warnings about it. This primarily concerns changes and deletions, since new values are already automatically detected (see item 4). With the help of these logs, it should be possible to correct the mappings manually.