All posts

Building MTGAuctioneer: OCR, Search, and the Architecture Behind a Card Listing Pipeline

12 min readengineering, computer-vision, architecture, case-study

We recently shipped MTGAuctioneer — a platform that takes a photo of a Magic: The Gathering card and turns it into a priced, formatted listing on eBay and TCGPlayer. The case study tells the business story. This post is about the engineering.

The Problem, Technically

A trading card resale operation needs to do three things per card: identify it, price it, and list it. Each step sounds simple. None of them are.

Identification means extracting the card name, set, edition, and condition from a photograph. Magic has over 250,000 card printings across 100+ sets, with reprints that share names but differ in art, frame, and set symbol. A human expert can do this in seconds. Teaching a machine to do it reliably is a different problem entirely.

Pricing means querying real-time market data where the same card can have wildly different valuations depending on condition, edition, and whether it's foil.

Listing means generating marketplace-compliant listings with the right title format, item specifics, condition mappings, and images — conforming to eBay's Trading API XML schema and TCGPlayer's catalog-based inventory model, which have fundamentally different listing paradigms.

The manual version of this workflow topped out at a few hundred cards per day per person. We needed to get that number into the thousands.

System Architecture

The platform runs as three separate Node.js services orchestrated by PM2, all backed by a React frontend with Apollo Client:

Browser (React + Apollo Client)
    |
    v
GraphQL API (Apollo Server, port 4000)
    |--- JWT auth, context injection
    |--- Mongoose ODM → MongoDB
    |--- Mongoosastic → Elasticsearch
    |
Image Processing Service (Express, port 3037)
    |--- Multer (file upload handling)
    |--- OpenCV4 (image cropping)
    |--- Google Cloud Vision (OCR)
    |--- AWS S3 (image storage)
    |
eBay Proxy Service (Express, port 3038)
    |--- eBay Trading API (XML)
    |--- xml2json (response parsing)

Three services, one codebase, no container orchestration. PM2 manages the processes. The GraphQL API is the primary interface for the frontend — the image and eBay services are internal, called server-side only.

The OCR Pipeline

This is where the interesting engineering lives. Card identification runs through a forked child process to avoid blocking the main GraphQL server during image work.

Image Cropping with OpenCV

Cards are photographed in duplex format — front and back side by side in a single image. OpenCV splits and crops them into regions of interest before OCR runs:

const getImageCoords = ([height, width], format = "DUPLEX") => {
    const divisor = format === "DUPLEX" ? 2 : 1;
    return {
        front: [0, 0, Math.floor(width / divisor), height],
        back: [Math.floor(width / divisor), 0,
               Math.floor(width / divisor), height],
        frontTop: [0, 0, Math.floor(width / divisor),
                   Math.floor(height / 2)],
        frontBottom: [0, Math.floor(height / 2),
                      Math.floor(width / divisor),
                      Math.floor(height / 2)],
        // ... back regions follow the same pattern
    };
};

The coordinate math is proportional, not absolute — it works regardless of the input image resolution. OpenCV's getRegion extracts each crop, which gets written to disk, then uploaded to S3 for permanent storage.

The entire image processing step runs in a fork()ed child process communicating over IPC. The parent sends the file path and format, the child sends back OCR results and S3 URLs. This isolation means a slow Vision API call or a large batch upload never starves the GraphQL server.

Google Cloud Vision for Text Detection

We use Google Cloud Vision's textDetection endpoint rather than running OCR locally. The API returns a FullTextAnnotation with the detected text and language confidence scores for each detected language — useful for handling foreign-language printings where the printed_name field differs from the English card name.

The raw OCR result is stored directly on the CardInstance document. This means we can reprocess cards later without re-running Vision API calls — a decision that paid for itself when we tuned the matching logic.

Card Matching: Scryfall First, Elasticsearch Fallback

Once we have OCR text, the matching pipeline extracts the card name from the first two lines of detected text (card names are always at the top of the frame) and tries to resolve it:

const getNameMatch = (text) => {
    const lines = text.split("\n").slice(0, 2);
    const regex = /^\D*$/; // alphabetic characters only
    return lines.find(line => regex.test(line));
};

The matching strategy is two-tier:

Tier 1: Scryfall API. We hit Scryfall's fuzzy search endpoint first. It handles misspellings well and returns canonical card data. All Scryfall calls go through a Bottleneck rate limiter — 100ms minimum between requests — to stay within their API guidelines.

Tier 2: Elasticsearch. If Scryfall doesn't return a match, we fall back to our local Elasticsearch index. The Card model uses the mongoosastic plugin to automatically sync MongoDB documents to ES. The fallback query uses multi-match with fuzziness:

const result = await esClient.search({
    index: "cards",
    body: {
        query: {
            multi_match: {
                query: name,
                fields: ["name", "printed_name"], 
                fuzziness: "AUTO", 
                operator: "and",
            },
        },
    },
});

Name matches get a 10x relevance boost over printed_name matches, so English-language cards resolve correctly even when foreign printings exist in the index.

If exactly one card matches, the system auto-links it and marks the CardInstance as COMPLETE. If zero or multiple matches come back, it stays PENDING for human review.

Set Symbol Classification

Name matching alone can't distinguish between reprints — a card printed in ten different sets has ten different prices. The initial release relied on OCR of the collector number and set code text at the bottom of the card, which worked but was fragile on older card frames where the text is small and often partially obscured.

In a later iteration, we added a dedicated ML classifier for set symbol recognition. The approach: crop only the set symbol glyph from the card scan (a small region on the right side between the art and text box), then run it through a convolutional neural network trained on the ~100+ unique set symbols in Magic's history.

This turned out to be a better signal than trying to OCR tiny text. Set symbols are designed to be visually distinct — they're essentially a logo recognition problem, which CNNs handle well. The classifier's output gets combined with the name match to narrow identification down to a specific printing, which directly determines the price.

The Human Verification Queue

The auto-matching pipeline catches most cards, but the ones it misses need human eyes. The verification queue is built into the same GraphQL API — no separate service.

Card instances move through a status workflow:

PENDING    → image uploaded, OCR complete, no confident match
PROCESSING → reviewer is looking at it
COMPLETE   → card identified and verified
ALLOCATED  → assigned to a listing

A reviewer queries cardList(status: PENDING) to see the queue, pulls up a card instance with its OCR text and cropped images, and either confirms the system's best guess or manually searches for the correct card. The processPendingInstance mutation handles both the auto-match batch job and individual manual reviews — same code path, different trigger.

Every manual correction updates the card's price data from Scryfall and regenerates inventory group aggregations. This keeps pricing current without a separate refresh job.

The Feedback Loop

The verification queue isn't just a fallback — it's a training signal. Every time a reviewer corrects a misidentification, the system logs what the OCR extracted versus what the human selected. Over time, this builds a dataset of failure modes: which card names the OCR consistently mangles, which set symbols the classifier struggles with, which card frames produce unreliable crops.

We used this data to tune confidence thresholds and improve the region extraction coordinates. In the first week, roughly 40% of cards needed human review. After a month of corrections feeding back into threshold adjustments and crop tuning, that dropped to under 15%. The system gets more accurate the more it's used — not through retraining the models, but through better understanding of where they fail and adjusting the pipeline around those edges.

Inventory Aggregation

Individual card instances are useful for tracking physical inventory, but the team needs to see aggregate views — "how many Near Mint Lightning Bolts do we have across all bins?" MongoDB's aggregation pipeline handles this:

// Stage 1: Match only COMPLETE instances
{ $match: { status: "COMPLETE" } }
 
// Stage 2: Group by card + condition + foil
{ $group: {
    _id: { cardId: "$cardId", condition: "$condition", foil: "$foil" },
    count: { $sum: 1 },
    ids: { $push: "$_id" }
}}
 
// Stage 3: Lookup card details
{ $lookup: {
    from: "cards",
    localField: "_id.cardId",
    foreignField: "_id",
    as: "cardDetail"
}}

The InventoryGroup model pairs these aggregations with a price history collection. Every time a card is processed, its current Scryfall prices are snapshot into a Price document. This gives the team a historical view of inventory value over time without relying on external API uptime.

Marketplace Integrations

The marketplace integrations were the most tedious part of the build. Not technically complex — just deeply specific. Each platform has its own API, listing model, and taxonomy. We built a common adapter interface so the listing pipeline doesn't care which marketplace it's targeting.

eBay

eBay's Trading API is XML-based, and every listing requires a precise set of item specifics mapped to their taxonomy. The listing creation flow works in stages:

1. Chunking. Card instances are grouped into lots. The system decides lot size based on inventory — single high-value cards get individual listings, bulk commons get batched.

2. SKU generation. Small batches (4 or fewer cards) use concatenated bin locations as the SKU — "Bin1,Bin2,Bin3" — so the team can physically find the cards when they sell. Larger batches get a nanoid.

3. XML construction. An eBay-specific builder maps card attributes to item specifics:

// Pricing logic
const startPrice = Math.max(price * 0.3, 1.49);
const buyItNow = Math.max(price * 0.75, startPrice * 1.3);

Auctions start at 30% of market price with a floor of $1.49. Buy-it-now is set at 75% of market or 130% of the start price, whichever is higher. These ratios were tuned through a few weeks of real auction data.

Item specifics include brand, set name, rarity, card type, colors, condition, and special attributes like foil, misprint, or artist signatures. Signed cards get a "Modified Item" flag with the artist name — eBay's taxonomy requires this for authenticated signatures.

4. Image attachment. Up to 12 images per listing — front, back, and original photos. The S3 URLs from the OCR pipeline are passed directly to the eBay API. No re-uploading.

The eBay proxy service on port 3038 handles the reverse direction — pulling listing status, active items, and sale data back from eBay's XML API and converting responses to JSON for the GraphQL layer.

TCGPlayer

TCGPlayer is a fundamentally different listing model. Where eBay has you create individual listings, TCGPlayer is catalog-based — you're adding inventory to an existing product page. You don't write a title or a description; you select the card from their catalog and specify condition, quantity, and price.

This made the integration simpler in some ways (no XML construction, no item specifics mapping) and more annoying in others (catalog matching has to be exact, and their product IDs don't always line up cleanly with Scryfall IDs). We store the TCGPlayer product mapping on the Card document alongside the Scryfall data, so catalog lookups are a local query rather than an API call at listing time.

The Data Model

Everything lives in MongoDB with Mongoose as the ODM. The core collections:

  • cards — 250k+ documents imported from Scryfall via streaming JSON. Full card metadata: name, set, colors, mana cost, type line, oracle text, image URIs, and cached prices. Auto-synced to Elasticsearch via mongoosastic.
  • cardinstances — Individual physical cards. Links to a card document, tracks condition (NM through Damaged, with foil variants), bin location, status, and the raw OCR annotation. Each instance has its own set of cropped image URLs.
  • listings — Marketplace listing records. References the card and its instances, stores the marketplace item ID (eBay or TCGPlayer), SKU, pricing, and listing status.
  • inventorygroups — Aggregated inventory views grouped by card, condition, and foil status. Paired with price history snapshots.

The card import deserves a mention — streaming 250k+ documents from Scryfall's bulk data endpoint through JSONStream, upserted one at a time with findOneAndUpdate. It's not fast, but it's resumable, and it only runs when we need to refresh the master card database.

What We'd Do Differently

Message queue for batch processing. The OCR pipeline runs in a forked process, which is fine for single cards. But batch uploads of 50+ cards would benefit from a proper job queue with parallel workers. We scoped this out to hit the timeline.

Webhook-based listing status. We poll eBay for listing updates. The Trading API supports notifications — we should have wired those up from the start instead of building a polling loop.

Secrets management. The JWT secret is hardcoded. In production this gets overridden, but it's a code smell that should have been an environment variable from day one.

The Stack

  • Frontend: React with Apollo Client.
  • API: Node.js with Apollo Server 2. GraphQL-first, JWT auth.
  • Image Processing: OpenCV4 for cropping, Google Cloud Vision for OCR. Forked child processes for isolation.
  • Search: Elasticsearch with mongoosastic auto-sync. Fuzzy multi-match with field boosting.
  • External APIs: Scryfall for card data and pricing, rate-limited with Bottleneck.
  • Database: MongoDB with Mongoose. Aggregation pipelines for inventory views.
  • Marketplaces: eBay Trading API via XML, TCGPlayer Catalog and Inventory APIs. Common adapter interface.
  • Storage: AWS S3 for card images. Public-read ACL, direct URL serving.
  • Process Management: PM2 running three services from a single codebase.

No Kubernetes, no serverless, no microservice mesh. Three Node processes, one database, and an Elasticsearch instance. For this scale — thousands of cards per day, single-digit concurrent users — simplicity wins.