Tabelog Robots.txt [better] Online

For SEOs: Tabelog will rank for restaurant names anyway, because user behavior (searching “Sushi Tokyo Tabelog”) overrides crawl directives. But for anyone wanting structured data at scale? The robots file says everything you need to know: “No.” Would you like a technical breakdown of how to ethically monitor Tabelog changes without violating their robots.txt ?

| Want to crawl? | Allowed? | |----------------|----------| | Restaurant detail pages | ✅ (implicitly, via no explicit block) | | Search results | ❌ | | Review pages | ❌ | | Photo galleries | ❌ | | Regional index pages | ❌ | | Ranking lists | ❌ | For a site built on user contributions and openness, Tabelog’s robots.txt is remarkably closed. But that’s the point. In a market where restaurant data is a strategic asset (competitors include Google Maps, Retty, and Gurunavi), a robots.txt becomes a legal-engineering hybrid: “We’ve told you not to crawl these paths. If you do, you’re violating our terms and potentially the Unfair Competition Prevention Act of Japan.” Final take If you’re building a crawler for Tabelog, don’t bother negotiating with robots.txt — it’s not a negotiation. It’s a warning. Real access requires official APIs or commercial partnerships. The robots.txt is just the polite “Keep Out” sign before the electric fence. tabelog robots.txt

At first glance, it looks like a standard robots.txt . But look closer. It tells a fascinating story about data protection, competitive moats, and Japan’s unique web culture. User-agent: * Disallow: /search/ Disallow: /rgsearch/ Disallow: /kw/ Disallow: /syop/ Disallow: /rr/ Disallow: /list/ Disallow: /rvw/ Disallow: /photo/ Disallow: /map/ Disallow: /guide/ Disallow: /sitemap/ Disallow: /navi/ Disallow: /rank/ Disallow: /shop/%A5%EA%A5%B9%A5%C8 Disallow: /bshop/ Disallow: /rstd/ Disallow: /west/ Disallow: /tokyo/ Disallow: /osaka/ Disallow: /aichi/ Disallow: /kyoto/ Disallow: /hyogo/ Disallow: /hokkaido/ Disallow: /fukuoka/ Disallow: /miyagi/ Disallow: /chiba/ Disallow: /saitama/ Disallow: /kanagawa/ Disallow: /shizuoka/ Disallow: /hiroshima/ What Tabelog is really saying 1. “Search results are off-limits.” The /search/ and /list/ paths are blocked. This is common for large sites to prevent infinite crawl loops, but for Tabelog, it’s strategic: search result pages contain ranked restaurant lists — their core IP. Letting search engines index those would let competitors reverse-engineer their ranking algorithm. For SEOs: Tabelog will rank for restaurant names

The list of Disallow: /tokyo/ , /osaka/ , /kyoto/ , etc., is unusual. Most sites want their city landing pages indexed. Tabelog explicitly blocks them. Why? Possibly because those pages are thin, auto-generated, or contain internal navigation that leads to disallowed content. More likely: Tabelog prefers to control how its regional authority is presented — via their own sitemap and internal linking, not via open-ended crawler access. | Want to crawl

/rvw/ (reviews) and /photo/ (user-uploaded images) are fully disallowed. Why? Because Tabelog’s value is user-generated trust. If Google indexed every review page, scrapers could steal structured opinions and star ratings without ever touching the site. Blocking them doesn’t stop determined scrapers, but it raises the bar.

If you’ve ever tried to crawl Tabelog (食��グ), Japan’s most authoritative restaurant review platform, you’ve met its first line of defense. It’s not a CAPTCHA. It’s not an IP ban. It’s a deceptively simple text file: https://tabelog.com/robots.txt .