Robots.txt

Robots.txt Disallow Rules Kill Canonical Consolidation on Parameter URLs

Blocking parameter-based duplicate URLs via robots.txt blinds crawlers to canonical tags, warns Jono Alderson.

• Robots.txt
• Canonical
• Technical SEO