AI Agent Fed Site Searches and Sitemaps Extracted 20 Page Topics a Week at Upwork
Manual keyword lookups in Ahrefs and customer interviews cannot keep pace when marketplace users demand new hiring categories.
Abby Gleason reports deploying an AI research agent at Upwork that cross-referenced internal demand signals and competitor sitemaps to output 20 page topics a week.
Rather than using AI to draft briefs or page copy, the workflow runs the agent every week to synthesize topic gaps across five inputs:
- Existing site inventory: scans current
Upworkpages every week to establish a baseline for deduplication. - Platform job postings: flags emerging skills clients post jobs for where no corresponding page exists.
- Internal site searches: extracts on-site queries where users search and come up short.
- Competitor sitemaps: tracks newly published competitor
URLsto expose topical coverage gaps. - External websites and searches: crawls industry sites and search results.
Furthermore, the team does not autogenerate pages from the weekly 20-topic output, instead reviewing every suggestion by hand before building new pages.
The agent recommends topics that appear to be rising gaps when an existing page already covers the skill under slightly different wording.
Starting a new role or freelance client engagement, Gleason audits the site manually upfront to build the baseline context needed to challenge those false positives at the back end.
The resulting pages achieved "very quick success," though specific traffic and ranking figures are not disclosed.