Crawl a whole site or just the pages you name, and score each one against 13 standards-based checks. Open any result to inspect its heading outline, the links inside its content, its structured data and how it will look when shared — then export the lot to a single Excel workbook.
Reads the HTML each page serves, so it sees what a search engine crawler sees rather than what a browser
builds afterwards. Honours robots.txt, and follows a staging access
link when you paste one in.
Crawls up to 500 pages on this domain. A staging access link works here too — paste it and the crawl follows it to the site it opens.
Each line is analysed exactly as given. No links are followed.
Ready to start...
Successful: 0
| URL | Title | Description | Slug |
|---|
Ready to start...
Successful: 0
| URL | Title | Description | Slug |
|---|
Ready to start...
Successful: 0
A sitemap is like a roadmap for your website that helps search engines find and understand all the pages on your site.
yoursite.com/sitemap.xml)Update your sitemap whenever you:
Best Practice: For active websites, regenerate weekly or monthly. For static sites, quarterly updates suffice.
Check these indicators:
<a href> in the page content, excluding the page header and footer. On the sample homepage that is 30 content links kept and 33 nav/footer links left out, and the panel says so rather than quietly dropping them: kept + excluded + skipped reconciles exactly against the 63 hrefs on the page, so the numbers can be checked rather than trusted. Each row shows the link type (internal, external, email, phone, on-page, other), its anchor text, the destination, and any rel value, with nofollow/sponsored/ugc called out.role="banner", role="contentinfo") and the usual id/class conventions (site-header, masthead, colophon, …). <header> and <footer> inside an <article> are kept — those are the article's own byline and tags, which are content. Class matching is space-padded so card-header is not mistaken for the site header, and regions are deduplicated by node identity, since one <header class="header"> satisfies three of the queries.alt, then aria-label, then title, and the row says which was used. Roughly a third of links on a real page wrap an image and carry no text of their own, so without this the column would be mostly blank. A link with genuinely no text is counted and flagged, since that is an SEO finding in itself.?inspect endpoint returning one page's inventory on demand. The link list measures ~50 KB on a real page, so carrying it in the crawl stream would add roughly 25 MB to a 500-page run and more again to the DOM copy the results table keeps — for data opened on a handful of rows. Bulk crawls are unchanged in size. It runs through the same assertFetchableUrl() guard as every other endpoint, verified against loopback, private, link-local, cloud-metadata, IPv6 loopback and non-http schemes.CompressionStream and stored uncompressed otherwise, both valid ZIP..xlsx with two sheets: SEO Analysis, one row per page across all 80 columns, and Links, one row per link across every page. A page can hold hundreds of links, which no single row can represent — the two-sheet shape is exactly what CSV could not express. Measured on a 48-page crawl: 49 × 80 and 380 × 8 in one file.Heading Outline, OG Values and Twitter Values (the actual tag contents, not just which tags exist), JSON-LD Retained and JSON-LD Omitted. From the link inventory: Links In Content, Unique Link Targets, Internal Links, External Links, Nofollow Links, Links Without Anchor Text, Links Excluded (Header/Footer), Header/Footer Regions, Links Not Read Reason and All Links.All Links cell is capped at 30,000 characters and says so when truncated — Excel rejects a cell over 32,767, which would have corrupted the file rather than trimming a value.lg they fall back to stacking underneath, so nothing is squeezed on tablet or phone (verified at 1500px, 900px and 420px).?inspect endpoint could return invalid JSON. It called applyAccessLink(), which echoes STATUS: / ERROR: lines for the newline-delimited crawl streams. Printed ahead of a JSON body those made the response unparseable, so a working access link would have surfaced as a parse error rather than a result. It now calls unlockSession() directly and reports failure inside the JSON.//cdn.example.com/x) resolved to nonsense. They fail FILTER_VALIDATE_URL and start with /, so they fell into the absolute-path branch and became https://ownhost//cdn.example.com/x. They now inherit the page's scheme. This affected the crawl queue too, not just the new tab.makeAbsoluteUrl() returned malformed URLs when the base had no scheme or host, instead of reporting failure.Links Not Read Reason and as a row on the Links sheet, so a gated or failing page is distinguishable from a page that genuinely has no links in its content. One failure does not abort the export, and the completion message says how many pages were affected.= in an inline string cell is inert, because a formula has to be an <f> element and the writer never emits one. Verified in Excel — a cell containing =SUM(A1:A2) reports HasFormula=False.CB), XML-illegal control characters stripped while tab/newline/carriage-return survive, multibyte text intact — then opened in Excel with no repair prompt, on both a synthetic worst case and a real 48-page export.?inspect does not consult it: a robots rule causes a URL to be skipped during crawling, never a 401. The 401 came from the environment's access gate.DEPLOYMENT.md guide, both included in the release zip. .htaccess denies .log/.json/.md over HTTP, blocks src/, and disables gzip for PHP so the result stream is not held back. .user.ini carries the PHP settings, because cPanel runs PHP as FPM/CGI where php_value lines in .htaccess are silently ignored.MDE_ERROR_LOG — define it before including function.php to move the error log above the document root. The log records every crawled URL and why it failed, and it lived inside the webroot where a live host would serve it: GET /php_errors.log returned 200 with internal staging hostnames.?v=<version>, so a released update is fetched rather than served from cache. Every previous change needed a manual hard refresh to appear.package.json was missing from the release zip, so appVersion() fell back to 'dev': packaged builds showed "dev" in the footer instead of the version, and would have pinned the new cache-busting query to a value that never changes.FcgidOutputBufferSize is server-context only, so .htaccess cannot fix it — PHP-FPM streams without special configuration. DEPLOYMENT.md covers the diagnosis.?seo-analysis-stream endpoint that tab used is left in place. It predates the tab, is SSRF-guarded, and works standalone — but it now has no caller in the UI, so it should be removed if nothing external depends on it.tag_analysis keys. It hangs off the one delegated .seo-button handler that already served all three result tables, so URL Crawling, the URL list and Page Check all gained it at once.twitter:card — X falls back to Open Graph"), plus a full table of every og:* and twitter:* tag with its value.@graph unwrapped, and URLs clickable. Strings, numbers, booleans and nulls are styled distinctly so a wrong type is visible at a glance.twitter:title, twitter:description or twitter:card. The values are now captured alongside the names.OG:Title and other mixed-case Open Graph properties were ignored entirely. The prefix test was case-sensitive while the Twitter branch beside it was not, so a page using capitalised properties was reported as having no Open Graph tags at all. Property names are now lowercased, and values trimmed, before matching.type="application/ld+json; charset=utf-8" script was skipped, reporting pages with valid structured data as having none. Media-type parameters are now stripped before comparison.null hid every sibling node. isset() returns false for a literal null, so the whole array fell through to the single-node branch and was reported as one untyped entry.@graph was never unwrapped — a shape several WordPress SEO plugins emit — so its nodes were invisible to the type list.innerHTML without escaping: heading text, Open Graph property names, Twitter tag names, and the robots/charset/schema-type badges. A page carrying <h1><img src=x onerror=…></h1> ran its payload as soon as you opened that page's SEO details. Confirmed by execution against the previous build, which set a global from the injected handler; the same probe is now inert. Heading text and OG property names are escaped at the call site, and mkBadge() escapes its own text so any future badge is safe by default rather than depending on each caller.javascript: canonical URL was a live clickable link. <link rel="canonical" href="javascript:…"> was rendered straight into an <a href>. Escaping does not help here — the payload needs no quotes or angle brackets — so hrefs now pass through a new safeUrl() scheme allowlist that permits only http, https and relative URLs. Control characters and whitespace are stripped before the scheme is tested, because browsers ignore them when parsing one (java\tscript: still executes). A refused value is shown as inert text with a note instead of being silently dropped, and surviving links carry rel="noopener noreferrer".substr(), which splits a multi-byte character in half; the resulting invalid UTF-8 made json_encode() return false, so the streamed line arrived empty, JSON.parse threw, and the row was discarded by the catch. The page never reached the results array — but SUCCESS: had already counted it, so the completion total silently disagreed with the number of rows actually present. Truncation is now character-safe, and all four streaming encoders use JSON_INVALID_UTF8_SUBSTITUTE via a shared streamJson() helper so no single bad byte can ever cost a whole page again. Non-English and accented headings were the common trigger.extractImagesFromHtml() returned array_unique() directly, and array_unique() preserves the original keys — so any page repeating an image (a logo in both the header and the footer, i.e. nearly every page) left gaps, and json_encode() emitted a JSON *object* instead of an array. The Array.isArray() guard on the counting side then contributed nothing rather than throwing, which is why it under-reported quietly. Now re-indexed with array_values().CHANGELOG.md instead of ~145 lines of hand-maintained markup that had drifted from it. The footer version badge reads from package.json.file://, gopher:// or dict://. Redirect chains and response size are bounded.set_time_limit(0) and ignore_user_abort(false) via a shared beginStreamingResponse() helper.extractLinksFromPage(), halving the number of HTTP requests per crawl.in_array() scan.SKIP: lines for URLs robots.txt disallows, counted in the UI. A crawl blocked entirely by robots.txt gets a dedicated notice in place of the success summary — headed *"This site asked not to be crawled"* — which ticks off the steps that did succeed (access link accepted, site reachable) before naming robots.txt as the one that stopped it, so a blocked run is not mistaken for a broken token. It offers Crawl anyway, ignoring robots.txt, which ticks the option and reruns.Access link accepted — session established, crawling <host>, so a token that was accepted is not mistaken for the cause when something later stops the crawl.seo-analysis-stream endpoint added in 3.3.0, which had no frontend.alt attribute, and flags file-name alt text and alt text over 125 characters. alt="" is treated as correct for decorative images, not as a failure.@type or @context, and lists the schema.org types found. Falls back to detecting microdata and RDFa at partial credit.noindex / nofollow / none from robots, googlebot and bingbot meta tags. A noindex page is reported as CRITICAL, since it silently undoes every other optimisation.<html lang> as a BCP 47 tag.robots.txt per RFC 9309 (longest match wins, Allow breaks ties, * and $ wildcards, a group naming the crawler overrides the wildcard group) and respects Crawl-delay, clamped to 0.2–5s. Skipped URLs are reported in the stream. The crawler identifies itself as MetaDataExtractor/3.4.assertFetchableUrl() enforces an http/https scheme allowlist, rejects credentials in URLs, and resolves hostnames to reject loopback, private, link-local, carrier-grade NAT and cloud metadata addresses. Redirect hops are re-validated so a redirect cannot reach an internal host.console.error() before testing for AbortError, so every user-initiated stop reported an error for an outcome that is expected and already handled.COMPLETE: line left the controls locked — a killed worker or dropped connection had no path back but a page reload. The reader now restores them after the stream ends however it ends.#urls-tab-pane to disableOtherTabs(), a selector that no longer matched anything once the two modes were merged, so no tab was treated as the active one.COMPLETE: handlers set a summary box on the success-count element and then called updateProgress(), which overwrote it with a single plain Successful: N line. Every explanation of why a run produced no results was lost this way. updateProgress() now leaves that element alone once a run has completed.getSeoAnalysis() never checked the HTTP status, so a 401 login wall or a 404 came back with a plausible-looking percentage. That is worse than reporting nothing, because the report looks valid. Non-2xx responses now report the status, and a 401/403 points at the Access link option.https://example.com and https://example.com/ address one resource but are different strings, so the homepage consumed two slots out of Max URLs and appeared twice in results and sitemaps. URLs are now canonicalised (default port dropped, empty path becomes /, scheme and host lowercased, fragment discarded) before deduplication. Query strings and path case are preserved, since ?page=2 is genuinely a different page and many servers treat paths case-sensitively.URL: result carrying SEO details is several KB, well beyond one chunk. Both readers now buffer partial lines, as the sitemap reader already did.max_execution_time and were killed mid-stream, leaving the progress bar stuck with no END marker.fetchSingleUrlMetaData()'s error return omitted images and _html, so the caller's array access raised a warning that the global error handler converted into an exception, discarding the row instead of showing its error.updateProgress() still assigned the Bootstrap classes removed in the 3.3.0 Tailwind migration (bg-success, bg-danger, bg-info, progress-bar-striped), so the bar had no state colour.npm run build:icons failing on a clean install — it copied node_modules/lucide/dist/umd/lucide.min.js, a path lucide 1.7.0 declares in package.json but does not ship. Fixed by updating lucide.date('F j, Y').=, +, - or @ are now prefixed with a single quote in both the JS export and the PHP sitemap generator. Crawled page titles are attacker-controlled and previously executed as formulas on open.curl_close() was skipped on every failure path, leaking the handle.?fetch and ?crawl endpoints and their fetchMetaData() / crawlWebsite() implementations — unreachable, since the UI only calls the -stream variants (~300 lines).setupSeoTableHeaders() and removeSeoTableHeaders(); all four table-header functions now share two helpers.clean-css-cli, unused since Tailwind CLI took over minification.Proper Hierarchy, Total Headings, and Empty Headings columns.maxPossibleScore calculation (was 70, now correctly 65 based on actual maximum scores).exportToCsv() function from JavaScript.npm run build to compile Tailwind CSS, minify JS via Terser, copy Lucide icons, and create production zip.getSeoAnalysis() now reuses already-fetched HTML instead of making a redundant second HTTP request per URL.php_errors.log for server-side debugging.pathinfo(null) deprecation warning on PHP 8.1+ — parse_url() calls now use ?? '' null coalescing to prevent null parameter warnings.trim(parse_url(..., PHP_URL_PATH), '/') passing null to trim().catch (Exception) blocks changed to catch (Throwable) to properly catch PHP fatal errors and type errors.set_error_handler breaking DOMDocument — Error handler now respects the @ suppression operator so HTML parsing warnings don't crash the process.seo-analysis-stream endpoint — The Page Check tab's backend handler was missing; added streaming handler.example.com → www.example.com) now crawl correctly. Added normalizeDomain() helper and redirect resolution before crawling..form-check selectors.mb_strlen fallback — Falls back to strlen if mbstring extension is not available on the server.src/input.css — Tailwind CSS source file with custom component classes.CHANGELOG.md — Version history tracking.npm run build:zip — Packages production files into a versioned zip after build.normalizeDomain() PHP helper for www-agnostic domain comparison.seo-analysis-stream PHP endpoint for the Page Check tab.width=device-width and initial-scale=1. Flags accessibility harm if zoom is disabled.<meta> tags, now correctly searches <link> elements. Added absolute URL format validation.<head> (iframe, img, div) that cause Google to stop parsing.terser (JS) and clean-css-cli (CSS) for production minification.