google.com/goto: read Location with HEAD

Google Search no longer always puts the final URL in the result href. Organic results point at https://www.google.com/goto?url=CAES…. This is not the old google.com/url?q=… wrapper: the parameter is not an encoded URL. You cannot decode it.

Google still has to know the destination. It puts it in the HTTP Location header when you request /goto. That is the read we use.

Examples in curl, Python, and Node: github.com/automdev/google-goto-url-fix. Autom’s Google Search API runs that hop on the server.

The rollout and why Google did this: google.com/goto: Google's anti-scraping update.

One sentence

Do not follow the redirect. Read Location.

GET /goto?url=CAES…  HTTP/1.1
Host: www.google.com

HTTP/1.1 302 Found
Location: https://www.linkedin.com/in/satyanadella

HEAD asks for headers only. GET with redirect: "manual" (or allow_redirects=False) is the same read. Google sometimes returns 402 (or another status) with a Location: read the header even when it is not a 3xx.

curl

curl -sI --max-redirs 0 \
  -H 'Referer: https://www.google.com/search' \
  'https://www.google.com/goto?url=CAES...' \
  | grep -i '^location:'

-I is HEAD. Without -L, curl does not follow. The Location: line is the real URL.

Same read with GET:

curl -sD - -o /dev/null --max-redirs 0 \
  -H 'Referer: https://www.google.com/search' \
  'https://www.google.com/goto?url=CAES...' \
  | grep -i '^location:'

Python

import requests

goto = "https://www.google.com/goto?url=CAES..."
r = requests.head(
    goto,
    allow_redirects=False,
    headers={"Referer": "https://www.google.com/search"},
    timeout=10,
)
print(r.status_code, r.headers.get("Location"))

If HEAD is empty, retry with GET and no follow:

r = requests.get(goto, allow_redirects=False, timeout=10)
print(r.headers.get("Location"))

Node

const res = await fetch(goto, {
  method: "HEAD",
  redirect: "manual",
  headers: { Referer: "https://www.google.com/search" },
});
console.log(res.status, res.headers.get("location"));

In a browser, a manual 3xx can show up as opaqueredirect and hide the header. In Node, the headers are visible.

Why not decode CAES?

The blob is not a public URL protobuf. It is an opaque reference. Only Google knows the target, and it puts it in Location on the hop. Rebuilding a URL from a visible name ("LinkedIn · Satya Nadella") is a bad idea.

The href is /goto. The page still knows the URL.

Google cannot hide the destination completely. It needs the real URL to render the SERP: the visible domain, the favicon, the attribution block. Those values are still on the page, usually in a data script at the end, and in an internal store (W_jd) that the SERP JavaScript reads.

That is why, in some browsers, the links "clean themselves up" after you move the mouse: a script reads W_jd and rewrites each /goto href with the real URL. It only covers organic results under #rso div[data-rpos] that have an attribution block. AI Overviews stay on /goto. It needs a live page and that first mouse move. A plain headless fetch of the HTML does not fire it. Chrome often leaves the /goto hrefs in place.

Another leak sits on the "Translate this page" link. If the UI language does not match the result language, Google adds that control and the clean URL is often on it. Latin (Accept-Language: la) shows up almost every time, because almost no pages are in Latin.

Those are explanations of what the page still contains. They are brittle: Google can rename W_jd, change the script, or drop the translate control. They also do not give you a stable API field.

The durable read is still Location on /goto. No live DOM, no mouse event, no language trick.

At Autom

The Google Search API already resolves /goto and returns the final URL in organic_results[].link.

curl -s https://api.autom.dev/v1/google/search \
  -H "x-api-key: YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{"query":"linkedin.com/in satya nadella","gl":"us","hl":"en"}'

Code and README: github.com/automdev/google-goto-url-fix.