Debug a web page error from the command line

One way to debug a web server is by using the wget command-line program.
Register or Login to like
Digital creative of a browser on the internet

Sometimes when managing a website, things can get messed up. You might remove some stale content and replace it with a redirect to other pages. Later, after making other changes, you find some web pages become entirely inaccessible. You might see an error in your browser that "The page isn't redirecting properly" with a suggestion to check your cookies.

One way to debug this situation is by using the wget command-line program, with the -S option to show all server responses. When using wget for debugging, I also prefer to save the output to some temporary file, using the -O option, in case I need to view its contents later.

$ wget -O /tmp/test.html -S http://10.0.0.11/announce/
--2021-08-24 17:09:49--  http://10.0.0.11/announce/
Connecting to 10.0.0.11:80... connected.
HTTP request sent, awaiting response... 
 
HTTP/1.1 302 Found
 
Date: Tue, 24 Aug 2021 22:09:49 GMT
 
Server: Apache/2.4.48 (Fedora)
 
X-Powered-By: PHP/7.4.21
 
Location: http://10.0.0.11/assets/
 
Content-Length: 0
 
Keep-Alive: timeout=5, max=100
 
Connection: Keep-Alive
 
Content-Type: text/html; charset=UTF-8
Location: http://10.0.0.11/assets/ [following]
--2021-08-24 17:09:49--  http://10.0.0.11/assets/
Reusing existing connection to 10.0.0.11:80.
HTTP request sent, awaiting response... 
 
HTTP/1.1 302 Found
 
Date: Tue, 24 Aug 2021 22:09:49 GMT
 
Server: Apache/2.4.48 (Fedora)
 
X-Powered-By: PHP/7.4.21
 
Location: http://10.0.0.11/announce/
 
Content-Length: 0
 
Keep-Alive: timeout=5, max=99
 
Connection: Keep-Alive
 
Content-Type: text/html; charset=UTF-8
Location: http://10.0.0.11/announce/ [following]
--2021-08-24 17:09:49--  http://10.0.0.11/announce/
Reusing existing connection to 10.0.0.11:80.
.
.
.
20 redirections exceeded.

I've omitted a lot of repetition in this output. By reading the server responses, you can see that http ://10.0.0.11/announce/ redirects immediately to http ://10.0.0.11/assets/, which then redirects back to http ://10.0.0.11/announce/. And so on. This is an endless loop and wget will exit after 20 redirections. But armed with this debugging information, you can fix the redirects and avoid the loop.

What to read next
photo of Jim Hall
Jim Hall is an open source software advocate and developer, best known for usability testing in GNOME and as the founder + project coordinator of FreeDOS.

1 Comment

Oooooooooooooh Jim what a great idea! I can think of about 200 times when I wish I had thought to use wget as you suggest.

Thank you thank you thank you!

Creative Commons LicenseThis work is licensed under a Creative Commons Attribution-Share Alike 4.0 International License.