OneAndOneIs2

Mon, Nov 26, 2007

Damage, and routing round it

There's a phrase that's been around for some time: The Internet interprets censorship as damage and routes around it

Firewalls have been really, REALLY hacking me off lately. Heavy-handed "lock down everything" attitudes have always gotten on my nerves. Today, I couldn't take any more: There was a page I *had* to get a look at, and the bloody firewall refused point-blank to let it.

So.. The problem: Direct access of a page is impossible

Resources: A webserver that can be accessed only via FTP, a certain amount of geeky knowledge

Limitations: Too damn busy to learn any programming languages properly

The solution: FTP a bit of PHP that uses wget to grab the web page you're after and save it as a new page on your own webserver. Simple, crude, primitive, etc. But what the hell, it worked. I got a huge sense of triumph as I watched the blocked page appear on my screen :)

<head>
<meta http-equiv="REFRESH" content="0;
url=http://www.oneandoneis2.org/file.html">
<title>Mirror, Mirror</title>
</head>
<body bgcolor="#ffffff" vlink="#0000ff" alink="#ff0000">

<?php
$foo = system('wget webpage -O ~/public_html/file.html',$output);
echo "<p>Working on it</p>";

?>


</body>

Somebody with more time to learn PHP than me could make this much more all-singing and all-dancing. And I daresay that setting up a proper proxy server on my web page could make the whole thing redundant. But for ten minutes work with nothing but a text editor and the default Windows FTP client, I reckon that's not bad :)

11 comments • Categories: Omni, Rant, Technology, My Life

Comments:

Comment from: Al [Visitor]

· http://eggfriedrice.com

There are times when a bit of geek comes in handy. I'm at a university in Edinburgh (no not that one, one of the others) and within the first week I was cheesed off with it's firewall. I couldn't even get my webmail because it runs on a weird port and pretty much all ports are blocked, don't even try port 25!

So now I have an SSH tunnel over port 443 to my Debian box set up using Portable Apps Putty and from there the world is mine! Well, IMAP, SMTP and SSH at least...

I've even got as far as remembering how to write a Windows batch file from the depths of my memory so that I can start the tunnel, then Thunderbird and Firefox. Only downside: I've filled my home directory with Portable Apps...

Glad to hear the teaching is going well (mostly), I'm glad it's working out at least a little!

Permalink 26/11/07 @ 18:27

Comment from: alison [Member] · http://www.creativehedghog.com

wait, so where is the bit that has the url of the blocked page? This looks like a handy script, I'd like to understand it a bit more.

Permalink 27/11/07 @ 03:58

Comment from: Ginny [Visitor]

· http://iamgenevieve.wordpress.com

Must have been an important page if you went to all that to get a look at it...

Thankfully, most of the things I want to look at at college are unblocked. But some friends can't go more than five clicks without coming up against a firewall.

I'd show them this page and see if that worked. But I haven't got a clue what the hell it means. And I don't think they would either :-P

Permalink 27/11/07 @ 12:03

Comment from: oneandoneis2 [Member] · http://geekblog.oneandoneis2.org/

Al - sounds a handy tactic, somebody should write a guide on doing that ;)

Sadly, even port 443 is firewalled where I am. I'm actually surprised I get even FTP..

Alison: The bit near the bottom that says "webpage" in italics should be replaced with the full URL. And the bit at the top that redirects to a oneandoneis2.org address would also need to be edited a bit if you wanted to use the script ;)

It's not nearly perfect as a remote file grabber - it only grabs the page, no images or anything else. But it works for what I need it for, so...

Permalink 27/11/07 @ 14:09

Comment from: alison [Member] · http://www.creativehedghog.com

groovy, thanks Dominic.

Permalink 27/11/07 @ 23:26

Comment from: andrewtheart [Member]

· http://www.freewebs.com/andrewtheart/

I could see the value in this for viewing block webpages in a static fashion - absolutely.

If you haven't already done this, you should make a file called "webpage.html" and use the following code I just cooked up to submit a variable to your nifty little wget script using a simple HTML form and php POST -

Click here to see the code I made for you - Enjoy.

http://stein.freetzi.com/dominicwget.html

Permalink 28/11/07 @ 02:05

Comment from: oneandoneis2 [Member] · http://geekblog.oneandoneis2.org/

Ooo, that's definitely a very cool refinement!

Cheers Andrew!

Permalink 28/11/07 @ 11:27

Comment from: andrewtheart [Member]

· http://www.freewebs.com/andrewtheart/

No problem, glad to help!

You should look into the recursive (-r) switch of wget, which would download all images/content/non-HTML entities from the web page you specify.

You're slowly building a primitive proxy from the ground up (=

Permalink 28/11/07 @ 17:12

Comment from: oneandoneis2 [Member] · http://geekblog.oneandoneis2.org/

Hmm.. could do.. but then I'd have to set up something to delete the downloaded files after a while to stop the drive from filling up.. and I might have to get sed involved to redirect the image URLs to my mirrored copies - or does wget automate that too..?

I hacked your HTML a bit to make it standards-compliant (I tend to be a bit OCD about getting the green tick from my HTML validator extension ;o) so it's now:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"

"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">

<html>

<head>

<title>wget-based mirroring frontend</title>

</head>

<body>

<form action="access.php" method="post">

<h1>Webpage URL:<br />

<input name="url" size="35" type="text" /><br />

<input value="Load Page" type="submit" /></h1>

</form>

</body>

</html>

And preserved for posterity, the access.php bit is:

<head>

<meta http-equiv="REFRESH" content="0;

url=http://www.oneandoneis2.org/file.html">

<title>Mirror, Mirror</title>

</head>

<body bgcolor="#ffffff">

<?php

$foo = system('wget '.$_POST["url"].' -O ~/public_html/file.html',$output);

echo "<p>Working on it</p>";

?>

</body>

'tis definitely a most cool bit of scripting :o)

Permalink 28/11/07 @ 20:02

Comment from: andrewtheart [Member]

· http://www.freewebs.com/andrewtheart/

wget automatically makes *relative* links to any type of content (pictures, etc) from the web page you specify when using the recursive switch, so it should work quite nicely actually. No sed or awk hacking necessary :P

I encourage you to visit this page for some useful wget command examples

http://linuxreviews.org/quicktips/wget/

Permalink 28/11/07 @ 23:37

Comment from: Johan [Visitor]

For times like these, there's also Site unblocked (http://siteunblocked.info/) but that's an interesting solution you've got there. :)

Permalink 05/12/07 @ 11:52