When I decided to migrate to this new blog I was confronted by a very important issue: how do I make sure that the (already modest) traffic going to http://www.detronizator.org/* would be redirected to this new URL?
301 Redirection
I started searching and I found this article by Danny Tuppeny on how to use a Google App Engine application to do the trick.
So, what I did was to take his code and started putting it in place for me at http://redirector.ivandemarino.me. But because I’m a Software Developer that likes to make elegant stuff, I noticed that quite few things were missing:
- It didn’t use any Memcache, so Destination URL were always regenerated
- Logging was poor
- Was not really configurable
- Destination URL were not Checked (i.e. test if it exists) before Redirection
Introducing gredirector#
gredirector is an App Engine project that lets you set up your personal HTTP Redirect HUB, with a lot of emphasys on:
- Speed
- Configurability (but it could be pushed a even more)
- Logging
It’s a very very simple application, but I think serves well it’s purpose. See here the main code:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
| # /main.py
# ...
# Main Request Handler
class MainHandler(webapp.RequestHandler):
def get(self):
# Perform redirect
url = get_redirect_url(self.request.url);
# Check that we were able to build a URL and that this URL actually exists
if url and (check_url_exists(url) if config.CHECK_URL_EXISTANCE else True):
logging.info("Redirecting URL '%s' to URL '%s'" % (self.request.url, url) );
self.redirect(url, permanent=True);
else:
# Log that we didn't know what this was, and redirect to a good default
logging.error("Unable to Redirect URL '%s'" % (self.request.url) );
if config.ERROR_EMAIL_ACTIVE:
mail.send_mail_to_admins(
sender=config.ERROR_EMAIL_SENDER,
subject=config.ERROR_EMAIL_SUBJECT,
body=config.ERROR_EMAIL_BODY + self.request.url
);
self.redirect(config.DEFAULT_URL, permanent=True);
# ...
|
As you can see, it works in few simple steps:
- Generate a Destination URL
- If it’s Valid and it Exists (optional) do 3. else 4.
- Redirect
- Log error and redirect to Default URL
URL generation#
To generate the URL, it uses the mapping from the config.py file. Here is a portion of an example configuration:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
| # /config.py
# ...
# Default URL: if no Redirect URL can be generated, traffic will be diverted here
DEFAULT_URL = 'http://default.new-domain.com/'
# Mapping of Source URLs to Target URLs.
# You can choose to map in 2 ways:
# 1) to the new URL maintaining the same Path (i.e. 'www.old.com/p/a/t/h/index.html -> www.new.com/p/a/t/h/index.html')
# 2) or to discard the Path (i.e. 'www.old.com/p/a/t/h/index.html -> www.new.com')
# Format:
# URLS = { 'Old Domain': ('New Domain', 'Discard URL Path part and Redirect to Root?'), ... }
URLS = {
'%s.appspot.com' % (os.environ['APPLICATION_ID']) : ('www.new-domain.com', False),
'www.old-domain.com' : ('www.new-domain.com', False),
'downloads.old-domain.com': ('downloads.new-domain.com', False),
'stuff.old-domain.com': ('stuff.new-domain.com', True),
};
# ...
|
The code that actually generates the URL is:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
| # /main.py
# ...
# Generates the URL to redirect to
def get_redirect_url(url):
# Search the Memcache
memcachekey = "get_redirect_url-" + url;
result = memcache.get(memcachekey) if config.MEMCACHE_ACTIVE else None;
# Nothing in Memcache
if ( result == None ):
logging.debug("Redirect for URL '%s' not in Memcache" % (url) );
scheme, netloc, path, query, fragment = urlparse.urlsplit(url);
# Discard any port number from the hostname
netloc = netloc.split(':', 1)[0];
# Fix empty paths to be just '/' for consistency
if path == '':
path = '/';
# Check if we have a mapping for this domain
if netloc in config.URLS:
# Grab the redirect info tuple
redirect_info = config.URLS[netloc];
# Root redirects
if redirect_info[1]:
result = 'http://' + redirect_info[0] + '/';
logging.debug("Redirecting to Root: " + result);
# Paths
else:
result = urlparse.urlunsplit(['http', redirect_info[0], path, query, fragment]);
logging.debug("Redirecting to Precise Path: " + result);
# No mapping, so return None
else:
logging.debug("No Mapping found for: " + url);
result = None;
# Store the result in Memcache for config.MEMCACHE_EXPIRES_IN_SECONDS
memcache.set(memcachekey, result, time=config.MEMCACHE_EXPIRES_IN_SECONDS);
return result;
# ...
|
As you can see, the results are stored in Memcache and reused until expired. A major speed improvement if you ask me.
How do you check the URL?#
Yes, I also said that I added a check to see if a Destination URL actually exists, before serving the Redirect. Well, App Engine provides a very nice package, google.appengine.api.urlfetch, and I used it this way:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
| # /main.py
# ...
# Check if a URL exists, trying a URL Fetch
def check_url_exists(url):
# Search the Memcache
memcachekey = "check_url_exists-" + url;
result = memcache.get(memcachekey) if config.MEMCACHE_ACTIVE else None;
# Nothing in Memcache
if ( result == None ):
logging.debug("Existence of URL '%s' not in Memcache" % (url) );
try:
response = urlfetch.fetch(url, allow_truncated=True, deadline=10); # Wait as much as possible: 10 sec
if ( response.status_code == 200 ):
logging.debug("Verified URL '%s' existence" % (url));
result = True;
else:
logging.error("Unable to Verify URL '%s' existence" % (url));
result = False;
except:
# In case of InvalidURLError or DownloadError
logging.error("Exception while Verifying URL '%s' existence" % (url));
result = False;
# Store the result in Memcache for config.MEMCACHE_EXPIRES_IN_SECONDS
memcache.set(memcachekey, result, time=config.MEMCACHE_EXPIRES_IN_SECONDS);
return result;
# ...
|
Try it out#
This is an App Engine application, so just clone/fork it from http://github.com/detro/gredirector, personalize the config.py and push it out on. If you are not a complete newbie to App Engine, it should be really really simple.
301 Happy Redirecting ;-).