Oct
10
Filed under: Asides | Tags: | October 10th, 2003

Wildcard DNS and Sub Domains

What follows is what I consider to be best practice for my personal sites and a guide for those who wish to do the same. Months ago I dropped the www. prefix from my domain in part because I think it’s redundant and also because I wanted to experiment with how Google treated valid HTTP redirect codes. The experiment has been a great success. Google seems to fully respect 301 Permanent Redirects and the change has taken my previously split PageRank has been combined and now I am at 7. There are other factors that have contributed to this, of course, and people still continue to link to my site and posts with a www. (or worse) in front of it, but overall it just feels so much cleaner to have one URI for one resource, all the time. I’m sure that’s the wrong way to say that, but the feeling is there nonetheless.

Now for the meat. What’s a good way to do this? Let’s look at our goals:

  • No links should break.
  • Visitors should be redirected using a permanent redirect, HTTP code 301, meaning that the address bar should update and intelligent user agents may change a stored URI
  • It should be transparent to the user.
  • It should also work for mistyped “sub domains” such as ww. or wwww. (I still get hits from Carrie’s bad link)

So we need a little magic in DNS and in our web server. In my case these are Bind and Apache. I am writing about this because at some point the code I put in to catch any subdomain stopped working and while I reimplemented it I decided to write about what I was doing. This method also works with virtual hosts on shared IPs where my previous method did not.

In Bind you need to set up a wildcard entry to catch anything that a misguided user or bad typist might enter in front of your domain name. Just like when searching or using regular expressions you use an asterisk (or splat) to match any number of any characters the same thing applies in Bind. So at the end of my zone DB file (/var/named/photomatt.net.db) I added the following line:

*.photomatt.net. 14400 IN A 64.246.62.114

Note the period after my domain. The IP is my shared IP address. That’s all you need, now restart bind. (For me /etc/init.d/named restart.)

Now you need to set up Apache to respond to requests on any hostname under photomatt.net. Before I just used the convinence of having a dedicated IP for this site and having the redirect VirtualHost entry occur first in my httpd.conf file. That works, but I have a better solution now. So we want to tell Apache to respond to any request on any subdomain (that does not already have an existing subdomain entry) and redirect it to photomatt.net. Here’s what I have:

<VirtualHost 64.246.62.114>
DocumentRoot /home/photomat/public_html
BytesLog domlogs/photomatt.net-bytes_log
ServerAlias *.photomatt.net
ServerName www.photomatt.net
CustomLog domlogs/photomatt.net combined
RedirectMatch 301 (.*) http://photomatt.net$1
</VirtualHost>

The two magic lines are the ServerAlias directive which is self explanitory and the RedirectMatch line which redirects all requests to photomatt.net in a permanent manner.

There is a catch though. The redirecting VirtualHost entry must come after any valid subdomain VirtualHost entries you may have, for example I have one for cvs.photomatt.net and I had to move that entry up in the httpd.conf because Apache just moves down that file and uses the first one it comes to that matches, so the wildcard should be last.

That is it, I’m open to comments and suggestions for improvement.

Possibly related posts:

  1. Random Redirect Plugin I just updated theĀ  Random Redirect plugin, with two extra...
  2. My WordPress Tattoo – Baker’s Hours Ed Morita writes about his new permanent WordPress tattoo. I...
  3. Velocity and the Bottom Line Velocity and the Bottom Line. How performance touches everything on...
  4. Blo.gs Lives On Do you guys remember Blo.gs? In addition to being a...
  5. Olympians on WP Shawn Johnson, a gold medalist and WordPress-powered blogger. Dreamhost says...

48 Responses

  • Cody | October 10th, 2003 @ 1:31 pm | Reply

    I’m glad I know a guy like you. Because I don’t understand a damn thing you just wrote and I’m glad there are people who do.

  • mriannnnnnn | October 12th, 2003 @ 9:54 am | Reply

    LOL

    well as one who does understand what was written. I can appreciate Cody’s comment. I can’t really talk tech like this to my friends without them getting cross-eyed either.

  • Matt | October 12th, 2003 @ 11:54 am | Reply

    Well hopefully someone appreciated it. Maybe I should tone down some of the posts.

  • Cody | October 12th, 2003 @ 9:34 pm | Reply

    No! Write whatever motivates you to write. Don’t go changin’.

  • Scott | December 2nd, 2003 @ 12:28 pm | Reply

    Hey, I’m looking to do something just like this but with one critical difference. I do not want the address basr updated, because I need to look at the original request and respond intelligently in my application code to the subdomain requested. Is there a way to change the VirtualHost directive so it will behave how I want?

    Thanks.

  • Scott | December 2nd, 2003 @ 12:31 pm | Reply

    I want to do basically the same thing you are describing but with one critical difference. I don’t want to change the address in the address bar because I need to respond intelligently in my application based on the original subdomain requested. Basically, I don’t want to use it to capture mistyped subdomains but instead to maintain virtual subdomains without having to create individual virtual hosts and dns entries for each.

    Do you know how I might do this?

  • Georg Zimmer | December 11th, 2003 @ 8:18 am | Reply

    Is this line correct? It did not work for me.

    *.photomatt.net. 14400 IN A 64.246.62.114

    This worked for me, i added it after all the other stuff in the zone file:
    * A 123.45.67.89

    Is that bad, or is there a reason for all the other stuff in the first line?

  • Georg Zimmer | December 11th, 2003 @ 8:20 am | Reply

    oops, i meant * IN A 123.45.67.89

  • Georg Zimmer | December 11th, 2003 @ 8:23 am | Reply

    DUH, i forgot the period after the domain. never mind

  • Matt | December 11th, 2003 @ 3:27 pm | Reply

    Georg: don’t feel bad, that’s a pretty common problem. Glad to hear it worked out for you.

  • Jason Schock | December 22nd, 2003 @ 5:30 pm | Reply

    Thanks. This helped deduce a solution to redirecting .net requests to .com. Searched forever til I found this. Nice work.

  • Jake | January 3rd, 2004 @ 8:48 am | Reply

    Hi Matt, great info on how to redirect http://www.domain.com to domain.com – but I’m not sure how to do it the other way — I want to have my websites do a 301 redirect from domain.com to http://www.domain.com (for some reason, I like that way better). Here’s the difficult part – I also use wildcard subdomains (handled by a CGI script to offer up the appropriate content) and don’t want those to be 301 redirected – just domain.com to http://www.domain.com. Any ideas? I’m taking a guess below with this example:RedirectMatch 301 http://domain.com(.*) http://www.domain.com$1Would that work? I’m always paranoid about messing around with server config files…Thanks!-Jake

  • Jake | January 3rd, 2004 @ 8:51 am | Reply

    Oops, let me try to put the example on its own line:

    RedirectMatch 301 http://domain.com(.*) http://www.domain.com$1

    …would that work for what I want to do in #12 above?

  • Steve | February 1st, 2004 @ 2:36 pm | Reply

    I was just trying to do the same thing…but it seems that I don’t have access to the httpd.conf on my shared hosting plan. Any idea what to do…?

  • Andy | March 20th, 2004 @ 3:28 pm | Reply

    It works! Exactly what I needed. – Thanks

  • James | April 14th, 2004 @ 9:05 pm | Reply

    I have been looking in to having wildcard domains on my site. But I have a question about this method. Will this actually change the URL in the browser? I need the URL in the addressbar to stay on the wildcard subdomain. What I would like it for the 404 page to be displayed on the screen but the URL in the address bar to remain on the ‘incorrect’ subdomain.
    Thanks,
    James

  • Thijs | June 7th, 2004 @ 2:57 pm | Reply

    Hi,

    My hosting-company offers cPanel to manage your web site, but I don’t have access to the httpd.conf nor to the DNS settings. When I visit xx.domain.nl, I get a standard cPanel-page that tells me: “There’s no website configured at this address”. Is it possible to redirect all those requests to the main domain? (probably with .htaccess)

  • Brad | September 8th, 2004 @ 9:25 am | Reply

    Hey!

    I have been looking for how to do this for a while with CPanel, and your instructions worked!

    Thanks!

    One note for me though, I didn’t need the RedirectMatch 301 (.*) http://photomatt.net$1 line to make it work, it worked with just adding *.hl2x.com. to the DNS zone, and the *.hl2x.com to the server alias in the httpd.conf file. Restarted the services, and it worked! Thanks again!

    -Bizzar

  • mas222 | October 12th, 2004 @ 6:12 pm | Reply

    This is great info Matt!! Thanks for helping all us numbskulls!

    I have a question that is much like Scott’s above, but unfortunately I don’t think he completed the thought and I was not able to contact him directly to confirm.

    He wrote:
    *****************************************************
    I want to do basically the same thing you are describing but with one critical difference. I don’t want to change the address in the address bar because I need to respond intelligently in my application based on the original subdomain requested. Basically, I don’t want to use it to capture mistyped subdomains but instead to maintain virtual subdomains without having to create individual virtual hosts and dns entries for each.
    *****************************************************
    I would like to extend the question to add: parse the subdomain out of the request (as sent to the webserver) then search a database for a match on the subdomain string. If a match is found, use that string as the user key and programmatically rebuild a replicated webpage for that user, populating all the fields with the data which that user has previously entered.

    This is a critical function I am looking for. If anyone can do this, I would like to further correspond with you.

  • Fong | November 12th, 2004 @ 1:34 pm | Reply

    mas222: This is what I did to accomplish your extended question. I used .htaccess to forward all the subdomain to one particular folder “members” within my public_html folder. I then use php to parse out the subdomain and then do a query in my database to verify if it’s a valid user or not. If it’s a valid user, do the other query stuff else displays an error or the default domain page.

    Here’s my code to get the subdomain name:

    < ?php

    //gets the http request string
    $serverhost = explode('.',$_SERVER["HTTP_HOST"]);

    //get the subdomain
    $subdomain = $serverhost[0];

    ?>

    pretty simple, good luck

  • Fong | November 12th, 2004 @ 1:37 pm | Reply

    < ? p h p

    / / get server http request info
    $serverhost = explode( '.' , $_SERVER["HTTP_HOST"]);

    / / get subdomain
    $subdomain = $serverhost[0];

    ? >

    sorry it won’t display my code on the previous post

  • blake | January 16th, 2005 @ 4:11 pm | Reply

    Exacccctly what I needed to close out my old site and redirect effectively; I found your solution through Google, thanks!!

  • Julian | January 31st, 2005 @ 4:05 pm | Reply

    Hey, everyone.

    I’m glad that someone finally got this idea out into the public. I have similar needs to that of everyone else that posted here.
    What I’d like to do is take my subdomain and point it at a folder with the subdomain value (i.e. foo.bar.com –> bar.com/special/foo).

    I’ve tried to figure out how to do this on my own, and I have yet to find a solution. Now, I read that a .htaccess might do the trick, but the problem is that my .htaccess is already riddled with so many redirects that I don’t think it can take one more.

    If anyone knows how to do this, please get back to me. Thanks.

  • Jason | March 10th, 2005 @ 9:11 pm | Reply

    Julian,
    You’re really going to have to use Redirects in .htaccess (or httpd.conf if you control it) if you want to do that.

    foo.bar.com/some/directory/hello.php -> bar.com/special/foo/some/directory/hello.php

    Otherwise your entire site would have to be dynamic and use a common header include that checked the subdomain on every page request and redirect with something like <?php header(”Location: http://bar.com/special/foo“) ?>; And if in the above example /some/directory/hello.php didn’t exist (because it’s under /special/foo/some/directory), your 404 php handler would also have to include the common header to do the check.

    Much better to just use the apache builtin redirects.

  • ogtool | April 14th, 2005 @ 3:11 am | Reply

    Julian,

    The other option is to use a php’d auto prepended file which saves you manually triggering the command each time. In the prepend file, just have it figure out if there is a subdomain that you want to process (exploding HTTP_HOST returns a 3 part array and part 1 isn’t www) then do a header Location.

    Hopefully that makes sense.

    C.

  • z | May 3rd, 2005 @ 1:32 pm | Reply

    Hi.

    Thanks a lot for providing this page. I had pretty much figured it out, but reading your advice helped it all came together.

    groovy

  • mike bailey | May 24th, 2005 @ 7:44 pm | Reply

    Thanks for the tutorial, i just set that up for testing at a project i’m working on. Now to look into it’s uses with mod_rewrite with a project i’m working on.

  • Joe | August 10th, 2005 @ 9:34 am | Reply

    I like the straightforward post. It gave me confidence with a project I’m doing that will need wildcard dns and sub-domains. (Hence my google search of “wild card dns sub-domain”.. you’re #4! haha)

  • Box | November 16th, 2005 @ 2:42 pm | Reply

    Hi,

    for those that haven’t solved the http://domain.com redirecting to cpanel page, you just have to add like this:

    DocumentRoot /home/photomat/public_html
    BytesLog domlogs/photomatt.net-bytes_log
    User photomat
    Group photomat
    ServerAlias photomatt.net *.photomatt.net
    ServerName http://www.photomatt.net
    CustomLog domlogs/photomatt.net combined
    RedirectMatch 301 (.*) http://photomatt.net$1

    on ServerAlias just add your domain without the (*) thing and restar httpd :D done..

    Thank you Matt.

  • Chirag | March 30th, 2006 @ 10:10 pm | Reply

    Found this page googling for “apache wildcard subdomains.” Works perfectly for me. Thanks!

    1. I only changed the bind config (/var/named/mydomain.com.db) as you said. No changes to my Apache conf.
    2. I have a server with cPanel/WHM. Restarted bind from WHM. That’s all I had to do.

    *.mydomain.com/file.php gets automatically forwarded to mydomain.com/file.php and the address bar still shows *.mydomain.com/file.php. Perfect!

    Here’s a little code for those who might be using PHP on how to get the subdomain name:

    $host = str_replace(array(”www.”, “.mydomain.com”), “”, strtolower(trim($_SERVER["HTTP_HOST"])));

    if($host == “mydomain.com”)
    echo “Welcome to My Domain.com!”;
    else if(strpos($host, “.”) === false)
    echo “Dynamic subdomain: $host”;
    else
    echo “Invalid Subdomain!”

  • Matt Whitlock | May 3rd, 2006 @ 3:24 pm | Reply

    It’s a little less overhead to do it like this:

    Redirect permanent / http://photomatt.net/

    That way, you don’t have to involve the whole big regular expression parsing engine. Any URL path beginning with “/” (that would be any URL for that virtual host at all) will be redirected to a URL beginning with http://photomatt.net/. So the URL http://www.photomatt.net/foo/bar.html would be redirected to http://photomatt.net/foo/bar.html.

  • rllq | September 13th, 2006 @ 9:41 pm | Reply

    good information. it was helpful in setting up our wordpress mu (multiple user) version

  • Roy | October 19th, 2006 @ 1:38 am | Reply

    Matt, any problems with search engines and duplicate content using your method? Seems like it could be a potential problem.

  • Corey | June 27th, 2007 @ 7:46 pm | Reply

    I know this is old, but I just have to give the Lighttpd equivalent to redirect like did in your httpd.conf since I know a lot of people are making the switch.

    $HTTP["host"] =~ “^www\.(.*)$” {
    url.redirect = ( “^/(.*)” => “http://%1/$1″ )
    }

  • George | November 18th, 2007 @ 4:57 pm | Reply

    Great tutorial!

  • bozhidar | April 23rd, 2008 @ 3:44 pm | Reply

    Actually it is on the spot for me, and it very nicely written THANKS ALLOT!! you saved me some time in configuration something :) please do continue the nice work :)

  • Kevin Paquet | May 2nd, 2008 @ 1:40 am | Reply

    It’s gonna take me a long long time to understand this, I’m a total noob. I want WP MU installed *sigh*

  • Rich Bowen | August 27th, 2008 @ 11:08 am | Reply

    You can’t put User and Group in a block and expect them to do anything useful, so it would be really helpful if you’d remove that from your example configuration, since it’s misleading people who are using this as an example configuration, and expecting that User directive to have an actual effect on the user permissions that the vhost runs with. Thanks.

    –Someone who provides user support on #apache.

  • Matt | August 30th, 2008 @ 11:11 am | Reply

    Thanks for the suggestion – this was just copied from my httpd.conf so it was doing something ,but it might have been cPanel specific. I’ve removed those two lines. Crazy that 5 years later people still use this.

  • Devon_O | March 1st, 2009 @ 6:44 pm | Reply

    As disappointed as I am by this fact, I just don't get it. :-(

Share your thoughts