doom'd net

still mucking around with the ol'computer


Moving From Wordpress to Static S3 Site

Using AWS s3 to host a web site is easy, and rather inexpensive. Also using S3, there is no code or database required, so not only is security better, but there is a lot less complexity than using Wordpress. If you have your own domain name, you can either use Nginx or Cloud Front with Route 53 to front end it. I chose the former as I didn’t want to give up control of my domain to AWS.

You’ll also need a static site generator, unless you want to write the HTML yourself. I started with Pelican, but switched to Hugo. Pelican just didn’t work the way I wanted it to. So far, except for a few minor issues, Hugo is working just fine.

NOTE The examples below will use the domain name domain.net. This will have to be updated to match your domain name.

Wordpress to Pelican

I’m sure there is a way to go strait to Hugo from Wordpress, but alas, I did not start out there.

First I exported the site ( using settings -> export ). Then using Pelican’s Wordpress importer:

$ pelican-import --wpfile -o content ~/Downloads/thedoom039dnet.WordPress.2022-03-13.xml

I’ll skip the rest of the Pelican stuff since I’m not actually using Pelican.

You’ll have to manually copy all your static files over. And the whole thing should be in the content directory of you Pelican folder.

In my case, that was ~/pelican.

Pelican to Hugo

First, install Hugo

$ brew install hugo

I copied my rst files from ~/pelican/content to a tmp convert directory, ~/hugo/convert.

Then using PelicanToHugo, I converted the rst files for Hugo:

$ cd ~/hugo/convert
$ python3 pelican-to-hugo.py

I converted them to markdown using pandoc:

$ for i in "*.rst" ; do  f=$(basename -s _hugo.rst $i) ; pandoc $i -r rst -t markdown -o ${f}.md ; done 

This will get 85% the way there. Now to handle the munged up Front Matter.

The title can be fixed with this:

$ or i in *.md ; do echo $i ;  sed '1 s/^#/title:/' $i > ~/hugo/content/posts/$i ; done

Once that is done, I just manually fixed the rest:

  • Make sure the whole header is valid yaml.
  • Remove Author, it’s not used.
  • Add three ‘-’s before and after Front Matter.
  • Change category to categories.
  • Fix other formatting
  • Add aliases for Wordpress and Pelican Slugs

When fixed, mine look like:

title: Some Blog Post 
date: 2019-06-08T12:14:00Z
categories:
  - Category 
tags:
  - tag 1 
  - tag 2 
  - tag 3 
slug: some-blog-post
aliases:
  - /2019/06/some-blog-post.html
  - /some-blog-post.html 

Another thing that needs to be fixed, is that all the image links need to have a ‘/’ added to them:

![Alt_test](images/image.jpg)

Needs to be changed to:

![Alt_test](/images/image.jpg)

After that, follow Hugo’s documentation for selecting a theme and setting up Hugo.

Useful Linux:

Be sure to set you base domain to your domain name. It will have to match the s3 bucket and Nginx settings.

AWS s3

NOTE: This assumes you already have an AWS account and know how to create and set up s3 buckets and ssh keys needed for management using the awscli command.

Using this guide, create an s3 bucket and enable it for web hosting. And then copy the contents of your Hugo public directory to the s3 bucket.

It should be the same name as your domain, in this case, domain.net That also has to be the base URL in your Hugo configuration.

$ cd ~/hugo
$ aws s3 sync public/ s3://domain.net

Nginx

I run Nginx on a small Lightsail instance, but you can run it anywhere.

First install Nginx and certbot:

$ apt-get install nginx-light certbot python3-certbot-nginx

Create a config file in /etc/nginx/sites-available/domain.net:

proxy_cache_path	   /var/lib/nginx/tmp/default/ levels=1:2 keys_zone=d_cache:10m max_size=500m inactive=60m use_temp_path=off;

server {
  server_name  domain.net www.domain.net;
  access_log  /var/log/nginx/domain.net-access.log  combined;
  error_log   /var/log/nginx/domain.net-error.log;
  set $bucket "domain-net.s3-website-us-east-1.amazonaws.com";
  sendfile on;

  location / {
    resolver 8.8.8.8;
    proxy_http_version     1.1;
    proxy_redirect off;
    proxy_set_header       Connection "";
    proxy_set_header       Authorization '';
    proxy_set_header       Host $bucket;
    # proxy_set_header       Host $host;
    proxy_set_header       Host $bucket;
    proxy_set_header       X-Real-IP $remote_addr;
    proxy_set_header       X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_hide_header      x-amz-id-2;
    proxy_hide_header      x-amz-request-id;
    proxy_hide_header      x-amz-meta-server-side-encryption;
    proxy_hide_header      x-amz-server-side-encryption;
    proxy_hide_header      Set-Cookie;
    proxy_ignore_headers   Set-Cookie;
    proxy_intercept_errors on;
    add_header             Cache-Control max-age=31536000;
    proxy_pass             http://$bucket; # without trailing slash
    # rewrite                ^/$ / last;
  }


# This configuration uses a 60 minute cache for files requested:
  location ^~ /cached/ {
    rewrite           /cached(.*) $1 break;
    resolver 8.8.8.8;
    proxy_cache            d_cache;
    proxy_http_version     1.1;
    proxy_redirect off;
    proxy_set_header       Connection "";
    proxy_set_header       Authorization '';
    proxy_set_header       Host $bucket;
    proxy_set_header       X-Real-IP $remote_addr;
    proxy_set_header       X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_hide_header      x-amz-id-2;
    proxy_hide_header      x-amz-request-id;
    proxy_hide_header      x-amz-meta-server-side-encryption;
    proxy_hide_header      x-amz-server-side-encryption;
    proxy_hide_header      Set-Cookie;
    proxy_ignore_headers   Set-Cookie;
    proxy_cache_revalidate on;
    proxy_intercept_errors on;
    proxy_cache_use_stale  error timeout updating http_500 http_502 http_503 http_504;
    proxy_cache_lock       on;
    proxy_cache_valid      200 304 60m;
    add_header             Cache-Control max-age=31536000;
    add_header             X-Cache-Status $upstream_cache_status;
    proxy_pass             http://$bucket; # without trailing slash
  } 

Set it as the default domain

$ cd /etc/nginx/sites-enabled
$ rm default
$ ln -s ../sites-available/domain.net default 

Then load the nginx config

$ nginx -t
$ systemctl reload nginx

Next, if you don’t already have a certificate, enable Lets encrypt

$ certbot --nginx -d doomd.net -d www.doomd.net

And with that, now your site is no longer on Wordpress and is hosted on s3, front-ended with Nginx and using Hugo as the content generator. It will be much more secure, faster, and more reliable.