2017-09-27 – Hotels, PageSpeed and AWS RDS

So I woke later than usual this morning after spending far too long staring at the laptop last night. Jumping in the shower was an experience – there was no way to adjust the temperature, so it was either hot or hot.

Nothing to report on the work front, same old, same old. Just getting on with stuff and making progress on the PageSpeed investigations. Seems like a number of our automated tests rely on specific URL paths in certain tags that are being re-written by PageSpeed. We’re slowly disabling the optimisations until we find a working solution. It certainly looks like most of the tests can be rewritten to work with the PageSpeed rewrites, but we don’t have the resources or the time to work through that at the moment.

Significant progress was made on the side project last night, and I got so carried away I neglected to write up my days thoughts. I now have a fully automated infrastructure build in AWS using Terraform and Ansible, including making my own AMIs for auto-scaling groups and an RDS Aurora Cluster for the backend. The only thing left is to get Ansible to play nice with Aurora and create the additional schemas and user accounts that my application requires.

Tonight, I am being messed around with the hotel Wifi, which seems to have a kink in the pipe. Mainly caused by Gamma Telecom according to this traceroute output;

~$ traceroute {target host}
traceroute to {target host} ({target ip}), 64 hops max, 52 byte packets
1 ( 5.687 ms 7.072 ms 4.213 ms
2 ( 47.813 ms 28.547 ms 23.928 ms
3 ( 46.187 ms 45.805 ms 46.369 ms
4 * * *
5 lonap-gw1.gammatelecom.com ( 2516.409 ms 4634.714 ms *
6 * * *
7 * ( 143.750 ms

And so it goes on… I’ve wasted so much time over the last few weeks with dodgy connectivity. So I’ve turned on my VPN tunnel provided by ProXPN, and hey presto, the problems virtually go away – I wish I’d thought of this before!

So Aurora now plays nicely with my Ansible code after some debugging on the bastion host. Now to get all the required updates into the main codebase and rebuild everything to test full functionality.

A good end to the day.


2017-09-25 – Frustrations and Woodworking

Frustrated that I missed posting over the weekend, but other than a family visit and a bunch of gardening, nothing much happened.

The family visit was nice, it was good to see my sisters and nieces again – we live a fair distance away, so don’t get to spend much time together.

Sunday was spent almost finishing off the work on our front garden, moving a number of small trees from one area to another, and shifting the best part of a ton of gravel to finish it all off.

We found an issue with our implementation of PageSpeed today, which was really frustrating – the website works fine when used manually, but the automated tests fail. So until we get more time to test it, PageSpeed has had to be disabled for now.

And this evening was back to working on the house. We’ve had a new window opener made to replace the old rotten one, so I needed to chisel out the recesses for the hinges – something I’ve not done for years, but I was pleased with the finished result 🙂

2017-09-22 – PageSpeed, window casements and Narcos

Finally the PageSpeed work is complete. Took a day or so longer than planned, but there were other things that got in the way.

We’ve had a new window casement built to replace one that was rotten, so the weekend will involve preparing and painting that, so I can fit the glass and furniture beforefitting the window. Don’t fancy having the opening boarded up for too long.

Not a lot else happened today, and tonight I’ll be relaxing with a cider and starting to watch the first series of Narcos on Netflix – heard good things about it, time to see what all the fuss is about.

Linux command pipeline using ‘|’

When writing bash scripts, I often connect a series of commands together into a pipeline, where the commands are separated by the pipe '|' character.

When using pipelines, the output from the first command is treated as the input to the second command, the output of the second command is treated as the input to the third command, and so on.

This can be useful in a number of situations, such as when you need to process the output of a command further before displaying or assigning to a variable.

For example, given a file containing a sequence of numbers

bash ~$ cat numbers.txt

We can find the numbers in the file with the largest distribution as follows

bash ~$ sort -n numbers.txt | \
          uniq -c | \
          sort -rn | \
141 2
 69 1685
 59 1
 53 2950
 11 1902
  4 2870
  4 2132
  3 9151
  3 4345
  3 1796

Where we first sort the contents of the file, using -n to sort them numerically, then pipe that output into the uniq command with the -c option to count the unique values, then sort again, this time with -rn for reverse numeric order, and finally take the first 10 entries in the output (10 is the default number of lines that head will return.)