Kevin's Worklog

Documenting the Daily Stream

Migrating Select Pairtree Files

The Pairtree file structure is a hiearchical structure into which files can be put. In my case I store different types of files in it: JPEGs, JP2, TIFFs, etc. I have a case for which I need to copy all the files of one type out of the existing structure and into a new Pairtree structure in a different location.

Here is a Bash script I whipped up to do that:

#! /bin/bash

find "${1}" -type f -name "*.jp2" -print0 | while IFS= read -r -d $'\0' LINE; do
  PATH="${LINE/#${1}/}"
  echo "cp ${1}${PATH} ${2}${PATH}"
done

It takes a “from” directory and a “to” directory as arguments. Right now it’s hard wired for JP2s but that could be a third argument to the script.

Including SNAPSHOT Dependencies in Maven Build

I try to not use SNAPSHOT dependencies in Maven but sometimes, as code is in process, I need to temporarily reference a SNAPSHOT dependency. If that SNAPSHOT dependency is in my local Maven repository, my local build will work. But, when the project gets built in another environment (like Travis-CI’s), the build fails. To work around this, the project needs to be aware of how to resolve those SNAPSHOT dependencies.

To accomplish this, add repositories (and pluginRepositories if the SNAPSHOT is also a plugin) to the project’s pom.xml file:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
<repositories>
  <repository>
    <id>dependency-snapshots-repo</id>
    <url>https://oss.sonatype.org/content/repositories/snapshots</url>
    <releases>
      <enabled>false</enabled>
    </releases>
    <snapshots>
      <enabled>true</enabled>
    </snapshots>
  </repository>
</repositories>
<pluginRepositories>
  <pluginRepository>
    <id>plugin-snapshots-repo</id>
    <url>https://oss.sonatype.org/content/repositories/snapshots</url>
    <releases>
      <enabled>false</enabled>
    </releases>
    <snapshots>
      <enabled>true</enabled>
    </snapshots>
  </pluginRepository>
</pluginRepositories>

With these two elements added, the Travis build will be able to resolve the SNAPSHOT dependencies. The elements can be removed again once the SNAPSHOT dependencies are replaced with stable versions.

Hangouts Screensharing Broken in Chromium

Screensharing within Google Plus Hangouts is broken with the Chromium browser, version 35 and greater. There is a ticket for it, but the temporary workaround is to run the browser with a different User Agent.

chromium-browser --user-agent="Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/34.0"

In looking at my system, it seems I’d already created a wrapper script in /usr/local/bin to workaround another Chromium bug (with WebGL). So, now, that script looks like:

#! /bin/bash

LIBGL_DRI3_DISABLE=1 /usr/bin/chromium-browser --user-agent="Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/34.0" "$@"

To Fix the Permissions of a Directory Structure

A fixperms.sh script that will recursively fix the directory permissions to 755 for directories and 644 for files.

#! /bin/bash

if [ $# -eq 0 ] ; then
  DIR=`pwd`
else
  DIR=${1}
fi

if [ ! -e ${1} ]; then
  echo "File or directory \"${1}\" doesn't exist"
  exit 1
fi

find ${DIR} -type d -print0 | xargs -0 chmod 0755
find ${DIR} -type f -print0 | xargs -0 chmod 0644

# Let's treat our bash scripts differently though
for FILE in $(find ${DIR} -name "*.sh"); do
  if [ -e ${FILE} ]; then
    echo "Making ${file} executable"
    chmod -f 0744 ${FILE}
  fi
done

Timing a cURL Download

A quick script to time a cURL download:

#!/bin/bash

CURL="/usr/bin/curl"

if [ -n "$1" ]; then
  URL="$1"
else
  echo -n "Please pass the URL you want to measure: "
  read URL
fi

RESULT=`$CURL -o /dev/null -s -w "%{time_connect}:%{time_starttransfer}:%{time_total}" $URL`

echo -e "To_Connect\tStart_Transfer\tTotal_Time"
sed s/\:/\\t\\t/g <<< $RESULT

Copying One MySQL Database Into Another

In the category of tidbits to remember… how to copy one MySQL/MariaDB database into another one (that already exists):

unset HISTFILE
mysqldump -u root -p[PASSWORD] [DATABASE] | mysql -u root -p[PASSWORD] [DATABASE]

A Quick and Dirty eXist-db AWS Startup Script

In lieu of using one of the more formal methods from the eXist-db Advanced Installation page, I just wanted a quick and dirty way to automatically spin up an eXist-db database, with code checked out from the eXist-db GitHub repository, when an EC2 instance was started.

My requirements? I wanted to run the Web application at port 80 and I wanted to do this on an EC2 instance that doesn’t have an Elastic IP. What follows is the quick and dirty solution I came up with. I’m not suggesting it’s better than the more formal methods, but it works and does what I wanted.

First, I started with installing some of the necessary software:

sudo aptitude install git screenie openjdk-8-jdk ant

Next, I checked out the eXist-db code and built it:

cd /home/ubuntu
git checkout https://github.com/eXist-db/exist.git
ant

Then I created a script that would configure a CNAME on my Route 53 DNS configuration, port map port 8080 to port 80, and startup the eXist database. I also created a configuration file for the AWS Route 53 connection. I put the configuration file at /etc/route53/config. It contains:

aws_access_key_id = [SOMETHING_SECRET_GOES_HERE]
aws_secret_access_key = [SOMETHING_SECRET_GOES_HERE]

The contents of the script are:

#! /bin/bash

# Make sure only root can run this script
if [ "$(id -u)" != "0" ]; then
  echo "This script must be run as root" 1>&2
  exit 1
fi

# Load AWS configuration
. /etc/route53/config

# Export access key ID and secret for cli53
export AWS_ACCESS_KEY_ID
export AWS_SECRET_ACCESS_KEY

# Use command line scripts to get the public hostname
PUBLIC_HOSTNAME=$(ec2metadata | grep 'public-hostname:' | cut -d ' ' -f 2)

# Create a new CNAME record on Route 53, replacing the old entry if necessary
/usr/local/bin/cli53 rrcreate lisforge.net c4lse CNAME "$PUBLIC_HOSTNAME" --replace --ttl "60"

# It's okay with me if the first fails because it's never been configured before
iptables -D PREROUTING -t nat -i eth0 -p tcp --dport 80 -j REDIRECT --to-port 8080
iptables -A PREROUTING -t nat -i eth0 -p tcp --dport 80 -j REDIRECT --to-port 8080

# At startup, this script is run as root, but we don't want to run eXist-db as root
cd /home/ubuntu/exist
sudo -u ubuntu bin/startup.sh

I put the script at /usr/local/sbin/exist-startup and then put the following the /etc/rc.local file:

screenie -j exist /usr/local/sbin/exist-startup

That’s it. Now each time I start the EC2 instance with eXist-db installed, I can go to c4lse.lisforge.net and find the application running there.

If I’m on the EC2 instance and want to see the console output, I can use screenie to reattach to the session. Quick and dirty.

Skype Woes

Skype never seems to work for me out of the box with my Ubuntu system. This time, with the latest version, at least there was a solution and it was pretty simple (this hasn’t always been true in the past). Before starting Skype, I dropped to the command line and typed:

pulseaudio -k
pulseaudio -D

Now on starting Skype, the microphone works. Sound was working fine for me before and the pulseaudio control panel was picking up my microphone but for some reason, without the daemon flag, Skype didn’t recognize my mic. Now I can participate in Skype conferences again. Yay?

I should also mention this useful blog post that I found while researching again. It documents “the top five wrong ways to fix your sound card” (many of which I’ve tried in the past).

List Bash Aliases

Discovered a neat, new-to-me trick to see the available aliases in my Bash shell:

compgen -a | grep searchstr

So if I have all my Docker aliases start with “dk”, I can run:

compgen -a | grep ^dk

To get a quick list of them.

There are other options too:

  • compgen -c # Will list all the commands you could run
  • compgen -a # Will list all the aliases you could run
  • compgen -b # Will list all the built-ins you could run
  • compgen -k # Will list all the keywords you could run
  • compgen -A function # Will list all the functions you could run
  • compgen -A function -abck # Will list all the above in one go

I learned about this from an answer on StackOverflow.

Octopress and Safe_YAML Tagged Classes

After a system upgrade on Ubuntu, Octopress seems to have broken. The error I get is:

`<module:SafeYAML>': undefined method `tagged_classes' for Psych:Module (NoMethodError)

I know next to no Ruby, but thankfully a Google search turned up this bug report on the safe_yaml Github page. It seems the safe_yaml developer has fixed the problem in the latest version so I want to upgrade to that. The first thing I do is add gem 'safe_yaml', '~> 1.0.4' to the Octopress Gemfile. Then, when I run rake generate, I get:

You have requested:
  safe_yaml ~> 1.0.4

The bundle currently has safe_yaml locked at 1.0.3.
Try running `bundle update safe_yaml`
Run `bundle install` to install missing gems.

So I try running bundle update safe_yaml and the problem is fixed. Yippee! After that, I can run rake generate and the site generation works.