19 Years

February 20, 2015 | posted in: site

Zanshin.net is 19 years old today.

$ whois zanshin.net | grep Creation
Creation Date: 1996-02-20T05:00:00.00Z

I've lost track of the number of major revisions my site has had in that time, but I do know that it has gone from hand-coded to Blogger to MoveableType to Wordpress to Octopress and finally to Jekyll.

In addition to the main domain, there are three subdomains. Geek a semi-frequently updated link-list of technology sites. You can drag-and-drop the sections. Books is a listing of all that I've read this year. Includes audiobooks too. And Health is a record of my daily exercise, calories, and weight.

In terms of numbers there are (approximately) 653,754 words in the postings (not counting this one), and this post will be number 2,158. The site's readership has steadily increased each year now for several years. Currently I average about 550 visits per day or 14,000 per month, give or take.

Here's to another 19 years.

Arch Linux on a MacBook Pro Part 4: System Configuration

February 05, 2015 | posted in: nerdliness

Introduction and Motivation

This is the fourth part in a series on dual booting Arch Linux and Mac OS X on a MacBook Pro. This part covers setting up Xorg, getting the Nvidia graphics sub-system working, and installing Gnome and Awesome. At the end the is a bibliography of some of the many sites I used as references. Part One covered creating a bootable USB drive with the Arch Linux installer. Part Two discussed how to prepare the MacBook Pro. Installing the base Arch Linux operating system and getting the dual boot between Arch and OS X working was covered in Part Three.

My day job has transitioned to system administration and infrastructure automation (Chef!). All of our virtual infrastructure (with rare exceptions) is based on Linux so I am keenly interested in learning more about its underpinnings. Setting up Arch Linux and sorting through its configuration challenges seems like a good way to jump into the deep end of the Linux pool.

Arch Linux on a MacBook Pro Part 1 - Creating a USB Installer
Arch Linux on a MacBook Pro Part 2 - Preparing for Dual Boot
Arch Linux on a MacBook Pro Part 3 - Base Installation
Arch Linux on a MacBook Pro Part 4 - System Configuration

Setup Wireless

You need to download the proprietary Broadcom driver, and then make a package, and finally install the package. (I did this under the chroot used to setup Arch, but didn't test it until the base install was completed and I was logging in via my account. Consequently you may need to use sudo for some of these commands.)

$ mkdir src
$ cd src
$ wget https://aur.archlinux.org/packages/br/broadcom-wl/broadcom-wl.tar.gz
$ tar xf broadcom-wl.tar.gz
$ cd broadcom-wl/
$ makepkg -s

Note: If you are doing this under the chroot you'll have to su to the user account you created earlier to run the makepkg command.

Next use the packman command below to install the newly made package.

$ pacman -U /tmp/usb/broadcom-wl-6.30.223.248-4-x86_64.pkg.tar.xz

After the package installation, load the kenel modules necessary for WPA2 connections

$ modprobe wl lib80211_crypt_tkip

Finally use the wifi-menu command to connect to a wireless network. Note: you may have to install some dependencies via pacman -S <dependency>.

64-bit Pacman

In order to make the library of 64-bit wrapped 32-bit applications available edit /etc/pacman.conf and uncomment the multilib lines. Then update the system using:

$ pacman -Syu

More packages

Next install all the basic utilities needed to build 32-bit or 64-bit packages.

$ sudo pacman -S multilib-devel fakeroot git jshon wget make pkg-config autoconf automake patch

Press enter to install all members in group multilib-devel. And say yes to removing those packages in conflict.

AUR

The "Arch User Repository" is a user-managed package repository with a huge number of packages. To access it install packer, which uses the same command syntax as pacman. I create a src directory in my home to hold such things as this.

$ mkdir src
$ cd src
$ wget http://aur.archlinux.org/packages/pa/packer/packer.tar.gz
$ tar xf packer.tar.gz
$ cd packer
$ makepkg -s
$ pacman -U packer-20140817-1-any.pkg.tar.xz

Sound

ALSA works out of the box with Macs so install it via:

$ pacman -S alsa-utils

Then use

$ alsamixer

to control the speakers. Make sure to disable channels for speakers you don't have. Test your speakers with

$ speaker-test -c 2

where 2 is the number of speakers.

Xorg

Setting up Xorg will enable your video card, trackpad, backlight, et etera.

Install the base packages for Xorg, using:

$ sudo pacman -S xorg-server xorg-xinit xorg-server-utils xorg-server-devel mesa. 

Note that the mesa package is included for 3D support, if you do not have a 3D capable graphics card you can skip this package.

NVIDIA

Getting the NVIDIA card working was the hardest part of the install for me. In the end I used the Nouveau driver as the Nvidia ones didn't work. Follow the instructions on the Nouveau page and you should be okay.

At this point you will need to reboot your machine to load the proper kernel modules and module blacklists.

Desktop

There are lots of choices in each of the three major desktop manager categories (stacking, tiling, and dynamic). To try something completely different I decided to install Awesome as well as Gnome.

Gnome

Awesome didn't so much work out of the box, so I installed Gnome.

$ pacman -S gnome gnome-extra

It will install lots of packages which, depending on your download speed, may take a while.

This command:

$ sudo systemctl enable gdm.service

will set Gnome to start when you boot your computer.

Awesome

One of the many postings I used to figure all of this out talked about Awesome, and since I'd never seen it before I thought I'd give it a try.

Start off by getting some fonts:

$ pacman -S ttf-dejavu ttf-ubuntu-font-family

Then install awesome from the official respoitory.

$ pacman -S awesome

Now you can select which desktop environment you want when you sign in, either Gnome or Awesome. You'll want to read the Awesome wiki and documentation, as out of the box it provides no configuration.

Conclusion

I've only scratched the surface of configuring and using Arch Linux. But I now have a capable machine running both OS X and Arch with which to learn.

I relied heavily on a large number of websites to figure all of this out. These are the most heavily used.

Dual Boot Arch Linux on MacBook Pro Installation

Arch Linux on MacBook Pro Retina 2014 with DM-Crypt, LVM and suspend to disk

Arch Linux: System Installation

Arch Linux: From Post-Install to Xorg

Awesome

Xorg

Swap

Disable clearing of boot messages

root partition read only after update systemd

Failed to start Remount Root and Kernel File System (systemd)

Blkid returns nothing, but has no error

I highly recommend reading the Arch Beginners's Guide. The #archlinux IRC channel is vibrant and can also be a source of help.

Feedback

If you've gotten this far in the series and or if you've successfully (or not) tried to dual boot a MacBook Pro with Arch, I'd love to hear from you. You can reach me via email link in my sidebar.

Arch Linux on a MacBook Pro Part 3: Base Installation

February 05, 2015 | posted in: nerdliness

Introduction and Motivation

This is the third part in a series on dual booting Arch Linux and Mac OS X on a MacBook Pro. This part covers partitioning the drive, the base Arch installation, and getting the dual boot setup. Part One covered creating a bootable USB drive with the Arch Linux installer. Part Two discussed how to prepare the MacBook Pro. Setting up the system, configuring Xorg, installing Gnome and Awesome are all covered in Part Four.

My day job has transitioned to system administration and infrastructure automation (Chef!). All of our virtual infrastructure (with rare exceptions) is based on Linux so I am keenly interested in learning more about its underpinnings. Setting up Arch Linux and sorting through its configuration challenges seems like a good way to jump into the deep end of the Linux pool.

Arch Linux on a MacBook Pro Part 1 - Creating a USB Installer
Arch Linux on a MacBook Pro Part 2 - Preparing for Dual Boot
Arch Linux on a MacBook Pro Part 3 - Base Installation
Arch Linux on a MacBook Pro Part 4 - System Configuration

Determine the MacBook Pro version

Determine the version of MacBook Pro you have. These instructions are for a Mid 2010 model or 6,2 in Apple parlance. Minor hardware differences may make these instructions not work if you have a significantly different MacBook Pro, e.g., one with a SSD.

Boot from the Arch USB Drive

Insert the USB drive containing the bootable Arch ISO (from Arch Linux on a MacBook Pro Part 1 - Creating a USB Installer) and boot or restart the MBP. Hold the left Option key down until the drive selection window appears.

Select the EFI Boot disk using the arrow keys, and press enter to start.

Select the Arch Linux option from the boot menu, or let the time out do it for you. Once the initial boot completes (should be fairly quick) you see a prompt like this:

Arch Linux 3.17.6-1-ARCH (tty1)

archiso login: root (automatic login)
root@archiso ~ #

Congratulations, you've got root.

Get connected

You'll need to be connected to the Internet to complete the setup. The easiest way is via Ethernet. If you don't have Ethernet available, skip ahead to the Setup Wireless section at the start of Arch Linux on a MacBook Pro Part 4 - System Configuration, and setup wireless now. These instructions assume you have working Ethernet.

Type:

$ dhcpcd

This will start the DHCP client and get an IP address lease from your LAN. You can verify that you have working connectivity by pinging something, Google for example.

$ ping -c 3 google.com

Determine drive mapping

In order to proceed you'll need to know the drive mapping scheme. The easiest way to get that information is via

$ fdisk -l

which lists the existing partitions. If you created two partitions when preparing the MBP, you should see a partition with a Type of Apple HFS/HFS+ with a size that matches the size you set aside for Arch. In my case this was /dev/sda4. All the partitioning commands below will use /dev/sda4, you should substitute the designation for your drive.

Partitioning

Now that we know the drive designation we can use cgdisk tp setup the partitions for our install. Run

$ cgdisk /dev/sda

to see the current state of the disk and to access the partitioning utility. The output will be similar to what is shown below.

Part #  Size      Partition type      Partition name
        3.0 Kib   free space
1       200 Mib   EFI System          EFI system partition
2       78.8 Gib  Apple Core Storage  Macintosh HD
3       619.9 MiB Apple boot          Recovery HD
4       223.4 Gib Apple HFS/HFS+      Untitled 1
        128.0 Mib free space

We are going to add 6 partitons:

Linux boot loader 128 Mib
swap 8 Gib
boot 256 Mib
root 40 Gib
var 12 Gib
home ~160 Gib

The Linux boot loader will later be blessed as bootable using OS X. The final result should look like this:

Part #  Size        Partition type      Partition name
        3.0 Kib     free space
1       200 Mib     EFI System          EFI system partition
2       78.8 Gib    Apple Core Storage  Macintosh HD
3       619.9 MiB   Apple boot          Recovery HD
4       128.0 Mib   Apple HFS+          Linux boot loader from Apple
5       8.0 Gib     Linux swap          swap
6       256.0 Mib   Linux filesystem    boot
7       40.0 Gib    Linux filesystem    root
8       12.0 Gib    Linux filesystem    var
9       163.1 Gib   Linux filesystem    home

To navigate around cfdisk, use the left and right arrow keys to select a command and use the up and down arrow keys to select the partition and/or the free space. Once you have the partitions set the way you desire, select the Write option to create the new partitions.

Begin by deleting the Untitled 1 HFS partition created when you setup the MBP. Next subdivide it into the 6 new partitions. You can use K, M, and G to specify sizes. Accept the starting location offered by the tool when creating a new partition, then enter the size desired. You can use the L option to look up filesystem type codes. Apple HFS+ is af00, Linux filesystem is 8300 (the default), and Linux swap is 8200. Finally give the new partition a name. Repeat for each partition. On the last partition accept the default size offered to use up all remaining space. This means you should do home last to give it as much space as possible.

Select Write and then confirm that you want to overwrite the disk. Once the display returns you can Quit from cgdisk.

Running fdisk -l again should show your new partition scheme. If it doesn't look like you want or expected, repeat the cgdisk process to fix things.

Format the partitions

Now it's time to format the partitions.

$ mkfs.ext2 /dev/sda6
$ mkfs.ext4 /dev/sda7
$ mkfs.ext4 /dev/sda8
$ mkfs.ext4 /dev/sda9

And create the swap and use it.

$ mkswap /dev/sda5
$ swapon /dev/sda5

Note that we skipped /dev/sda4 as that will be setup using OS X later.

Mount the filesystem

In order to acces the new partitions they must be mounted. First mount your root partition.

$ mount /dev/sda7 /mnt

And within the root mount just created, create mount points and mount boot, var, and home.

$ mkdir /mnt/boot && mount /dev/sda6 /mnt/boot
$ mkdir /mnt/var && mount /dev/sda8 /mnt/var
$ mkdir /mnt/home && mount /dev/dsdc9 /mnt/home

Install Arch

With the partitions created and formated, and with the filesystem mounted, the operating system can now be installed using pacstrap.

$ pacstrap /mnt base base-devel

This will install ~140 packages and could take a while based on your download speeds. Sit back and watch the progress.

Set fstab

Once the packages are installed it's time to record your file system settings in fstab. Use the genfstab command to generate your fstab

$ genfstab -p /mnt >> /mnt/etc/fstab

You can view the fstab using less

$ less /mnt/etc/fstab

Initial Configuration

Now you can set up your desired hostname, proper time zone, the hardware clock, create a user account for yourself, and enable sudo.

First you need to chroot into the new system.

$ arch-chroot /mnt /bin/bash

Set your hostname.

$ echo arch > /etc/hostname

Set your time zone. I'm in the US Central time zone so I used Chicago. Adjust this to your location.

$ ln -s /usr/share/zoneinfo/America/Chicago /etc/localtime

And set the hardware clock.

$ hwclock --systohc --utc

Create your user account and generate a password for it.

$ useradd -m -g users -G wheel -s /bin/bash you && passwd you

While you're at it, create a root password too.

$ passwd

You'll want sudo installed so that when you are signed in to your account you can run commands requiring root privilege. pacman is the command line package manager for Arch, so we'll use that.

$ sudo pacman -S sudo

Edit /etc/sudoers and uncomment the wheel line. The -G wheel portion of the useradd command above added you to the wheel group. By uncommenting this group in the sudoers file you'll allow all wheel members to run commands as sudo. You need to use visudo to edit this file.

$ visudo

Set your locale.

sudo vi /etc/locale.gen

and uncomment the locales you want. For me this was en_US.UTF-8 UTF-8 and en_US ISO-8859-1.

Generate the locale.

$ locale-gen

And set the /etc/locale.conf and export your settings.

$ echo LANG=en_US.UTF8 > /etc/locale.conf
$ export LANG=en_US.UTF-8

Double check /etc/mkinitcpio.conf to make sure keyboard is after autodetect in the HOOK section.

$ less /etc/mkinitcpio.conf

Then run:

$ mkinitcpio -p linux

Bootloader configuration and setup

This is somewhat confusing. The best way to do this is to boot directly from your MBP's EFI boot loader, which means creating a boot.efi.

Grab the package:

$ pacman -S grub-efi-x86_64

And then, edit /etc/default/grub and alter GRUB\_CMDLINE\_LINUX_DEFAULT to look like this:

GRUB_CMDLINE_LINUX_DEFAULT="quiet"

which I believe is the default setting.

Then you can generate the boot.efi with GRUB which you just installed. You’ll want to put this on a USB device because you’re going to be switching into OS X in a minute.

$ grub-mkconfig -o boot/grub/grub.cfg
$ grub-mkstandalone -o boot.efi -d usr/lib/grub/x86_64-efi -O x86_64-efi --compress=xz boot/grub/grub.cfg

This is going to create a file in the current directory called boot.efi. Copy it to a USB device. Check your devices then make a directory to mount your USB. Copy the boot.efi file onto your USB drive. You may have to determine the path to the USB drive.

$ mkdir /mnt/usbdisk && mount /dev/sdb /mnt/usbdisk
$ cp boot.efi /mnt/usbdisk/

Exit and reboot to OS X

Now you can exit chroot and unmount everything, and reboot back to OS X.

Exit chroot.

$ exit

Unmount all filesystems. And reboot.

$ reboot

Restart the computer and boot into OS X. Run Disk Utility and select /dev/sda4 which is the "Linux boot loader from Apple" partition created earlier. Using Disk Utility erase this partition by selecting "Mac OS X Journaled" and clicking on Erase. This is where the Grub2 image will go.

Open up Terminal and issue the following sequence of commands to create directories and files necessary for this partition.

$ cd /Volumes/disk0s4
$ mkdir System mach_kernel
$ cd System
$ mkdir -p Library/CoreServices
$ cd Library/CoreServices
$ touch SystemVersion.plist

Copy the boot.efi image from the USB drive as a peer to SystemVersion.plist.

$ cp /Volumes/usbdrive/boot.efi .

Edit SystemVersion.plist to look like this:

<xml version="1.0" encoding="utf-8"?>
<plist version="1.0">
<dict>
    <key>ProductBuildVersion</key>
    <string></string>
    <key>ProductName</key>
    <string>Linux</string>
    <key>ProductVersion</key>
    <string>Arch Linux</string>
</dict>
</plist>

Next "bless" the partition so that it is bootable.

$ sudo bless --device /dev/disk0s4 --setBoot

Now when you boot or restart the MBP, a Grub menu with Arch Linux should appear. To boot into OS X you'll need to hold down the left Option key. Test this and make sure it works before proceeding to Arch Linux on a MacBook Pro Part 4 - System Configuration.

Arch Linux on a MacBook Pro Part 2: Preparing for Dual-Boot

February 05, 2015 | posted in: nerdliness

Introduction and Motivation

This is the second part in a series on dual booting Arch Linux and Mac OS X on a MacBook Pro. This installment covers the preparation of the MacBook Pro for dual booting. Part One covered creating a bootable USB drive with the Arch Linux installer. Installing the base Arch Linux operating system and getting the dual boot between Arch and OS X working is covered in Part Three. Setting up the system, configuring Xorg, installing Gnome and Awesome are all covered in Part Four.

My day job has transitioned to system administration and infrastructure automation (Chef!). All of our virtual infrastructure (with rare exceptions) is based on Linux so I am keenly interested in learning more about its underpinnings. Setting up Arch Linux and sorting through its configuration challenges seems like a good way to jump into the deep end of the Linux pool.

Arch Linux on a MacBook Pro Part 1 - Creating a USB Installer
Arch Linux on a MacBook Pro Part 2 - Preparing for Dual Boot
Arch Linux on a MacBook Pro Part 3 - Base Installation
Arch Linux on a MacBook Pro Part 4 - System Configuration

Prepare the MacBook Pro

In my case I am maintaining a small OS X installation on the MacBook Pro (MBP) to allow for firmware updates. I used a Yosemite USB boot drive to format the drive and re-partition it. Initially it had one 320 GB partiton, I split that into two: an OS X drive ~80 GB and a second partition for Arch that is about 240 GB.

If you want to keep what is already on the OS X partition you can use Disk Utility to shrink that partition, creating free space for the Arch Linux installation. Just be careful to leave enough free space in the OS X partition for future growth and about 25% unused disk for swapping.

You can also use About This Mac and its System Report option to note the make and model of your video chipset. This will be essential when setting up Xorg at the end of the Arch install.

Once you have the MBP partitioned the way you like you are ready to start the Arch install process. Insert the USB drive you set up in Arch Linux on a MacBook Pro Part 1 - Creating a USB Installer and reboot the laptop. Once the screen goes black, hold down the left Option key under the list of available drives appears. Select the USB drive with the arrow keys and press Enter. Arch Linux on a MacBook Pro Part 3 - Base Installation will take you through the basic Arch install.

Arch Linux on a MacBook Pro Part 1: Creating a USB Installer

February 05, 2015 | posted in: nerdliness

Introduction and Motivation

This is part one in series on dual booting Arch Linux and Mac OS X on a MacBook Pro. This part covers creating a bootalbe USB installer. Part Two discusses how to prepare the MacBook Pro. Installing the base Arch Linux operating system and getting the dual boot between Arch and OS X working is covered in Part Three. Setting up the system, configuring Xorg, installing Gnome and Awesome are all covered in Part Four.

My day job has transitioned to system administration and infrastructure automation (Chef!). All of our virtual infrastructure (with rare exceptions) is based on Linux so I am keenly interested in learning more about its underpinnings. Setting up Arch Linux and sorting through its configuration challenges seems like a good way to jump into the deep end of the Linux pool.

Arch Linux on a MacBook Pro Part 1 - Creating a USB Installer
Arch Linux on a MacBook Pro Part 2 - Preparing for Dual Boot
Arch Linux on a MacBook Pro Part 3 - Base Installation
Arch Linux on a MacBook Pro Part 4 - System Configuration

Download an Arch Linux ISO

Visit the Arch download site and grab an ISO.

Convert the ISO to UDRW Format

The image needs to be in Read/Write Universal Disk Image Format (UDRW) format. Use the command line tool hdiutil to convert the file.

$ hdiutil convert -format UDRW -o destination_file.img source_file.iso

The resulting image will actually be called destination_file.img.dmg as hdiutil will append that suffix. The file format extension won't matter, so you can ignore it.

Prepare the USB Drive

While some of this can be done using GUI tools, not all of it can, so I'll describe all the steps using the command line.

Find the name of the USB drive

Insert the drive and then run this command:

$ diskutil list

The output will look something like this:

/dev/disk0
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *500.1 GB   disk0
   1:                        EFI                         209.7 MB   disk0s1
   2:          Apple_CoreStorage                         399.5 GB   disk0s2
   3:                 Apple_Boot Recovery HD             650.0 MB   disk0s3
   5:                 Apple_Boot Boot OS X               134.2 MB   disk0s5
/dev/disk1
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:                  Apple_HFS MacOSX                 *399.2 GB   disk1
/dev/disk2
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *2.0 GB     disk2
   1:       Microsoft Basic Data UNTITLED 1              2.0 GB     disk2s1

In this case the USB is /dev/disk2.

Prepare the drive

Prepare the USB by making it all free space. This will delete all data on the USB drive.

$ diskutil partitionDisk /dev/disk2 1 "Free Space" "unused" "100%"

Substitute the proper designation for your USB drive.

Copy the ISO

Now we can copy the converted image to the USB drive.

$ dd if=destination_file.img.dmg of=/dev/disk2 bs=1m

The dd command does not show any output before it has finished the copy process, so be patient and wait for it to complete. When the command does complete OS X will try to mount the drive and fail as it won't recognize the formatting. Click ignore or eject.

Eject the drive

If you clicked ignore above you can eject the drive from the command line.

$ diskutil eject /dev/disk2

Now the bootable image is ready to be used. Proceed to Arch Linux on a MacBook Pro Part 2 - Preparing for Dual Boot.

How to Dual Boot Mac OS X and Arch Linux on a MacBook Pro

February 05, 2015 | posted in: nerdliness

Introduction and Motivation

This is a four-part series on dual booting a MacBook Pro with Arch Linux and Mac OS X. Part One covers creating a bootable USB drive with the Arch Linux installer. Part Two discusses how to prepare the MacBook Pro. Installing the base Arch Linux operating system and getting the dual boot between Arch and OS X working is covered in Part Three. Setting up the system, configuring Xorg, installing Gnome and Awesome are all covered in Part Four.

My day job has transitioned to system administration and infrastructure automation (Chef!). All of our virtual infrastructure (with rare exceptions) is based on Linux so I am keenly interested in learning more about its underpinnings. Setting up Arch Linux and sorting through its configuration challenges seems like a good way to jump into the deep end of the Linux pool.

Arch Linux on a MacBook Pro Part 1 - Creating a USB Installer
Arch Linux on a MacBook Pro Part 2 - Preparing for Dual Boot
Arch Linux on a MacBook Pro Part 3 - Base Installation
Arch Linux on a MacBook Pro Part 4 - System Configuration

Teaching A Homely Mutt New Tricks

January 19, 2015 | posted in: nerdliness

About two years ago I read Steve Losh's excellent The Homely Mutt. It's a fantastic how-to on setting up Mutt as your email client. Mostly to see if I could, I set aside a day and managed to get it all working. Having scratched my technology itch I set Mutt aside and continuted on with Mail.app.

About a year ago my employer switched to Office365 for email. While I find the webmail interface to be clunky and difficult to use, getting my mail through Mail.app worked so I wasn't too bothered by the switch. However when I upgraded to Yosemite in October I started having problems with my Office365 mail. The 10.10.1 release did improve things some, but it's still a pain to use.

This past weekend I decided to resurrect my Mutt setup, and see if I could make it work -- not just for one account, but for six accounts. The Homely Mutt (THM) posting describes how to setup Mutt to work with a single GMail account. I have 3 domain accounts, two Gmail accounts, and the afore mentioned Office365 (Exchange) account.

My dotfiles repository had the Mutt configuration I had created in 2013, and I had managed to get that working with multiple accounts. Not having touched it in nearly two years I decided to start over from scratch.

Mutt for Gmail

I reread THM and started in from the top. This allowed me to setup one of my two Gmail accounts. I picked the one with the least amount of mail to reduce the time Offline imap would need to complete the initial download of my mail.

Offlineimap for Multiple Computers

I have three computers that I use on a daily or near daily basis and I want to have my mail available on all three. So I wanted to modify the original offlineimap.py script to work using the current user account, and not a hard coded account as shown in THM. Digging around in Mr. Losh's dotfiles I discoverd his solution was to simply use whoami in place of the hard coded user account name. The resulting script looks like this:

#!/usr/bin/python

import re, subprocess

def get_keychain_pass(account=None, server=None):
    params = {
        'security': '/usr/bin/security',
        'command': 'find-internet-password',
        'account': account,
        'server': server,
        'keychain': '/Users/`whoami`/Library/Keychains/login.keychain',
    }

    command = "sudo -u `whoami` %(security)s -v %(command)s -g -a %(account)s -s %(server)s %(keychain)s" %params
    output = subprocess.check_output(command, shell=True, stderr=subprocess.STDOUT)
    outtext = [l for l in output.splitlines()
                     if l.startswith('password: ')][0]

    return re.match(r'password: "(.*)"', outtext).group(1)

Mutt with sidebar-patch

On OS X the easiest way to install Mutt is using Homebrew. Unfortunately the formula for Mutt no longer contains the sidebar patch. Having a sidebar where your accounts and their mailboxes are displayed is, in my opinion, essential. A short Google search lead me to how to apply this mutt sidebar patch on StackOverflow. You just edit the Mutt formula and insert the missing option.

$ brew edit mutt

And then, scroll down to a section of commands that all start with "option", sort of like:

option "with-debug", "Build with debug option enabled"
option "with-trash-patch", "Apply trash folder patch"
option "with-s-lang", "Build against slang instead of ncurses"
option "with-ignore-thread-patch", "Apply ignore-thread patch"
option "with-pgp-verbose-mime-patch", "Apply PGP verbose mime patch"
option "with-confirm-attachment-patch", "Apply confirm attachment patch"

Add this line to the bottom of the options:

option "with-sidebar-patch", "Apply sidebar patch"

Scroll down further to the section with all the patches, e.g.

patch do
  url "http://patch-tracker.debian.org/patch/series/dl/mutt/1.5.21-6.2+deb7u1/features/trash-folder"
  sha1 "6c8ce66021d89a063e67975a3730215c20cf2859"
end if build.with? "trash-patch"

And add this block:

patch do
  url "https://raw.github.com/nedos/mutt-sidebar-patch/7ba0d8db829fe54c4940a7471ac2ebc2283ecb15/mutt-sidebar.patch"
  sha1 "1e151d4ff3ce83d635cf794acf0c781e1b748ff1"
end if build.with? "sidebar-patch"

Exit the editor and run

$ brew install mutt --with-sidebbar-patch

Multiple Accounts in Mutt

Much of the configuration described in THM still applies for multiple mail accounts. Below I've described where there are differences.

Offlineimap

You'll need to add the account identifier for each account to the accounts = line. And you'll need to create new [Account ...], [Repository ...-Local], and [Repository ...-Remote] sections for the new email account as well. I've included a shortened version of my .offlineimaprc file showing two accounts. The complete version is a bit long.

# vim: ft=rc:

[general]
# ui = ttyui
ui = blinkenlights
#ui = quiet
accounts = mark,root
pythonfile=~/.mutt/offlineimap.py
fsync = False

[Account mark]
localrepository = mark-Local
remoterepository = mark-Remote

[Repository mark-Local]
type = Maildir
localfolders = ~/.mail/mark

[Repository mark-Remote]
maxconnections = 3
type = IMAP
remoteuser = mark_zanshin
remotehost = mail.example.com
remoteport = 993
ssl = yes
remotepasseval = get_keychain_pass(account="mark_zanshin", server="mail.webfaction.com")
realdelete = no

[Account root]
localrepository = root-Local
remoterepository = root-Remote

[Repository root-Local]
type = Maildir
localfolders = ~/.mail/root

[Repository root-Remote]
maxconnections = 3
type = IMAP
remoteuser = root_zanshin
remotehost = mail.example.com
remoteport = 993
ssl = yes
remotepasseval = get_keychain_pass(account="root_zanshin", server="mail.webfaction.com")
realdelete = no

You'll also need to visit Keychain Access and create new password entries for incoming (imap) and outoging (smtp) messages. This process is unchanged from THM, you just need to do it for each new account. With all the accounts setup in .offlineimaprc and new Keychain entries I was able to kick off the rather lenghty process of downloading my mail.

.muttrc Changes

Some of the changes to the .muttrc file happen in that file and others I farmed out to a new directory called accounts.

The .muttrc changes are fairly straight forward. You need to edit the list of mailboxes you want to have appear in the sidebar. I've added what amounts to a comment or tag before each new set of mailbaoxes to make identifying which account is which easier.

# Mailboxes to show in the sidebar.
mailboxes "+-- mark ---------------" \
          +mark/INBOX \
          +mark/archive \
          +mark/Drafts \
          +mark/Sent\ Messages \
          "+-- root ---------------" \
          +root/INBOX \
          +root/archive \
          +root/Drafts \
          +root/Sent\ Messages \
          "+-- MarkNichols ----------" \
          +marknichols/INBOX \
          +marknichols/archive \
          +marknichols/drafts \
          +marknichols/sent \
          "+-- CodeProle ----------" \
          +codeprole/INBOX \
          +codeprole/archive \
          +codeprole/drafts \
          +codeprole/sent 
          "+-- mhn (ksu) ----------" \
          +mhn/INBOX \
          +mhn/Archive \
          +mhn/Drafts \
          +mhn/Sent\ Items \
          +mhn/github \
          +mhn/bugzilla \
          +mhn/ome-dt-l \
          +mhn/Junk\ Email \
          +mhn/Deleted\ Items \\
          "+-- Chef ---------------" \
          +chef/INBOX \
          +chef/archive \
          +chef/Drafts \
          +chef/Sent\ Messages \

Using the folder-hook feature of Mutt I was able to setup account specifc settings, each in their own file. When an account is selected in the sidebar and one of its mailboxes opened the folder hook sources a short chunk of muttrc code to set the account specific values.

Under the .mutt directory I created a new directory called accounts. In accounts I put a file for each email account. Here's an example file:

# Account Settings ------------------------------

set from      = "mark@zanshin.net"
set sendmail  = "/usr/local/bin/msmtp -a mark"

# Default inbox.
set spoolfile = "+mark/INBOX"

# Other special folders.
set mbox      = "+mark/archive"
set postponed = "+mark/drafts"
set record    = "+mark/Sent Messages"

color status yellow default

The folder-hook lines in the .muttrc file look like this:

# Set account specific options on folder change
folder-hook mark/*        source ~/.mutt/accounts/mark
folder-hook root/*        source ~/.mutt/accounts/root
folder-hook chef/*        source ~/.mutt/accounts/chef
folder-hook marknichols/* source ~/.mutt/accounts/marknichols
folder-hook codeprole/*   source ~/.mutt/accounts/codeprole
folder-hook mhn/*         source ~/.mutt/accounts/mhn

.msmtprc

The final piece of the puzzle is sending email. Edit the .msmtprc file and add the necessary information for each of the additional accounts. Again, here is a shortened version of my .msmtprc file:

account mark
host smtp.webfaction.com
port 587
protocol smtp
auth on
from mark@zanshin.net
user mark_zanshin
tls on
tls_trust_file ~/.mutt/Equifax_Secure_CA.cert

account root
host smtp.webfaction.com
port 587
protocol smtp
auth on
from root@zanshin.net
user root_zanshin
tls on
tls_trust_file ~/.mutt/Equifax_Secure_CA.cert

Conclusion and Caveats

You can view my complete Mutt setup on Github.

A word of caution applies. Unlike configuring tmux or Vim, making a mistake with Offlineimap or Mutt could potential wipe out some or all of your mail and cause the end of Western civilization. Proceed at your own risk.

A Guide for Setting Up Weechat and Bitlbee

January 10, 2015 | posted in: nerdliness

Introduction

With polished chat and IRC clients like Adium and Textual readily available why would you want to use a terminal-based, largely text-only IRC client for instant messaging? For me the reason was to have the same chat regardless of which computer I was using at the moment. Not the same chat application, the same chat. By using Weechat and Bitlbee in conjunction with tmux I'm able to create a singleton instance of my chat, complete with logs and all my chat providers. This posting is a guide to how I accomplished this setup.

The Setup

At my employment I have a desktop with a static IP address. By installing both Weechat and Bitlbee on that machine, and launching Weechat inside a tmux session, I can secure shell to my desktop from any of my other computers and attach to the chat tmux session. With one single instance of weechat+bitlbee installed this way I've centralized my instant message and IRC experience. One set of logs, one set of configuration files, one of everything.

The steps that follow were all done on OS X. Your experience will certainly vary on other operating systems.

Install bitlbee

$ brew install bitlbee

Install weechat

$ brew install weechat --with-lua --with-perl --with-python --with-ruby --HEAD --with-aspell

Add Lua, Perl, Python, and Ruby flags to support scripts (plug ins) in those languages. The --HEAD flag installs the latest available version of weechat. --with-aspell provides spell check support. To use this you first need to install aspell (using brew or method of your choice) along with any languages you wish. See the weechat documentation.

Start bitlbee

$ bitlbee -F -u <your-user-id>

The -F flag starts bitlbee as a daemon. And the -u flag followed by your user id starts it as you and not as root. It is recommended to not run bitlbee as root.

Start weechat

$ weechat

Configure weechat

There are almost a thousand settings in weechat, so configuration option combinations are nearly limitless.

/set *
...
980 options (matching with "*")

Getting help

The best way to get help for any weechat configuration option is to use the /help command.

/help <command>

You can also see "settable" options using the /set command.

/set *

or use the * wildcard

/set *nick*

bitlbee also has excellent built-in help, accessed by using the help (no leading slash) command in the &bitlbee control channel.

Also, be sure to periodically save your settings or they won't be persisted when you quit weechat.

/save

You should also save your bitlbee work periodically.

save

Basic weechat setup and configuration

These are the configuration settings I've made.

Saner time format:

/set weechat.look.buffer_time_format “%H:%M:%S”

Use a tilde (~) between panes within weechat:

/set weechat.look.separator_horizontal “~”

Position the weechat nicklist bar on the left:

/set weechat.bar.nicklist.position left

And limit the nicklist bar width to 15 characters:

/set weechat.bar.nicklist.size 15

Mute the time color:

/set weechat.color.chat_time gray

Set the background color for the status bar (at the bottom):

/set weechat.bar.status.color_bg black

Turn off highlighting of your own nick, it gets annoying otherwise.

/set weechat.color.chat_nick_self *!lightgreen

Limit the width of the buffers side bar:

/set weechat.bar.buffers.size 20

Status bar settings

Don't hide the status bar:

/set weechat.bar.status.hidden off

Mute the status bar background:

/set weechat.bar.status.color_bg darkgray

Configure the contents of the status bar:

/set weechat.bar.status.items “buffer_number+:+buffer_name+{buffer_nicklist_count}+buffer_filter,completion,scroll”

Allow for multi-line input:

/set weechat.bar.input.size 0
/set weechat.bar.input.size_max 3

Miscellaneous weechat settings

Hide join/leave messages in IRC (After enabling freenode)

/set irc.look.smart_filter on
/filter add irc_smart * irc_smart_filter *

Use array of colors for participants in channels or chat rooms.

/set weechat.color.chat_nick_colors red,green,brown,blue,magenta,cyan,white,lightred,lightgreen,yellow,lightblue,lightmagenta,lightcyan

Colorize nick list away status. First check every x minutes, second limits to nicklists of a size or smaller (less processing)

/set irc.server_default.away_check 5
/set irc.server_default.away_check_max_nicks 25

One of my favorites, substitue the UTF-8 hookrightarrow character for the nick when the same person has multiple back-to-back messages.

/set weechat.look.prefix_same_nick ↪

Scripts (plugins)

weechat has a large library of scripts (think plugins). Here are the 4 that I use.

  • buffers.pl - Sidebar with list of buffers
  • highmon.pl - Highlight monitor
  • iset.pl - Interactive Set for configuration options
  • autosort.py - Automatically or manually keep buffers sorted and grouped by server

buffers

buffers.pl provides a nice sidebar presentation of all the open buffers (channels or individual chats) you have open. There are a number of settings for buffers (64!); here are the ones I use:

/set buffers.color.number white
/set buffers.look.hotlist_counter on
/set buffers.name_size_max 20
/set buffers.color.current_bg  default
/set buffers.color.current_fg = blue
/set buffers.color.hotlist_message_bg default
/set buffers.color.hotlist_message_fg yellow

These are fairly self explanatory. Show the number for each buffer, and indicate how many new messages are in a buffer. Limit the length of buffer names to 20 characters. Make the name of the current buffer blue against a default background -- my terminals are black, and make any buffer with new activity yellow against the default background.

highmon

highmon.pl creates a highligh monitor allowing you to see mentions of your nick, or any other key word you choose. Since my given name is Mark I actually have my highlight monitor set to ignore uses of mark, e.g., bookmark, to prevent highlight notification I don't want. Like buffers, highmon has a number settings. I used Pascal Poitras's Weechat Hightlight to get my highligh monitor setup.

iset

iset.pl creates an interactive set interface for configuration options. It has a search feature that makes find obscure options a bit easier.

autosort

autosort.py by default groups your channels by server. I'm using this script to mimic GUI chat clients in grouping chatrooms and individual chats under their owning server. More on this in the Advanced Topics section at the end of this article.

Create a sidebar with list of all people from all servers

These steps will create a sidebar containing a list of all people in all your networks: IRC, Jabber, et cetera. It makes use of the weechat /bar command. Read the /help bar output to learn more.

Create the bar and populate it

/bar add bitlist root right 15 on "@irc.Bitlbee.&bitlbee:buffer_nicklist"

Configure the bar. I no longer remember where I found these settings. Your mileage may vary.

/set weechat.bar.bitlist.color_bg = default
/set weechat.bar.bitlist.color_delim = default
/set weechat.bar.bitlist.color_fg = 37
/set weechat.bar.bitlist.conditions = ""
/set weechat.bar.bitlist.filling_left_right = vertical
/set weechat.bar.bitlist.filling_top_bottom = columns_vertical
/set weechat.bar.bitlist.hidden = off
/set weechat.bar.bitlist.items = "@irc.Bitlbee.&bitlbee:buffer_nicklist"  (default: "")
/set weechat.bar.bitlist.position = right
/set weechat.bar.bitlist.priority = 0
/set weechat.bar.bitlist.separator = on
/set weechat.bar.bitlist.size = 15
/set weechat.bar.bitlist.size_max = 0
/set weechat.bar.bitlist.type = root

The result of this will be a column on the right side of the terminal showing the ids of all the people connected to your IRC and chat servers. People online will be indicated by a plus sign, e.g., +somebuddy.

Configure Weechat for IRC

Out of the box weechat is an IRC client so by default it connects you to freenode using generic connection settings. You can customize your connection like this.

Set your IRC nicks

/set irc.server.freenode.nicks “your-irc-nick”

This can be a comma delimited list.

Set your IRC username

/set irc.server.freenode.username “your-irc-username”

Set your real name or pseudonym

/set irc.server.freenode.realname “Your Name”

Configure things to automatically connect

/set irc.server.freenode.autoconnect on

Add the list of channels you'd like to always join

/set irc.server.freenode.autojoin “#channel1,#channel2,#channel3,#weechat”

Change your IRC passphrase

If you ever need or want to change your IRC account passphrase, here's how.

/set irc.server.freenode.command /msg nickserv identify NEWPASSHERE

Joining/parting channels and being away

To join a channel use the /join command:

/join #channel

or use the short form

/j #channel

Or to leave a channel use the /part command:

/part [quit message]

You can mark yourself as away:

/away <away message>

You can also mark yourself as away for -all connected servers

/away -all <away message>

And you can return from being away

/away [-all]

Configure bitlbee in weechat

bitlbee is a gateway to servers that aren't IRC servers. Jabber, Twitter, Yahoo! and others are possible. The setup described below is for a Google Talk and Jabber account.

Connect to bitlbee server

bitlbee runs on port 6667

/connect localhost 6667

The connection will use a default user name based on your IRC name. Since you haven't created or signed into a nick yet the id used will likely be your operating system account.

Autoconnect Bitlbee

If you'd like to have weechat automatically connect to bitlbee, create a server in weechat for the connection. Here the tag im for instant message is used as the server name.

/server add im localhost -autoconnect

Register a password for bitlbee

Change to the control channel called &bitlbee and register a password for the account. This creates an XML file on the host system named for the user to store connections and chat room information.

register <password>

Create a "instant message" server

Combining the two previous commands, here's how you would create a server called im for "instant message" that automatically connects when you start weechat, and further identifies you to bitlbee using the previously registered password, do this:

/server add im localhost/6667 -autoconnect
/set irc.server.im.command "/msg &bitlbee identify YOUR_PASS"

Add accounts

While in the &bitlbee control channel (denoted by the &) add a Jabber account:

account add jabber you@server.tld <password>

Or add a Google Talk account.

account add jabber first.last@gmail.com <password>

Use the ac list (ac is short for account) to see your newly created accounts.

ac list

If you want to see the full name of an buddies on GTalk and not just their account short name, you can do this:

ac gtalk set nick_format %full_name

Conclusion

I've been using weechat+bitlbee as my primary chat interface for the better part of a year now. I'm still learning the ins and outs of how to tweak this setup. The most recent change has been to have individual bitlbee connections for each service rather than having all services merged into a single &bitlbee control channel. Combined with some buffers.pl settings changes the result is quite nice.

Advanced Topics

Multiple bitlbee users

If you want to separate individual and group chats by service (MSN, Yahoo!, Jabber, GTalk, et cetera) you need to create separate bitlbee users and register a password for each.

In a &bitlbee channel use /nick to change your identity and then use register to set a password for that identity.

/nick gtalk
register <password>

Next create a new weechat server and connect to it.

/server add gtalk  localhost/6667 -autoconnect
/connect gtalk

Move to the &bitlbee control channel belonging to gtalk and switch to to new nick and identify yourself.

/nick gtalk
identify <password>

Now you can add your Google Talk account as described above. Repeat these steps for Jabber or MSN or what have you.

Adjusting the buffer list

These settings changes will make use of the multiple bitlbee (and IRC for that matter) connections.

/set irc.look.server_buffer independent
/set buffers.look.indenting on
/set buffers.look.show_number off

By default the server buffers are all merged together (merge_with_core), using independent splits them apart. Turning indenting on causes the channels belonging to a service to be indented. And turning buffering numbering off cleans up the look.

Here's an image of the final result.

weechat

New Year

January 01, 2015 | posted in: site

The start of a new year always inspires people to start new ventures, set new goals, and make resolutions. Quite without prior planning or thought I've added two new sub-domains to my site today.

Books

I read. I read a lot. I frequently am reading more than one book at a time. Books is where I plan to track all the books I read or listen to this year.

Health

New year beginnings seem to attract health resolutions. I'm not making a resolution. What I am doing is keeping score; tracking my exercise and eating habits. I want to have a more active lifestyle overall and to increase my general fitness. Toward that end I bought a treadmill desk for my office where I hope to walk at least a couple, three, hours every work day. At home we also have a treadmill, and my wife and I take walks most days around the block. By cataloging my health metrics for a year I hope to see patterns, good and not so good, and develop better habits. You can follow along on the new health sub-domain.

Changing My tmux Command Prefix to Tic

December 27, 2014 | posted in: nerdliness

By default the tmux Command Prefix is Control-B (<C-b>). As this isn't the easiest key combination on the standard US layout QWERTY keyboard most tutorials suggest remapping it to Control-A (<C-a>). I went one step further and remapped my Caps Lock key to be a control key (I wasn't using it anyway). Now any time I need to communicate to the tmux server I can <C-a> plus a command key.

The tmux commands I issue frequently are:

<C-a> c - create a new pane inside the current window
<C-a> , - rename the current pane
<C-a> # - switch to the pane numbered #
<C-a> d - detach from the current session
<C-a> s - synchronization toggle, syncs all panes or turns sync off
<C-a> o - maximizes current pane by minimizing all others
<C-a> i - equalizes size of all panes
<C-a> z - hides all but current pane (great for copying)

Some of these -- creating a new pane or renaming a pane -- happen infrequently in the life of a tmux session. Others -- switching panes using their index number -- happen dozens of times a day.

Using <C-a> as the Command Prefix adds a two-key combination to each of the tmux sever commands I issue. Until quite recently this wasn't an issue. However I watched several installments of Learn tmux the other day and one of the first suggestions made was to use the backtic character: ` as the Command Prefix.

The trick to this mapping is to set up your .tmux.conf file to pass along a ` when two are pressed back-to-back. This way you can still utilize the backtic character (as I have creating this posting).

unbind C-b
set -g prefix `
bind ` send-prefix

The unbind C-b removes the default Command Prefix binding (Control-b). The set -g prefix ` make ` the new Command Prefix key. And bind ` send-prefix uses send-prefix to pass ` along to the application. Without the last line above you'll lose the ability to type `.

After having used tmux with <C-a> as my Command Prefix for the better part of two years now, switching to ` will take some effort, but I think the savings in key strokes over time will be well worth it.