I have 2 SmartOS servers at home with a total of 17 zones running persistently - a majority of these provisioned with the latest LTS support release of pkgsrc: 2014Q4. All zones are setup to download a similar set of “bootstrap” packages to make them ready to be used. Packages like gcc, git, etc. I pull to all of my zones upon creation.

The problem, however, is my home internet is not the fastest - it usually can pull packages at around 100KB/s. git and gcc alone are responsible for over 100MB of compiled binary data, meaning these initial package downloads can take over 15 minutes per new zone.

A possible solution to speed up initial package downloads I investigated was to rsync the entire pkgsrc tree to a local server, and host it over an internal HTTP server. This, however, required a massive amount of storage dedicated to packages I am almost guaranteed to never use. I could have just pulled 2014Q4, but then I would have been required to add new releases whenever I wanted to upgrade, and keep around the old packages until all my old zones have been upgraded.

Instead, I created a Node.JS HTTP caching server to proxy GET and HEAD requests to Joyent’s pkgsrc server, and cache the data on the local filesystem. The requests will be proxied if the file does not exist locally, and will be dual-written to the requesting client, and the local filesystem. Subsequent requests for the same resource will be streamed from the local filesystem without every making an outbound request.

fs-caching-server

https://github.com/bahamas10/node-fs-caching-server

The program is written to be generic - it allows it to act as a caching proxy to any website, and also allows the user to specify a regex to test a URL when deciding if the request should be cached, or just proxied directly with no cache.

Read More...

For the last couple of years I’ve used PFSense as my home router. It’s been great - it’s easy to manage with the web interface, and really easy to add features like an OpenVPN server, bandwidth monitoring, etc.

But, I like to manage all of my servers and zones at home using Chef, and my router has always been left out as it required being configured manually through the web interface. So now, I’ve replaced PFSense with 2 SmartOS zones: 1 for NAT and the other for DHCP - both managed by Chef, and both monitored with Nagios. This also had the added effect of reducing two physical servers at home down to one, for a cheaper power bill.

There are 3 steps required to configure a SmartOS server as a home router replacement.

1. Add the External Interface
2. Create the NAT Zone
3. Create the DHCP Zone

1. Add the External Interface

The first thing is to ensure that you have 2 (or more) NICs on the SmartOS server - one for the external network (The Internet) and one for the internal network that will be behind NAT. To list the interfaces run

root - datadyne sunos ~ # dladm show-phys -m
LINK         SLOT     ADDRESS            INUSE CLIENT
rge0         primary  f4:6d:4:X:XX:XX    yes  rge0
e1000g0      primary  0:4:23:XX:XX:XX    yes  e1000g0

Cross-referencing this information with output from ifconfig(1M) reveals that the e1000g0 interface is not currently in use, and can be used as the external interface.

root - datadyne sunos ~ # ifconfig e1000g0
ifconfig: status: SIOCGLIFFLAGS: e1000g0: no such interface

Read More...

Sonos is a fantastic product that makes it easy to set up whole-house audio in a piecemeal fashion. I started with one speaker, then a couple weeks later was up to 4 speakers, and a couple months after that the whole house is equipped with enough speakers for music to be heard anywhere.

In order to stream your own music collection to Sonos, it must be available over the network using a supported protocol - sadly at the time of this writing NFS is unsupported by Sonos. In order to get Sonos up and running on my network, I setup a cifs/samba read-only share with my music collection inside a SmartOS zone.

This post is basically a modified version of Jonathan Perkin’s and Thomas Merkel’s posts covering mounting in a shared directory with music using lofs.

create the zone

To create the zone you have to first create a JSON manifest to be used by the vmadm(1M) command. This is the JSON manifest for the cifs zones called cifs.json I used. The filesystems array mounts in /goliath/entertainment from the Global Zone to /entertainment inside the zone. I do this because my entertainment directory is used by multiple zones - for example my plex zone uses it to share my plex library.

Read More...

djbdns is a software package for running a secure, fast, and simple DNS server.

djbdns is not actually a program itself, but instead is a collection of programs that can be used together to create a full caching, forwarding, and authoritative DNS system - this post will show how to setup all the necessary programs on SmartOS to mimic my home setup.

I use djbdns at home to give me:

1. DNS lookups for my internal network: rapture.com and 10.X.X.X
2. DNS caching daemon for quick look ups: it forwards to OpenDNS and caches the results

Installation

To install the suite of tools, run

pkgin in djbdns gmake

Note: the GNU version of make isn’t needed specifically, any implementation will suffice.

This will install a lot of programs, but the most important are:

• tinydns: a DNS server daemon
• dnscache: a recursive DNS caching daemon

The next step is to create the configuration directory which will be used later

mkdir -p /opt/local/etc/djbdns

Read More...

__dirname is a variable that is available to all scripts that are run by Node.JS - it contains a string that refers to the directory name where the currently executing script is contained. The variable is set on a per-file basis, so any script that is sourced using require will have its own __dirname variable that points to the directory where the script itself is contained.

Example In Node

Take the following node script located at /tmp/node-one

console.log(__dirname);

When executed, we can see

$ node /tmp/node-one
/tmp
$ cp /tmp/node-one /var/tmp/node-one
$ node /var/tmp/node-one
/var/tmp
$ cp /tmp/node-one ~/node-one
$ node ~/node-one
/home/dave

And, when using require, given the following script in ~/two.js

require('/tmp/node-one');

When executed, we can see

$ node ~/two.js
/tmp

Even though ~/two.js resides in /home/dave, it prints /tmp becasue the script being sourced resides in /tmp.

Because of this behavior, it is very easy and elegant for node scripts to require one another without needing to know an absolute path ahead of time. By using only relative require statements, all paths will be made relative to __dirname implicitly.

Problems

This behavior however, relies on very specific situations for the script to be executed. Imagine the case where node does not know where the JavaScript bytes are coming from. For example, using the original node-one script above:

Read More...

Here’s a quick hack to get the perl plugin working for WeeChat on the latest (2014Q4) [SmartOS] (http://smartos.org)

weechat is now in pkgsrc, but it appears to have been built without perl support, which renders certain plugins not working with errors like:

/script install pushover.pl
script: script "pushover.pl" can not be installed because plugin "perl" is not loaded
/plugin load perl
Error: unable to load plugin "perl": ld.so.1: weechat: fatal: perl: open failed: No such file or directory
If you're trying to load a script and not a C plugin, try command to load scripts (/perl, /python, ...)

How

To quickly work around this:

1 install weechat from pkgsrc

pkgin in weechat

This will install weechat without the perl plugin

2 compile weechat at the same version from source

wget https://weechat.org/files/src/weechat-1.0.tar.gz
tar xzf weechat-1.0.tar.gz
cd weechat-1.0
mkdir build
cd build
cmake ../ -DPREFIX=/opt/local
make

THIS WILL FAIL, version 1.0 does not compile cleanly on SmartOS. However, perl.so has been built, which is all we need

3 move the plugin into place

sudo cp ./src/plugins/perl/perl.so /opt/local/lib/weechat/plugins/

4 it works

/plugin load perl will now work as expected

todo

1. have weechat in pkgsrc built with perl support - https://github.com/joyent/pkgsrc/issues/252
2. fix weechat build problems on SmartOS - https://github.com/weechat/weechat/issues/381

These are some of the configs I’m using on my Amazon Fire TV to make XBMC / Kodi work on it without problems.

Buffering

Add the following to advancedsettings.xml to eliminate sporadic buffering

/mnt/shell/emulated/0/Android/data/org.xbmc.xbmc/files/.xbmc/userdata/advancedsettings.xml

<advancedsettings>
        <network>
                <cachemembuffersize>209715200</cachemembuffersize>
                <readbufferfactor>4.0</readbufferfactor>
                <buffermode>1</buffermode>
        </network>
</advancedsettings>

Keymap

Add this to dave.xml (or anything .xml) to get a sane keymap.

• back button stops the current video
• rewind and fast-forward work as expected

/mnt/shell/emulated/0/Android/data/org.xbmc.xbmc/files/.xbmc/userdata/keymaps/dave.xml

<?xml version="1.0" encoding="UTF-8"?>
<keymap>
        <FullscreenVideo>
                <keyboard>
                        <backspace>Stop</backspace>
                        <rewind>Rewind</rewind>
                        <fastforward>FastForward</fastforward>
                </keyboard>
        </FullscreenVideo>
</keymap>

I’ve accepted a new challenge: every week for the next 52 weeks, I will compose, record and publish a new a song. I’ll have a week to work on each song, and they will be “due” each Sunday for the next year.

1. 12/7/2014 - The Guardian
2. 12/14/2014 - Carol of the Bells
3. 12/21/2014 - Retro Drive
4. 12/29/2014 - High-Rise Lights
5. 1/5/2015 - Vagrant
6. 1/13/2015 - Winter Lullaby
7. 1/19/2015 - The Knight
8. 1/25/2015 - Long Exposure
9. 2/2/2015 - Icicle Falls
10. 2/10/2015 - Pipe Dream
11. 2/16/2015 - Clematis
12. 2/23/2015 - Crow’s Nest
13. 3/2/2015 - Flutter
14. 3/9/2015 - Heliograph
15. 3/16/2015 - Beacon
16. 3/23/2015 - Beacon (8bit)
17. 3/30/2015 - Aura
18. 6/14/2015 - Nine Valiants
19. 7/25/2015 - Heliograph Beacon

Update

I gave up after 19 tracks… sorry!

Checkout my album Memories, released July 16th, 2014, on iTunes and BandCamp

Also available for free on my new website music.daveeddy.com

This is the culmination of ~10 years of music composition and recording, and you can buy all 19 tracks (68 minutes of music) for only $10.

Red, yellow, and blue are the primary colors… right?

A couple years ago, I was talking with Skye, and she was telling me how she was annoyed with color pickers on computers found in most image processing and manipulation software, and how she wanted to use a real color wheel or color picker.

I was confused. I’ve seen plenty of color pickers online, all showing the standard rainbow array of colors you are used to seeing with plenty of output formats like hex, rgb, hsl, etc. When I asked what she meant, that started me on a huge journey into color theory and how it relates to computers.

In school, I was taught the primary colors are red, yellow, and blue. With these colors, all other colors can be created by mixing them together. However, with monitors, TVs, projectors, etc., the colors that are used are red, green, and blue, typically referred to as RGB. Why is this different? How can you substitute out yellow for green and still get the same array of colors?

Read More...