I just upgraded my OpenSolaris box from 2009.06 (snv_111b) to the latest development build (snv_131 from January 14th, 2010). The main reason was the broken CIFS server that comes with 2009.06. And although the next release is pretty much around the corner (2010.02), I needed the beast in a working state now.

The good thing about such upgrades is that thanks to ZFS and its snapshot functionality, a roll-back is trivial. The package manager will automatically create the snapshot and also create a new entry in the Grub menu for the upgraded OS. So if things go badly wrong, you can just select the old version and continue from there.

The upgrade itself is a three-step process

• Change repository URL from release to development builds
• Upgrade package manager
• Upgrade other packages

The URL change for the repository is done with the following command
$ pfexec pkg set-publisher -O http://pkg.opensolaris.org/dev opensolaris.org
The update of the package manager is done like that
$ pfexec pkg install SUNWipkg
To start the upgrade from a shell, run this command
$ pfexec pkg image-update
If you want the GUI version, please execute
$ gksu packagemanagerand press the “Update All” button.

Part of the upgrade (bug or feature?) is that the permissions of /dev/ptmx will be changed. The result is that shell access via an SSH login is not possible any more. Since my box is headless, SSH is the only way to work on it. So it’s a real problem. Fortunately, it is only the shell access that is blocked (and this not only via SSH but universally) and giving the error message:
PTY allocation request failed on channel 0

The execution of commands via SSH is still working. So you just have to issue the following command:
$ ssh -l user host "pfexec chmod 666 /dev/ptmx"

which will fix the problem. It seems that for people working directly on the machine, the solution is to blindly enter the command from above.