…40 Hours Later, the Joy of NFS

For the past 40 hours or so, my little notebook has been serving an NFS volume to my desktop.  Both of them are running recovery CDs.  The desktop is running Photorec, the notebook is running an NFS server.

This morning, when I was looking for the power supply for my workstation, I pulled the cord and saw the little notebook turn off.

Unfortunately, the little notebook has no battery in it.  The existing battery got toasted ages ago.   40 hours into scouring a drive with that utility could be lost… if not for the fact that NFS and EXT3 are so absurdly robust.

I powered on the notebook, mounted the ext3 FS, edited the exports file, ran dhcpcd, and poof.  The NFS mount was restored to its former glory.

15 hours or so to go before I get to see what kind of images come out of a data recovery attempt on that mostly virgin drive.

a bad day.

So I backed everything up, rebuilt the OS and restored what I could.

What I could?

Yep.

For some reason my tar command got the first 40GB of my data, but stopped thereafter.  I don’t care so much about the recoverable stuff, and I got most of the unrecoverable stuff, but a complete, organized set of photos from Greece, my entire MP3 collection and a few other things are missing.

The photos tick me off the most.  I only have a copy of half of them on a drive that I originally unloaded a lot of them from and a few others on an SD card.  Burning this to disk was on the to-do-list

I’m experimenting with PhotoRec to scour my drive for jpg and mov files.  Yeah, a new OS was loaded on there, but we’ll see what I can get.   This thing is interesting from a forensic perspective.  Awesome tool.

http://www.cgsecurity.org/wiki/PhotoRec

There are Two Copies…

… of the NTFS boot record. One at the beginning of the NTFS volume, one at the end.

http://www.heavygravity.com/2007/01/24/easily-restore-a-corrupt-ntfs-boot-sector/

“Could not find a valid backup boot sector”

Pah.

I found a recovery boot disk, it came with TestDisk 6.6.

• Recover boot sector
• search mft
• Rebuild Boot Sector

No go.  hmmm…

Downloaded 6.9

http://www.cgsecurity.org/wiki/TestDisk_Download

executable binary. Nice.

An exhaustive search using the tool found the second file table.  I was able to write out the partition table, although the machine still won’t boot.  Given this machine evolved from chaos more than being intelligently partitioned, and the resulting partition table doesn’t look quite right, I’m going to back up the contents of the drive onto that little 250GB notebook drive I just picked up before I go any further.*

This’ll take a while.

* yes, I know I should have taken a dd of the drive before doing anything…

Windows has Betrayed Me

I was getting so tired of fighting, that I simply had to try reactivating Windows. Something changed at the MS end, so the automated line worked just fine.

It was a breath of fresh air to be able to adjust my monitors to reflect my physical desktop layout, to hear more than one sound being played at a time, etc, etc, you’ve heard it all.

Anyway, for some reason, Windows tagged my Linux disk as a FAT12 disk and called it “C:”. This shifted all my Windows drive letters and trashed all my apps. Annoying. A bit of finagling, and a touch in the registry later, I managed to get it back to C:, but it would not boot properly.

Now while it is nice that Ubuntu gave me the option to boot Windows, it did not direct it to the Windows boot loader, it directed it to the OS boot. This means I’m SOL if I’m trying to get to the recovery console or safe mode.

So I pull out my grey-market DVD to try to get a recovery console together for Windows. Then I make a…. mistake.

fixboot. I’ve got to look into this. FAT12 encoded bits were vomited on to my pristine NTFS and EXT3 partitions. Stupid underdocumented Microsoft command line utils.

This includes a lot of personal data (there’s plenty backed up, but given that I was fighting with Linux to play sound properly and meet my basic needs for a month, backup was unfortunately not on the top of my mind. After all, do you *see* anything about backups in this blog?

The good news is that I merely toasted what I think would be the boot record, partition table and boot block. Most of NTFS and hopefully EXT3 is kept in the “middle” of the disk. Both my Windows boot partition and my Ubuntu partition are trashed. The partition table on the Ubuntu side is truly jibberish, but I have trouble imagining how Windows vomiting some FAT12 data on to a EXT3 and NTFS volume would result in *real* data loss. I’ll probably have to rebuild the OSes, but the data should be fine. It’ll just be a trick to figure out how exactly to extract it.

Playing an AVI

Okay. I did some flirting with the dark side again.

Somebody told me that I should see the film “Wickerman” but under no circumstances should I pay for it to contribute to the rottenness and depravity that is the film. It would be unethical, so I was directed to leech it through a torrent.

So, I leeched a nice 700MB AVI through a Torrent.

To play the AVI, I had a problem. There was a luminous multihued ghost image appearing on screen. I tried to manually adjust the video modes, but regardless of the app or the mode either it would show me nothing but a small rectangle of a letterbox in the middle of my screen, it would show me an image with a luminous multihued ghost image a few inches off to the left, or it would play back so slowly and unstable, that it was absurd. To top it all off, the audio would also play with that weird crackle or snap every minute or so.

I was frustrated. Very frustrated. I didn’t want to spend the night hacking with my machine, I just wanted to watch the fricking film so that I could see the horror which is a bad film.

So here’s how I watched the film. Something Linux just was utterly unable to do for me:

1. I copied the film to a USB key
2. I plugged the USB key into my circa 2002 X24 laptop
3. I plugged the 24″ monitor into my circa 2002 X24 laptop
4. I plugged my USB speakers into my circa 2002 X24 laptop
5. I installed Virtualdub, by clicking, downloading and running it

The machine doesn’t do USB 2.0 (it’s too old!), so I didn’t want to wait for it to copy back to the little notebook.

To top it off, when Linux copied the film TO the USB key, it didn’t close the device. Yeah, I was frustrated, but even after the LED on the drive stopped flashing, I was surprised that it didn’t actively try to sync removable devices. Maybe Windows does it by eliminating or continuously flushing the write cache. A write cache on a device which wasn’t around at boot time like that is a bit silly. It didn’t warn me about “unsafe device removal” nor did it offer a method obvious to me to flush the cache. The closest I could find was to right-click the drive icon and unmount it. I had to wait a fair while after the copying “completed” to be able to unmount it without a nebulous “this is in use” error.

Anyway. No word of a lie. Windows automatically set the USB speakers to my default, automatically set up mixing, automatically detected and started using my external monitor, warned me that I was using a USB 1.1 port for a 4GB USB key, and allowed me to install and run Virtualdub, all without even thinking about asking me to reboot.

The video played flawlessly right from the key. Flawlessly. Simultaneously driving a pair of USB speakers and reading the video driectly from the flash key. This machine has 1/4 the computing power, 1/100th the video computing power, yet it completely floored my modern Linux box for ease of use and performance.

I won’t conclude that Linux itself is crap. None of this has anything to do with anything I use Linux for on a regular basis professionally. As a server OS, it is awesome. As a server OS, you don’t touch ATI drivers, you don’t touch USB speakers, plug and play is not an issue, audio mixing is not an issue. You want simple, stable and reproducable. While Package managemnt and stupid config scripts have challenged that, for the mostpart, Linux still kicks ass there.

However, for the desktop, I’m deeply disappointed.

Searching for the OO doc I wrote on speedcontrol.c

Last month, I wrote some blurb about the program speedcontrol.c. I won’t get in to what it does here, hopefully I can find that article.

It’s not obvious to me that Nautilus will let me do this. So I’m trying the command line.

mike@whitetower:~/Documents/lotd$ grep -r speedcontrol.c *

mike@whitetower:~/Documents/lotd$ grep -r -i vlc *
Binary file day 1/sound-daemon.odt matches
Binary file day 4/dmix.odt matches


Well, that’s wrong. I forgot, odt files are zipped XML.

mike@whitetower:~/Documents/lotd$ find ./ -name *.odt -exec zgrep -i vlc '{}' \;


Must not be searching all the contents

mike@whitetower:~/Documents/lotd$ zcat "./day 1/vlc.odt"
application/vnd.oasis.opendocument.textgzip: ./day 1/vlc.odt has more than one entry--rest ignored


Uh huh.

There must be an easier way.

mike@whitetower:~/Documents/lotd$ OFS=$IFS; IFS='
' ; for i in `find ./ -name *.odt `; do unzip -p "$i" content.xml | grep -i vlc > /tmp/$$.find ; if [ -s /tmp/$$.find ]; then echo "$i"; fi; done; IFS=$OFS; rm /tmp/$$.find
./day 3/flash.odt
./day 4/moresound.odt
./0208/today.odt
./day3/flash.odt
./day 1/vlc.odt
./day 1/morestuff.odt
./day 1/sound-daemon.odt
./published/day 0/linux-on-the-desktop2.odt


So a search on “vlc” works.

mike@whitetower:~/Documents/lotd$ OFS=$IFS; IFS='
' ; for i in `find ./ -name *.odt `; do unzip -p "$i" content.xml | grep -i speedcontrol > /tmp/$$.find ; if [ -s /tmp/$$.find ]; then echo "$i"; fi; done; IFS=$OFS; rm /tmp/$$.find
mike@whitetower:~/Documents/lotd$


ok. No reference to it. Meh. I should be able to recall most of it and re-write.

I Miss Windows

I’m having a moment of weakness. I miss Windows. My video is faster, my Youtube can run full screen, my DVDs play smoothly, my sound… works, I can run video smoothing filters, the clipboard isn’t crap, keystrokes are consistent, fonts are better, the machine doesn’t crash when suspended. Honestly, for my Desktop, there is nothing I can do in Linux that I can’t do in Windows, and there are plenty of things in Windows that I can’t do in Linux.

In the next few months, I’m going to be going on a year-long trip, so this desktop machine I’m working with will be hidden in storage for a while. In the meantime, I have a small X24 notebook which will need a bit of touching up to make it good for travel. The battery is toasted on it, I’m not sure if I’ll replace that. It’s got an honest-to-goodness valid Windows XP license on it.

That said, I’m going to pick up a USB memory key, a 160GB drive and probably load Ubuntu on to a main partition. I’m going to do a dual-boot with probably 100GB of shared ext3 disk (Windows XP can load ext3 drivers). In theory, I should be able to use the Windows drive for video filters and all that.

Given that the machine has less exotic video and less exotic sound hardware, it should work fairly smoothly out-of-box.

Dual-boot is partially to get out of potential airport security issues. For those that don’t know, the airports have been asking people to boot their machines to search them for porn. It will allow me to apply Truecrypt, and the command line will be handy for remote backups. In the event that airport security demands to search my machine, I can help them by booting Windows. Given that I won’t use it for my personal finances, email or MP3s, I won’t have to worry about them freaking out because I won’t give them my banking info, email, MPAA/RIAA/SOCAN/Whatever collusion etc. I figure the worst they should do is confiscate my machine, the worst they might do at a U.S. border is to send me to Guantanamo to get my password.

I might have to set it to use the Windows boot loader and set Linux to boot from a seemingly uninteresting menu item. I’ll have to remember to shut down and not hibernate when crossing the border.

A tech with an iota of knowledge would know something’s not quite right. But the border guards don’t have an iota of knowledge, and the forensic analysis by back-room customs officials should be thwarted by reaonsable encryption… all that and I’m not doing anything wrong.

I guess I’ll be fighting with Linux a bit longer.  Truecrypt might actually be a killer app.

Wresting with gvim to post Old Experiences

Today I’m going back and adding a lot of what I experienced back in late January while working with this machine. To do this, I threw gvim on to my machine.

Now in Windows, I could just go into something like WordPress, do ctrl-a, ctrl-c, alt-tab, esc, i, shift-ins and I would have the full text of whatever I was doing in WordPress in gvim.

It seems that this version of gvim doesn’t like the Windows editing keys. Grr. That’s the first I’ve seen of that. Ctrl-x, Ctrl-v and Ctrl-c I can understand, but Ctrl-Insert and Shift-Insert?
Consistency in a UI is a key part of user friendly interfaces. It’s okay to break it to enhance speed, but is [quote], +, g, Shift-P faster or more intuitive than Shift-Insert?

I guess it’s all a pointless gripe. I don’t have the time or energy to try to figure out how to reconfigure/recompile/repackage or whatever to get my editing keys to have some semblence of normalcy.

Time for a different tactic…

It looks like installing gvim through Add/Remove… has set my systemwide vi to vim, and gnome-terminal seems to not have a problem with Ctrl-Insert/Shift-Insert. Although I can’t do a Ctrl-a to select all, which makes gathering the edited text to paste back into a browser text editor kind of absurd.

So I ask Google… “read a file into the clipboard command line linux”

http://elcasey.wordpress.com/2008/02/12/xclip-use-the-clipboard-from-the-command-line/

I’m impressed. Another WordPress blogger.


mike@whitetower:~$ sudo apt-get install xclip
[sudo] password for mike:
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
xclip
0 upgraded, 1 newly installed, 0 to remove and 9 not upgraded.
Need to get 17.2kB of archives.
After unpacking, 77.8kB of additional disk space will be used.
Get:1 http://ca.archive.ubuntu.com gutsy/universe xclip 0.08-7 [17.2kB]
Fetched 17.2kB in 0s (47.1kB/s)
Selecting previously deselected package xclip.
(Reading database ... 98177 files and directories currently installed.)
Unpacking xclip (from .../xclip_0.08-7_amd64.deb) ...
Setting up xclip (0.08-7) ...
mike@whitetower:~$

Let’s see if this works…

And we paste…. nothing.

middle click?


xxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxx

Oh feck. It uses the X buffer not the clipboard. I am NOT going ot reach my hand over to the mouse to do a delicate middle-click every time I want to move text from vim back to my browser!

grr… more reading.

Found something in that guy’s blog.

Of course…


:%!xclip -o -selection c

vim remembers these commands so I can type :, up-arrow multiple times. It’s a beginning. I might have to toss it in a script.

All that said, anyone know how to fix this dumb gvim default?

ATI Video Card Driver Patch

Ever since I installed the video driver on Jan 29, I’ve had problems with X crashing on logout. I was just hacking around, trying to get Compiz to work last night and it crashed again while testing video resolutions. I decided to check for a video card driver update.

It seems that on Feb 13, they produced a new one:

Linux Proprietary Driver – Catalyst 8-2 Release: x86 and x86_64 variants
Driver version: 8.455.2
ISSUES RESOLVED: (1) Fixed Xserver crash on screen resolution change in horizontal or vertical desktop setup with a monitor that does not support DDC. (2) Fixed problems related to BusID entries in xorg.conf in an unexpected format. Symptoms could range from failure to initialize 3D acceleration to Xserver crashes on startup. (3) Fexed intermittent Xserver hangs on logout if atieventsd was running. (4) Fixed hang of the first OpenGL application in an Xsession on Xserver version 1.4.

Maybe this will prevent my system from freezing on logout and WoW might work! Woohoo!


mike@whitetower:~$ bash ./ati-driver-installer-8-02-x86.x86_64.run
Created directory fglrx-install.vD7255
Verifying archive integrity... All good.
Uncompressing ATI Proprietary Linux Driver-8.455.2............................................................

Hmph. That’s new. The ATI package should only output a deb file which I should be able to install as root. Running this massive ATI script as root is not a good idea, it’s why the “fake root” package was created. Running it the same way as I did on the 29th makes no difference. (i.e., chmod +x ./…. )

It looks like the ATI documentation doesn’t actually say that they support Ubuntu, but from the Ubuntu site, there seems to be an undocumented way to do this:

https://help.ubuntu.com/community/BinaryDriverHowto/ATI


bash ./ati-driver-installer-8-02-x86.x86_64.run --buildpkg Ubuntu/gutsy


Ahhh, the WordPress blockquote truncates long lines of preformatted text


==================================================
ATI Technologies Linux Driver Installer/Packager
==================================================
Generating package: Ubuntu/gutsy
Package /home/mike/xorg-driver-fglrx_8.455.2-0ubuntu1_amd64.deb has been successfully generated
Package /home/mike/xorg-driver-fglrx-dev_8.455.2-0ubuntu1_amd64.deb has been successfully generated
Package /home/mike/fglrx-kernel-source_8.455.2-0ubuntu1_amd64.deb has been successfully generated
Package /home/mike/fglrx-amdcccle_8.455.2-0ubuntu1_amd64.deb has been successfully generated
Removing temporary directory: fglrx-install.a15276
mike@whitetower:~$ man dpkg
mike@whitetower:~$ dpkg -l | grep fglrx
ii fglrx-kernel-2.6.22-14-generic 8.452.1-1+2.6.22-14.47 ATI binary kernel module for Linux 2.6.22-14
ii fglrx-kernel-source 8.452.1-1 Kernel module source for the ATI graphics ac
ii xorg-driver-fglrx 8.452.1-1 Video driver for the ATI graphics accelerato
ii xorg-driver-fglrx-dev 8.452.1-1 Video driver for the ATI graphics accelerato
mike@whitetower:~$ dpkg -r fglrx-kernel-2.6.22-14-generic
dpkg: requested operation requires superuser privilege
mike@whitetower:~$ sudo dpkg -r fglrx-kernel-2.6.22-14-generic
[sudo] password for mike:
(Reading database ... 96630 files and directories currently installed.)
Removing fglrx-kernel-2.6.22-14-generic ...
mike@whitetower:~$ sudo dpkg -r fglrx-kernel-source
(Reading database ... 96626 files and directories currently installed.)
Removing fglrx-kernel-source ...
mike@whitetower:~$ sudo dpkg -r xorg-driver-fglrx
dpkg: dependency problems prevent removal of xorg-driver-fglrx:
xorg-driver-fglrx-dev depends on xorg-driver-fglrx.
dpkg: error processing xorg-driver-fglrx (--remove):
dependency problems - not removing
Errors were encountered while processing:
xorg-driver-fglrx
mike@whitetower:~$ sudo dpkg -r xorg-driver-fglrx-dev
(Reading database ... 96622 files and directories currently installed.)
Removing xorg-driver-fglrx-dev ...
mike@whitetower:~$ sudo dpkg -r xorg-driver-fglrx
(Reading database ... 96611 files and directories currently installed.)
Removing xorg-driver-fglrx ...
Stopping atieventsd: done.
rmdir: /usr/lib/fglrx: Directory not empty
rmdir: /usr/lib32/fglrx: Directory not empty
Processing triggers for libc6 ...
ldconfig deferred processing now taking place
mike@whitetower:~$


This is a serious PITA.


mike@whitetower:~$ sudo dpkg -i fglrx-kernel-source_8.455.2-0ubuntu1_amd64.deb
Selecting previously deselected package fglrx-kernel-source.
(Reading database ... 96525 files and directories currently installed.)
Unpacking fglrx-kernel-source (from fglrx-kernel-source_8.455.2-0ubuntu1_amd64.deb) ...
Setting up fglrx-kernel-source (8.455.2-0ubuntu1) ...
Adding Module to DKMS build system
Doing initial module build
Installing initial module
Done.mike@whitetower:~$ sudo dpkg -i xorg-driver-fglrx_8.455.2-0ubuntu1_amd64.deb
Selecting previously deselected package xorg-driver-fglrx.
(Reading database … 96542 files and directories currently installed.)
Unpacking xorg-driver-fglrx (from xorg-driver-fglrx_8.455.2-0ubuntu1_amd64.deb) …
Setting up xorg-driver-fglrx (8.455.2-0ubuntu1) …
Installing new version of config file /etc/ati/atiogl.xml …
Installing new version of config file /etc/ati/signature …
Installing new version of config file /etc/init.d/atieventsd …
* Starting atieventsd [ OK ]Processing triggers for libc6 …
ldconfig deferred processing now taking place
mike@whitetower:~$

Reboot…… we’re back. I wonder if this did anything? I’ll try logging out.

We’re good.  No WoW though.  that’s probably some other issue.   Meh. Win some, lose some.

lazy umount

NFS has some capabilities which are really quite slick. One of these features is that if you mount a drive by NFS, it doesn’t matter if the server is unavailable, your kernel will continue to wait for it to become available.

Nautilus has been causing me some grief the past few days. Freezing and requiring me to force it to quit through the GUI. I first thought it was my digital camera. Plugging it in seemed to cause this.

Today the camera wasn’t hooked up and I realized it was still freezing.

One of the pains of NFS is that if the server is unavailable, your kernel will continue to wait for an NFS mount to become available.  When this happens, something as simple as an “ls /mnt/lotd” will cause the prompt to sit idle, immune to ctrl-c, ctrl-z, or similar interruptions.

root@whitetower:~# umount -f /mnt/lotd
umount2: Device or resource busy
umount: /mnt/lotd: device is busy
root@whitetower:~# man umount
root@whitetower:~# umount -l /mnt/lotd
root@whitetower:~#

It’s working again. I guess I can’t hold it against Linux because honestly, it’s a pretty neat feature, it would just be nice if there were a GUI timeout on a resource so that it wouldn’t foul up everything unrelated to the mount point.

Can’t have everything I guess.

I used take advantage of this to reboot NFS servers that were serving active thin clients. Even a 5 minute reboot of their root filesystem wouldn’t phase the clients. Despite the PITA that NFS is for firewalls and security, I don’t know of any other mount method that’s quite so robust.

The “force” (-f) is usually enough, but if a process is holding a handle in the filesystem, it won’t allow for the umount. “force” just avoids contacting the remote (down) NFS server. This “lazy” (-l) tells it to break the mount point for all new connections, but existing handles remain open. Now when Nautilus tries to get info for my mounted filesystems, it won’t try to access the dead NFS share.

Rebooting would have fixed this too. Now how do I restart the Nautilus desktop?