How I recovered a SATA hard drive

Last night, I had the unpleasant experience of having a hard drive “crash” on me.  The computer turned on, but just sat there, with the ubuntu boot screen essentially stuck indefinitely.  The cause wasn’t too hard to figure out – one of my hard drives had died suddenly. Unfortunately, it was the hard drive to which I’d been diligently moving all my data, preparing the computer to be given away. (Anyone want a giant tower, two monitors and a four year old computer that supports up to 8 hard drives?)

In any case, since I was able to determine that the failed drive was SATA and not IDE, the solution turned out to be a reasonably scary, but simple solution.  (Btw, determining that the drive was broken consisted of unplugging the drive, and working out that the computer was able to boot without it, as it was a storage volume and did not have any system files on it.)

First, I allowed the computer to boot up to where it was stalled.  At this point, I unplugged the SATA cable (NOT the power cable) connecting the drive to the mother board.  The computer was then able to recognize that the boot process was stalled and was unable to mount the drive.  A prompt comes on the boot screen and asked if I wanted to manually repair (R) the drive or skip mounting (S) it.  Picking manual repair by pressing r, Ubuntu then drops you down to a shell prompt.

Once at the shell prompt, you can do something strange – plug the SATA cable back in.  For me, the drive was recognized, but error messages flashed up. (Alas, I didn’t write them down.)

At this point, I tried a manual mount on the drive – and it stalled.  20 minutes later, the drive was not mounted, which seemed unusual.  My suspicion was that the drive was being read but that the file system was misbehaving.  If the drive was truly dead, it should have failed much faster than that.

The solution then seemed somewhat intuitive: running “fsck -v /dev/<drivename>”.  Surprisingly, It was able to see the drive and began recovering the journaling.  After about 35 minutes, the drive was restored to a readable state, and all of my data appeared to be intact.

Not that there’s a moral to this story or anything, but I’m glad I didn’t give up on this disk as it turned out to be a file system problem, not a hardware problem.

However, there were two important things I did learn: 1) SATA hot swapping actually works on desktop computers. 2) If your EXT3/EXT4 file system fails, it may be indistinguishable from a hardware failure at first glance.  When in doubt, try a file system recovery – and SATA is pretty nifty!

(K)Ubuntu – Oneiric Ocelot is out… meh.

Ok, I wasn’t going to blog until I’ve finished my thesis corrections, but I’ve spent the whole day formatting protein/gene names correctly, and I’m showing signs of brain calcification.  So, I’m going to do one post that has nothing to do with science.  I’m saving those for later.

Instead, I thought I’d give up a few thoughts on Ubuntu’s latest release: Oneiric Ocelot (11.10).

First, I have a few caveats.  I’ve been using Onieric for several months.  It’s something I like to do: load up alpha versions and watch them develop.  It keeps you sharp: when you have to trouble shoot things, you learn lots about how the operating system works and you get the pleasure of finding new software improvements all the time.  What’s not to like? At this point, I’ve been running alpha and beta versions on my laptops for about 4 years now, and it has been an enjoyable process – well, at least it was until Natty (11.04).

For the past year, I’ve been increasingly disappointed in Ubuntu because more things get broken than fixed as the development goes along. But, I should explain a few things:  I don’t use the vanilla Ubuntu.

First, I use Kubuntu.  I’ve flipped-flopped between KDE and GNOME a few times, but I always find myself gravitating back to KDE.  The ability to customize things so deeply has always kept me coming back. (For instance, I remapped the Eject button on my macbook to the eject command last night in the KDE settings panel, and I think that’s the cat’s pajamas.  I’ve done the similar things to enable my keyboard backlight as well.  Stuff like that just makes me happy.)

Second, I love compiz.  I find KDE’s effects kinda lackluster, whereas compiz has a few modules that make me more productive.  I love scale and the desktop cube, because I think well in terms of a 3D desk, and it just makes it easy for me to remember where I left my windows.

So, with that in mind, My ideal desktop is Ubuntu + KDE + compiz.  A combination that ran REALLY well in 10.10.  Actually, that was the last time it did run well – which is a big part of my beef.

After several months of watching Oneiric evolve, I’m really disappointed. I was hoping for many more bug fixes before today’s release, but I just got more and more bugs.  Here are some of the ones that annoy me on a frequent basis:

  • Compiz no longer runs smoothly in KDE.  Flickering and artifacts that were all cleaned up in Maverick are all back with a vengeance.
  • Compiz has been broken so that it hangs repeatedly whenever KDE panels are used – particularly stuff like the new activity manager, or even just resizing a panel.
  • Compiz itself now prevents panels from appearing when it’s set as the default window manager, so that you must kill compiz and start the KDE window manager on every boot just to see your panels.
  • KDE itself has been broken so that the window manager crashes on EVERY single exit.  You can’t shut down the computer without having to hit close on two separate “KDE window manager has crashed” windows
  • There’s the dreaded “Kernel is taking too much power” bug that was only recently “fixed”.  Actually, it wasn’t fixed, but with a few custom kernel parameters, becomes manageable.
  • Oneiric decided to go with the brcmsmac driver for my wireless card.  The driver works fine, but it’s in developmental stage, so there’s no power management for the driver, making it take up 3-5W of power.  For a laptop, that’s just inexcusable – it’s about 30-50% of my total battery draw (usually 11-15W after a lot of manual tuning)!
  • Something totally botched up the NVidia dual monitor support. Even a month ago I was able to drive a second monitor from my video card, but it now fails to do so reliably.  I gave up the second monitor because turning the second monitor on and off 10-15 times in a row in the Nvidia settings panel just to get it to work is no longer a reasonable solution I’m willing to engage in.
  • Then there is the move to “alternate architecture” – I’m still not sure how this is supposed to work, but try getting skype to work.  It’s “fun”.  (Fun being defined as a pain in the ass that involves manually installing i386 libraries that are automagically removed every time you upgrade a package, because the 64-bit version just plain fails to work at all.)
  • Oh, and now Skype can’t see my microphone, but I’m not sure that’s an Ubuntu problem, although I doubt anything will be fixed now that Microsoft has bought Skype. (Again, not an Ubuntu problem…)

Anyhow, you get the idea.

It seems that KDE isn’t a priority for Ubuntu developers, and worse, I don’t think that Ubuntu devs are even aware of the breakage they’ve caused in compiz and KDE while re-purposing it for Unity.  Lack of testing might be one problem, but I suspect that they’re not really even interested in keeping compatibility – which was always one of the core virtues of GNU/Linux for me: interchangable parts.  I have no interest in switching to Unity, but I wish they wouldn’t break everything else for me in the rush to get Unity working for themselves.

Alas, while I’m going to keep using Kubuntu for a little while longer, my love for the Ubuntu distros is fading.  I love bleeding edge, but I’m not a fan of this rampant (avoidable) breakage.

So, my advice – stay away from Kubuntu Oneiric Ocelot – it’s not worth the pain.  With any luck, some of these bugs will be fixed for the LTS release Pulverized Penguin Precise Pangolin.  But I won’t be holding my breath.

Upgrading directly from Ubuntu Maverick to Oneiric Alpha 3.

First of all, this is my experience on macbook pro.  Let me begin by saying this is not recommended, my notes are practically non-existent (I’m just typing this from memory and the “history” command) and Oneiric Alpha three isn’t officially ready for production systems.  That is to say, DO NOT TRY THIS AT HOME!

The other issue that you must be aware of is the ability to put your computer in a non-bootable state very easily.  If you start this process, you must finish it.  If you turn off your computer part way through or before all of the dependencies are configured, you may find yourself with an inoperable computer, or with some subsystems (such as networking) in a completely unusable state – forcing a complete reinstall.

Ok, those disclaimers out of the way, it IS possible to go directly from Maverick (Ubuntu 10.10) to Oneiric (Ubuntu 11.11), but I don’t recommend it.  The process took hours (for me, 5 hours) of (learning how to) manually straighten out dependencies.  It was absolutely not worth the pain.  (It took approximately 500 install/remove/configure commands to straighten out, so consider yourself forewarned!)

To get it started, I brute forced the upgrade by opening my /etc/apt/sources.list file and doing a global search and replace, changing “maverick” to “oneiric”.  At that point, you can issue the command:

sudo apt-get update; sudo apt-get dist-upgrade

That will get the upgrade started.  If you’re ready for 5 hours of agony, press Y.  Before your trial by fire can begin, you may have to wait about while the nearly 1.3Gb of updated packages download. It took about 2 hours for me.

At first, the process will look fine, installing packages quickly, with the update manager doing what it should. However, at some point, the whole upgrade crapped out on me with unresolvable dependencies.  At this point, I used two tools:

sudo apt-get dist-upgrade

and

sudo dpkg --install [deb filename]
sudo dpkg --remove [package name]

Between those two tools (and three commands), you can work through all but one of the broken package problems you’ll face. (I used synaptic to bypass that one, as mentioned below.)

Unfortunately, I don’t have any screen captures, but if you try, you’ll see a message that reads something like:

Package [name1] can't be installed, depends: [name2] (>version).
  [name2] not configured.

Each time, you’ll need to manually try to add package [name2] yourself.  The trick is that all of these files are already downloaded, so you can add them relatively easily by moving into the directory in which apt downloaded them:

cd /var/cache/apt/archives/

The (not so hard) challenge is to figure out the name of the .deb file you need to install – which is surprisingly easy.  Once you’re in the apt archives directory, all of the packages will start with the name of the package as described by the error message above.

The bigger problem is that the dependency relations can be pretty deep.  Sometimes trying to install one package will result in another message that it, in turn, depends on another package.  One by one, you have to follow each lead to figure out what the right package is, and worse, the right order to install them in.

The bigger issue is that periodically, you will find that there is a set of dependencies that you just can’t resolve, in which case, you need to remove the blocking package from the system, install the new package, then re-install the new version of the one you’ve just un-installed.  For instance, I found I had to remove the kubuntu-desktop package to get some of the kdebase packages to install.  It’s not a big deal to do this (although you need to be aware you can accidentally remove a lot of important stuff if you aren’t careful), but you’ll need to be aware of what you remove so you can add it back if (and only if!) there is an updated package for it in the archives directory.

I also found, at one point, that there were packages I could not resolve (icedtea-6-jre-jamvm and openjdk-6-jre, if I remember right) and ended up turning to synaptic. There was a simple “resolve broken packages” command or some such. That was able to get around the set I couldn’t fix manually. I still have no idea how. However, when Synaptic works, it can finish an install for you – while it can’t/doesn’t fix everything, it can be an incredibly useful tool for navigating some complex dependency nightmares.

Periodically, you’ll also discover that you have a bunch of things that are installed, but not configured because their dependencies weren’t satisfied.  You can either re-install them manually, or use the command:

sudo dpkg --configure -a

that will reconfigure any packages in that state.

Other notes – I had to remove xserver-xorg-video-all and xserver-xorg-video-nouveau to get X to work again, as well as blacklist the nouveau driver.  You can see my post here, where I describe it in more detail.  I also had to reinstall kubuntu-desktop at the end, as it was removed earlier.

Finally, I also found I needed to recompile my kernel modules (eg, for nvida-current)at some point when I did a further dist-upgrade.  For those who are familiar with the process, I discovered this on a later reboot when x refused to restart and had to drop to the terminal to make that change.

To give the complete picture, even now, not everything is now working.  Adjusting KDE plasma panels can cause significant instability in compiz (driving it up to 100% cpu), and I had to reset the compiz settings to default before compiz would stop becoming unstable. (I have a diff between the before and after settings, and nothing jumps out at me other than the active plugins and the s0_outputs resolution:

< s0_active_plugins = core;bailer;detection;composite;opengl;imgjpeg;
            debugspew;compiztoolbox;decor;vpswitch;mousepoll;resize;
            commands;imgpng;imgsvg;staticswitcher;move;dbus;resizeinfo;
            regex;text;wobbly;place;session;animation;kdecompat;fade;
            cube;rotate;expo;scale;scaleaddon;
---
> s0_active_plugins = core;bailer;composite;opengl;debugspew;decor;
            cube;kdecompat;vpswitch;resize;imgpng;rotate;move;scale;
            snap;place;

and

< s0_outputs = 1680x1050+0+0; 
--- 
> s0_outputs = 640x480+0+0;

I can’t see that helping anyone, but perhaps one of the packages previously enabled was the cause of the instability.

Hopefully the above is useful to someone else, either by pointing you in the right directions if you follow my footsteps, OR, by keeping someone from following. Either one would be a good outcome!

Good luck, and happy migrating.

Compiz running on Ubuntu 11.10 (Oneiric)

NOTE: October 18, 2011.  I see that this page is the top hit on my blog this week.  I think if you’re looking for more up to date information, you should see my current opinion on Compiz on Kubuntu here.

After months of waiting for things to be resolved with the nvidia drivers and such, I finally buckled down to see if I could get compiz running again smoothly on Ubuntu 11.10.  Happily, I can report that the answer is yes!

Using the latest packages, the trick was to follow some advice in this thread.

First, you need to blacklist the nouveau driver.  It hasn’t been working for me, and it has been preventing the nvidia driver from being loaded.

sudo nano /etc/modprobe.d/blacklist.conf

add “blacklist nouveau” to the end of the file, then save and close it.

Next, you do need to update initramfs and  reinstall the nvidia-current driver:

sudo update-initramfs -u -v
sudo apt-get install --reinstall nvidia-current

From that point on, a simple reboot (and making sure that your xorg.conf actually uses the nvidia driver) should be enough to get you back into Xorg with the Nvidia binary blob… and by opening a terminal and typing:

compiz --replace

You should be good to go again. Things appear to be stable, but I’m sure this will need a lot more testing before I switch back to compiz/kde/nvidia as my default setup.

>xorg.conf file for vostro 1000 using compiz in Ubuntu 9.04

>I’m sure most people aren’t interested in this, but I finally got my laptop (a Dell Vostro 1000) to work nicely with Compiz under Ubuntu 9.04 (Jaunty). I think the key steps were removing every fglrx package on the computer (apt-get remove fglrx*), switching to the “ati” driver in the xorg.conf, and getting the BusID right (I tried copying it from my earlier xorg.conf file, but the value seems to have changed.) However, I added a lot of other things along the way, which sees to have helped the performance, so, for those who are interested, this is the Ubuntu 9.04, jaunty alpha 5 xorg.conf file for the vostro 1000:


Section "Device"
Identifier "Configured Video Device"
Driver "ati"
BusID "PCI:1:5:0"
Option "DRI" "true"
Option "ColorTiling" "on"
Option "EnablePageFlip" "true"
Option "AccelMethod" "EXA"
Option "RenderAccel" "true"

EndSection

Section "Monitor"
Identifier "Configured Monitor"
EndSection

Section "Screen"
Identifier "Default Screen"
Monitor "Configured Monitor"
Device "Configured Video Device"
Defaultdepth 24
Option "AddARGBGLXVisuals" "True"
SubSection "Display"
Modes "1280x800"
EndSubSection
EndSection

Section "Module"
Load "glx"
Load "dri"
EndSection

Section "DRI"
Group "video"
Mode 0660
EndSection

Section "ServerFlags"
Option "DontZap" "false"
EndSection

Section "Extensions"
Option "Composite" "Enable"
EndSection

>Ubuntu Jaunty Alpha

>Wow. Really, wow. I upgraded to Jaunty (Ubuntu 9.04 Alpha) over the weekend on my work computer. It wasn’t flawless, and I wouldn’t recommend anyone else do it yet since it’s far from a trivial process, but I’m very impressed with the results. The computer is more responsive, several bugs that were annoying me have disappeared, and the monitor just looks nice.

But, since I had problems, I figured I should leave a trail for those who are following in my footsteps.

I had problems installing the nvidia drivers. My old xorg.conf failed miserably on the new configuration, and no end of tweaking seemed to fix it. In the end, I settled for using the standby:

sudo dpkg-reconfigure -phigh xserver-xorg

You’ll note that without using -phigh, the command no longer works. Either way, this fixed several issues (multiple instances of X trying to start, bad configurations, inability to install new nvidia drivers, etc), and left me with a much cleaner xorg.conf file.

Once the new xorg.conf file was installed, I was able to install the new nvidia drivers, which had to be done manually (drop to terminal by pressing “ctl-alt-F1”, stop the current gdm with the command “sudo /etc/init.d/gdm stop”, and then running the nvidia binary with “sudo sh NVIDIA-Linux-x86-96.43.11-pkg1.run”. Substitute your appropriate version/file name as necessary.)

You’ll have to enable the nvidia driver as well, which can be done with the command:

sudo nvidia-xconfig

From that point on, I had to manually add back a couple of things:

To the end of the file, I put the following, to re-enable the control-alt-backspace combination, which is, for no discernable reason, removed from Jaunty.

Section "ServerFlags"
Option "DontZap" "false"
EndSection

At one point, I also tried adding dontzap from the repositories, which didn’t help, but may have been necessary for that code to work:

 sudo apt-get install dontzap

To the Screen section, I had to add back

    Option  "AddARGBGLXVisuals" "true"

in order to use compiz again, which seems to require this flag.

To get my monitor showing the right resolution, I also had to edit the “Modes” line in the Screen section:

Modes      "1920x1200" "1600x1200" "1280x1024" "1024x768" "800x600" "640x480"

as my monitor’s native resolution (1920×1200) wasn’t included in the list. Putting it to the front makes it the default mode.

Compiz itself required several libraries which weren’t installed:

sudo apt-get install compiz compiz-core compiz-fusion-plugins-extra
compiz-fusion-plugins-main compiz-gnome compiz-plugins compiz-wrapper
compizconfig-backend-gconf compizconfig-settings-manager libcompizconfig0

Some of which were present, others weren’t, so I asked it to reinstall the whole stack.

Java, for some reason, became really slow – probably because it changed the default away from Sun’s version 1.6 to something else. That should be switched back with:

sudo update-alternatives --config java

Firefox was upgraded at one point, to version 3.0.7, which seems not to be backwards compatible. (it was no longer able to go back a page, or remember previous pages or bookmarks.) Blowing away the ~/.mozilla/firefox directory fixed it, but lost all my bookmarks and settings. I also had to remove the lock files, but if you’re blowing away the firefox directory, that will be included as part of it.

I’m leaving out several other steps I did that shouldn’t be necessary for anyone else: the computer hung midway through the install, and had to be recovered at the command line, several repositories were tested out, as well as multiple rounds of update/upgrade/autoremove from the command line were also used, and a bit of tweaking of other features.

I would also give a couple of points to people who are considering doing this upgrade: backup everything you value. config settings, xorg files, your compiz settings, etc. Several config files were reset back to default or had to be blown away to get things working again, and, although minor, took some time to recover.

Was all that worth the 6 hours of tweaking? So far, I think so!

>Ubuntu Intrepid Beta

>I just upgraded my laptop from Ubuntu 8.04 to Ubuntu 8.10, and I have to say it was – by far – the easiest OS upgrade I have ever done in my life. I had been waiting for the new fglrx driver for my video card, as I use compiz heavily on this notebook for application switching… but now that it’s here, I just gave the command to upgrade “update-manager -d” and off it went.

Going from 7.04 to 7.10 and then to 8.04 gave me all sorts of weird, but entertaining problems – wireless died, or the monitor changed resolutions or whatever. This time – Nothing! I didn’t even see the usual set of weird errors (such as incompatible fonts and locale settings) in the upgrade terminal.

It asked a few relevant questions – kde vs gnome (gnome), if I want to install the proprietary fwcutter for my wireless card (yes) and a password for mysql-5, which it later uninstalled (strange…). And off it went.

It took about an hour and a half, of which most of it was entirely unsupervised. (A couple config file changes were offered for my approval, where I had changed system settings.) Otherwise, it took care of all those things I would normally do, such as uninstalling old kernel versions and clearing out unused packages. All in all, a very pleasant experience. (I even had my IM programs running throughout the process, and continued talking with people throughout.)

On reboot, I instantly noticed a speedup to the boot – there were fewer pauses, and the process seemed somewhat quicker because of it.

Once back into Gnome, you’ll notice not much has changed – nothing broke during the install! That’s a nice change.

When things do change unexpectedly, menus pop up to warn you. (eg. converting my exit Ubuntu icon to a user switcher/msn status combined icon.) Information was offered right away with the lightbulb icon, and the option to retain the old icon was provided as well. Very considerate of a new OS to give you a choice!

Finally, one of the things I though would break would be my previous hack to get the screen brightness keys working, which I deliberately overwrite during the upgrade. Instead, the screen brightness buttons now work even better than before. Where my previous solution had given uneven increments of brightness, which were hard to adjust, it now actually goes from light to dark in pretty even steps. Very nice!

All this in the first 5 minutes after the install… Anyhow, now it’s time to get back to work and actually enjoy Ubuntu 8.10!

>Hardy Heron, Ubuntu 8.04 beta

>So I took the bait. I can’t refuse Beta operating systems. I’ve always enjoyed living on the bleeding edge when it comes to new and shiny OSes, so I upgraded my laptop just to see what would happen.

Well, no surprise, I broke a few things, though surprisingly, it’s a pretty small list. Going from Gutsy Gibbon to Hardy Heron on my Vostro 1000 was very smooth, and required very little effort to get it up and running again, for the most part.

Compiz (3d graphics): Well, no surprise, It was somewhat of a hack to get it going on my laptop, originally, but the only thing that broke during the upgrade was a weird error about gnome-settings-daemon not being able to start. The fix:

sudo apt-get remove xserver-xgl

Unfortunately, I’d tried someone else’s fix of blowing away all of the .gconf, .gconf2, etc settings in my home directory, which obviously didn’t help, but now has me reconfiguring all of my settings… oh well. Just use the line above, and save yourself some of the pain.

However, removing xserver-xgl will probably kill your ability to run Compiz, and you may need to turn composite on in your xorg.conf file (/etc/X11/xorg.conf). That can be done by finding the lines:

Section “Extensions”
    Option “Composite” “0”
EndSection

and changing the “0” to “1”. Tada, Compiz works again!

Wireless: This one was interesting: there’s a new driver for the wireless card, so it tries to install that over what you’ve already got going. The new driver is much better than the old, so its a good thing, but you still need the (not-open-source) firmware from broadcom. Installing it couldn’t be easier.

Follow the first 3 lines here to uninstall ndiswrapper, if you had it running before. (You don’t want the new driver running at the same time as the old: this may mean you might want to take the out any lines saying ndiswrapper or bwcml43, or similar from /etc/modules.)

Follow the instructions here, to install the b43 driver firmware, although my simplified version would be:

sudo apt-get install b43-fwcutter
wget http://downloads.openwrt.org/sources/broadcom-wl-4.80.53.0.tar.bz2
tar xjf broadcom-wl-4.80.53.0.tar.bz2
cd broadcom-wl-4.80.53.0/kmod
b43-fwcutter -w /lib/firmware wl_apsta.o

You’ll notice I’ve skipped a few steps compared to what’s on the linked page. “b43-fwcutter” can be installed through apt-get, so there’s no point in building it yourself, and /lib/firmware is the right place to put the firmware in Ubuntu, so you don’t need to be fancy about exporting it and then using a shell variable. And that did it for me.

You may also need to go to System->Administration->Hardware Drivers to tell Ubuntu to use the b43 driver. After that, my FN+F2 key works again, and I was able to connect to my wireless network.

All is good again with my laptop.