The site might be a bit temperamental as I have switched my blog from Linode to DigitalOcean.
I had to re-learn how to setup PHP/MySQL/NGINX etc, plus import my blog from an export of a few days ago.
My biggest problem has been permissions – real pain in the ass!
Back to normal in a few days
As much for my own reference, this is a diagram of the technologies I now use. I never did get round to making a post about the switch to Ruby on Rails. If I get some time free I will do an expo on how these all fit together. For now, here is what is busting my brain at the moment:
I have just spent in excess of 200 hours trying to get intel ICH type Raid (controlled partly in the intel ICH BIOS) to work with Ubuntu. Luckily I didn’t gnaw all my fingers off in frustration, so I felt I had a duty to write it up in the hope of saving some other poor soul the nightmare I had. These are instructions based on my dual boot Ubuntu 12.10 and Windows 7 x64 setup, though I expect they will be very useful for anyone trying to get this working. You could go all software raid, but why not have the best of both worlds?
Before I start I would like to point out a few interesting observations:
- Currently in Ubuntu 12.10 mdadm is started after the OS uses FSTAB to mount the drives, resulting in Ubuntu not being able to mount your valid array. This is why I have written this guide. I consider this a bug.
- Ubuntu 12.10 uses mdadm version > 3.2. This is very important as post 3.2 supports imsm (Intel chipset raid) metadata properly.
- Ubuntu pre 12.10 I cannot vouch for and previously I had to go software raid, but you could compile mdadm from source.
- MDRAID (mdadm) has absolutely appalling documentation. Getting this working took me to have about 20 different how-to’s open at once. Most sources of information contradict each other in some small way, leading to confusion. There are often 2 ways to do the same thing. If ever something needed a manual, it’s this (I don’t consider The Manual to be a proper manual!)
- Currently in Ubuntu 12.10, on reboot, your assembly is lost and you have to scan and assemble again. I had to hack a workaround for this. Even outputting the scan result to mdadm.conf is not enough to allow it to find stuff on boot.
- You don’t need to worry about the raid initialising and can continue to make changes and reboot safely.
- I cannot comment on DMRAID as, according to Intel, you should use mdadm as dmraid is not developed actively.
- Using this method you can access the array in both Windows 7 and Ubuntu.
- mdadm installs postfix for mail – don’t be spooked by this, it is just a choice of the developer and you don’t need to use it.
- I am a programmer, not a linux expert: if anyone wants to be a smart ass about my method, fine by me, flame away!
- sudo gedit will allow you to edit things like fstab, mdadm.conf, rc.local etc.
- I take no responsibility for your RAID collapsing, computer catching fire, etc.
I will list my mdadm.conf and rc.local at the end for you to look at. Here are the steps I took to get it all working:
- First of all set up the raid volume in the Intel RAID BIOS by booting and pressing CTRL + I (sometimes K or L). The volume name should have no spaces. I went with a RAID 5 array with 5 members of 1TB each.
- If you are doing this at install time, install Ubuntu 12.10 from a live CD/USB and install gparted. There will be a link on the desktop to install Ubuntu once it boots up. You can get gparted from the software centre. kpartx already seems to be installed. Install to the device and not a particular partition. Make sure you selected to format any existing partitions that need wiping at this time as well (from the advanced options in install in the GUI). Reboot!
- Launch a terminal and: ‘sudo apt-get remove dmraid‘ then ‘sudo apt-get install mdadm’ to replace the shipped dmraid with mdraid. Install gparted as well from the Ubuntu Software Centre. Keep this terminal open for the rest of the steps.
- At this point change to root user so that you are not constantly typing sudo, type ‘sudo -i‘ (when done you can type ‘exit’).
- Assemble the array with a simple ‘mdadm –assemble –scan‘. You might also want to check it has worked using ‘cat /proc/mdstat’. You should see your array as something like md126 and it is probably re-syncing. At this stage if you don’t have a Raid array then something has gone wrong: send me a bullet in the post!
- If you launch the disk utility from dashboard you should see your new raid array. Note where it is (e.g. dev/md126) as we will need this for the mount point. Also, you may as well format it now from the disk utility, as it means we can specify the file system for mount in the script we are about to alter in rc.local.
- Next we need to set mdadm.conf: ‘mdadm -E -s –config=mdadm.conf >> /etc/mdadm/mdadm.conf‘ (see intel paper).
- Unfortunately this will not survive a boot even with ’update-initramfs -u‘, so we will need to add something to the boot process to make sure Ubuntu finds our array. From your still open terminal ‘gedit /etc/rc.local‘ and add the lines: ‘mdadm –assemble –scan‘ and ‘mount -t ext4 /dev/md126 /store‘ to the file before the ‘exit 0‘. Obviously you will want to replace ‘ext4‘ with your file system, ‘/dev/md126‘ with the name found in step 6 above, and ‘/store‘ with the location of your mount point. I just created /store in the root file system. This should now start up your array and mount it on boot.
- Now is the time to type ‘exit’ in your terminal to return you to you… um… you know what I mean! If you are getting errors about permissions then maybe you missed the underlined bit in 4 above?
- As a final step you should install samba, launch it and set it up if you want to share the mounted volume with other windows computers. Also, you may want to edit the permissions (chmod/chown) of the mount point. I simply sudo launched nautilus and did it that way (‘sudo nautilus’ from a terminal). Check your share is there first at this point before rebooting. You may also want to install Ext2Fsd on the windows dual boot system so it can see the volume (or you could have set it up using ntfs3g anyway).
- Reboot and pray!
NB: I can’t work formatting in WordPress to get a mono spaced font for the code. Note the use of double hyphens – cut and paste if you have trouble reading it, sorry about that!
I think I have included every step I used here, but am happy to answer any questions you have. Click for rc.local and mdadm.conf and Intel’s Linux Raid Paper. You may also find this cheat sheet helpful. The Ubuntu forums have lots of disaster stories on trying to get Intel raid working too, but it is important to make sure the information is relevant to Ubuntu 12.10.
I found this whole thing a total nightmare and I wish someone with better editing skills than me could do a good article on it. Good luck!
EDIT: Ubuntu would not shut down after this, so you will also need to add a shutdown script that unmounts, then stops the raid. Here are the commands and here is the script. You will need to adjust depending on where your raid is (i.e. it might well not be md126).
EDIT 2: Here is the pdf that I reference from Intel. It seems not to be available any more:
Intel Linux Raid
I am writing this post for just one reason: for the poor souls searching for hours, trying to find out why password_confirmation works when included in attr_accessible, even though it appears not to be defined anywhere.
I was doing Michael Hardl’s great Rails Tutorial and was stumped at where the hell password_confirmation came from. It wasn’t in the database, I had no declaration of it anywhere in my code, so I naturally started searching from the has_secure_password method, which is in the new (in Rails 3.1) authentication module SecurePassword. I see password declared but not password_confirmation – how can this be?
Well it turns out that Rails “automagically” (to quote Michael) creates this variable (or method – I’m not sure which) in the module SecurePassword, even though it does not appear in the code!
There is an ambiguous line in the documentation that could mean they will declare it, but I believe this type of ambiguity is part of Rails biggest problem. For all the goodness that is Rails, there are, in my humble opinion, far too many magical or poorly documented methods that are difficult to figure out by logical investigation. This leads to frequent cries of frustration in the Rails user newsgroup, and I have bumped into the phenomenon many times on my journey.
So, rather than just moan, I am writing about it in the hopes that another person doing hours of searching will find this and have his or her question answered!
Here is my copy of the repair manual for the Samsung T220HD. I am posting it here in the hope that some others will find it useful for repairing a monitor that does not switch on or has some other fault. Usually it is an electrolytic capacitor on the board.
Feel free to post here if you want any advice, though I warn you I am not an expert! Thanks to Michael Livingston for his extremely useful page on this monitor, and for giving the rest of us the idea of having a go ourselves to fix it.
Click here for the manual in zip form.
The last 15 years or so has been an exercise in ‘catch-up’ for governments around the world, to get to grips with The Internet ™. Now they are ahead of the game, and working hard to remove anything they perceive as a threat to their right to control. That means your freedom on the internet is under threat.
The straw that probably broke the proverbial camels’ back was the leaking of American diplomatic information, causing huge embarrassment to the United States government, and the changing of attitudes forever, to the phenomenon of The Internet – politicians and power brokers hate embarrassment. Get ready now for your freedom online to drip away.
At the same time, Capitalism is in world crisis, desperately attempting to hang on to antiquated business models, whilst prosecuting others for their own failure to market efficiently over the web. Companies like Netflix and Love Film plough a lonely furrow whilst the might of the Media Mafia is busy elsewhere introducing authoritarian, Nazi-like ‘law’, written in haste and probably not repented to at leisure.
Do you want a world where International Media Barons draft draconian laws to protect their profits? Well you already do if you live in The USA, UK and many other western so-called ‘democracies’. The USA has a copyright act that now means it has made places like the UK sub-territories, where it can swoop on people, who have not broken law in their own country, but may have done in the USA. As a UK citizen, even though I like the US, I find this law abhorrent and objectionable.
I am just a programmer, but when my product is ready I will offer it at a decent price, knowing that if it a good enough offer most decent people will pay for it if they need it, just like millions now do on Steam. I am sure I will make a good amount of money if my product is worth it. I do not want to see my, and your, freedom on the internet taken away so some ‘Fat-Cats’ can make a bit more money. Enough is a Feast: Please join the fight at the Internet Defense League.
This is just a quick informational post, made in the hopes that one day someone else will not have to wade through seven pages of Google search to find an obscure blog answer to: The Program can’t start because msvcrt-ruby18.dll is missing.
What is happening is you have probably typed:
gem install wxrubylike I did. In that funny way that Ruby/Rails likes to punish you every day, this has now been silently updated to:
gem install wxruby-ruby19Remember to uninstall the old gem first. In true Ruby/Rails fashion this is not made obvious, but has now been updated somewhere in the wiki for wxRuby. In my opinion wxRuby should be defaulted for 1.9 and a separate gem should be made for older versions. Even better – have a little look and see which Ruby is running and install the right version automatically. This would save thousands of man hours and make the world a better place. I would do it, but do not yet know the inner workings of Ruby. I am dedicated, however, to one day help updating wxRuby.
Here is a picture of the error, I hope you found it quickly instead of the hell I went through:
It’s been ages since I last wrote – I was ill again for some months and unable to update the site. I am now back to posting more often.
One of the big changes has been my decision to switch from MySQL to PostgreSQL. I will come back to the installation of this later, but suffice it to say it was not an easy ride given that I am running a compiled version of PHP.
There were several reasons:
- PostgreSQL 9.1 (beta) supports Master/Slave replication and streaming natively with fallover. This is simply fantastic news for anyone running a high availability platform.
- It is more future-proof especially in terms of scalability. I have read horror stories of production projects getting bigger then having to move to PostgreSQL etc.
- I don’t trust Oracle to do right by MySQL. I already fear for Netbeans.
- All the other normal reasons for PostgreSQL vs. MySQL
As I am using Nginx/Suhosin/PHP compiled (using Vlad’s guide
) this also meant recompiling PHP to include the pgsql module. I did this by compiling PostgreSQL first, then adding –with pgsql
to the PHP compile options. In both cases I would strongly recommend doing what I didn’t do first time, and reading installation instructions for both in their respective documentation.
You will also need to setup PostgreSQL with the recommended username of postgres. I didn’t the first time and it just led to more problems. PgAdmin III is a good windows interface to manage the database remotely, though it isn’t for the feint hearted, especially if you use SSH to connect to your server. So, my overall setup is:
- Ubunti 10.04 LTS
- Nginx 1.04
- PHP 5.3.5 with PHP-FPM, APC & Suhosin
- PostgreSQL 9.1 Beta
I am still wrestling with the database design, mostly due to foreign key issues and getting a design that normalises. But, I am going ahead with some site coding in order to learn more about PostgreSQL with PHP.
If anyone has a similar setup I would love to hear how it works so I can compare.
EDIT: My script is deprecated as I am now working in Ruby. See Vlad’s site for further information.
Any perfectionists out there? Paralysed revisionists? Or just plain indecisive? Well if you are any of these then don’t try database design! I think I’m a little bit of all three because I have just spent weeks trying to nail down my database design.
The difficulty is that once The Project ™ goes live, if there is an underlying fault in the logic, or missing bit of logic, this means changes to the database. If it’s not designed properly then it could mean major re-writes. So I have been plodding away trying to make the thing modular, open-ended and future-proof.
But, at some stage I have to finalise and I have set myself next week as a deadline for this. When it’s done I will post a diagram of what it looks like, minus a few labels so as I don’t give too much away! I think this must be as much an art form as a science, but I think it’s pretty much done and dusted.
In other news, I found out at last that my joint problems are due to something called “Fibromyalgia” The last couple of months has been about learning to adjust to a different regime as well as databases and the project! I don’t think it counts as good therapy, but I have started drinking coffee again to help with the fatigue, and discovered and amazing device… an Aeropress coffee maker! Go look it up on Amazon – it’s the best cup of coffee you will ever have!
It's all over!
I finally finished my dissertation this week! I had to finish my degree part-time due to working and the last couple of bits seem to have dragged on for centuries. For some reason, my writing only flows when I’m under pressure, I don’t know why, and coming close to the deadline I just found the words somehow. I still can’t believe it’s all over – huge weight off my mind (waking and sleeping)!
I can now finally relax and put all my time into The Project ™ Expect lots of new stuff on php and general web application development as I get the back end for this working.
Company mugs etc on the way. It’s over! Wahey!
UPDATE: For anyone interested, I got a 2.1!