Click Technology Limited

Linux, Windows, Macs – it's all good!

Create Virtual Disk from physical hardware


This came up the other day, how do you add a disk to VirtualBox that physically exists? E.G. you have created a test vbox and would like to plug in your old laptop hard drive and have a look at the contents..

Here goes..

sudo lsblk

And get the following similar output..

NAME                       MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
sda                          8:0    0 465.8G  0 disk  
|-sda1                       8:1    0   243M  0 part  /boot
|-sda2                       8:2    0     1K  0 part  
|-sda5                       8:5    0 465.4G  0 part  
  +-sda5_crypt (dm-0)      252:0    0 465.4G  0 crypt 
    +-mint--vg-root (dm-1) 252:1    0 465.4G  0 lvm   /
sdb                          8:16   1   7.5G  0 disk  
|-sdb1                       8:17   1   459M  0 part  /media/your-username/usbstick
sdc                          8:32   0 465.8G  0 disk  
|-sdc1                       8:33   0   243M  0 part  
|-sdc2                       8:34   0     1K  0 part  
|-sdc5                       8:37   0 111.6G  0 part  

Examine the tree and find out what your old hardware is called, e.g. /dev/sdc in this case.

Now create a directory for the virtual disk..

mkdir -p /home/your-username-here/VirtualBox/spare_disks

And use vBox’s commandline tools to create the image, changing /dev/sdX for your device location, prolly /dev/sdc.

sudo VBoxManage internalcommands createrawvmdk -filename "/home/your-username-here/VirtualBox/spare_disks/500GB-hard-disk.vmdk" -rawdisk /dev/sdX

Now chown and chmod the varioius relevant locations and that’s it..

sudo chown your-username-here:your-username-here /home/your-username-here/VirtualBox/spare_disks/500GB-hard-disk.vmdk
sudo chmod 777 "/home/your-username-here/VirtualBox/spare_disks/500GB-hard-disk.vmdk"
sudo chmod 777 /dev/sdX

Now open VirtualBox and add in the disk in the usual way (Select your box, then –> settings –> storage –> select disk controller –> click ‘adds hard disk’ icon –> Select vmdk file and that’s it. Fire up your machine and voila, the machine sees the external disk. Nice.

posted under Linux Tips | Comments Off on Create Virtual Disk from physical hardware

Burn an ISO to a USB stick


Every time I want to do this it seems to take waaay longer than it should. Here then, the instructions in n steps under Linux.

sudo lsblk

Check and see which device is your USB stick. In the commands below, the device is listed as /dev/sdaX, but in reality it will be /dev/sdc probably. Be careful. If you mis-select the device you could potentially DESTROY all your data, in which case you will have to go through all five stages of data recovery grief, namely..

Denial = “It wasn’t me

Anger = “How the fuck did I just do that? What was I thinking, cutting and pasting commands off the interwebs??

Bargaining = “If I reboot the machine, the data will probably come back. Actually, thats almost a certainty.

Depression = “Who suggested rebooting? What a stupid idea that was. I’m completely screwed. Why didn’t I do routine backups, especially of my thesis, my research, my entire music collection and all my photos since 2003.

Acceptance = “Either fork out for a data recovery company or start fresh with nothing.

Consider yourself warned. And also, make regular backups.

So, unmount the USB drive in question..

sudo umount /dev/sdX

Format it as FAT32

sudo mkdosfs -n 'usb-bootable' -I /dev/sdX -F 32

Run isohybrid..

sudo isohybrid /home/username/Downloads/systemrescuecd-x86-4.7.1.iso

Now burn the iso image to USB

sudo dd if=/home/username/Downloads/systemrescuecd-x86-4.7.1.iso of=/dev/sdX bs=4k

Flush data to the USB drive..

sudo sync

And eject the USB drive..

sudo eject /dev/sdX

Finished. Your USB pendrive is bootable and ready to go.

posted under Linux Tips | Comments Off on Burn an ISO to a USB stick



One of the problems with information technology is the rate at which the subject matter changes. This captures it nicely!

Ain't that the truth!

posted under Linux Tips | Comments Off on Classic.

CSV mish mash


I’ve just found a fantastic new set of tools for handling CSV data.  CSVkit allows you to chop, change, splice and cut any csv file you like without all the Excel style overhead. All command line driven on Linux, this is a really fast tool for preparing CSV data for upload to a database.  Perhaps even cooler, once you have your data nicely prepared, you can even generate the SQL statements to create the database on your database server.  Very very sweet and massively time saving too.

As it’s a python too, get these in first..

sudo apt-get install python-dev python-pip python-setuptools build-essential

Then install with pip

sudo pip install --upgrade setuptools
sudo pip install --upgrade csvkit

I just downloaded a huge excel spreadsheet from the LSE with the list of all company names and details on the exchange.

Let’s convert that Excel file into csv

in2csv list-of-all-companies.xls > list-of-all-companies.csv

Nice. The first six or so lines are fluff, so let’s remove them

sed -e '1,6d' < list-of-all-companies.csv > list-of-all-companies-trimmed.csv

OK, great. Now there are a lot more things you could do, such as select the columns you want etc, but I’m just going to create the MySQL table for the CSV file. It can convert to any of these database formats – access, sybase, sqlite, informix, firebird, mysql, oracle, maxdb, postgresql & mssql. Sick.

Right, let’s create the table.

csvsql -i mysql list-of-all-companies-trimmed.csv


CREATE TABLE `list-of-all-companies-trimmed` (
	`List Date` FLOAT NOT NULL, 
	`Company` VARCHAR(35) NOT NULL, 
	`Group` FLOAT, 
	`Sector` VARCHAR(38) NOT NULL, 
	`Sub Sector` VARCHAR(39) NOT NULL, 
	`Country of Incorporation` VARCHAR(2) NOT NULL, 
	`Market` VARCHAR(11) NOT NULL, 
	`Mkt Cap £m` FLOAT, 
	`International Main Market` BOOL NOT NULL, 
	`FCA Listing Category` VARCHAR(8), 
	CHECK (`International Main Market` IN (0, 1))

Cut and paste this to your favourite SQL editor and execute on the server and voilà, instant data table. You can also have csvsql upload the table and data direct to the server too. Crazy sweet. A brilliant tool.

Time to read the documentation and tutorial for more info. Worth every minute.

posted under Linux Tips | Comments Off on CSV mish mash
« Older Entries