Click Technology

Linux, Windows, Mac it's all good

YouTube Downloading

May15

I’ve been using

youtube-dl

for ages now and use it to download videos, podcasts and playlists I want to listen to offline. To install it on Linux, just use

sudo apt-get install youtube-dl

Once in, all that’s needed is the URL you want to download, and, once you have it, you just use it like this..

youtube-dl https://www.youtube.com/watch?v=USg3NR76XpQ

Simple.

But the video in this case is a three hour podcast, so that ends up being around 3GB. A lot. Can we get a smaller version?

Yes. Use the -F switch to list all the available audio/video and audio only formats..

youtube-dl -F https://www.youtube.com/watch?v=USg3NR76XpQ

You get this..

user@host $ youtube-dl -F https://youtu.be/USg3NR76XpQ
[youtube] USg3NR76XpQ: Downloading webpage
[youtube] USg3NR76XpQ: Downloading video info webpage
[youtube] USg3NR76XpQ: Extracting video information
[youtube] USg3NR76XpQ: Downloading MPD manifest
[info] Available formats for USg3NR76XpQ:
format code  extension  resolution note
249          webm       audio only DASH audio   55k , opus @ 50k, 64.26MiB
139          m4a        audio only DASH audio   58k , m4a_dash container, mp4a.40.5@ 48k (22050Hz), 60.43MiB
250          webm       audio only DASH audio   73k , opus @ 70k, 79.10MiB
171          webm       audio only DASH audio  110k , vorbis@128k, 125.32MiB
140          m4a        audio only DASH audio  137k , m4a_dash container, mp4a.40.2@128k (44100Hz), 161.38MiB
251          webm       audio only DASH audio  139k , opus @160k, 147.56MiB
160          mp4        256x144    DASH video  148k , avc1.4d400c, 30fps, video only, 42.91MiB
278          webm       256x144    144p  154k , webm container, vp9, 30fps, video only, 119.24MiB
242          webm       426x240    240p  278k , vp9, 30fps, video only, 154.07MiB
133          mp4        426x240    DASH video  282k , avc1.4d4015, 30fps, video only, 136.58MiB
243          webm       640x360    360p  469k , vp9, 30fps, video only, 346.58MiB
134          mp4        640x360    DASH video  610k , avc1.4d401e, 30fps, video only, 204.77MiB
244          webm       854x480    480p  856k , vp9, 30fps, video only, 673.62MiB
135          mp4        854x480    DASH video 1024k , avc1.4d401f, 30fps, video only, 521.78MiB
247          webm       1280x720   720p 1729k , vp9, 30fps, video only, 1.79GiB
136          mp4        1280x720   DASH video 1875k , avc1.4d401f, 30fps, video only, 1.20GiB
248          webm       1920x1080  1080p 3188k , vp9, 30fps, video only, 3.58GiB
137          mp4        1920x1080  DASH video 3569k , avc1.640028, 30fps, video only, 3.06GiB
17           3gp        176x144    small , mp4v.20.3, mp4a.40.2@ 24k
36           3gp        320x180    small , mp4v.20.3, mp4a.40.2
43           webm       640x360    medium , vp8.0, vorbis@128k
18           mp4        640x360    medium , avc1.42001E, mp4a.40.2@ 96k
22           mp4        1280x720   hd720 , avc1.64001F, mp4a.40.2@192k (best)

This extensive list of options allows you to pick a preferred format. I’m going to go for option 139

139          m4a        audio only DASH audio   58k , m4a_dash container, mp4a.40.5@ 48k (22050Hz), 60.43MiB

which is a more manageable 60MB. Let’s download that.

youtube-dl -f 139 https://www.youtube.com/watch?v=USg3NR76XpQ

Nice. Job done.

posted under Linux Tips | No Comments »

Open source file sharing..

January25

Happy New Year to all. Here’s to a great New Year! Really! 🙂

This is pretty awesome. I just found a piece of open source file sharing web software called Bozon for when you want to host your own web files, but just make it simple and not have any admin overhead. Once running, all you need to do is upload the files you want to share to your server and then email the link. It’s that simple. Setup takes about five minutes, end to end.

To install Bozon, on your Linux box, assuming you have apache or nginx running and in operation, just do this..

cd /var/www
sudo su
git clone https://github.com/broncowdd/BoZoN.git bozon

It pulls the git repository down. If you need to install git, it’s just

sudo apt install git

.

[sudo] password for username: 
Cloning into 'bozon'...
remote: Counting objects: 1478, done.
remote: Total 1478 (delta 0), reused 0 (delta 0), pack-reused 1478
Receiving objects: 100% (1478/1478), 1.95 MiB | 567 KiB/s, done.
Resolving deltas: 100% (783/783), done.

You now have

/var/www/bozon/

with all the relevant files therein. Set the privs.

sudo chown www-data -R bozon/

Now link to this directory from your server. Let’s say you have a site structure like this and you want to put bozon in site03.

|-- var
\-- www
    |-- bozon
    |-- site01
    |-- site02
    |-- site03
        |-- code
        |-- js
        \-- html
    \-- site04

Let’s add a link.

cd /var/www/site03
sudo ln -s /var/www/bozon/

Right, that should be everything. Install is complete. Surf to https://www.yoursite.com/bozon and create a first time user and just begin using it.

Upload the files you want and when they upload, click the link share icon to reveal the URL to the file. Mail your friends the link. Simple.

Naturally, you are strongly advised to use https for reasons that should be fairly obvious.

First post best post.

posted under Linux Tips | No Comments »

Programming Best Practices Tidbits

October21

While writing some Python code to calculate PAYE and publish it to GutHub, I found this absolutely awesome set of advisory snippets that have really enlightened me on the crux of development work.

https://github.com/thomasdavis/best-practices

Enjoy.

posted under Linux Tips | Comments Off on Programming Best Practices Tidbits

Which version of Linux am I running?

June15

How many times have I tried to find out _exactly_ which version of Linux I’m running when installing something?  I either get the kernel version (no good for adding debs etc) when I want to find out what the Debian underlying codename is and so on. Here then is the solution. All of it.

uname -a && cat /etc/*release
posted under Linux Tips | Comments Off on Which version of Linux am I running?

Happy Ed Balls day

April28


Happy Ed Balls day

posted under General | Comments Off on Happy Ed Balls day

UK postcode regular expressions

March17

I found these two regexps expressions for UK postcodes today. The first is check if the number and letter sequence is correct. I’ve modified it slightly to look for one or two spaces between the groups of letters and numbers. Worked pretty well under PCRE. Noice. The second regex is to validate the postcode in its entirety. It worked for all the postcodes I had, but this is a constantly changing situation.

Just the basics – format is valid?

Here it is and requires the g(lobal) switch.

([A-Z]{1,2}[0-9][0-9A-Z]?\s{1,2}?[0-9][A-Z]{2})

Sample data…

BA2 3BH
UB11 1FW
CR0 4ZS
This is a B73 6AZ postcode
SW1E 5JD
LS1 2JZ
W1W 8AG
WF16 0HL
London too, EC2A 2BB, nice.
RG21 4EA
BA15 1AJ
CB4 0FW
EC4M 9HH
AB10 1XL
EH2 2BY
EC4M 9HH
EH2 2BY

And a PHP snippet of the above..

    $re = "/([A-Z]{1,2}[0-9][0-9A-Z]?\\s{1,2}?[0-9][A-Z]{2})/"; 
    $str = "BA2 3BH\nUB11 1FW\nCR0 4ZS\nThis is a B73 6AZ postcode\nSW1E 5JD\nLS1 2JZ\nW1W 8AG\nWF16 0HL\nLondon too, EC2A 2BB, nice.\nRG21 4EA\nBA15 1AJ\nCB4 0FW\nEC4M 9HH\nAB10 1XL\nEH2 2BY\nEC4M 9HH\nEH2 2BY"; 
     
    preg_match_all($re, $str, $matches);

Full Monty – Postcode itself is valid?

This regexp should be able to validate most postcodes. Worked well on all the data I had.

^GIR[ ]?0AA|((AB|AL|B|BA|BB|BD|BH|BL|BN|BR|BS|BT|BX|CA|CB|CF|CH|CM|CO|CR|CT|CV|CW|DA|DD|DE|DG|DH|DL|DN|DT|DY|E|EC|EH|EN|EX|FK|FY|G|GL|GY|GU|HA|HD|HG|HP|HR|HS|HU|HX|IG|IM|IP|IV|JE|KA|KT|KW|KY|L|LA|LD|LE|LL|LN|LS|LU|M|ME|MK|ML|N|NE|NG|NN|NP|NR|NW|OL|OX|PA|PE|PH|PL|PO|PR|RG|RH|RM|S|SA|SE|SG|SK|SL|SM|SN|SO|SP|SR|SS|ST|SW|SY|TA|TD|TF|TN|TQ|TR|TS|TW|UB|W|WA|WC|WD|WF|WN|WR|WS|WV|YO|ZE)(\d[\dA-Z]?[ ]?\d[ABD-HJLN-UW-Z]{2}))|BFPO[ ]?\d{1,4}$

And here then is the code snipped in Python, the most flexible scripting language there is. By far.

    import re
    p = re.compile(ur'^GIR[ ]?0AA|((AB|AL|B|BA|BB|BD|BH|BL|BN|BR|BS|BT|BX|CA|CB|CF|CH|CM|CO|CR|CT|CV|CW|DA|DD|DE|DG|DH|DL|DN|DT|DY|E|EC|EH|EN|EX|FK|FY|G|GL|GY|GU|HA|HD|HG|HP|HR|HS|HU|HX|IG|IM|IP|IV|JE|KA|KT|KW|KY|L|LA|LD|LE|LL|LN|LS|LU|M|ME|MK|ML|N|NE|NG|NN|NP|NR|NW|OL|OX|PA|PE|PH|PL|PO|PR|RG|RH|RM|S|SA|SE|SG|SK|SL|SM|SN|SO|SP|SR|SS|ST|SW|SY|TA|TD|TF|TN|TQ|TR|TS|TW|UB|W|WA|WC|WD|WF|WN|WR|WS|WV|YO|ZE)(\d[\dA-Z]?[ ]?\d[ABD-HJLN-UW-Z]{2}))|BFPO[ ]?\d{1,4}$')
    test_str = u"BA2 3BH\nUB11 1FW\nCR0 4ZS\nB73 6AZ\nSW1E 5JD\nLS1 2JZ\nW1W 8AG\nWF16 0HL\nEC2A 2BB\nRG21 4EA\nBA15 1AJ\nCB4 0FW\nEC4M 9HH\nAB10 1XL\nEH2 2BY\nEC4M 9HH\nEC4M 9HH\nEC4M 9HH\nEH2 2BY\nEH3 7NS\nEH3 7NS\nCB22 3AT\nGU2 7AH\nEC1M 6EH\nRG10 9NN\nSL4 1BE\nKT10 9AD\nW1W 8DH\nBA1 5BB\nTN4 8BS\nCF10 2EH\nCW7 3RT\nW1U 6HP\nEC2N 2AN\nW1W 7PA\nEC3A 7NH\nGU6 8TB\nB60 4JE\nW1S 2LG\nG2 7JS\nRH6 0PA\nWF5 0AN\nHA1 1BQ\nEC2R 7AF\nEC2R 7AF\nEC2R 7AF\nEC2R 7AF\nRG1 1AX\nWC2R 1DJ\nW1J 0DW\n"
     
    re.findall(p, test_str)

By the way, some awesome regex testing websites are…

posted under Linux Tips | Comments Off on UK postcode regular expressions
« Older EntriesNewer Entries »