Octave - computing large number of norms

From time to time, I have to compute the the square of norms for a large number of vectors. This usually happens when I'm modeling some physic phenomenon, such as a solar system plus a few extra bodies or a EM system.

Over the time - and my knowledge of Octave growing - I've used a few different techniques for this. The first one was the obvious for loops over the all the vectors, then, using a single for loop, simply calling norm on each line and squaring.  

Lately, I found two interesting techniques: if all my vectors are the lines in a matrix M, calculating the square norms is also possible by extracting the diagonal of M*M'.  The second one is to take advantadge of the .* operator and use sum to get, well, the sum.

This works fine, and recently, I've just started wondering which method would be the fastest. I deliberately skipped the very first one (the two nested for loops), and timed the three others, for sets of size 1,2,5,10,20,50,100,200,500,1000,2000,5000 and 10000. 

An interesting note is that diag(M*M') doesn't work well with values above  16000, with a limit on my machine of 16383. I assume the explanation lies in the fact that this generates a resulting matrix 16383x16383, or a matrix containing 268402689 elements. 

Here are the results. All the times are in second.

Size	Using norm	Using diag	Using .* + sum
1	9.0804e-09	8.8476e-09	8.7311e-09
2	5.7044e-09	6.5193e-09	5.1223e-09
5	6.4028e-09	8.2655e-09	5.0059e-09
10	7.7998e-09	6.5193e-09	5.0059e-09
20	1.0477e-08	5.3551e-09	5.0059e-09
50	1.9209e-08	5.4715e-09	5.2387e-09
100	3.3295e-08	6.1700e-09	5.4715e-09
200	6.0885e-08	8.7311e-09	6.9849e-09
500	1.4703e-07	3.5390e-08	5.8208e-09
1000	2.9348e-07	1.3306e-07	5.3551e-09
2000	5.9220e-07	5.4494e-07	5.2387e-09
5000	1.5607e-06	4.8558e-06	5.7044e-09
10000	4.1565e-06	2.4022e-05	6.1700e-09

A few interesting things there.

1. The time to run the computation using for/norm is almost linear in size(input)
2. The time to run the computation using diag is quadratic in size(input)
3. The time to run the computation using .*/sum has an initial bump then is almost linear in size(input)

I continued the comparison past 16000 for the first and last methods. For size(input)=500000, the for/norm method took 0.0054697 seconds, the .*/sum 1.6822e-07. So far, I haven't seen a profiler for Octave, so I can just either guess or imagine why such a difference. 

Bottom line, this test was interesting in multiple aspects. It also helped me decide which square norm function to choose. Starting now, my function is:

function N=sqnorm(X)
N=sum(X.*X);
endfunction

Bitcoins mining - there you go!

Not so long ago, I wrote a short post about my suspicions toward the BitCoin system, and that I thought someone would just harness others' machines' power to generate the random numbers associated with the "bitcoin mining". In short, every ten minutes, the system creates 50 units of currency that are given to the account whose machine created a number matching certain criterion.

My gut feeling at that time was that it wouldn't be long before someone runs a trojan to generate these numbers on hundreds or thousands of machines, to get the loot as many times as possible.

That's now a thing of the past ... someone did it. In his case, it didn't work well, but I'm pretty sure the bad guys will find something else. After all, there is money to be taken.

EXT4, OEL 5.5, Kernel 2.6.18 => kernel panic

Ok, not bleeding edge, but it made my day a bit less sunnier: I was wondering why my shiny Linux server running OEL 5.5 was not coming back to live ...

Well, kernel panic due to mounting the ext4. Tried it manually, same result. Mounting the same device formatted with ext3 doesn't trigger the same issue.

As it's an old kernel, I'll just let it die peacefully.

multipathd woes!

Today, playing with multipathd - it's not often I have a server, FC switches AND the storage at the same time to lay my beer-tainted hands on - I ran into an issue: no matter what, multipathd wouldn't hear a thing about my LUNs.

Tried a lot, installing some scsi debugging packages - which showed me that the storage was indeed presenting the LUNs, modprobing some modules and things.

Until I ran "udevinfo -e", which exports all the devices the kernel sees. Which also told me that the WWID have to be lower case!

I had done a couple of copy-paste between the SAN configuration screen and my /etc/multipathd.conf: the SAN configuration page gives me the WWID with capital letters (ex 35001ADE...), where udev reports lower case letters (35001ade...).

A quick trip to /etc/multipath.conf, change all the letters to lower case and ... voila! Both multipathd and I are happy.

Bad guys go after Bitcoins

Highly predictable: the bad guys now go after bitcoins.

For these who don't know what Bitcoins are, it's a new form a virtual, distributed currency. Instead of being managed by a state, a public entity or a government, the users manage it. Some people already accept them as a mean of payment for purchase on the Internet.

With the ability to convert hard cash into bitcoins and back, of course there is the ability to steal virtual money, which can translate into real money.

It's now done. When I first read the article in New Scientist, my first reaction was that it wouldn't be long before someone gets an "interest" at it. But my guess was mostly bad guys hacking into computers to steal processing resources to earn the random generation process.

It seems they found another way: stealing the purse.

My laptop has been upgraded to Ubuntu 11.04 (Natty Narwhal)

Yup, just made the switch. Ubuntu 11.04 has a completely changed user interface, closer to a netbook than to a laptop.

So far, so good: no bad surprises, though I had to make an extensive use of the search function to find my often used applications (terminal, wireshark, octave and a few other tools), no crash and no problem yet.

I also read a few users totally frustrated by Gnome 3, which is more tied to relearning the interface than using the interface itself.

A morning with Kevin Mitnick

A vendor of my company sent us invitation for their annual innovation event. And this year, the speaker is ... Kevin Mitnick. Yes, _the_ Mitnick from back in the days.

He spoke passionately for a good two hours of different hacking techniques, from the known stuff - spear phishing, technical exploits, abusing autoruns and social engineering - to tomorrow's techniques: advanced social engineering, phone system-man-in-the-middle or asterisk-in-the-middle, keystrokes injection and so forth.

A few tendencies:

The bad guys have increasingly more time to spend on devising new ways of perpetrating their bad deeds;

The exploits shifted from hugely technical to more focused on the human side;

Most of the people still don't have a clue about what data concerns their privacy, and are willing to give that even for small rewards;

The rate at which computers take over all our life is greater than the rate at which people learn how to use it correctly.

The audience was mesmerized, and during the presentation I saw a few people turning their phone off or with round eyes. Possibly some memories back in mind or recollection of recent events.

Kevin is truly a Hacker (with a capital H - not a mistake) and a very good speaker. Two hours was way too short.

Kevin Mitnick's company