Bayes network, variable independence and AI

In one of the Stanford AI class homework (now closed), the following question was asked. Given the following Bayes network, are B and C independent knowing A and D?

Following the usual rules, we are presented with a dilemna: A being known would imply that B and C are independent, but D being known implies that B and C are dependent.

From a probabilistic point of view, two variables A and B are independent if

Pr[A=v]=Pr[A=v|B=u]

So, to check the independence of two variables, one has to compute the conditional probability and compare to the probability without the added condition.

A quick solution is to model the Bayes network with a python script you may find on my github. The result shows that B and C are dependent if A and D are known, but independent if only A is known.



P(B)= 0.590436
P(B|C)= 0.464383811907
P(C)= 0.629475
P(C|B)= 0.49508837537

P(C,(D))= 0.650509052716
P(C|B,(D))= 0.554403493324
P(B,(D))= 0.704196855276
P(B|C,(D))= 0.600159513419

P(C,(A))= 0.448997199205
P(C|B,(A))= 0.449069612702
P(B,(A))= 0.949959858907
P(B|C,(A))= 0.9501130668

P(C,(A,D))= 0.174715296242
P(C|B,(A,D))= 0.139904742675
P(B,(A,D))= 0.844759945818
P(B|C,(A,D))= 0.676448630334

In order to double check, I did the formal calculation for the case where A and D are known, which gave me the same results. 

Python rocks! And so does the Stanford AI course!

NMAP - using nmap scripting engine (NSE)

NMAP is one of the tools I find super useful. No need to present it, it's powerful, it's fast, it has a ton of functions a features.

Recently, I've been playing with the NSE, or scripts, to offload some of my discovery to nmap rather than combine multiple tools. However, I got an error for "citrixxml" not being found. I tried to update the DB, same issue.

# export NMAPDIR=/usr/share/nmap
# nmap --script-updatedb
Starting Nmap 5.21 ( http://nmap.org ) at 2011-10-19 16:40 EDT
NSE: Updating rule database.
NSE: error while updating Script Database:
[string "local nse = ......"]:17: /usr/share/nmap/scripts//citrix-brute-xml.nse:35: module 'citrixxml' not found:
no field package.preload['citrixxml']
no file './citrixxml.lua'
no file '/usr/local/share/lua/5.1/citrixxml.lua'
no file '/usr/local/share/lua/5.1/citrixxml/init.lua'
no file '/usr/local/lib/lua/5.1/citrixxml.lua'
no file '/usr/local/lib/lua/5.1/citrixxml/init.lua'
no file '/usr/share/lua/5.1/citrixxml.lua'
no file '/usr/share/lua/5.1/citrixxml/init.lua'
no file '/usr/share/nmap/nselib/citrixxml.lua'
no file './citrixxml.so'
no file '/usr/local/lib/lua/5.1/citrixxml.so'
no file '/usr/lib/lua/5.1/citrixxml.so'
no file '/usr/local/lib/lua/5.1/loadall.so'
stack traceback:
[C]: in function 'assert'
[string "local nse = ......"]:17: in main chunk

A quick trip to /usr/share/nmap/nselib revealed that that particular file was missing. It's available however on the nmap website.

# cd $NMAPDIR/nselib
# wget http://nmap.org/svn/nselib/citrixxml.lua

The following "nmap --script-updatedb" ran like a charm. 

"Best Practices" ...

Ok, I got one "best practices" too many. Consultants, colleagues, vendors, they all swear, breath and live by these magic words. "Best Practices".

When I hear that expression, I can't help but have a bunch of questions popping in my mind: who made them? What is the reference platform? What's the test scenario? What are the constraints and trade-offs? What are the limits?

But it seems that these "Best Practices" are universal. Got that software? Here are the best practices, they cover everything, all scenarios, all cases, have no constraints and have no limits. From Security, going through file servers, web servers and finally to database servers, Best Practices are everywhere. You're going to deploy that server with that OS? Here are the best practices, everything will run nice and smooth and you'll never have to change anything.

Too often, I'm under the impression that these best practices are just a substitute for some people's inability to understand what they're doing, what they're working with or their use cases, that these "Best Practices" are little short than "cook books" aimed at giving users a way to have something that will work OK in most of the cases, but that will never work "great".

Here are my list of "Best Practices". To be used with everything.

• Understand your systems, most of all, know what the constraints and trade-offs are;
• Understand your use cases, if possible, have a set of tests in a handy;
• Read the "Best Practices", don't be ruled by them;
• Read all the white papers and user cases you can. Try to find similarities;
• If possible, have a test system you can tweak and break;
• Document what you did, and when possible, share with the community at large.

Windows 7 and Cached Credentials

Recently, I provided some help in assessing the security of a Windows 7 image for a client. Quite fun, given that I'm not a windows specialist. As usual, I took that as a good opportunity to learn new stuff. All I was given was the laptop, the BitLocker PIN and the admin password. 


First approach: global and local policies. In order to do that, I used a spreadsheet from the National Checklist Program. This is very comprehensive and covers several domains. Some of them are not applicable in all cases, but the bulk is really interesting and things I wouldn't have thought of at first. I also used some resources from the SANS/CIS.


Second: let's test the beast. It was really tighten down, with lots of restrictions, a service that prevents running programs that are not white listed and things, and disabling that service was greeted with an "Access Denied". How rude. In addition, the firewall is up and running and Forefront protects the whole stuff. These two are pieces of cake: services, disable ...


What about the safe mode in which only the necessary services are started? Booting and pressing F8 like crazy doesn't help. But what about running "msconfig" and changing the boot option to "safe mode"? Ok, it complains that BitLocker will ask for the recovery token. So? Let's go to the BitLocker Manager and let's get it. Then, reboot, BitLocker PIN, BitLocker Recovery Token, and voila! Safe mode, the service is not started and I can disable it. Next ...

After a quick passage into msconfig to restore the boot options to normal, and another reboot, I'm free and I can run executables as I wish. One down.


My weapon of choice in this is usually Metasploit. First because it gives me a very convenient CLI to access the machine plus a bunch of scripts to extract information. So "psexec" and "meterpreter" it is. To find that the exploit runs into a wall.


By default, Windows 7 machines have the "ADMIN$" share disabled. Whatever, a quick trip to regedit to add a DWORD key (LocalAccountTokenFilterPolicy) with value 1 into  HKLM\Software\Microsoft\Windows\CurrentVersion\Policies\System and I'm good to go! psexec runs fine and I'm in.


Next stop: password and cache dumps. If the former works fine, the latter doesn't. Hmmm ... What about Cain&Abel? Same problem ... error. Another trip to the registry to find that HKLM\Security is empty. So no cached credentials there. Unfortunately, my knowledge of windows - and the time I could spend with the machine - didn't allow me to find whether these were stored somewhere else.

File entropy calculator

A long time ago ... I got a request from a colleague: how could we, given a bunch of files, sort out the ones that could be encrypted?

I remembered that encrypted files tend to exhibit an entropy that's higher than the usual file, so I wrote a quick python script - I was learning the language - and used it on our large dataset. This was really useful and we were able to quickly find all the encrypted files.

A few false positives were caught: mostly compressed files. Feel free to drop me a line if you find this useful or if you find any bug.

Git repository

IPv6 - playing with the stuff

As I got my shiny new Cisco 877 router, I started playing (again) with IPv6.  Setting an IPv6-in-IPv4 to a Hurricane Electric was real easy.

Also, I started taking their tests and I got:



:)

Google Chrome crashes on Fedora 15 when accessing a google document or calendar

I just had this: when opening a document in Google Docs or going to my calendar, Chrome would display the "Woops ..." page.

In /var/log/messages, a few lines point to SELinux:



Messages in /var/log/messages

Aug 21 18:59:02 jeff-fedora setroubleshoot: SELinux is preventing /opt/google/chrome/chrome from read access on the file /home/jeff/.config/google-chrome/Dictionaries/en-US-2-1.bdic. For complete SELinux messages. run sealert -l 56257509-1d9e-49a4-8b31-de14161c5c2c
Aug 21 18:59:04 jeff-fedora setroubleshoot: SELinux is preventing /opt/google/chrome/chrome from read access on the file /home/jeff/.config/google-chrome/Dictionaries/en-US-2-1.bdic. For complete SELinux messages. run sealert -l 56257509-1d9e-49a4-8b31-de14161c5c2c

I disabled SELinux and rebooted, no more crashes, proving its something with SELinux.