KISMACK KISmet Merge And Convert to Kml

Scruge · 02-05-2006

The initial results sound promising. So I'm giving it another shot.

KisMaCK is a utility to merge, maintain and convert Kismet CSV, XML and GPS logs into a master database and produce KML files for Google earth.

..features
1) Maintains a unique set of data for both net and gps data, meaning no duplicate records.
2) Updates the master records each time new data is processed.
3) Recovers orphan nets.
4) Provides a report for nets not having gps data.
5) Provides a stat report of each log file processed
6) Provides a source record for all nets processed
7) Consolidates all 3 file types into a unique set of data and converted to a KML, based on drive date/time.
8) IEEE hardware vendor info is provided, via MAC vendor registration.
9) Uses a std .dbf file structure that can be accessed via various means MS Access, MS Excel, SQL etc.

Kismack has been renamed to
KNSGEM

If the feed back continues to be positive I'll add NS1 file support.

theprez98 · 02-05-2006

Quote:

Originally Posted by Scruge

3. Put a copy of the icon file "lwfw.png" in your "C:/Program Files/Google/Google Earth/res/ " folder

Where do I find this file?

Also, when I run "Kismack.exe" I get "Registry entry missing or access denied."

(WinXP SP2)

Scruge · 02-05-2006

Quote:

Originally Posted by theprez98

Where do I find this file?

Also, when I run "Kismack.exe" I get "Registry entry missing or access denied."

(WinXP SP2)

Oops, give it another try. Should be there now.

Looks like the balloon popped. I'll have to check out the "registy" problem.

theprez98 · 02-05-2006

Quote:

Originally Posted by Scruge

Oops, give it another try. Should be there now.

Got it. Still getting the error though. I guess some sort of registry or permission issue.

beakmyn · 02-05-2006

Quote:

Originally Posted by theprez98

Got it. Still getting the error though. I guess some sort of registry or permission issue.

Tech Support says to run this and filter by "kismack" to see what's going wrong
http://www.sysinternals.com/Utilities/Regmon.html

theprez98 · 02-05-2006

Quote:

Originally Posted by beakmyn

Tech Support says to run this and filter by "kismack" to see what's going wrong
http://www.sysinternals.com/Utilities/Regmon.html

Ok I guess I'm a little slow. It seems when I run Kismack and filter regmon to "Kismack" I get about 70 entries. About half the results are "Success" while the other half are "Not Found." So by the error "Registry entry missing or access denied" I am supposed to add some sort of entry to the registry?

EDIT: I read Scruge's edited first post, I'll guess I'll wait until the issue gets fixed...

Looking forward to another cool tool!

fregniacciaro · 02-05-2006

Scruge - I just sent the CSV files, not the GPS or XML. The number of networks is different, but not by much (at least in my instances) as you can tell from Dutch's post here. Please let me know if you want those files to help with your program. I don't think I have them from that same drive, but I can send you a couple that were tar'd & gzip'd from a later date

Scruge · 02-05-2006

Works great on my system and any of the others that have had the compiler installed on them.

I'm such a tight-wad I haven't updated my "installshield express" so its not building the exe completely.

I'll check around next week to see if I can find a replacement or upgrade.

@ Frag
The problem isn't with your files. Once I get the installer fixed I'll take a look at the gps and xml files to see if they can be added.

@ Mods
If you guys want to dump this thread feel free to. I can always make another attempt later.

Dutch · 02-06-2006

Quote:

Originally Posted by Scruge

Okay, here's my first trial balloon for KISMACK, KISmet Merge And Convert to Kml.

Due to an out of date installer I'll have to repost at a later time.

Sorry for the trouble.

Looking forward to test it out Scruge.
Are you checking for duplicate networks in the xml files, and are you updating or just overwriting data, if there are duplicate networks ?
As for merging the .gps files, no checking is necessary. I manually copy the first .gps file to a new file, edit it to remove the </gps-run> at the end, then run a cat/grep command to add the <gps-point> data to the new file :
I.E. three files to concatenate : Kismet-Feb-02-2006.1.gps, Kismet-Feb-03-2006.1.gps, Kismet-Feb-04-2006.1.gps
copy Kismet-Feb-02-2006.1.gps to new.gps, edit new.gps to remove the </gps-run> at the end, then execute

Code:

cat Kismet-Feb-03*.gps Kismet-Feb-04*.gps |grep gpspoint>>new.gps && echo "</gps-run>">>new.gps

Currently I'm using a revised version of revised original perl klc.pl script (search kismetwireless.net forums for it) for the .xml files. I made some bugfixes to it, as well as a couple of changes to the loop checking for duplicates, as it was quite slow.
Even with the changes to the loop it is still quite slow, taking about 12 hours to merge two files with 1200+ and 800+ networks (about 25 % duplicates) on the wardriving laptop (a 850Mhz PIII), so if your application can do it faster, I'm definitely going to use your application.

A short pseudocode outline of how the revised klc.pl file works :

Code:

Read the first xml logfile;
populate an array with the <wireless-network> data from it;
keep a counter of the number of networks;

start of loop
{
  Read one <wireless-network> data segment from the next files.
  For each network in the array, compare the BSSID with the BSSID just read in.
 If the BSSID just read in, matches an existing BSSID 
   { Update the <wireless-network> data fields in the array, with the <wireless-network> data fields just read in;
      Exit current iteration of loop, thereby reading the next <wireless-network> data segment from the second file;
    }
else
  { Increment the network counter;
     Add all the <wireless-network> data just read in, as a new item in the array;
   }
end if block;

} end the loop when all <wireless-network> data segments from the next files has been processed;
Write the data from the array to the new xml file.

For those who want to use the revised revised klc.pl file, untill Scruge gets his app out, I'm including it here.
Perl, Linux and libXML required. Might work on Windows also, if perl and libXML is available, but not tested.

Code:

#!/usr/bin/perl
# copyright 2005 tud.thaunderdog
# minor bugfix and speedup attempt by Dutch 2006

use XML::LibXML;

unless ( @ARGV > 2 ) {
    print "Usage: $0 <input-1st> <input-Nth> <output>\n";
    exit;
}

$start_time = localtime(time);

$out_file_name = pop @ARGV;
@log_files = @ARGV;
$net_num = 0;
$firstlog = 0;
foreach $this_log (@log_files) {
	$firstlog++;
	print "KLV: Reading Logfile $this_log....\n";
	

	$doc = XML::LibXML::Document->new();
	$root = $doc->createElement('html');
	$doc->setDocumentElement($root);
	$body = $doc->createElement('body');
	$root->appendChild($body);
	
	
	
	if (!$xml_body) {
		open(LOG_FILE,"<$this_log");
		@log_file = <LOG_FILE>;
		close(LOG_FILE);
		
		foreach $line (@log_file) {
			if ($line=~/<?xml/) { $xml_head .= $line; }
			elsif ($line=~/<!DOCTYPE/) {$xml_head .= $line; }
			elsif ($line=~/<detection-run/) { $xml_head .= $line; }
			elsif ($line=~/<\/detection-run/) { $xml_foot .= $line; }
		}
		
		open(OUT_FILE,">$out_file_name");
		print OUT_FILE ("$xml_head"); 
		print OUT_FILE ("$xml_foot");
		close(OUT_FILE);
		$xml_body = 1;
	}
	
		
	$parser = XML::LibXML->new();
	$parser->expand_entities(0);
	$tree = $parser->parse_file($this_log);
	$root = $tree->getDocumentElement;
	
	@networks = $root->getElementsByTagName('wireless-network');
	
	foreach $this_network (@networks) {
	
		$net_bssid = $this_network->getElementsByTagName('BSSID')->[0]->getFirstChild->getData;
		
		$out_parser = XML::LibXML->new();
		$out_parser->expand_entities(0);
		$out_tree = $out_parser->parse_file($out_file_name);
		$out_root = $out_tree->getDocumentElement;
		
		if ( $temp = $out_root->getElementsByTagName('wireless-network') ) { @out_networks = $out_root->getElementsByTagName('wireless-network'); }
		
		$unique = 1;
		
		if ("$firstlog" > "1") {
		if (@out_networks) {

			foreach $this_out_network (@out_networks) {
				
				$out_net_bssid = $this_out_network->getElementsByTagName('BSSID')->[0]->getFirstChild->getData;
				
				if ( "$net_bssid" eq "$out_net_bssid" ) {
					undef $unique;
					$out_net = $this_out_network;
					last;
				} 
		
			} # end foreach $this_out_network 
		
		 } # end if there are networks in out file
		}
		if ($unique) {
			print "KLV: Merging $net_bssid  -  $net_num - $firstlog\n";
			
			## Is this faster?...
			#$net_num = 0;
			#foreach $this_out_network (@out_networks) {
			#	
			#	$this_network_xml = $this_out_network->toString;
			#	if ( $this_network_xml =~ /<wireless-network number="\d+"/ ) {
			#		$net_num++;
			#		$this_network_xml=~ s/<wireless-network number="\d+"/<wireless-network number="$net_num"/;
			#	} else {
			#		$networks .= "\n$this_network_xml";
			#	}
			#}
			#$this_network_xml = $this_network->toString;
			#$networks .= "\n$this_network_xml";
			#
			## ...Or is this?
			 $net_num++;
			 $this_network_xml = $this_network->toString;
			 $this_network_xml=~ s/<wireless-network number="\d+"/  <wireless-network number="$net_num"/;
			 $networks .= "\n$this_network_xml";
			
			
			
		} else {
			print "KLV: Merging $net_bssid  -  Updated\n";
			undef $networks;
			undef $gps;
			
			$net_packets_LLC = $this_network->getElementsByTagName('LLC')->[0]->getFirstChild->getData;
			$out_net_packets_LLC = $out_net->getElementsByTagName('LLC')->[0]->getFirstChild->getData;
			$net_packets_LLC += $out_net_packets_LLC;
			
			$net_packets_data = $this_network->getElementsByTagName('data')->[0]->getFirstChild->getData;
			$out_net_packets_data = $out_net->getElementsByTagName('data')->[0]->getFirstChild->getData;
			$net_packets_data += $out_net_packets_data;
			
			$net_packets_crypt = $this_network->getElementsByTagName('crypt')->[0]->getFirstChild->getData;
			$out_net_packets_crypt = $out_net->getElementsByTagName('crypt')->[0]->getFirstChild->getData;
			$net_packets_crypt += $out_net_packets_crypt;
			
			$net_packets_weak = $this_network->getElementsByTagName('weak')->[0]->getFirstChild->getData;
			$out_net_packets_weak = $out_net->getElementsByTagName('weak')->[0]->getFirstChild->getData;
			$net_packets_weak += $out_net_packets_weak;
			
			$net_packets_dupeiv = $this_network->getElementsByTagName('dupeiv')->[0]->getFirstChild->getData;
			$out_net_packets_dupeiv = $out_net->getElementsByTagName('dupeiv')->[0]->getFirstChild->getData;
			$net_packets_dupeiv += $out_net_packets_dupeiv;
			
			$net_packets_total = $this_network->getElementsByTagName('total')->[0]->getFirstChild->getData;
			$out_net_packets_total = $out_net->getElementsByTagName('total')->[0]->getFirstChild->getData;
			$net_packets_total += $out_net_packets_total;
			
			$net_packets_datasize = $this_network->getElementsByTagName('datasize')->[0]->getFirstChild->getData;
			$out_net_packets_datasize = $out_net->getElementsByTagName('datasize')->[0]->getFirstChild->getData;
			$net_packets_datasize += $out_net_packets_datasize;
			
			if ( ( $temp = $this_network->getElementsByTagName('min-lat')->[0] ) && ( $temp = $out_net->getElementsByTagName('min-lat')->[0] ) ) {
				$gps = 1;
				
				$net_gps_min_lat = $this_network->getElementsByTagName('min-lat')->[0]->getFirstChild->getData;
				$out_net_gps_min_lat = $out_net->getElementsByTagName('min-lat')->[0]->getFirstChild->getData;
				if ( $net_gps_min_lat > $out_net_gps_min_lat ) { $net_gps_min_lat = $out_net_gps_min_lat; }
				
				$net_gps_min_lon = $this_network->getElementsByTagName('min-lon')->[0]->getFirstChild->getData;
				$out_net_gps_min_lon = $out_net->getElementsByTagName('min-lon')->[0]->getFirstChild->getData;
				if ( $net_gps_min_lon > $out_net_gps_min_lon ) { $net_gps_min_lon = $out_net_gps_min_lon; }
				
				$net_gps_min_alt = $this_network->getElementsByTagName('min-alt')->[0]->getFirstChild->getData;
				$out_net_gps_min_alt = $out_net->getElementsByTagName('min-alt')->[0]->getFirstChild->getData;
				if ( $net_gps_min_alt > $out_net_gps_min_alt ) { $net_gps_min_alt = $out_net_gps_min_alt; }
				
				$net_gps_min_spd = $this_network->getElementsByTagName('min-spd')->[0]->getFirstChild->getData;
				$out_net_gps_min_spd = $out_net->getElementsByTagName('min-spd')->[0]->getFirstChild->getData;
				if ( $net_gps_min_spd > $out_net_gps_min_spd ) { $net_gps_min_spd = $out_net_gps_min_spd; }
				
				$net_gps_max_lat = $this_network->getElementsByTagName('max-lat')->[0]->getFirstChild->getData;
				$out_net_gps_max_lat = $out_net->getElementsByTagName('max-lat')->[0]->getFirstChild->getData;
				if ( $net_gps_max_lat < $out_net_gps_max_lat ) { $net_gps_max_lat = $out_net_gps_max_lat; }
				
				$net_gps_max_lon = $this_network->getElementsByTagName('max-lon')->[0]->getFirstChild->getData;
				$out_net_gps_max_lon = $out_net->getElementsByTagName('max-lon')->[0]->getFirstChild->getData;
				if ( $net_gps_max_lon < $out_net_gps_max_lon ) { $net_gps_max_lon = $out_net_gps_max_lon; }
				
				$net_gps_max_alt = $this_network->getElementsByTagName('max-alt')->[0]->getFirstChild->getData;
				$out_net_gps_max_alt = $out_net->getElementsByTagName('max-alt')->[0]->getFirstChild->getData;
				if ( $net_gps_max_alt < $out_net_gps_max_alt ) { $net_gps_max_alt = $out_net_gps_max_alt; }
				
				$net_gps_max_spd = $this_network->getElementsByTagName('max-spd')->[0]->getFirstChild->getData;
				$out_net_gps_max_spd = $out_net->getElementsByTagName('max-spd')->[0]->getFirstChild->getData;
				if ( $net_gps_max_spd < $out_net_gps_max_spd ) { $net_gps_max_spd = $out_net_gps_max_spd; }
				
			}
			
			foreach $this_out_network (@out_networks) {
				$this_network_xml = $this_out_network->toString;
				if ($this_out_network eq $out_net) {
					$this_network_xml=~ s/<LLC>\d*<\/LLC>/<LLC>$net_packets_LLC<\/LLC>/;
					$this_network_xml=~ s/<data>\d*<\/data>/<data>$net_packets_data<\/data>/;
					$this_network_xml=~ s/<crypt>\d*<\/crypt>/<crypt>$net_packets_crypt<\/crypt>/;
					$this_network_xml=~ s/<weak>\d*<\/weak>/<weak>$net_packets_weak<\/weak>/;
					$this_network_xml=~ s/<dupeiv>\d*<\/dupeiv>/<dupeiv>$net_packets_dupeiv<\/dupeiv>/;
					$this_network_xml=~ s/<total>\d*<\/total>/<total>$net_packets_total<\/total>/;
					$this_network_xml=~ s/<datasize>\d*<\/datasize>/<datasize>$net_packets_datasize<\/datasize>/;
					
					if ($gps) {
						$this_network_xml=~ s/<min-lat>\d*\.\d*<\/min-lat>/<min-lat>$net_gps_min_lat<\/min-lat>/;
						$this_network_xml=~ s/<min-lon>\d*\.\d*<\/min-lon>/<min-lon>$net_gps_min_lon<\/min-lon>/;
						$this_network_xml=~ s/<min-alt>\d*\.\d*<\/min-alt>/<min-alt>$net_gps_min_alt<\/min-alt>/;
						$this_network_xml=~ s/<min-spd>\d*\.\d*<\/min-spd>/<min-spd>$net_gps_min_spd<\/min-spd>/;
						$this_network_xml=~ s/<max-lat>\d*\.\d*<\/max-lat>/<max-lat>$net_gps_max_lat<\/max-lat>/;
						$this_network_xml=~ s/<max-lon>\d*\.\d*<\/max-lon>/<max-lon>$net_gps_max_lon<\/max-lon>/;
						$this_network_xml=~ s/<max-alt>\d*\.\d*<\/max-alt>/<max-alt>$net_gps_max_alt<\/max-alt>/;
						$this_network_xml=~ s/<max-spd>\d*\.\d*<\/max-spd>/<max-spd>$net_gps_max_spd<\/max-spd>/;
					}
					
					
					$client_num = @net_clients = $this_network->getElementsByTagName('wireless-client');
					undef $client_num;
					if (@net_clients) {
					
						while ($this_network_xml=~ /.*client.*\n/) { $this_network_xml=~ s/.*client.*\n//; }
						$this_network_xml=~ s/.*<\/wireless-network>.*//;
						
						
						foreach $this_client (@net_clients) {
							$unique = 1;
							$client_mac = $this_client->getElementsByTagName('client-mac')->[0]->getFirstChild->getData;
							$out_net_clients_total = @out_net_clients = $out_net->getElementsByTagName('wireless-client');
							
							foreach $out_this_client (@out_net_clients) {
								$out_client_mac = $out_this_client->getElementsByTagName('client-mac')->[0]->getFirstChild->getData;
								
								if ( "$client_mac" eq "$out_client_mac" ) {
									undef $unique;
									$out_client = $this_out_client;
									last;
								}

							}
							
							if ($unique) {
								print "     Client  $client_mac\n";
								$this_client_xml = $this_client->toString;
								$client_num++;
								$this_client_xml=~ s/<wireless-client number="\d+"/<wireless-client number="$client_num"/;
								
							} else {
								print "     Client  $client_mac  -  Updated\n";
								undef $gps;
								$client_packets_data = $this_client->getElementsByTagName('client-data')->[0]->getFirstChild->getData;
								$out_net_packets_data = $out_net->getElementsByTagName('client-data')->[0]->getFirstChild->getData;
								$client_packets_data += $out_net_packets_data;
			
								$client_packets_crypt = $this_client->getElementsByTagName('client-crypt')->[0]->getFirstChild->getData;
								$out_net_packets_crypt = $out_net->getElementsByTagName('client-crypt')->[0]->getFirstChild->getData;
								$client_packets_crypt += $out_net_packets_crypt;
			
								$client_packets_weak = $this_client->getElementsByTagName('client-weak')->[0]->getFirstChild->getData;
								$out_net_packets_weak = $out_net->getElementsByTagName('client-weak')->[0]->getFirstChild->getData;
								$client_packets_weak += $out_net_packets_weak;
								
								$client_packets_datasize = $this_client->getElementsByTagName('client-datasize')->[0]->getFirstChild->getData;
								$out_client_packets_datasize = $out_net->getElementsByTagName('client-datasize')->[0]->getFirstChild->getData;
								$client_packets_datasize += $out_cliet_packets_datasize;
								
								if ( ( $temp = $this_client->getElementsByTagName('client-min-lat')->[0] ) && ( $temp = $out_net->getElementsByTagName('client-min-lat')->[0] ) ) {
									$gps = 1;
				
									$client_gps_min_lat = $this_client->getElementsByTagName('client-min-lat')->[0]->getFirstChild->getData;
									$out_net_gps_min_lat = $out_net->getElementsByTagName('client-min-lat')->[0]->getFirstChild->getData;
									if ( $client_gps_min_lat > $out_net_gps_min_lat ) { $client_gps_min_lat = $out_net_gps_min_lat; }
							
									$client_gps_min_lon = $this_client->getElementsByTagName('client-min-lon')->[0]->getFirstChild->getData;
									$out_net_gps_min_lon = $out_net->getElementsByTagName('client-min-lon')->[0]->getFirstChild->getData;
									if ( $client_gps_min_lon > $out_net_gps_min_lon ) { $client_gps_min_lon = $out_net_gps_min_lon; }
				
									$client_gps_min_alt = $this_client->getElementsByTagName('client-min-alt')->[0]->getFirstChild->getData;
									$out_net_gps_min_alt = $out_net->getElementsByTagName('client-min-alt')->[0]->getFirstChild->getData;
									if ( $client_gps_min_alt > $out_net_gps_min_alt ) { $client_gps_min_alt = $out_net_gps_min_alt; }
				
									$client_gps_min_spd = $this_client->getElementsByTagName('client-min-spd')->[0]->getFirstChild->getData;
									$out_net_gps_min_spd = $out_net->getElementsByTagName('client-min-spd')->[0]->getFirstChild->getData;
									if ( $client_gps_min_spd > $out_net_gps_min_spd ) { $client_gps_min_spd = $out_net_gps_min_spd; }
				
									$client_gps_max_lat = $this_client->getElementsByTagName('client-max-lat')->[0]->getFirstChild->getData;
									$out_net_gps_max_lat = $out_net->getElementsByTagName('client-max-lat')->[0]->getFirstChild->getData;
									if ( $client_gps_max_lat < $out_net_gps_max_lat ) { $client_gps_max_lat = $out_net_gps_max_lat; }
				
									$client_gps_max_lon = $this_client->getElementsByTagName('client-max-lon')->[0]->getFirstChild->getData;
									$out_net_gps_max_lon = $out_net->getElementsByTagName('client-max-lon')->[0]->getFirstChild->getData;
									if ( $client_gps_max_lon < $out_net_gps_max_lon ) { $client_gps_max_lon = $out_net_gps_max_lon; }
				
									$client_gps_max_alt = $this_client->getElementsByTagName('client-max-alt')->[0]->getFirstChild->getData;
									$out_net_gps_max_alt = $out_net->getElementsByTagName('client-max-alt')->[0]->getFirstChild->getData;
									if ( $client_gps_max_alt < $out_net_gps_max_alt ) { $client_gps_max_alt = $out_net_gps_max_alt; }
				
									$client_gps_max_spd = $this_client->getElementsByTagName('client-max-spd')->[0]->getFirstChild->getData;
									$out_net_gps_max_spd = $out_net->getElementsByTagName('client-max-spd')->[0]->getFirstChild->getData;
									if ( $client_gps_max_spd < $out_net_gps_max_spd ) { $client_gps_max_spd = $out_net_gps_max_spd; }
								} # enf client if gps
								
								$this_client_xml = $this_client->toString;
								$this_client_xml=~ s/<client-data>\d*<\/client-data>/<client-data>$client_packets_data<\/client-data>/;
								$this_client_xml=~ s/<client-crypt>\d*<\/client-crypt>/<client-crypt>$client_packets_crypt<\/client-crypt>/;
								$this_client_xml=~ s/<client-weak>\d*<\/client-weak>/<client-weak>$client_packets_weak<\/client-weak>/;
								$this_client_xml=~ s/<client-datasize>\d*<\/client-datasize>/<client-datasize>$client_packets_datasize<\/client-datasize>/;
								if ($gps) {
									$this_client_xml=~ s/<client-min-lat>\d*\.\d*<\/client-min-lat>/<client-min-lat>$client_gps_min_lat<\/client-min-lat>/;
									$this_client_xml=~ s/<client-min-lon>\d*\.\d*<\/client-min-lon>/<client-min-lon>$client_gps_min_lon<\/client-min-lon>/;
									$this_client_xml=~ s/<client-min-alt>\d*\.\d*<\/client-min-alt>/<client-min-alt>$client_gps_min_alt<\/client-min-alt>/;
									$this_client_xml=~ s/<client-min-spd>\d*\.\d*<\/client-min-spd>/<client-min-spd>$client_gps_min_spd<\/client-min-spd>/;
									$this_client_xml=~ s/<client-max-lat>\d*\.\d*<\/client-max-lat>/<client-max-lat>$client_gps_max_lat<\/client-max-lat>/;
									$this_client_xml=~ s/<client-max-lon>\d*\.\d*<\/client-max-lon>/<client-max-lon>$client_gps_max_lon<\/client-max-lon>/;
									$this_client_xml=~ s/<client-max-alt>\d*\.\d*<\/client-max-alt>/<client-max-alt>$client_gps_max_alt<\/client-max-alt>/;
									$this_client_xml=~ s/<client-max-spd>\d*\.\d*<\/client-max-spd>/<client-max-spd>$client_gps_max_spd<\/client-max-spd>/;
								} # end gps
								$client_num++;
								$this_client_xml=~ s/<wireless-client number="\d+"/<wireless-client number="$client_num"/;
																
								
								
							} # end if client unique
							
							$this_network_xml .= "$this_client_xml\n";
							
						} # end foreach client this net
						$this_network_xml.= "</wireless-network>\n";
					} # end if net has clients
					
					$networks .= "\n$this_network_xml";
					
				} else {
					$networks .= "\n$this_network_xml";
				} # end if found unique net
			} # end foreach out network
			
		} # end if net unique
					
		open(OUT_FILE,">$out_file_name");
		print OUT_FILE ("$xml_head");
		print OUT_FILE ("$networks\n");
		print OUT_FILE ("$xml_foot\n");
		close(OUT_FILE);
			 
		

		
	} # end foreach $this_network
	
} # end foreach $this_log

$end_time = localtime(time);

print "KLV: Combined Logs to $out_file_name\n";
print "KLV: Operation Started: $start_time\n";
print "KLV: Operation Ended:   $end_time\n";
print "KLV: Done\n";
# EOF

To produce a KML or KMZ file from kismet xml and gps files, there are several solutions available. I'm using another self revised version of the php based converter made by P. Niquille, and posted on www.niquille.com.
I like this one because you can use it to visualize the coverage by any specific detected network, based on the data from the .gps file. I changed the visualization list to show the ESSID just like the AP list, iso BSSID, making it easier to select the AP to visualize, as well as some minor bugfixes.
Works under both windows and Linux. Requires PHP5 installed.

Code:

<?php

/* 
Kismet to kml (Google Earth) log file converter alias "Kismet Earth"
Released by Philippe Niquille, philippe at niquille dot com, www.niquille.com

******************************
* version 0.1 beta3 - 28.9.2005
******************************

features
--------
parses Kismet log files (.xml and .gps) to generate a google earth kml file
- draws WarDrive Path as tesselated line
- adds discovered access points to specific location on the map, parses meta information (mac, channel, packets, clients, etc.)
- draws 3D polygons instead of simple AP images. The shape should represent the signal area as far as every corner has been
captured correctly. The height depends on the channel. The color is yellow (open), green (closed) or blue when open and cloaked.
- parses ap_manuf and client_manuf for vendor information
- saves .kml as .kmz (zipped) upon request

requirements
------------
PHP5 because of simple_xml, file_get_content functions

usage
-----
Use it as command line script as followed:
/usr/bin/php -f parse.php <Kismet-Sep-16-2005-5> <kmz>

The first argument is the kismet log file name without extension. The second one is optional.

changelog
---------
v0.1 beta3-ds - Small bugfixes and changed the Visualized list to show ESSID iso BSSID making it easier to select both
the AP icon and the corresponding visualized polygon.
v0.1 beta3 - added pack as kmz functionality (zip compression)
v0.1 beta2 - replaced real slow for loop with foreach and gps coordinates index array, huge speed boost
- added vendor information parser, added to description
- tweaked description section
v0.1 beta1 - initial release

todo/wishlist
-------------
- parse default password list
- give the WarSession a name
- merge multiple log files
- get better icons
- better polygons... (interpolation?)
- do dynamic (network connection) stuff

*/

/*******************
* set vars
********************/
$ap_manuf_file = "ap_manuf";
$client_manuf_file = "client_manuf";

//$argv[1] = "Kismet-20041223-10"; //debug
//$argv[2] = "kmz";

/*******************
* execution time
********************/
$mtime = explode(" ",microtime());
$mtime = $mtime[1] + $mtime[0];
$starttime = $mtime;

/*******************
* open files
********************/
if(isset($argv[1]))//get filenames
{
$xml_file = $argv[1].'.xml';
$gps_file = $argv[1].'.gps';
}else
exit("No argument specified\n");

if(file_exists($xml_file) && file_exists($gps_file))
{
$xml = simplexml_load_file($xml_file); //load kismet log
$gps = simplexml_load_file($gps_file); //load corresponding detailed gps log
}else
exit("Files couldn't be opened: ".$xml_file.", ".$gps_file."\n");

if(file_exists($ap_manuf_file) && file_exists($client_manuf_file))
{
$ap_manuf = parse_vendor($ap_manuf_file); //load access point vendor data
$client_manuf = parse_vendor($client_manuf_file); //load client vendor data
}else
exit("Files couldn't be opened: ".$ap_manuf_file.", ".$client_manuf_file."\n");

$array = object2array($xml); //convert simplexml object to php array, easier to work on..

/*******************
* do meta stuff
********************/

$meta['session-duration'] = strtotime($xml['end-time'])-strtotime($xml['start-time']); //get duration (unix timestamp, in seconds)
$meta['session-duration'] = floor($meta['session-duration']/3600).' h, '.floor($meta['session-duration']/60).' min'; //format

$meta['net-count'] = count($array['wireless-network']);
$meta['parsed'] = $meta['net-count'];

/*******************
* parse .gps file and build gps coordinates index
********************/
foreach($gps->{'gps-point'} as $value)
{
$coordinates[(string) $value['bssid']][] = array((string) $value['lon'], (string) $value['lat']);
}

/*******************
* loop trough kismet wireless nets - types: infrastructure, probe
********************/
foreach($array['wireless-network'] as $key => $network)
{
if(
($network['gps-info']['min-lat'] == '90.000000' && $network['gps-info']['min-lon'] == '180.000000') //remove nets with no gps coordinates, no sense to display them on a map
|| $network['BSSID'] == '00:00:00:00:00:00'
|| $xml->{'wireless-network'}[$key]['type'] != 'infrastructure') //remove zero bssid nets, just junk
{
if($xml->{'wireless-network'}[$key]['type'] == 'probe') //count probes
$meta['probe-count']++;

$meta['net-count']--; //remove non infrastructure from total net count, exit foreach
continue;

}elseif(!isset($lookatnet)) //get the first usable net to set LookAt coordinates
$lookatnet = $key;

$kml .= "<Placemark>\n<name>";
$kml .= "<![CDATA[";
if(empty($network['SSID'])) $kml .= "no ssid";
else $kml .= $network['SSID'];
$kml .= "]]></name>\n";

/*******************
* build description html
********************/
$seenfor = strtotime($xml->{'wireless-network'}[$key]['last-time'])-strtotime($xml->{'wireless-network'}[$key]['first-time']);
$kml .= "\t<description><![CDATA[SSID :";
if(empty($network['SSID'])) $kml .= "no ssid";
else $kml .= $network['SSID']."<br>\n";

$kml .= "seen for ".floor($seenfor/60)." min (".floor($seenfor)." sec)<br>"; //get duration
$kml .= "first seen: ".$xml->{'wireless-network'}[$key]['first-time']."<br>last seen: ".$xml->{'wireless-network'}[$key]['last-time']."<br><hr>\n";
$kml .= "BSSID: ".$network['BSSID']."<br>\nchannel: ".$network['channel']."<br>\n";

if($xml->{'wireless-network'}[$key]['wep'] == 'true') //net encrypton on?
{
if($network['encryption'] == 'WEP')
$kml .= "<font color=\"green\">encryption: WEP</font>";

elseif(is_array($network['encryption']))
{
$kml .= "<font color=\"green\">encryption: ";

foreach($network['encryption'] as $enc)
{
$kml .= $enc." ";
}
$kml .= "</font>";
}
}else
$kml .= "<font color=\"red\">no encryption</font>";
$kml .= "<br>\ncloaked: ".$xml->{'wireless-network'}['cloaked'];
$kml .= "<br>\nmax rate: ".$xml->{'wireless-network'}['maxrate'];

if(isset($network['ip-address']))
$kml .= "<br>IP range: ".$network['ip-address']['ip-range'];

/*******************
* get vendor info
********************/
if(get_manuf((string) $network['BSSID'],$ap_manuf) != false)
{
$manuf = get_manuf((string) $network['BSSID'],$ap_manuf);
$kml .= "<hr>\n<b>Vendor Info</b><br>Type: ".$manuf[0]." ".$manuf[1];
if(!empty($manuf[2]))
$kml .= "<br>default SSID: ".$manuf[2].", channel ".$manuf[3];
if(!empty($manuf[4]))
$kml .= "<br>default ip range: ".$manuf[4];
}

$kml .= "<hr>\n<b>GPS coordinates</b><br>\nfirst-seen: ".$network['gps-info']['min-lat']." lat, ".$network['gps-info']['min-lon']." lon, ".round($network['gps-info']['min-alt'],1)." alt";
$kml .= "\n<br>last-seen: ".$network['gps-info']['max-lat']." lat, ".$network['gps-info']['max-lon']." lon, ".round($network['gps-info']['max-alt'],1)." alt";
$kml .= "<hr>\n<b>captured packets</b><br>\n";

/*******************
* list the packets section (IV's, LLC, Data, etc.)
********************/
foreach($network['packets'] as $tag => $value)
{
$kml .= $tag.": ".$value."<br>\n";
}
$kml .= "total datasize captured: ".$network['datasize']."\n";


/*******************
* parse clients, if existant - types: fromds, tods
********************/
if(isset($network['wireless-client']))
{
$kml .="<hr>\n<b>captured ".count($network['wireless-client'])." attached clients</b>\n<br>\n<ul>\n";

foreach($network['wireless-client'] as $client) //list all attached and captured clients
{
$kml .= "<li>MAC: ".$client['client-mac'];

if(get_manuf((string) $client['client-mac'],$client_manuf) != false)
{
$manuf = get_manuf((string) $client['client-mac'],$client_manuf);
$kml .= " (".$manuf[0]." ".$manuf[1].")";
}

$kml .= "<br>IP: ";
if(isset($client['client-ip-address'])) //IP discovered?
$kml .= $client['client-ip-address'];
else
$kml .= "none discovered";
$kml .= "</li>\n";
}
}

$kml .= "</ul>]]>\n</description>\n\t<View>\n\t<longitude>".$network['gps-info']['min-lon']."</longitude>\n";
$kml .= "\t<latitude>".$network['gps-info']['min-lat']."</latitude>\n</View>\n";

$kml .= "\t<visibility>1</visibility>\n\t<styleUrl>root://styleMaps#default?iconId=0x307</styleUrl>\n";

if($xml->{'wireless-network'}[$key]['wep'] == 'true') //net encrypton on?
{
if($argv[2] == 'kmz') //if kmz requested point to local icons
$kml .= "\t<Style>\t<icon>node_closed.png</icon></Style>";
else
$kml .= "\t<Style>\t<icon>http://www.niquille.com/wp-content/node_closed.png</icon></Style>";

$meta['wep-count']++;
}else{
if($argv[2] == 'kmz')
$kml .= "\t<Style>\t<icon>node_open.png</icon></Style>";
else
$kml .= "\t<Style>\t<icon>http://www.niquille.com/wp-content/node_open.png</icon></Style>";
}

$kml .= "\t<Point>\n\t<coordinates>".$network['gps-info']['min-lon'].", ".$network['gps-info']['min-lat'].", ".$network['gps-info']['min-alt']."</coordinates>\n</Point>\n</Placemark>\n";

/*******************
* add gps coordinates to path source
********************/
$line .= $network['gps-info']['min-lon'].", ".$network['gps-info']['min-lat'].", ".$network['gps-info']['min-alt']." ";

/*******************
* assemble polygon placemark
********************/

if($network['channel'] != '0') //we don't display networks with no height
{

$poly .= "<Placemark>\n<name><![CDATA[";
if(empty($network['SSID'])) $poly .= "no ssid";
else $poly .= $network['SSID'];
echo "SSID:".$network['SSID']." BSSID:".$network['BSSID']."\n";
$poly .= "]]></name>\n<visibility>0</visibility>\n<open>0</open>\n";
$poly .= "<Style>\n\t<LineStyle>\n\t<width>1.5</width></LineStyle>\n\t<PolyStyle>";

if($xml->{'wireless-network'}[$key]['wep'] == 'true') //set color 
$poly .= "<color>8f00ff00</color>\n"; //green, closed
elseif($meta[$key]['cloaked'] == 'true')
$poly .= "<color>7dff0000</color>\n"; //blue, cloaked
else
$poly .= "<color>7d00ff00</color>\n"; //red, open

$poly .= "</PolyStyle>\n</Style>\n<Polygon>\n<extrude>1</extrude>\n<tessellate>0</tessellate>\n";
$poly .= "<altitudeMode>relativeToGround</altitudeMode>\n<outerBoundaryIs>\n<LinearRing>\n<extrude>0</extrude>";
$poly .= "<tessellate>0</tessellate>\n<altitudeMode>clampToGround</altitudeMode>\n<coordinates>";

$poly_alt = $network['channel']*10;

foreach($coordinates[$network['BSSID']] as $index => $gpoints)
{
if($gpoints[0] != $coordinates[$network['BSSID']][$index-1][0]
&& $gpoints[1] != $coordinates[$network['BSSID']][$index-1][1]) //remove duplicates
$poly .= $gpoints[0].", ".$gpoints[1].", ".$poly_alt." ";
}

$poly .= $coordinates[$network['BSSID']][0][0].", ".$coordinates[$network['BSSID']][0][1].", ".$poly_alt." "; //finish polygon at first coordinate
$poly .= "</coordinates>\n</LinearRing>\n</outerBoundaryIs>\n</Polygon>\n</Placemark>";

}
} //end foreach

/*******************
* assemble final KML xml file
********************/
$meta['wep-count'] = ($meta['wep-count']/$meta['net-count'])*100; //encrypted percentage of total nets

$kml_final = "<?xml version='1.0' encoding='UTF-8'?>\n<kml xmlns='http://earth.google.com/kml/2.0'>\n<Folder>\n<name>";
$kml_final .= $meta['session-duration'].', '.$meta['net-count'].' nets, '.$meta['probe-count'].' probes ('.$meta['parsed'].' total parsed) - '.date('d.m.Y',strtotime($xml['start-time']));
$kml_final .= "</name>\n<visibility>1</visibility><description>encrypted: ".round($meta['wep-count'],2)."%</description>\n";

//fly to first network
//range: zoom level, tilt: view angle
$kml_final .= "<LookAt>\n\t<longitude>".$array['wireless-network'][$lookatnet]['gps-info']['min-lon']."</longitude>\n\t<latitude>".$array['wireless-network'][$lookatnet]['gps-info']['min-lat']."</latitude>";
$kml_final .= "<range>1000</range><tilt>54</tilt><heading>-35</heading></LookAt>";

// output the WarDrive Path in a separate placemark
$kml_final .= "\n<Placemark>\n<name>WarSession Path</name>\n<description></description>\n<visibility>0</visibility>\n";
$kml_final .= "\t<Style>\n\t<geomColor>BB0000FF</geomColor>\n<geomScale>3</geomScale>\n</Style>\n\t<LineString>";
$kml_final .= "\n\t<tessellate>1</tessellate>\n\t<coordinates>$line</coordinates>\n</LineString>\n</Placemark>\n"; //tesselate adjusts to terrain

//output 3D visualized polygons
$kml_final .= "<Folder>\n<name>3D visualized view</name><open>0</open>\n";
$kml_final .= $poly;
$kml_final .= "</Folder>\n";

$kml_final .= $kml;

$kml_final .= "</Folder>\n</kml>";

//$kml_final = simplexml_load_string($kml_final); //check kml syntax (xml), fails because of <br>, <li> in description tag
//$kml_final->asXML('new.kml');







writeXML($argv[1].".kml",$kml_final); //write final XML file as KML to disk
echo $argv[1].".kml saved.\n";

if($argv[2] == 'kmz') //output as kmz (zipped) if second command line argument supplied
{
$zip = new zipfile();
$zip->addFiles(array($argv[1].".kml","node_open.png","node_closed.png")); //also include icons
$zip->output($argv[1].".kmz");
echo $argv[1].".kmz saved.\n";
}

/*******************
* execution time
********************/
$mtime = explode(" ",microtime());
$mtime = $mtime[1] + $mtime[0];
$endtime = $mtime;
$totaltime = ($endtime - $starttime);
echo "Parsed in ".round($totaltime,2)." seconds\n";

/*******************
* functions
********************/
function parse_vendor($filename)
{
$array = explode("\n",file_get_contents($filename));

foreach($array as $row)
{
$tabs = explode("\t",$row);
$mac = explode("/",$tabs[0]);
array_shift($tabs); //pop off first element (mac) 

if($mac[1] == "FF:FF:FF:FF:00:00")
$mac = rtrim($mac[0],":00:00");
elseif($mac[1] == "FF:FF:FF:00:00:00")
$mac = rtrim($mac[0],":00:00:00").":*";

$macarray[(string) $mac] = $tabs;

} //end foreach
return $macarray;
/*
get's array with all the different mac ranges and vendor/manufactor info:

Array ( [00:00:22:22] => 
Array ( [0] => Lucent 
[1] => Orinoco Silver 
[2] => 
[3] => 0 
[4] => )
etc.
*/
}

function get_manuf($mac,$macarray)
{
$longmac = substr($mac,0,11); //format to array style
$shortmac = substr($mac,0,9)."*";

if(isset($macarray[$longmac]))
return $macarray[$longmac];

elseif(isset($macarray[$shortmac]))
return $macarray[$shortmac];

else
return false; //no info found
}

function writeXML($filename, $kml)
{
$handle = fopen($filename,'w+'); //open xml file for writing
// fwrite($handle, header("Content-type: text/xml")); //write header to xml file
fwrite($handle, $kml);
fclose($handle);
}

/*******************
* pulled from php.net
********************/
function object2array($object)
{
$return = NULL;

if(is_array($object))
{
foreach($object as $key => $value)
$return[$key] = object2array($value);
}
else
{
$var = get_object_vars($object);

if($var)
{
foreach($var as $key => $value)
$return[$key] = ($key && !$value) ? NULL : object2array($value);
}
else return $object;
}

return $return;
}

/*
* zipLib v1.1
*
* Last Modification and Extension By :
*
* Hasin Hayder
* HomePage : www.hasinme.info
* Email : countdraculla@gmail.com
* IDE : PHP Designer 2005
*
*
* Originally Based on :
*
* http://www.zend.com/codex.php?id=535&single=1
* By Eric Mueller <eric@themepark.com>
*
* http://www.zend.com/codex.php?id=470&single=1
* by Denis125 <webmaster@atlant.ru>
*
* a patch from Peter Listiak <mlady@users.sourceforge.net> for last modified
* date and time of the compressed file
*/

class zipfile
{
/**
* Array to store compressed data
*
* @var array $datasec
*/
var $datasec = array();

/**
* Central directory
*
* @var array $ctrl_dir
*/
var $ctrl_dir = array();

/**
* End of central directory record
*
* @var string $eof_ctrl_dir
*/
var $eof_ctrl_dir = "\x50\x4b\x05\x06\x00\x00\x00\x00";

/**
* Last offset position
*
* @var integer $old_offset
*/
var $old_offset = 0;


/**
* Converts an Unix timestamp to a four byte DOS date and time format (date
* in high two bytes, time in low two bytes allowing magnitude comparison).
*
* @param integer the current Unix timestamp
*
* @return integer the current date in a four byte DOS format
*
* @access private
*/
function unix2DosTime($unixtime = 0) {
$timearray = ($unixtime == 0) ? getdate() : getdate($unixtime);

if ($timearray['year'] < 1980) {
$timearray['year'] = 1980;
$timearray['mon'] = 1;
$timearray['mday'] = 1;
$timearray['hours'] = 0;
$timearray['minutes'] = 0;
$timearray['seconds'] = 0;
} // end if

return (($timearray['year'] - 1980) << 25) | ($timearray['mon'] << 21) | ($timearray['mday'] << 16) |
($timearray['hours'] << 11) | ($timearray['minutes'] << 5) | ($timearray['seconds'] >> 1);
} // end of the 'unix2DosTime()' method


/**
* Adds "file" to archive
*
* @param string file contents
* @param string name of the file in the archive (may contains the path)
* @param integer the current timestamp
*
* @access public
*/
function addFile($data, $name, $time = 0)
{
$name = str_replace('\\', '/', $name);

$dtime = dechex($this->unix2DosTime($time));
$hexdtime = '\x' . $dtime[6] . $dtime[7]
. '\x' . $dtime[4] . $dtime[5]
. '\x' . $dtime[2] . $dtime[3]
. '\x' . $dtime[0] . $dtime[1];
eval('$hexdtime = "' . $hexdtime . '";');

$fr = "\x50\x4b\x03\x04";
$fr .= "\x14\x00"; // ver needed to extract
$fr .= "\x00\x00"; // gen purpose bit flag
$fr .= "\x08\x00"; // compression method
$fr .= $hexdtime; // last mod time and date

// "local file header" segment
$unc_len = strlen($data);
$crc = crc32($data);
$zdata = gzcompress($data);
$zdata = substr(substr($zdata, 0, strlen($zdata) - 4), 2); // fix crc bug
$c_len = strlen($zdata);
$fr .= pack('V', $crc); // crc32
$fr .= pack('V', $c_len); // compressed filesize
$fr .= pack('V', $unc_len); // uncompressed filesize
$fr .= pack('v', strlen($name)); // length of filename
$fr .= pack('v', 0); // extra field length
$fr .= $name;

// "file data" segment
$fr .= $zdata;

// "data descriptor" segment (optional but necessary if archive is not
// served as file)
$fr .= pack('V', $crc); // crc32
$fr .= pack('V', $c_len); // compressed filesize
$fr .= pack('V', $unc_len); // uncompressed filesize

// add this entry to array
$this -> datasec[] = $fr;

// now add to central directory record
$cdrec = "\x50\x4b\x01\x02";
$cdrec .= "\x00\x00"; // version made by
$cdrec .= "\x14\x00"; // version needed to extract
$cdrec .= "\x00\x00"; // gen purpose bit flag
$cdrec .= "\x08\x00"; // compression method
$cdrec .= $hexdtime; // last mod time & date
$cdrec .= pack('V', $crc); // crc32
$cdrec .= pack('V', $c_len); // compressed filesize
$cdrec .= pack('V', $unc_len); // uncompressed filesize
$cdrec .= pack('v', strlen($name) ); // length of filename
$cdrec .= pack('v', 0 ); // extra field length
$cdrec .= pack('v', 0 ); // file comment length
$cdrec .= pack('v', 0 ); // disk number start
$cdrec .= pack('v', 0 ); // internal file attributes
$cdrec .= pack('V', 32 ); // external file attributes - 'archive' bit set

$cdrec .= pack('V', $this -> old_offset ); // relative offset of local header
$this -> old_offset += strlen($fr);

$cdrec .= $name;

// optional extra field, file comment goes here
// save to central directory
$this -> ctrl_dir[] = $cdrec;
} // end of the 'addFile()' method


/**
* Dumps out file
*
* @return string the zipped file
*
* @access public
*/
function file()
{
$data = implode('', $this -> datasec);
$ctrldir = implode('', $this -> ctrl_dir);

return
$data .
$ctrldir .
$this -> eof_ctrl_dir .
pack('v', sizeof($this -> ctrl_dir)) . // total # of entries "on this disk"
pack('v', sizeof($this -> ctrl_dir)) . // total # of entries overall
pack('V', strlen($ctrldir)) . // size of central dir
pack('V', strlen($data)) . // offset to start of central dir
"\x00\x00"; // .zip file comment length
} // end of the 'file()' method


/**
* A Wrapper of original addFile Function
*
* Created By Hasin Hayder at 29th Jan, 1:29 AM
*
* @param array An Array of files with relative/absolute path to be added in Zip File
*
* @access public
*/
function addFiles($files /*Only Pass Array*/)
{
foreach($files as $file)
{
if (is_file($file)) //directory check
{
$data = implode("",file($file));
$this->addFile($data,$file);
}
}
}

/**
* A Wrapper of original file Function
*
* Created By Hasin Hayder at 29th Jan, 1:29 AM
*
* @param string Output file name
*
* @access public
*/
function output($file)
{
$fp=fopen($file,"w");
fwrite($fp,$this->file());
fclose($fp);
}



} // end of the 'zipfile' class

?>

Dutch

EDIT : New much faster optimized Kismet Log Combiner version available from this thread : New Dutch revised version of KLC

Mark57 · 02-06-2006

Good stuff here guys. I'm looking forward to testing.

Scruge · 02-06-2006

@ Dutch

Actually I don't have any use for the csv converter, but because several others here had a need and it was easy to add to my NS1 and Wigle cache converter I proceeded with the addition. The stupid mistake I made was never testing the executables on machines that hadn't had dbase installed on them first.

I used to develop in xbase many years ago and about 6 months ago I dusted off one of my old compilers (db2k) to write a few programs for my own needs. Although I first I tried developing a few things with the Visual languages I had, I kept going back to xbase because it was 20-50x faster.

db2k is based on the 32 bit Borland Database Engine (BDE 5.1). It screams with speed being its all hard coded in assembler. As an example it will physically sort a 5 million record 1GB file in less than 5 minutes, typical is less than 3 mins depending on the sort key. It can build an index in less than 90 seconds on the same file. All test were on machines running 1.4ghz cpus or less. I've got another ap that has a very complex relational structure between several tables/dbf files. It processes network densities and some other things plus produces 4 different output files that runs almost linear with your cpu speed at ~ 1000 records per second per 1ghz. It takes my 1.1ghz machine ~1.4hrs to process 5 kk records.

However, from what I can tell it looks like my "Installshield Express" is out of date and therefore isn't including all necessary files when it compiles. The "Installshield" determines whether the executable is deployable. So I'll have to see what's necessary to fix that before I post anymore non functioning apps.

However if anyone wants to test the object code I can provide. You'll need to find a copy of "db2k" or "dbase 7.5" and install. There are several trial and student versions floating around. FYI don't bother with versions earlier like 5, 5.5 or 7 or later like vdb2.1, vdb2.5 etc. Borland sold the product off to DBI in the late 90s which resulted in a name version change.

edit...
Dutch I almost forgot.. yes it removes all duplicates from the kml output files and processed Frag's sample 3000+ nets .csv files in 2.5seconds.

Dutch · 02-06-2006

Quote:

Originally Posted by Scruge

However if anyone wants to test the object code I can provide. You'll need to find a copy of "db2k" or "dbase 7.5" and install. There are several trial and student versions floating around. FYI don't bother with versions earlier like 5, 5.5 or 7 or later like vdb2.1, vdb2.5 etc. Borland sold the product off to DBI in the late 90s which resulted in a name version change.

I know where you're coming from. I have kindda the same relation with Filemaker 4. I still use it to test various DB structures, and do q'n'd import, sorting, displaying and exports. Why ? Because I know the inner and outer workings of that program. The best tool for the job, is allways the tool that you know how to use.

Quote:

Originally Posted by Scruge

edit...
Dutch I almost forgot.. yes it removes all duplicates from the kml output files and processed Frag's sample 3000+ nets .csv files in 2.5seconds.

Ahh.. Somehow I thought you were using the .xml and .gps files. I prefer those, over the .csv files, as I then can get data on which clients belongs to which AP's, as well as getting the area coverage visualized.
I'll post a screenshot, when I reboot to Windows, from a G.E. display of the coverage of an AP, produced from the .xml and .gps data, with the above php script. Pretty nifty, and a cheap addition to a site survey to show how much a sites network bleeds into the surroundings.
The screendump will be of a local school, who just setup WiFi, and are polluting the airwaves for much further than necessary.

It'll be some hours though, as I'm busy getting the Evil Bastard project documentation finished and the E.B package out the door.
Which reminds me.. Anybody got a heads up on when the CoWF site will be back up ? Streaker ?

Dutch

Dutch · 02-06-2006

Quote:

02-05-2006	#1 (permalink)
Scruge Nyuk nyuk! Join Date: Jan 2005 Location: TX Posts: 1,357	KISMACK KISmet Merge And Convert to Kml The initial results sound promising. So I'm giving it another shot. KisMaCK is a utility to merge, maintain and convert Kismet CSV, XML and GPS logs into a master database and produce KML files for Google earth. ..features 1) Maintains a unique set of data for both net and gps data, meaning no duplicate records. 2) Updates the master records each time new data is processed. 3) Recovers orphan nets. 4) Provides a report for nets not having gps data. 5) Provides a stat report of each log file processed 6) Provides a source record for all nets processed 7) Consolidates all 3 file types into a unique set of data and converted to a KML, based on drive date/time. 8) IEEE hardware vendor info is provided, via MAC vendor registration. 9) Uses a std .dbf file structure that can be accessed via various means MS Access, MS Excel, SQL etc. Kismack has been renamed to KNSGEM If the feed back continues to be positive I'll add NS1 file support. Last edited by Scruge : 05-02-2006 at 04:59 AM.

02-05-2006	#7 (permalink)
fregniacciaro Where's my Guinness? Join Date: Jun 2002 Location: Sq'rat on the loose!! Shoot 'im! (not you G8t) Posts: 1,943	Scruge - I just sent the CSV files, not the GPS or XML. The number of networks is different, but not by much (at least in my instances) as you can tell from Dutch's post here. Please let me know if you want those files to help with your program. I don't think I have them from that same drive, but I can send you a couple that were tar'd & gzip'd from a later date __________________ When I was your age, television was called books. And this is a special book. It was the book my father used to read to me when I was sick, and I used to read it to your father. And today, I'm gonna read it to you. Rompipalle della Chiesa del WiFi SHAMELESS PLUG! "Are you trying to irritate my colon ? Because if you are, I'll shit all over you in such a way you'll end up in a sanatorium, never to be heard from again." --Dutch

02-06-2006	#11 (permalink)
Scruge Nyuk nyuk! Join Date: Jan 2005 Location: TX Posts: 1,357	@ Dutch Actually I don't have any use for the csv converter, but because several others here had a need and it was easy to add to my NS1 and Wigle cache converter I proceeded with the addition. The stupid mistake I made was never testing the executables on machines that hadn't had dbase installed on them first. I used to develop in xbase many years ago and about 6 months ago I dusted off one of my old compilers (db2k) to write a few programs for my own needs. Although I first I tried developing a few things with the Visual languages I had, I kept going back to xbase because it was 20-50x faster. db2k is based on the 32 bit Borland Database Engine (BDE 5.1). It screams with speed being its all hard coded in assembler. As an example it will physically sort a 5 million record 1GB file in less than 5 minutes, typical is less than 3 mins depending on the sort key. It can build an index in less than 90 seconds on the same file. All test were on machines running 1.4ghz cpus or less. I've got another ap that has a very complex relational structure between several tables/dbf files. It processes network densities and some other things plus produces 4 different output files that runs almost linear with your cpu speed at ~ 1000 records per second per 1ghz. It takes my 1.1ghz machine ~1.4hrs to process 5 kk records. However, from what I can tell it looks like my "Installshield Express" is out of date and therefore isn't including all necessary files when it compiles. The "Installshield" determines whether the executable is deployable. So I'll have to see what's necessary to fix that before I post anymore non functioning apps. However if anyone wants to test the object code I can provide. You'll need to find a copy of "db2k" or "dbase 7.5" and install. There are several trial and student versions floating around. FYI don't bother with versions earlier like 5, 5.5 or 7 or later like vdb2.1, vdb2.5 etc. Borland sold the product off to DBI in the late 90s which resulted in a name version change. edit... Dutch I almost forgot.. yes it removes all duplicates from the kml output files and processed Frag's sample 3000+ nets .csv files in 2.5seconds. Last edited by Scruge : 02-06-2006 at 08:29 AM.

02-05-2006	#8 (permalink)
Scruge Nyuk nyuk! Join Date: Jan 2005 Location: TX Posts: 1,357	Works great on my system and any of the others that have had the compiler installed on them. I'm such a tight-wad I haven't updated my "installshield express" so its not building the exe completely. I'll check around next week to see if I can find a replacement or upgrade. @ Frag The problem isn't with your files. Once I get the installer fixed I'll take a look at the gps and xml files to see if they can be added. @ Mods If you guys want to dump this thread feel free to. I can always make another attempt later.

02-06-2006	#10 (permalink)
Mark57 What?? Join Date: May 2005 Location: OKC Posts: 1,240	Good stuff here guys. I'm looking forward to testing.

02-06-2006	#13 (permalink)
Dutch Humourless EuroMod. Join Date: Mar 2004 Location: City of Mermaids, Denmark Posts: 6,813	Quote: